Metadata-Version: 2.4
Name: reap-cli
Version: 0.3.0
Summary: Recursive extraction and parsing of firmware and partition images
Author: Blackbox Research
License-Expression: Apache-2.0
Keywords: firmware,forensics,reverse-engineering,android,partition,extraction,unpacker
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: System :: Archiving
Classifier: Topic :: System :: Filesystems
Classifier: Topic :: Utilities
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: ext4>=1.2.2
Requires-Dist: lz4>=4.0.0
Requires-Dist: zstandard>=0.22.0
Provides-Extra: dev
Requires-Dist: bandit>=1.7; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Requires-Dist: hypothesis>=6.100; extra == "dev"
Requires-Dist: pip-audit>=2.7; extra == "dev"
Requires-Dist: py7zr>=0.21; extra == "dev"
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"
Requires-Dist: twine>=6.1; extra == "dev"
Requires-Dist: tomli>=2.0; python_version < "3.11" and extra == "dev"
Requires-Dist: cryptography>=43.0; extra == "dev"
Provides-Extra: release
Requires-Dist: build==1.5.0; extra == "release"
Requires-Dist: id==1.6.1; extra == "release"
Requires-Dist: pypi-attestations==0.0.30; extra == "release"
Requires-Dist: setuptools==83.0.0; extra == "release"
Requires-Dist: twine==7.0.0; extra == "release"
Provides-Extra: signing
Requires-Dist: cryptography>=43.0; extra == "signing"
Provides-Extra: fuzz
Requires-Dist: atheris>=2.3; platform_system != "Windows" and extra == "fuzz"
Dynamic: license-file

# REAP

**Recursive Extraction And Parsing** — a general-purpose CLI tool for identifying and recursively extracting firmware and partition images. Works with raw eMMC/flash dumps, individual partition images, full-disk images (GPT or Rockchip PARM), and forensic disk images from a wide range of embedded Linux and Android devices.

Pure Python. No root, no FUSE, no mounting, no Linux kernel modules. Runs on macOS, Linux, and Windows.

## What it does

Point it at a directory of partition `.bin` files, a single image, or a set of 7z archives and it will:

1. **Identify** each image's format through more than 60 ordered core detectors
2. **Annotate** what partition it is (boot, recovery, system, userdata, etc.) by reading ext-family superblock metadata and analyzing ramdisk contents
3. **Extract** contents recursively -- e.g. a boot image yields a kernel + ramdisk; the ramdisk decompresses to a cpio archive; the cpio extracts to a filesystem tree
4. **Analyze** kernels (version, config, build paths, kallsyms symbol table), bootloaders (U-Boot environment, embedded DTBs), and unknown partitions (forensic hex dump, strings, SHA256)
5. **Report** everything found in text, JSON, CSV, normalized findings/SARIF,
   opt-in CycloneDX inventory, and deterministic SHA-256 forensic manifests,
   including FBE encryption detection

## Supported formats

### Partition tables and disk layouts

| Format | Detection | Extraction |
|--------|-----------|------------|
| GPT partition table | `EFI PART` at offset 0x200 or 0x1000 (UFS 4K sectors) | Individual partition images |
| MBR / DOS partition table | `0x55AA` at offset 510 + a valid primary entry (not a FAT VBR or GPT protective MBR) | Primary and bounded EBR logical partitions carved at their LBA-start offsets, with cycle/overlap checks, then recursed. |
| Apple Partition Map | Consistent `ER` Driver Descriptor Map and declared `PM` entry stride | Validates the complete map, driver coverage, logical/physical spans, overlap, and exact provider geometry before atomically carving data partitions |
| FreeBSD BSD disklabel | Redundant little-endian disklabel magic in sector 1 with an explicit 512--8192-byte sector size | Validates checksum, raw-provider geometry, relative offsets, and overlap before atomically carving nonempty partitions; embedded absolute-offset labels fail closed without parent-origin evidence |
| Rockchip PARM partition table | `PARM` at offset 0 | Individual partition images (RK29xx/RK3xxx flash dumps) |
| Android super.img (LP metadata) | `0x67446C70` at offset 0x1000 | Logical partition images (system, vendor, product, etc.) |
| QCOW2 v2/v3 disk image | `QFI\xFB` at offset 0 | Integrity-checked flattening of internal, uncompressed data and deterministic holes/zero clusters to a raw child image; unsupported backing, encryption, snapshots, external data, compression, and extended-L2 layouts fail closed |
| VirtualBox VDI v1.1 disk image | `0xBEDA107F` signature at offset 0x40 | Flattens standard 1 MiB-block dynamic and static images to an exact raw child; unallocated/discarded dynamic blocks become zeros, while links/parents, saved/trailing regions, nonstandard extras, and ambiguous maps fail closed |
| VMware VMDK hosted sparse v1 | `KDMV` at offset 0 | Flattens a self-contained `monolithicSparse` extent to an exact raw child after authenticating its embedded descriptor, redundant directories/tables, physical ownership, and exact file envelope; split/flat, stream-optimized, compressed, snapshot/parent, external-extent, unclean, and trailing-data variants fail closed |
| Classic VHD fixed/dynamic disk image | Checksum-valid `conectix` footer at byte 0 or EOF; a tail-only dynamic match additionally requires an authenticated header | Flattens fixed and expandable type-2/type-3 images to an exact sparse raw child after footer, header, BAT, bitmap, bounds, and overlap validation; one damaged dynamic footer copy can be recovered unambiguously, saved-state sources are read without mutation, and differencing/parent/VHDX images fail closed |
| LVM2 physical volume | CRC-valid `LABELONE` / `LVM2 001` label in sector 0--3 | Extracts exact logical-volume children from a single PV with one data area, one or two validated metadata areas, and linear (`striped`, count 1) mappings; unsupported or ambiguous layouts fail closed before atomic publication |

### Android boot and kernel

| Format | Detection | Extraction |
|--------|-----------|------------|
| Android Boot Image (v0--v4) | `ANDROID!` magic | Kernel, ramdisk, second-stage, recovery DTBO, DTB, and v4 GKI boot signature; legacy AOSP SHA-1 IDs are checked when present |
| Android Init Boot Image (v4) | `ANDROID!` v4 with a generic ramdisk and no kernel | Generic init ramdisk, distinguished from ordinary boot/recovery images |
| Android Vendor Boot Image (v3/v4) | `VNDRBOOT` magic | v3 vendor ramdisk + DTB; v4 bounded ramdisk-table fragments, DTB, and bootconfig with names/types/board IDs retained as metadata |
| Android update_engine full OTA | `CrAU` header (major versions 1/2 identified) | Major-version-2, minor-version-0 full payloads reconstruct exact target partition images from bounded `REPLACE`, bzip2, and XZ operations; delta/partial payloads fail closed without a validated source |
| ARM zImage | `0x016F2818` at offset 0x24 | Decompressed vmlinux, kernel config, version string, source paths, kallsyms, all strings |
| ARM64 Image | `ARM\x64` at offset 0x38 | Kernel config, version string, source paths, kallsyms, all strings |
| Raw ARM kernel binary | MSR CPSR instruction + `Linux version` string | Kernel config, version string, source paths, kallsyms, all strings |
| Device Tree Blob (DTB) | `0xD00DFEED` | Extracted DTB, optional `dtc` decompile to DTS |
| DTBO container | `0xD7B7AB1E` | Individual DT overlay entries |

### Bootloaders and firmware

| Format | Detection | Extraction |
|--------|-----------|------------|
| U-Boot FIT/ITB | Structurally valid version-17 FDT with unique `/images` and `/configurations` nodes | Atomically publishes every inline or supported external image, verifies CRC32/SHA-1/SHA-2 hashes, inventories untrusted signature metadata, and validates configuration references; see [FIT support](docs/FIT.md) |
| Microsoft UF2 | Dual start magic in each 512-byte transport block | Reorders authenticated sequence blocks and publishes exact, address-preserving contiguous flash segments; unsupported flags, family conflicts, overlaps, padding, and malformed blocks fail closed |
| U-Boot uImage | `0x27051956` | Unwrapped payload (kernel, ramdisk, firmware, device tree, etc.) |
| U-Boot binary | `U-Boot <version>` string, 64 KB--4 MB | Default environment, embedded DTBs, strings |
| U-Boot environment | CRC32 + key=value pairs, power-of-2 size | Parsed environment variables |
| Samsung Exynos boot partition | BL1 header pointer + `Exynos BL` label | bl1.bin, u-boot.bin, tzsw.bin |
| Rockchip KRNL wrapper | `KRNL` at offset 0 | Unwrapped payload (re-identified as gzip, zImage, etc.) |
| ELF binary | `\x7fELF` magic | Metadata dump (class, machine, entry point), strings |
| AVB vbmeta | `AVB0` / `AVBf` | Metadata dump (version, algorithm, rollback index, flags) |

### Encrypted firmware containers

| Format | Detection | Extraction |
|--------|-----------|------------|
| IM\*H firmware container | `IM*H` at offset 0 or 0x400 | Header parse (version, module name/type, chunk table, key family). Successful extraction preserves encrypted chunks (RTOS, kernel, TZOS, DTB, etc.) as raw `.bin` files; an explicitly selected and locked API-v2 `dji_imah` provider may additionally publish verified plaintext. |
| Ambarella environment (UNR0) | `UNR0` + `0x5AA5` flags | Boot config, A/B slot status, firmware versions, bootloader logs |
| BitLocker (BDE) volume | `-FVE-FS-` OEM ID at offset 3, *or* BitLocker volume GUID at offset 0x10 of an NTFS / FAT-style boot sector | Boot-sector + FVE metadata parse: variant (Vista vs. Win 7+ / To-Go), volume GUID, encryption method (AES-128/256, CBC ± Elephant diffuser, XTS), dataset GUID, FVE block locations, full inventory of key protectors (TPM, TPM+PIN, password, recovery password, startup key, smart card, clear key, etc.). Surfaces clear-key suspended volumes prominently. Decryption is out of scope. |
| LUKS1 / LUKS2 volume | `LUKS\xBA\xBE` magic at offset 0; version word distinguishes LUKS1 from LUKS2 | LUKS1: cipher / mode / hash / key size / UUID + per-keyslot PBKDF2 iterations, salt, active flag. LUKS2: binary header + JSON metadata — segments (cipher), keyslots (PBKDF2 vs. Argon2i / Argon2id with memory + parallelism), tokens (systemd-cryptenroll TPM2 / FIDO2 / PKCS11 bindings), digests. Decryption is out of scope. |

### Filesystems

| Format | Detection | Extraction |
|--------|-----------|------------|
| ext2 / ext3 / ext4 | `0xEF53` at offset 0x438 plus authoritative feature masks | Full filesystem tree with checked legacy direct/indirect traversal and ext4 extents; journals and unsupported features fail closed; ext4 FBE is detected |
| FAT12/16/32 | `0xEB`/`0xE9` + `0x55AA` at 510 | Full filesystem tree (LFN support) |
| exFAT | `EXFAT   ` OEM ID at offset 3 | Full filesystem tree (long names, fragmented + NoFatChain runs) |
| NTFS | `NTFS    ` OEM ID at offset 3 + `0x55AA` at 510 | Full filesystem tree (resident + non-resident `$DATA`, LZNT1 compression, `$ATTRIBUTE_LIST`, `$INDEX_ROOT` + `$INDEX_ALLOCATION`, ADS, USA fixup; EFS-encrypted files marked, reparse points recorded as text) |
| cramfs | `0x28cd3d45` at offset 0 + `Compressed ROMFS` at 16 | Full filesystem tree (zlib blocks, little- and big-endian) |
| romfs | `-rom1fs-` at offset 0 | Full filesystem tree (files, dirs, symlinks, hard links) |
| squashfs | `hsqs` (little-endian) / `sqsh` (big-endian) at offset 0 | Full filesystem tree for v4.0 little-endian images — inode/directory metadata + data/fragment blocks. Compressors: gzip, xz/lzma, lz4, zstd (if `zstandard` installed); lzo and big-endian images are flagged, not decoded |
| JFFS2 | node magic `0x1985` (`85 19` LE / `19 85` BE) at offset 0 | CRC-validated filesystem recovery (common headers, inode/dirent nodes, names, and compressed data); newest valid versions win; none/zero/copy/zlib/rtime compressors are decoded, while lzo/rubin are flagged |
| ISO 9660 / Joliet / Rock Ridge | `CD001` primary descriptor in the sector-16 descriptor set | Bounded single-volume tree recovery with Joliet UCS-2 and RRIP names, attributes, symlinks, and continuations; validated El Torito catalogs yield exact floppy images or explicitly bounded no-emulation loader spans |
| UFS1 / UFS2 (FFS) | Version magic at the canonical 8 KiB or 64 KiB primary-superblock location | Extracts clean standalone FreeBSD-compatible filesystems after validating cylinder groups, backup superblocks, allocation maps, inode reachability/link counts, direct/indirect blocks, paths, and source stability; journals, active recovery, ACL/xattr policy, and special nodes fail closed |
| XFS v5 | `XFSB` at offset 0 | CRC-validates every allocation-group superblock and inventories stable geometry, UUIDs, features, log placement, and counters. Namespace/file extraction is explicitly not yet supported and is reported as metadata-only. |
| UBI | `UBI#` erase-counter header at offset 0 (per PEB) | Reassembles each logical volume from its PEBs (infers PEB size, orders LEBs, parses volume names) and carves each volume as a child image for recursion |
| UBIFS | node magic `0x06101831` at offset 0 | CRC-validated linear node-scan recovery (tolerates mid-commit/partial dumps where the committed index is inconsistent): rebuilds the tree from surviving newest-`sqnum` inode/dentry/data nodes, decompresses none/zlib (lzo/zstd if the optional lib is present), zero-fills + flags missing blocks, and puts unreferenced inodes under `__ORPHANS__/` |
| EROFS | `0xE0F5E1E2` at offset 0x400 | Flat plain/inline files plus conservative LZ4 full-index and 4-byte compact-index, one-pcluster mappings; advanced/chunk/fragment/2-byte layouts fail closed with typed warnings. |
| F2FS | `0xF2F52010` at offset 0x400 | CRC-validated checkpoint-pack recovery, checkpoint-authoritative NAT selection, direct/indirect/double-indirect nodes, sparse files, inline data and inline directories. Encrypted files, compressed clusters, and special inodes fail closed. |
| littlefs v2.0/v2.1 | Canonical block-0 compacted superblock tags, exact declared logical geometry, and a commit-CRC-valid metadata-pair copy; damaged bootstrap routing requires a source-hash-bound geometry manifest | Read-only recovery of nested directories, inline files, CTZ skip-list files, revisions, splices, hard/soft tails, contiguous expanded-superblock prefixes, commit CRCs, and v2.1 forward CRCs; ambiguous geometry, interrupted global state, future semantics, overlap, and corruption fail closed before grouped atomic publication |
| SPIFFS | No geometry-free detector: a source-bound geometry manifest must authenticate the exact byte region and all 16 compile-time/layout parameters | Read-only regular-file recovery across object-index header/continuation pages and data pages, including slash-containing names, non-power-of-two geometry, deleted-page accounting, bounded validation, and grouped atomic no-clobber publication; geometry and unsupported object semantics are never guessed |
| YAFFS2 | No geometry-free detector: a source-bound geometry manifest must authenticate the exact byte region and all 14 NAND/tag/ECC parameters | Conservative regular-file/directory recovery from OOB or in-band tags with sequence/revision selection and optional `yaffs_ecc_other` correction; checkpoint, link/special/shadow/shrink/bad-block, graph, path, and ambiguous-layout cases fail closed |

### Compression and archives

| Format | Detection | Extraction |
|--------|-----------|------------|
| gzip | `1F 8B` | Decompressed content |
| LZ4 frame | `04 22 4D 18` | Decompressed content |
| LZ4 legacy | `02 21 4C 18` | Decompressed content (Android ramdisk format) |
| LZMA | `5D 00 00` | Decompressed content |
| bzip2 | `BZh` | Decompressed content |
| XZ | `FD 37 7A 58 5A 00` | Decompressed content |
| cpio newc | `070701` / `070702` | Files, directories, symlinks (as text files with `-> target`) |
| 7z archive | `37 7A BC AF 27 1C` | Full decompression (supports split `.7z.001` parts) |
| ZIP / TAR / Odin `.tar.md5` | Native signatures and validated TAR trailer | Safely extracts regular members; rejects traversal, links/devices, collisions, corruption, excessive members and expansion; Odin MD5 is verified. |
| Zstandard | `28 B5 2F FD` | Bounded streamed decompression |
| Android sparse image | `0xED26FF3A` | Converted to raw image, then re-identified and extracted |

### Device-specific partitions

| Format | Detection | Extraction |
|--------|-----------|------------|
| Android devinfo | `ANDROID-BOOT!` magic | Lock status, tamper flags |
| ArduPilot / Autel DataFlash flight log | `0xA3 0x95` head + FMT type `0x80` | Self-describing FMT schema, per-message CSVs, decoded firmware versions, and a GPS track (GPX + KML). Auto-detects the Autel 8-byte FMT name field vs. stock ArduPilot's 4-byte field. |
| Qualcomm modemst (EFS) | `IMGEFS` marker in first 64 bytes | Forensic scan (SHA256, strings, hex dump) |
| BMP image | `BM` + valid DIB header | Trimmed BMP (strips partition padding) |
| Boot logo container | ASCII count/sizes header + BMP at 0x200 | Individual BMP images |
| Empty / zeroed | All-zero content | Verified-empty marker with likely purpose annotation |

## Installation

Requires **Python 3.10+** (supported through Python 3.13).

From PyPI:

```bash
pip install reap-cli
```

> The PyPI distribution is `reap-cli` because the bare `reap` name on PyPI is held by an unrelated, long-abandoned 2012 package. We are pursuing a [PEP 541](https://peps.python.org/pep-0541/) transfer. The installed CLI command is `reap` regardless.

From source (for development):

```bash
git clone https://gitlab.com/blackbox-research/reap
cd reap
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
```

Dependencies (installed automatically):
- `ext2` / `ext3` / `ext4` -- pure-Python ext-family filesystem reader (no FUSE/mounting)
- `lz4` -- LZ4 decompression for Android ramdisks
- `zstandard` -- Zstandard archives and filesystem blocks

Normal runs are core-only: REAP neither enumerates nor imports the
`reap.plugins` entry-point group unless at least one repeatable
`--enable-plugin NAME` is supplied. Inspect installed plugin metadata without
importing plugin code using `reap plugins`; even there, `--load` executes code
only when one or more names are explicitly selected with `--enable-plugin`.
Extraction also accepts repeatable `--disable-plugin NAME` options.

Object-based plugin API v1 remains a trusted, serialized, in-process
compatibility interface. Plugin API v2 is a closed subprocess protocol for the
single `decrypt-artifact` capability: an explicitly selected provider must also
match `--plugin-lock`, including its launcher size and SHA-256. Supplying a lock
never enables a plugin by itself. The core hands ciphertext by descriptor,
enforces finite control, time, staging, and output limits, verifies both sides'
size/digest/identity claims, and atomically ingests plaintext. Provenance is
core-owned and excludes key references and control streams. Process separation
is not a same-UID security sandbox, and a launcher digest does not transitively
pin an interpreter or its libraries; see [docs/API.md](docs/API.md) and
[docs/LIMITATIONS.md](docs/LIMITATIONS.md).

The full frozen plugin inventory, including selection state,
API/isolation/capability data, distribution and launcher identities, and lock
digest, is bound into resume snapshots. Forensic manifests bind selected-plugin
evidence and embed the canonical lock digest and document. Incompatible,
conflicting, unknown, or unauthenticated selections fail closed with exit 7.

## Usage

```
reap <input_path> [options]
```

`input_path` can be a single image file, a directory containing partition images, or a set of 7z archives.

### Options

| Flag | Description |
|------|-------------|
| `-o DIR` | Exact output directory for a single file; output root for a directory (defaults: `<file>_unpacked/` or `<directory>/unpacked/`) |
| `--identify-only` | Print format identification only, no extraction |
| `--skip-ext4` | Skip ext4 filesystem extraction (useful for huge partitions) |
| `--skip-archives` | Skip 7z archive extraction |
| `--force-archives` | Force archive extraction even when `physicalImage/` already exists |
| `--no-recursive` | Don't recurse into extracted children |
| `--salvage`, `--carve-scan` | For images with an unparseable partition table (identified as `Unknown`), scan for embedded ext-family/FAT/exFAT/NTFS filesystems and a UBI area (`UBI#` after a boot region) and carve them out for extraction |
| `--max-depth N` | Maximum recursion depth (default: 10) |
| `--max-items N` | Total identified-item admission budget; sorted top-level wave positions are reserved before workers start (default: 100000; 0 = unlimited) |
| `--max-output-bytes N` | Per-artifact extraction cap (default: max(1 GiB, 200× input); 0 = unlimited) |
| `--max-total-output-bytes N` | Thread-safe whole-run output cap (default: 20 GiB; 0 = unlimited) |
| `--timeout SECONDS` | Cooperative deadline; timeout exits 124 and cleans partial files |
| `--progress none\|human\|json` | Progress on stderr; JSON mode is JSON Lines |
| `--resume` | Resume only from a matching hash-validated journal; stale/missing state exits 3 |
| `--geometry-manifest PATH` | Use a versioned, portable-path/size/SHA-256-bound geometry contract for raw littlefs, SPIFFS, or YAFFS2; unknown geometry is never guessed |
| `--enable-plugin NAME` | Explicitly opt into one installed plugin by entry-point name (repeatable); ordinary runs remain core-only |
| `--disable-plugin NAME` | Explicitly exclude one named plugin (repeatable) |
| `--plugin-lock PATH` | Authenticate explicitly enabled API-v2 subprocess providers; the lock never enables a provider by itself |
| `-y`, `--assume-yes` | Use the default archive-skip answer without prompting |
| `-j, --jobs N` | Parallel extraction workers (0=auto, 1=sequential; default: auto) |
| `--fail-fast` | Stop scheduling new deterministic worker waves after the first failed item tree; the failing wave is allowed to finish safely |
| `-v` | Verbose output (INFO level) |
| `-vv` | Debug output |
| `--report text\|json\|csv\|both\|all` | Report format (`all` emits all three; default: both) |
| `--findings none\|json\|sarif\|both` | Publish normalized findings JSON and/or SARIF (default: json) |
| `--fail-on-finding none\|info\|warning\|error` | After publishing evidence, exit 4 when a finding meets the selected severity threshold (default: none) |
| `--sbom` | Opt in to deterministic CycloneDX package and ELF inventory of completed output files |
| `--sbom-max-files N` | Maximum regular files inspected by `--sbom` (default: 100000) |
| `--sbom-max-bytes N` | Maximum aggregate bytes inspected by `--sbom` (default: 4 GiB) |
| `--sbom-max-packages N` | Maximum package-database records observed by `--sbom` before deduplication (default: 100000) |
| `--sbom-max-elf-entries N` | Maximum program-header, section-header, and dynamic-table entries inspected per ELF file (default: 8192) |
| `--manifest none\|json\|csv\|both` | Portable SHA-256 evidence manifest (default: both) |

The JSON report always embeds the normalized `findings` list and severity
counts. Standalone `findings.json` is emitted by default; SARIF is available for
scanner integrations. `--findings none` suppresses only standalone files, not
report findings or policy evaluation. A finding policy is evaluated only after
requested reports, findings, SBOM, and manifest evidence have been published,
so exit 4 does not suppress the evidence that explains it; extraction-failure
exit 2 takes precedence. `--sbom` is deliberately opt-in and all four inventory
limits must be finite positive integers.

Resume diagnostics are separate read-only commands. They require explicit
roots so portable journal paths cannot be resolved against ambient state:

```console
reap resume status output/.reap-resume.json \
  --input evidence/ --output output/ --format json
reap resume explain output/.reap-resume.json '$INPUT/system.img' \
  --input evidence/ --output output/
reap resume prune output/.reap-resume.json \
  --input evidence/ --output output/              # dry-run
reap resume prune output/.reap-resume.json \
  --input evidence/ --output output/ --apply      # journal records only
```

`status` and `explain` never write. `prune` is a dry-run unless `--apply` is
present, and even then it atomically removes only unusable journal records;
it never deletes extracted files. Unusable records exit 4, while unsafe or
malformed journals exit 5. JSON output carries stable `reason_code` values.

Directory output names are allocated before concurrency begins. REAP uses full
Unicode compatibility caseless matching plus stable source-name hashes to avoid
case, normalization, same-stem, and long-name collisions on portable evidence
media. Existing output roots are rejected if any scanned component is a link or
special file; REAP never follows or removes such entries.

Verify both recorded content and inventory completeness without modifying the
case:

```console
reap verify output/manifest.json --input evidence/ --output output/ --strict
```

New manifests fingerprint every regular file in the selected input scope and
output tree, including non-recursed extracted artifacts, reports, and the
resume journal. Each entry identifies its artifact class. `--strict` fails for
any later unrecorded input or output; use the independent `--added-inputs` and
`--added-outputs` policies when warning-only or ignore behavior is required.
Geometry, diff, manifest, JSON-report, findings, CycloneDX SBOM, plugin-lock,
plugin-result, and resume-journal Draft 2020-12 schemas are included in the
installed package under `reap/schemas`; their stable IDs and versioning policy
are documented in [docs/SCHEMAS.md](docs/SCHEMAS.md).

Compare two current manifest or JSON-report documents deterministically:

```console
reap diff baseline/manifest.json candidate/manifest.json --format json
reap diff baseline/report.json candidate/report.json \
  --fail-on added --fail-on removed --fail-on findings
```

REAP auto-detects a common supported document type and validates both inputs
before comparing them. Output can be `human`, `json`, or `csv`; `--output`
publishes atomically and refuses an existing path unless `--replace` is given.
Differences alone exit 0. A selected `--fail-on` category exits 4, malformed,
unsupported, ambiguous, or unsafe input/output exits 5, and invalid CLI usage
exits 2. Policy categories are `added`, `removed`, `changed`, `moved`,
`partition`, `path`, `format`, `metadata`, `finding_added`, `finding_removed`,
`finding_changed`, `findings`, and `any`.

Detached authenticity is opt-in and never creates or discovers keys:

```console
pip install 'reap-cli[signing]'
chmod 600 investigator-ed25519-private.pem
reap sign-manifest output/manifest.json --private-key investigator-ed25519-private.pem --signature output/manifest.sig.json
reap verify-signature output/manifest.json --signature output/manifest.sig.json --public-key trusted-investigator-ed25519.pem
```

Private keys must be unencrypted Ed25519 PEM files with owner-only permissions
on POSIX systems. Keep them outside evidence/output trees. Public trust is
always an explicit `--public-key` path; key generation, persistence, lookup,
and logging are deliberately out of scope. Signature mismatch exits 6;
malformed inputs, unsafe key permissions, missing signing support, and I/O
errors exit 5. Existing `reap verify` content verification and exit codes are
unchanged and can be run before or after authenticity verification.

### Examples

**Identify all partitions in a dump:**
```bash
reap ./physicalImage --identify-only
```

**Full extraction (skip large ext4 partitions):**
```bash
reap ./physicalImage --skip-ext4 -v
```

**Extract a single boot image:**
```bash
reap boot.img -o ./boot_extracted -v
```

**Extract a directory of 7z archives (split parts supported):**
```bash
reap ./archives/ -v
```

**Parallel extraction with 4 workers:**
```bash
reap ./physicalImage -j 4 -v
```

### Output structure

For a boot image, the recursive extraction produces:

```
boot_unpacked/
    kernel_info.txt          # Kernel analysis summary
    kernel_config.txt        # Build-time .config (if IKCONFIG enabled)
    kernel_source_paths.txt  # Build-time source paths
    kernel_strings.txt       # All embedded ASCII strings
    kallsyms.txt             # Kernel symbol table (if present)
    vmlinux                  # Decompressed kernel binary
    ramdisk_unpacked/
        init                 # cpio filesystem tree
        init.rc
        fstab.*
        sbin/
        ...
```

For a directory of partitions, you get a subdirectory per partition plus reports:

```
physicalImage/unpacked/
    report.txt               # Human-readable report
    report.json              # Machine-readable report
    report.csv               # Flat machine-readable report (with --report csv/all)
    findings.json            # Normalized findings (default)
    findings.sarif.json      # SARIF findings (with --findings sarif/both)
    sbom.cdx.json            # CycloneDX package/ELF inventory (with --sbom)
    manifest.json            # Versioned portable evidence manifest
    manifest.csv             # Flat evidence manifest
    .reap-resume.json        # Private bounded resume journal
    mmcblk0p1_unpacked/      # boot image contents
    mmcblk0p2_unpacked/      # DTB contents
    mmcblk0p3_unpacked/      # recovery image contents
    mmcblk0p4_unpacked/      # system filesystem tree
    ...
```

### Partition annotation

The tool automatically identifies partition roles by:
- Reading the ext2/3/4 superblock `s_last_mounted` field (e.g. `/system`, `/data`, `/cache`)
- Analyzing boot image ramdisks for `/sbin/recovery` to distinguish boot vs recovery
- Parsing U-Boot uImage type fields (kernel, ramdisk, firmware, device tree)
- Parsing IM\*H firmware module names and types (bootloader, kernel, RTOS)
- Recognizing format-specific roles (DTB, vbmeta, DTBO, sparse, super, modemst)
- Inferring empty partition purpose from size (<=4 MB zeroed = likely misc or metadata)

Annotations appear in reports and verbose output as labels like `(recovery)`, `(system)`, `(userdata)`, etc.

### FBE encryption detection

When extracting ext4 filesystems with File-Based Encryption (FBE), the tool:
- Detects the encryption superblock flag and per-inode encryption flags
- Hex-encodes encrypted filenames for safe extraction
- Writes `encrypted_paths.txt` listing all encrypted files and directories
- Reports encryption algorithms (AES-256-XTS, AES-256-GCM, etc.) in JSON output

## Architecture

```
reap/
    cli.py              # Argument parsing, entry point
    identify.py         # Ordered detection registry (65 core formats)
    annotate.py         # Partition role inference
    pipeline.py         # Recursive extraction orchestrator (parallel workers)
    report.py           # Atomic text + JSON + CSV reports (FBE-aware)
    findings.py         # Normalized JSON/SARIF findings and policy evaluation
    inventory.py        # Bounded CycloneDX package/ELF inventory
    manifest.py         # Deterministic portable JSON + CSV evidence manifests
    diff.py             # Validated deterministic manifest/report comparisons
    limits.py           # Output budgets, bounded writers, cancellation
    resume.py           # Bounded flat v2 resume journals and diagnostics
    plugin_lock.py      # Closed API-v2 provider/launcher lock contract
    plugin_protocol.py  # Descriptor handoff and verified atomic ingest
    handlers/
        __init__.py     # BaseHandler ABC, handler registry
        ambarella_env.py # Ambarella UNR0 boot environment
        avb.py          # AVB vbmeta metadata
        bitlocker.py    # BitLocker (BDE) detection + FVE metadata + protectors
        bmp.py          # BMP image (partition padding trim)
        boot_img.py     # Android boot image (v0--v4)
        bootlogo.py     # Boot logo container (multiple BMPs)
        compression.py  # gzip, LZ4, LZMA, bzip2, XZ
        cpio_handler.py # cpio newc archives
        dataflash.py    # ArduPilot/Autel DataFlash flight log (schema, CSVs, GPS track)
        devinfo.py      # Android devinfo (lock status)
        dji_imah.py     # IM*H ciphertext preservation + optional locked provider
        dtb.py          # Device Tree Blob
        dtbo.py         # DTBO container
        elf.py          # ELF binary metadata + strings
        ext4_handler.py # ext2/3/4 (legacy blocks, extents, FBE detection)
        exynos_boot.py  # Samsung Exynos eMMC boot partition
        fit.py          # Validated U-Boot FIT/ITB images and configurations
        apm.py          # Apple Partition Map carving
        bsd_disklabel.py # FreeBSD BSD disklabel carving
        mbr.py          # MBR/DOS partition table (splits primary partitions)
        fat.py          # FAT12/16/32 filesystem
        gpt.py          # GPT partition table (512-byte + 4K UFS sectors)
        luks.py         # LUKS1 + LUKS2 detection + keyslot / token metadata
        lvm2.py         # Single-PV linear LVM2 logical volumes
        littlefs.py     # Conservative littlefs v2.0/v2.1 read-only recovery
        spiffs.py       # Explicit-geometry, read-only SPIFFS recovery
        yaffs2.py       # Explicit-geometry, read-only YAFFS2 recovery
        ota_payload.py  # Android update_engine full OTA target partitions
        qcow2.py        # Integrity-checked QCOW2 v2/v3 flattening
        vdi.py          # Standard dynamic/static VDI v1.1 flattening
        vmdk.py         # Self-contained hosted sparse VMDK v1 flattening
        vhd.py          # Fixed/dynamic classic VHD flattening
        uf2.py          # Address-preserving Microsoft UF2 extraction
        ufs.py          # Clean standalone UFS1/UFS2 recovery
        xfs.py          # XFS v5 redundant-superblock metadata validation
        modemst.py      # Qualcomm modem EFS partition
        ntfs.py         # NTFS filesystem (MFT + runlists + LZNT1 + indexes)
        raw.py          # Empty + unknown fallback (forensic scan)
        salvage.py      # --salvage carve-by-scan for unparseable partition tables
        raw_kernel.py   # Raw ARM kernel binary
        rk_krnl.py      # Rockchip KRNL wrapper
        rkparm.py       # Rockchip PARM partition table
        seven_zip.py    # 7z archive (split-part support)
        sparse_img.py   # Android sparse -> raw conversion
        super_img.py    # super.img LP metadata
        uboot_bin.py    # U-Boot binary (environment, embedded DTBs)
        uboot_env.py    # U-Boot environment block
        uimage.py       # U-Boot uImage wrapper
        zimage.py       # ARM zImage / ARM64 Image kernel extraction
        _kernel_utils.py # Shared kernel analysis (version, config, kallsyms)
```

Each handler implements `BaseHandler.extract()` and returns an `ExtractionResult` with optional children for recursive processing. Versioned public interfaces, plugin keys and conflict rules are documented in [docs/API.md](docs/API.md).

The pipeline orchestrator (`pipeline.py`) drives identify → annotate → dispatch → recurse. Children can run concurrently while sharing one reservation-based output ledger and cancellation token. See [architecture](docs/ARCHITECTURE.md), [schemas](docs/SCHEMAS.md), and [limitations](docs/LIMITATIONS.md).

## Symlink handling

Symlinks found inside ext-family filesystems and cpio archives are **not** created as OS symlinks (which can cause issues on some platforms and create security risks with path traversal). Instead, they're written as small text files containing `-> target` and recorded in the extraction metadata / JSON report.

## Running tests

```bash
pip install -e ".[dev]"
pytest tests/ -q
```

The compact, risk-based suite covers format contracts, hostile-input bounds,
pipeline orchestration, and forensic scanning. Provenance-pinned real-corpus
fixtures provide authentic-tool differentials without requiring external tools
during an ordinary test run. A lean-policy checkpoint consolidated expansion-
heavy suites without dropping their behavioral tables. CI folds coverage and
generated-corpus validation into the Python
3.13 compatibility job instead of repeating a fifth full suite. The release
harness runs one full/static preflight, the minimum two-build reproducibility
comparison, and then only the 20 concurrent installed-package workflows.
Protected tag pipelines reuse the exact commit's successful `main` validation
instead of rerunning the compatibility suite and quality build. See
[the testing policy](docs/TESTING.md).

## Known limitations

- **EROFS, F2FS, littlefs, SPIFFS, YAFFS2, UFS, VDI, and VMDK**: Useful conservative subsets are extracted. SPIFFS and YAFFS2 require authenticated complete geometry manifests; advanced or ambiguous layouts intentionally fail closed. XFS currently validates and inventories v5 superblock metadata but does not extract its namespace. See [the exact support matrix](docs/LIMITATIONS.md).
- **Encrypted partitions**: FBE-encrypted ext4 partitions are detected and documented, but file contents remain encrypted. The tool does not perform Android FDE/FBE decryption. EFS-encrypted files on NTFS volumes are similarly skipped, with a per-file marker recording the encryption. BitLocker and LUKS volumes are detected and their full encryption-scheme metadata is surfaced, but the encrypted payload itself is not decrypted — pipe the source image through `dislocker` / `libbde` (BitLocker) or `cryptsetup` (LUKS) for that.
- **IM\*H decryption**: On a successful extraction, core parsing preserves encrypted chunks. Optional plaintext requires an explicitly selected, lock-authenticated API-v2 `dji_imah` provider and operator-supplied key reference; REAP distributes no AES keys. A provider protocol failure fails the handler transaction closed and publishes neither raw nor plaintext outputs.
- **Large partitions**: Extracting a 54 GB ext4 partition takes time and disk space. Use `--skip-ext4` to skip these, or extract individual partitions as needed.
- **HFS+ and other deferred filesystems**: Native HFS+ extraction is not claimed. Journal state, HFS wrappers, catalog/extents B-trees, and Unicode/case semantics make a rushed core parser disproportionate to the recovery evidence goal; a future route should be an explicit, audited read-only adapter. APFS, Btrfs, and multi-device filesystem assembly remain deferred.
- **External tools**: Split 7z requires system `7z`; single files can fall back to `py7zr`, and DTB decompilation can invoke `dtc`. REAP bounds these subprocesses and validates staged output, but does not digest-pin the `7z` or `dtc` executable.
- **Symlinks**: Recorded as text files, not created as actual OS symlinks.
- **Text files**: Plain-text metadata files (.txt, .sha256, .xml, README) in the input directory are detected and skipped rather than subjected to forensic extraction.
