# Pallet inventory v3

Source: `/root/private/data/pallet` (read only). Regenerate the file inventory and then resolve archive content with:

```bash
.venv/bin/python -m pallet.data.inventory /root/private/data/pallet outputs/inventory_v3 --hash-cache outputs/content_hashes.json
.venv/bin/python -m pallet.data.inventory /root/private/data/pallet outputs/inventory_v3 --hash-cache outputs/content_hashes.json --resolve-pending --rar-decoder outputs/inventory_v3/tooling/usr/bin/unrar-nonfree
```

`items.jsonl` contains one row for every disk file and every regular file member listed from ZIP, TAR.GZ, RAR, and the nested TAR.GZ inside a ZIP. Rows carry `classification` (`used`, `duplicate`, `damaged`, or `pending`) and a machine-readable `reason`. Archive directory entries are metadata, not data-file members. `summary.json` contains totals. `unique_sensor_records.jsonl` records the one sensor payload not found on disk, including its extracted path, original collection, scene, archive, member, name, size, and SHA-256.

| Kind | Used | Duplicate | Damaged | Pending | Total |
|---|---:|---:|---:|---:|---:|
| Disk files | 25,816 | 128 | 1,152 | 214 | 27,310 |
| Archive members | 0 | 24,138 | 421 | 53 | 24,612 |

The 1,152 damaged disk files comprise 734 zero-byte files, 118 reported all-zero intensity YAMLs from `2023_10_7`, and 300 reported truncated XYZ YAMLs from `2024_11_27`. The 421 damaged archive members are zero-byte sensor files. The 128 disk duplicates use valid cached SHA-256 entries with matching size and modification time. Archive member duplicates use matching relative path, size, and either CRC32 (ZIP) or SHA-256 (TAR/GZIP), or matching size and SHA-256 across any disk path. Names and sizes alone are never treated as content equality. RAR member listings and decompression use the locally unpacked decoder under `tooling/`. Source archives are read without extracting into the source tree.

The 15 non-timestamped PCD candidates have valid, bounded-read structural headers. Fourteen are organized `352x264` point clouds; one is a `1009x1` segmentation cloud. They remain pending because they are outside the timestamped manifest, and header validity does not establish semantic suitability or full payload integrity.

The sole unique archived sensor is `2024_1_10/point_cloud_xyzi_10-01-2024_03-36-12.section.pcd` inside `2024_1_10.zip`. It is a structurally valid, unorganized `16460x1` binary PCD section cloud, so it is pending geometry review and is **not** counted as a usable new frame. Its 329,396-byte payload is cached under `extracted/`. The cache uses a 20 GiB cap and preserves at least 50 GiB of free disk space; this run used 329,396 bytes.

The 53 pending archive member rows are the one section cloud and 52 ancillary files: 50 logs, one text file, and one nested TAR.GZ container. The nested container's 1,245 regular sensor members were scanned separately and all matched disk files by SHA-256. The 214 pending disk files include 68 archive containers, 15 non-timestamped PCD candidates, and 131 other files outside the manifest. No sensor archive member remains unresolved for missing content evidence or decoder availability.
