Tool · Data & provenance

Provenance Inventory

Count a file collection and find exact duplicates before deciding what to move or index.

Repository: CinvanaAI/provenance-inventory

Explore the repository ↗Try or inspect it ↓

Recorded synthetic example. Source with a complete offline example.

See the idea in action.

Three file records, one duplicate pair, one project marker, zero errors, and unchanged source bytes.

Example input

Three synthetic files: two identical notes and a project marker.

Captured result

{
  "source_unchanged": true,
  "files": {
    "copy.txt": "d137d7338abb45f5c4ddbb49a04e8bbce54054add05622655a27c9684c645e35",
    "one.txt": "d137d7338abb45f5c4ddbb49a04e8bbce54054add05622655a27c9684c645e35",
    "pyproject.toml": "cf837afc3ffc8810781328700db8a5fc60ff2242e33fb909a94c5df3e46221cf"
  },
  "summary": {
    "status": "complete",
    "file_count": 3,
    "project_marker_count": 1,
    "duplicate_group_count": 1,
    "duplicate_file_count": 2,
    "error_count": 0
  },
  "manifest_fields": [
    "birth_time_ns",
    "created_or_changed_time_ns",
    "extension",
    "hash_status",
    "is_project_marker",
    "media_kind",
    "mime_hint",
    "modified_time_ns",
    "name",
    "original_path",
    "relative_path",
    "run_id",
    "scanner_version",
    "schema_version",
    "sha256",
    "size_bytes",
    "source_id",
    "source_label"
  ]
}

Try the example.

From the repository root, follow the dependency requirements in the README. This example uses supplied synthetic material.

python -m pip install -e .
python -m examples.walkthrough

Three file records, one duplicate pair, one project marker, zero errors, and unchanged source bytes.

Complete setup and instructions ↗

The interesting part.

Full-file hashes and immutable run receipts preserve source identity; duplicate detection never implies deletion.

Focused extraction

Where it came from.

Source-census engine from Personal Intelligence Dashboard.

Makes the destination an explicit dedicated directory while retaining the read-only provenance rules.

The Dashboard keeps its own copy; this package is independently usable.

Source ↗

Follow the family: Personal dashboard and source census

Extracted fromPersonal Intelligence Dashboard
See the integrated setting from which this independently useful mechanism was separated. The parent retains its census implementation; the extract changes the output boundary.
Source ↗