Corpus Inventory & Evidence Chain

What This Document Is

This is the foundational accounting of every piece of source material underlying this
investigation. Every report in this repository traces back to specific documents within
the corpus described here. If you want to verify a claim, this is where you start.

Bottom line: 1,380,937 PDF documents containing 2,731,785 pages and 3.18 billion
characters of text, plus 3,234 media files, 1,530 audio/video transcripts, 2,587,102
redaction records, and a 1,536-person entity registry. All derived from 194.5 GB of
publicly released U.S. Department of Justice files.

Source Material

All source data was released by the U.S. Department of Justice at
justice.gov/epstein between December 2025 and
February 2026. Bulk downloads were obtained from archive.org
mirrors of the DOJ release. No data was obtained through leaks, hacking, FOIA requests,
or any non-public channel.

DOJ Release Timeline

Complete Document Inventory

PDF Documents by Dataset

Disk-to-database reconciliation: Every PDF file on disk has a corresponding entry
in the database. Zero discrepancies across 11 of 12 datasets. DS8 has one duplicate
file (EFTA00022173 exists in two subdirectories at identical size — a packaging artifact,
not a data issue).

EFTA Number Gaps

EFTA numbers are not contiguous. The DOJ did not release every number — there are gaps
within each dataset's range. For example, DS9 spans EFTA00039025 to EFTA01262781
(a range of 1,223,757 possible numbers) but contains only 531,284 documents. These gaps
are part of the DOJ's release structure, not missing data on our end. The same pattern
holds across all datasets.

Media Files (Non-PDF)

Zero-Page Documents (Corrupted/Non-Standard PDFs)

Five documents across the entire corpus returned zero extractable pages from standard
PDF tools. Byte-level forensic analysis revealed these are not simply corrupted — they
are forensic disk image fragments and truncated scans. All five have been recovered.
See CORRUPTED_PDF_FORENSICS for full details.

Derived Databases

All analysis in this repository is built on four databases derived from the source PDFs.

full_text_corpus.db (6.08 GB)

The primary analytical database. Contains the full text of every page of every document.

How it was built: PyMuPDF (fitz) text extraction on every PDF. For scanned documents,
this captures the invisible OCR text layer (rendering mode Tr=3) that the DOJ's scanning
vendor applied. Documents where PyMuPDF returned zero text were flagged for manual review
(this is how the 5 corrupted PDFs were identified).

redaction_analysis_v2.db (0.95 GB)

Spatial analysis of every redaction rectangle in the corpus, with the text found at
each redaction's coordinates.

Important caveat: The vast majority of bad_overlay redaction records (~98%) are
OCR noise — the scanner's OCR engine attempted to read black redaction bars and produced
garbage text. Only 12 documents contain genuinely failed redactions (Apple Mail PLIST
metadata exposed behind incompletely flattened overlays). See
DATA_QUALITY_AUDIT and EVIDENCE_RELIABILITY_AUDIT
for the full audit. The redaction database remains useful as a searchable index of text
found near redaction zones, but its hidden_text field should not be interpreted as
"recovered secret content."

transcripts.db (2.5 MB)

GPU-transcribed audio/video content using faster-whisper large-v3 on NVIDIA A100.

Pre-screening classified 2,581 unique media files: 903 were processed, 1,633 were skipped
(silent surveillance footage — 77+ hours of MCC/facility video with no audio stream).

Notable content: BOP Warden OIG interview, 3 MCC prison phone calls, 20+ Grand Jury
testimony recordings, Deepak Chopra voicemails.

persons_registry.json

Unified entity registry merging three sources: our pipeline extraction, la-rana-chicana
community research CSV, and the knowledge_graph.db entity table.

What's NOT in This Corpus

For transparency, here is what we do NOT have access to:

1. Sealed court filings — Multiple cases have sealed dockets (SDNY, SDFL, USVI).
   We only analyze publicly released material.
2. Grand jury transcripts (full text) — We have audio recordings of some testimony
   sessions (transcribed in transcripts.db) but not official transcripts.
3. Classified intelligence material — References to intelligence connections are
   derived from what appears in the public DOJ files, not from classified sources.
4. Victim statements beyond what DOJ released — The DOJ's release is selective.
   Many victim interviews, depositions, and statements referenced in the documents are
   not included in the EFTA corpus.
5. The "missing" EFTA numbers — 692,473 EFTA numbers in DS9's range are not present.
   Whether these represent withheld documents, unimaged evidence, or simply unused
   numbers in the task force's tracking system is unknown.
6. Post-February 10, 2026 releases — This inventory reflects data through DS12.
   Additional datasets may be released.

Verification

Anyone can independently verify any finding in this repository:

1. Obtain the source PDFs from justice.gov/epstein
   or the archive.org mirrors linked in each report
2. Check any EFTA citation by constructing the URL:
   https://www.justice.gov/epstein/files/DataSet%20{N}/EFTA{########}.pdf
3. Reproduce the text extraction using PyMuPDF: fitz.open(path)[page].get_text()
4. Reproduce the redaction analysis using the methodology in
   REDACTION_TEXT_LAYER_ANALYSIS

The EFTA-to-dataset mapping table is in the main README.

Processing Pipeline

DOJ PDFs (194.5 GB, 1.42M files)
    │
    ├─→ PyMuPDF text extraction ──→ full_text_corpus.db (6.08 GB)
    │       └─→ FTS5 full-text search index
    │
    ├─→ Redaction rectangle analysis ──→ redaction_analysis_v2.db (0.95 GB)
    │       ├─→ Reconstructed pages (39,588)
    │       └─→ Entity extraction (107,422 entities)
    │
    ├─→ Media file pre-screening + GPU transcription ──→ transcripts.db
    │
    ├─→ Person registry unification ──→ persons_registry.json (1,536 persons)
    │
    └─→ Byte-level forensic recovery of 5 zero-page PDFs
            └─→ recovered_corrupted_pdfs/ (Apple Address Book, LSJ photo, etc.)

All processing was performed locally. No documents were uploaded to cloud services
or third-party APIs for analysis. Text extraction, OCR, transcription, and entity
extraction were all run on local hardware.