Corpus Admission¶
A document is real-corpus evidence only after its source, rights, bytes, parsing result, and independently reviewed truth have passed separate recorded decisions. A URL, downloaded file, parser success, fixture, generated paragraph, or system-produced label is not an admitted corpus.
Admission States¶
Corpus records move through explicit states. Each transition appends a receipt; it does not overwrite an earlier decision.
identified -> metadata_verified -> license_reviewed -> acquired
-> checksummed -> parsed -> truth_annotated -> admitted
-> held_out -> published
superseded may follow any retained state when a corrected source, license
decision, parser contract, or truth set replaces it. A failed transition keeps
the last accepted state and records a typed rejection. No state is inferred
from the existence of a later-looking file.
| State | Decision and required evidence | Owner |
|---|---|---|
identified |
stable source ID, DOI or canonical URI, candidate format, and discovery provenance | curator |
metadata_verified |
title, authors, journal or publisher, publication year, canonical landing page, and identifier agree with an authoritative record | curator |
license_reviewed |
the license for this exact work and version permits the planned acquisition, transformation, storage, evaluation, and redistribution | license reviewer |
acquired |
retrieval timestamp, response identity, media type, byte count, acquisition command, and protected local relative path identify the materialized bytes | acquisition operator |
checksummed |
a cryptographic digest binds the exact bytes; content checks reject abstracts, login pages, error pages, and unexpected media | admission automation |
parsed |
a qualified parser emits source-faithful text, stable document identity, exact locator mappings, structured rejects, and transformation provenance | ingest maintainer |
truth_annotated |
independent reviewers record locators, graded qrels, claims, citation relations, conflict and abstention labels, adjudication, and reviewer provenance | truth-review team |
admitted |
all required prior receipts agree, limitations are explicit, and the snapshot manifest locks content, parser, truth, configuration, and split identities | admission authority |
held_out |
the case assignment was frozen before final tuning and its labels are access-controlled from prompt, parameter, reranking, threshold, and selection work | evaluation custodian |
published |
only redistribution-permitted bytes and approved metadata, acquisition recipes, hashes, truth, and attribution are included in the declared publication | publication reviewer |
superseded |
the replacement identity and reason are recorded without deleting the historical bytes, decisions, or evaluation lineage | admission authority |
Discovery, review, acquisition, and admission must use different event types even when one operator performs more than one role. This preserves the question each decision answered and prevents an acquisition success from masquerading as legal approval.
Source And Rights Record¶
Every candidate source record includes:
- stable source ID, DOI or canonical URI, title, authors, journal or publisher, publication year, and canonical landing page;
- retrieval timestamp, acquisition method, final resolved URI, media type, byte count, byte hash, and corpus-root-relative storage path;
- exact license expression, license URL, attribution text, access terms, redistribution terms, and the evidence used for the decision;
- transformations applied to the acquired bytes and the tool, version, configuration, and output hash for each transformation; and
- known limitations, sensitive-content classification, disclosure controls, and any prohibition on committing or publishing the bytes.
License review applies to the exact article and materialized version. Publisher, journal, collection, or historical defaults do not establish the license of an individual work. When redistribution is not permitted, retain protected local bytes and publish only lawful metadata, acquisition instructions, checksums, and evaluation records that do not disclose restricted content.
Acquisition And Materialization¶
Acquisition is a network-enabled, policy-bound operation. It must reject redirects outside approved origins, authentication or error pages, media-type mismatches, excessive size, archive expansion beyond declared bounds, and content whose hash or identity conflicts with an existing source record.
The admitted snapshot is immutable and later runs offline. Materialization therefore retains the exact bytes rather than depending on a live URL during parsing, indexing, evaluation, inspect, or replay. A changed upstream document creates a new source-byte identity and snapshot generation; it does not mutate the admitted generation.
The flagship ancient-DNA portfolio requires full-text JATS for each specified
PLOS article. An abstract, citation export, landing page, or publisher error
body is a typed acquisition rejection. Parser qualification for digital PDF,
HTML, Markdown, plain text, and DOCX uses separately lawful real documents.
Image-only or scanned material returns an ocr_required refusal until an OCR
implementation and its provenance contract are separately admitted.
Parsing Admission¶
A parser result qualifies only when it preserves:
- source byte identity and parser/configuration identity;
- source-faithful normalized text with every transformation declared;
- exact locators capable of resolving a chunk or citation back to immutable source text and its hash;
- deterministic document and chunk identities independent of absolute paths, timestamps, and process IDs; and
- explicit rejects for unsupported, malformed, encrypted, oversized, expansion-heavy, or OCR-required input.
Success on a sample fixture does not admit a format. Qualification includes real documents, malformed and boundary cases, deterministic repetition, and a restart from retained bytes and configuration.
Independent Truth¶
Truth is authored without copying or approving system output. Each evaluation case records the source snapshot and exact locators, graded qrels, expected and optional atomic claims, opposed and forbidden claims, claim-citation support or opposition relations, conflict expectations, abstention expectations, and the allowed agent-policy outcome.
At least 120 manually reviewed cases are stratified by source, question type, evidence density, conflict, abstention, and retrieval difficulty. Two reviewers and an adjudication record resolve disagreements. Numerators, denominators, excluded cases, and label changes remain auditable.
Development and held-out assignments are frozen before final tuning. Held-out labels cannot influence prompts, model or index parameters, fusion and reranking, thresholds, budgets, or case selection. Access to held-out labels is recorded, and any premature disclosure invalidates the affected held-out evaluation generation.
Fixtures And Synthetic Material¶
Fixtures and synthetic text are useful for parser branches, serialization, failure injection, fuzzing, and deterministic unit contracts. They never satisfy a real-corpus, parser-qualification, retrieval-quality, grounding, flagship, or publication claim. Derived or transformed real text is still not independent truth when labels come from the system under evaluation.
Reports classify every input as real_admitted, real_not_admitted,
fixture, or synthetic and fail closed when a production metric includes an
ineligible class.
Admission Decision¶
An admission manifest binds the source records, byte hashes, license receipts, parser outputs, locator validation, truth generation, split manifest, limitations, and governing tool and configuration locks. Admission fails when any required receipt is absent, stale, mismatched, or produced from a different source generation.
The accepted manifest publishes counts for candidates, each state, typed rejections, admitted documents, truth cases, and held-out cases. A nonzero file count or successful command exit is not an admission criterion. Downstream index, reason, agent, and runtime evidence identifies the exact admitted snapshot; mixing generations invalidates the run.
Publication Review¶
Publication is a new decision, not an automatic consequence of admission. The reviewer verifies redistribution rights for every published byte, required attribution, privacy and sensitive-content handling, license compatibility of the aggregate, reproducible acquisition instructions, checksum completeness, and separation of held-out labels.
The publication record lists omitted restricted sources and explains how an authorized evaluator can rematerialize them. Superseded versions remain addressable for historical replay but are excluded from current capability and quality claims.