Skip to content

Risk register

Core risk is scientific semantic erosion: software can remain executable while input accounting, score meaning, provenance, or workflow claims become wrong.

Risk Early signal Consequence Required control
silent row loss accepted totals change without matching rejection records biased downstream evidence field accounting, rejected rows, reason codes
score-orientation drift adapter or dependency changes “better” direction thresholds and FDR invert orientation fixture, boundaries, reference values
target/decoy policy drift labels, strata, competition, or denominator moves error control becomes incomparable serialized policy and sensitivity tests
missingness collapse absent, censored, filtered, failed, and zero merge quantification and contrasts are biased explicit states and adversarial sparse fixtures
provenance loss producer, version, parameters, database, or transformation disappears imported output appears native or irreproducible lineage invariant and artifact review
circular benchmark implementation generates its own expected answer regression evidence is mistaken for independent validation external reference or independently specified truth
transfer overclaim one corpus or workflow family sets broader posture public language exceeds evidence companion, holdout, and family-specific ceiling
nondeterministic optimization chunking or parallelism changes order, acceptance, or hash repeated runs disagree serial equivalence and stable artifact tests
policy leakage Runtime, Knowledge, Intelligence, or Lab rules move into Core scientific functions depend on context outside their contract dependency and ownership guards
format overclaim one producer fixture is described as full standard support unseen dialects parse incorrectly producer/version scope and mutation corpus
flowchart LR
    I["input accounting drift"] --> S["scientific semantic drift"]
    P["policy or provenance loss"] --> S
    S --> B["benchmark misinterpretation"]
    B --> C["overstated public claim"]

Silent acceptance drift, reversed score meaning, broken error control, lost provenance, and benchmark circularity are release-blocking. A large green suite does not reduce a risk whose relevant negative, reference, transfer, or consumer evidence is absent.