Skip to content

Test Strategy

Receiver evidence spans pure DSP properties, stage contracts, streaming lifecycle, scientific truth, and cross-stage artifacts. Test size is not a measure of confidence: each claim needs the narrowest test that isolates its cause and the smallest boundary test that proves its public effect.

Proof Layers

flowchart LR
    property["numeric property or invariant"]
    stage["acquisition, tracking, or observation contract"]
    handoff["stage handoff"]
    scenario["streaming or synthetic scenario"]
    truth["truth, reference, or statistical evidence"]
    artifact["returned artifact and refusal evidence"]

    property --> stage
    stage --> handoff
    handoff --> scenario
    scenario --> truth
    truth --> artifact

Not every change needs every layer. A loop coefficient can stop after property and focused tracking proof if no public behavior changes. A new channel state must continue through handoff and artifact evidence. An accuracy or robustness claim must reach independent truth or a clearly bounded statistical model.

What Each Family Can Establish

family strongest justified claim common overclaim representative evidence
guardrails source layout, module limits, and pipeline purity rules hold receiver runtime is correct receiver source guardrail
public boundary imports and dependency direction remain within the intended surface every re-export has stable semantics navigation boundary guardrail
property and numerical sanity bounded mathematical invariants hold over generated inputs or tolerances acquisition or tracking works end to end receiver properties
stage integration one stage reports expected behavior under a controlled input neighboring stage handoff remains correct acquisition uncertainty evidence
handoff and determinism public records preserve ordering, identity, or repeatability across a boundary physical accuracy or broad runtime determinism observation determinism evidence
scenario and truth a declared signal condition produces bounded runtime behavior unmodeled field conditions are covered tracking truth evidence
operating envelope or statistics behavior meets a sampled probability, noise, or dynamics budget all conditions between or outside sampled points are safe acquisition operating envelope
golden or fixture reviewed output remains byte- or value-stable for one fixture the frozen output is scientifically correct acquisition golden evidence

Test names are not contracts. Inspect assertions before citing a test. For example, the current pipeline determinism test proves repeatable observation output and events from constructed tracking results; it does not run the entire sample-to-navigation receiver.

Select Evidence By Failure Mode

flowchart TD
    claim["changed receiver claim"]
    math{"numeric helper or threshold?"}
    property["property and edge cases"]
    stage{"single stage behavior?"}
    focused["focused positive and negative stage proof"]
    state{"state crosses epoch or stage?"}
    lifecycle["transition, continuity, and handoff proof"]
    accuracy{"accuracy, probability, or robustness?"}
    reference["truth, reference, or statistical budget"]
    output{"public artifact or refusal changed?"}
    artifact["typed artifact assertion"]

    claim --> math
    math -- yes --> property
    math -- no --> stage
    property --> stage
    stage -- yes --> focused
    stage -- no --> state
    focused --> state
    state -- yes --> lifecycle
    state -- no --> accuracy
    lifecycle --> accuracy
    accuracy -- yes --> reference
    accuracy -- no --> output
    reference --> output
    output -- yes --> artifact

Negative evidence is part of the contract:

  • acquisition must preserve absent, ambiguous, below-threshold, and unsupported outcomes;
  • tracking must expose loss, degradation, instability, phase slip, failed reacquisition, and refusal;
  • observations must retain rejection and uncertainty rather than silently dropping unusable measurements;
  • navigation handoff must preserve unavailable, refused, degraded, and integrity-failed results.

The completion gate maps these outcomes to review requirements.

Fast And Slow Evidence

Long-duration tracking, probability characterization, broad operating envelopes, and expensive truth captures belong in the governed slow lane when they exceed the fast budget. The slow-test roster is a scheduling contract, not a waiver. A change whose claim depends on long duration, weak signal, statistical confidence, or multi-stage capture behavior still requires that evidence before release.

During development, run focused evidence first so failures remain attributable. Record slow evidence separately rather than replacing it with a faster test that proves a narrower claim.

Evidence Independence

Synthetic truth is strongest when the assertion does not reuse the algorithm under test. Prefer:

  • checked-in external or independently generated references;
  • closed-form expectations for local computations;
  • separate truth builders with explicit provenance;
  • multiple rates, constellations, dynamics, or noise points where the claim spans them.

Golden fixtures protect compatibility only after their scientific meaning has been reviewed. Updating a golden file to match new output is not proof that the new output is correct.

Use the change validation guide to scope a receiver change and the known limitations to state what the selected evidence cannot support.