Known Limitations¶
bijux-canon-reason verifies declared relationships among plans, tool events,
evidence bytes, claims, and artifacts. It does not verify reality. A trace can
be structurally impeccable and still rely on a false, stale, biased, or
irrelevant source.
What Verification Means¶
flowchart LR
structure["structural validity"] --> provenance["provenance validity"]
provenance --> grounding["declared grounding"]
grounding --> adequacy["domain adequacy"]
adequacy --> truth["real-world truth"]
package["package verification"] -. "covers" .-> structure
package -. "covers" .-> provenance
package -. "covers bounded rules" .-> grounding
reviewer["domain review"] -. "required" .-> adequacy
world["external validation"] -. "required" .-> truth
| Verification layer | What a passing check establishes | What remains open |
|---|---|---|
| structure | plan topology, event lifecycle, identifiers, and schemas satisfy implemented invariants | whether the selected plan is sufficient |
| provenance | recorded bytes, digests, paths, runtime identity, and manifests agree | whether the source was authorized, authoritative, or current |
| grounding | claims have the support relationships required by the selected policy | whether support is persuasive, complete, or correctly interpreted |
| domain adequacy | not established automatically | counterevidence, assumptions, calibration, and decision thresholds |
| truth | not established automatically | the state of the world and consequences of acting |
A support span proves byte linkage. A hash proves content identity. A confidence field records a claim made by a producer. None of these is, by itself, a truth test or a calibrated probability.
insufficient_evidence is a governed refusal to complete a claim under the
available evidence. It does not prove that no answer exists.
Reference Reasoning And Retrieval¶
The bundled reasoner is extractive, and the local retrieval path uses BM25. They provide an inspectable offline reference, not general-purpose reasoning or state-of-the-art retrieval. Corpus composition, tokenization, chunk size, overlap, BM25 parameters, and query wording all change which evidence becomes available.
Corpus byte guards constrain reads but do not make the package a distributed search service. Large, changing, remote, or multi-tenant corpora require an index integration with explicit availability, freshness, authorization, and provenance contracts.
Replay Is Snapshot Replay¶
Replay reuses recorded tool results; it does not call live tools again. Equal replay fingerprints establish that the recorded inputs and snapshots produce the same governed trace under the supported canonicalization and runtime protocol. They do not establish that a provider, URI, or corpus would return the same information today.
Local evidence, corpus, index, and provenance artifacts are checked for drift when available. A URI without archived bytes cannot be re-attested. If future review depends on exact evidence, archive those bytes and their source metadata with the run.
Runtime Composition Boundary¶
Runtime's live reasoning executor asks the bijux_canon_reason package root
for reason(agent_outputs, evidence, seed) and requires the result to be the
runtime-owned ReasoningBundle type. The canonical reason root exports its own
plans, claims, evidence references, traces, verification reports, serializers,
and validators; it does not export that callable.
A reason-owned wrapper that imports runtime models would reverse the current
dependency direction because runtime already depends on reason. The adapter
therefore belongs in runtime or a separate integration boundary and must
preserve claim identity, exact support, verification status, steps, producer,
trace, and manifest custody. bijux-rar mirrors the canonical root and does not
make the handoff executable.
Evaluation Metrics Are Bounded Proxies¶
Current evaluation summaries measure properties of produced traces:
- alignment rate is the share of emitted claims with at least one evidence support link;
- the reported faithfulness value is the mean support-link count among supported claims, not a semantic entailment score;
recall_at_kandmrrcurrently indicate whether any evidence was registered, not relevance-judged information-retrieval recall or reciprocal rank;- verification failure counts summarize implemented checks, not the complete error space.
Do not publish these values under conventional retrieval or faithfulness names without their definitions. A domain evaluation must add relevance judgments, expected claims or refusals, counterevidence cases, and consequences appropriate to the intended use.
Operational Guards¶
Artifact-workflow disk, wall-time, and CPU budgets are process-level checks, not a scheduler, sandbox, hard deadline, or remote-resource limit. The interface rate limiter is best-effort, in-memory, and process-local; it does not coordinate workers or survive restart. API size guards do not provide authentication, tenant isolation, malware screening, or network egress policy.
The HTTP boundary also has explicit fail-open configuration and transport limits:
| Boundary | Implemented behavior | Consequence |
|---|---|---|
| API token | RAR_API_TOKEN enables exact x-api-token comparison; an unset value disables the check |
deployments must fail closed in their own configuration policy |
| request size | the 8 KiB guard checks a parseable Content-Length header |
missing or malformed length metadata is not a streaming body limit |
| response size | item responses are serialized in memory and capped at 2 MiB | this is a payload cap, not backpressure or admission control |
| rate limit | optional per-token or anonymous in-memory minute bucket | limits are neither shared across workers nor durable across restart |
| media type | XML content types are denied | other content is not thereby trusted or malware-scanned |
Deploy the API behind transport security, authenticated ingress, body-size enforcement, coordinated rate limiting, and tenant authorization appropriate to the environment. The package guards are defense-in-depth checks, not that perimeter.
The evaluation command supports workflow and metrics artifacts, but named suite discovery is not yet a stable public catalogue. Callers must supply and version the exact suite material rather than depend on an implied packaged benchmark name.
See the risk register for observable hazards and the test strategy for the evidence behind the bounded package claims.