Skip to content

Reviewing Workflow Contracts and Current Truth

The fastest way to waste time in a mature repository is to propose a redesign before you can describe what the current system is promising.

That is why Module 10 starts with review.

Review here does not mean style cleanup. It means naming the boundaries another maintainer would need to trust before changing anything.

If you are learning this without the live class

Read this lesson in five passes instead of trying to absorb everything at once:

  1. identify what the repository is already promising today
  2. separate public contract from internal convenience
  3. separate workflow meaning from operating policy
  4. identify the proof route and the socially trusted weak spots
  5. write one current-truth note before you imagine any redesign

If you skip that order, you will redesign a repository you have not yet described honestly.

How to use this page

Read this page before you write a migration plan, propose a refactor, or argue for a new tool boundary.

Keep one small review packet beside you as you read:

  • one list of trusted outputs
  • one list of internal-but-useful surfaces
  • one note on policy versus workflow meaning
  • one proof-route inventory
  • one short list of socially trusted weak spots

That packet is the minimum evidence another maintainer needs before they can judge whether change is safe.

What a real review should settle

Before you recommend migration, refactoring, or replacement, a workflow review should answer five questions:

  1. what files or reports are actually public and trusted
  2. which internal surfaces are useful but not contract surfaces
  3. where policy lives and how it differs from workflow meaning
  4. which evidence surfaces still let you explain behavior under pressure
  5. which parts of the repository are socially trusted but not yet technically reviewable

If those are still fuzzy, the repository is not ready for big change.

A useful review frame

flowchart TD
  files["file and publish contracts"] --> policy["policy and profile boundaries"]
  policy --> evidence["proof and observability surfaces"]
  evidence --> ownership["code and ownership boundaries"]
  ownership --> risk["named risks and next review questions"]

This is not paperwork. It is the shortest route to an honest migration plan.

Start with the public truth

Ask first:

  • which outputs are safe for downstream trust
  • where that promise is documented
  • how a reviewer confirms the published bundle is complete and parseable

In the capstone, this route is visible:

  • capstone/docs/file-api.md
  • make verify-report
  • capstone/docs/publish-review-guide.md

That is a good pattern. It keeps public truth visible without forcing reviewers to infer it from the whole repository tree.

The first classroom question to force on yourself

Before you improve anything, make yourself answer:

  • which files another team is safe to depend on
  • which outputs are merely useful during execution
  • which routes prove the published bundle is still trustworthy

Those are not cleanup details. They are the current contract.

A current-truth card you should be able to fill quickly

Use a table like this before you keep reading:

Surface Current honest description Why it matters
publish bundle files downstream users are safe to depend on public trust starts here
internal results useful execution surfaces, not public promises migration must not confuse convenience with contract
policy surfaces executor, storage, retries, placement, infrastructure these change operation, not workflow meaning
semantic surfaces targets, discovery, thresholds, published meaning these change what the workflow actually does
proof routes dry-run, file API, publish verification, profile audit these keep review honest under pressure

If you cannot fill one row, that row is already a review gap.

Separate internal usefulness from public contract

Many repositories blur these together:

  • detailed per-sample results become treated like published APIs
  • scratch or intermediate files get inspected as if they were stable outputs
  • notebooks or downstream scripts read internal paths because the public contract is vague

A strong review names that boundary plainly.

Example:

results/ is a useful internal execution surface, but the downstream trust boundary is publish/v1/ plus the file API and verification bundle.

That one sentence saves a lot of bad migration ideas.

Review policy separately from workflow meaning

Profiles, executor settings, storage placement, and retry policy matter.

But they do not answer the same questions as the workflow contract.

When you review a repository, ask:

  • which settings are pure operating policy
  • which settings would change target selection or published meaning
  • whether the profile comparison route is visible enough for others to inspect

This is where make profile-audit earns its keep. It turns policy comparison into a reviewable artifact instead of a memory exercise.

Review the proof route, not just the code

A repository can look organized and still be hard to trust because its proof route is weak.

Look for these questions:

  • can a maintainer dry-run safely
  • can they explain why work will rerun
  • can they inspect one compact publish review bundle
  • can they compare operating contexts without reading everything manually

If the answer is no, the governance problem may be bigger than the code-organization problem.

What goes wrong when learners mistake neatness for trust

Use this card before you praise the structure:

If you notice... The hidden problem may be... Smallest review surface
the tree looks clean, but downstream readers are still surprising contract drift file API plus consumer inventory
profiles look tidy, but sample sets differ silently policy leakage profile audit plus dry-run comparison
one helper is "known to be delicate" socially trusted ownership one helper-boundary note plus proof route
proofs are spread across many files and manual habits weak review route one compact proof inventory

If the smallest review surface is missing, the repository is less mature than it looks.

Look for socially trusted areas

Every mature repository has zones people "just know" are delicate:

  • a helper script nobody wants to touch
  • a profile nobody fully understands
  • a publish report everyone trusts but few could rebuild mentally
  • a wrapper or checkpoint whose behavior is explained orally, not in review artifacts

These are not merely social problems. They are review debt.

A good Module 10 review turns them into named technical risks.

A small example

Imagine a repository with:

  • a clean Snakefile
  • organized rule files
  • a popular report/index.html
  • no file API
  • no clear difference between results/ and published outputs

That repository may look professional, but the review should still flag:

  • public contract ambiguity
  • downstream trust risk
  • high migration risk because any change may break hidden consumers

This is why "looks organized" is not a sufficient review outcome.

One concrete review scenario

Imagine a repository where:

  • the publish directory is versioned and widely trusted
  • several notebooks still read internal results/ paths
  • one profile quietly changes sample filtering
  • one helper script expands discovery in ways few reviewers can explain

A weak review says, "the repository needs modernization."

A strong review says:

  1. the public contract is partially clear because the publish bundle is versioned
  2. downstream trust is still leaking onto internal paths
  3. policy and semantics are blurred because profile settings affect sample selection
  4. discovery ownership is too social and not yet reviewable enough for safe migration

That kind of review gives you a real starting point for change.

Write the review in boundary language

A helpful review note sounds like this:

The repository's strongest visible boundary is the versioned publish bundle, but the contract is weakened because downstream readers still depend on results/. Profile comparison is reviewable through the audit bundle, which is a strength. The weakest area is sample discovery, which still depends on helper behavior that is easier to trust socially than to inspect from the repository review route.

That is much stronger than:

The repository should be cleaned up and modernized.

The beginner mistake to catch early

The most common beginner mistake in repository review is confusing visible mess with real risk.

A repository can look messy and still have a trustworthy contract. A repository can also look clean while hiding downstream dependence, policy leaks, and socially trusted helpers.

The repair standard for this course is simple:

  1. name the public contract
  2. name the policy boundary
  3. name the proof route
  4. name the most dangerous socially trusted area
  5. only then discuss migration or redesign

A five-line review note you should be able to write from memory

If this lesson has landed, you should be able to write:

  1. which files or artifacts are truly public
  2. which internal surfaces are useful but not promised
  3. which settings are policy and which are semantic
  4. which proof route another maintainer can trust today
  5. which socially trusted area creates the highest migration risk

If you cannot do that, do not discuss modernization yet.

Keep this standard

Do not approve a migration proposal until the current review names:

  • the public contract
  • the policy boundary
  • the proof route
  • the highest-risk social trust area

If those are not written down first, the redesign is already outrunning the evidence.