Config as Data, Profiles as Policy¶

This page separates two ideas people often mix together: workflow meaning and execution policy.

How to study this page¶

Read this page with one question in mind:

if I change this setting, did I change the result I am trying to compute, or only the conditions under which Snakemake runs?

If you keep that question visible, config and profiles stop feeling like arbitrary containers and start feeling like different responsibilities.

Why this lesson belongs in Module 01¶

Beginners often postpone this boundary because it sounds advanced. That is a mistake.

If the boundary is blurry on day one, the workflow becomes harder to explain on day two:

target sets start depending on machine-local files
semantic choices hide inside executor bundles
debugging turns into guesswork because meaning and operating context are mixed together

This is still a first-principles lesson. It protects the file-contract model from becoming dishonest too early.

The sentence to keep¶

When you add a setting, ask:

does this change what the workflow computes, or only how Snakemake executes it?

That one question prevents a lot of confusion.

The boundary in plain language¶

Module 01 wants a simple split:

config describes semantic workflow choices
profiles describe execution policy

Config changes meaning. Profiles change operating conditions.

This is not bureaucratic tidiness. It is how you keep workflows explainable.

A quick self-check before you keep reading¶

Take one setting you already know from another tool.

If it chooses which data, threshold, or reference the workflow uses, expect it to land in config.
If it changes concurrency, logging, retries, or executor behavior, expect it to land in a profile.

That small prediction habit matters more than memorizing a definition.

What belongs in config¶

Config is for values that affect the outputs you mean to compute.

Typical examples:

sample lists
thresholds
reference paths
selected assays or panels
output modes that change report meaning

If changing the value should produce a meaningfully different result, config is a good home for it.

What belongs in profiles¶

Profiles are for execution behavior.

Typical examples:

core count
latency wait
retry policy
printing shell commands
cluster or executor settings

These change how the workflow runs, not what the final outputs mean.

That distinction matters because you need to know whether a change should alter the DAG or just alter scheduling and execution policy.

A small table that helps¶

Question	If yes, prefer...
does changing this alter output meaning	config
does changing this alter execution behavior only	profile
should another reader review this as part of workflow semantics	config
should this vary by machine or execution environment without changing the science or logic	profile

This is not mathematically perfect, but it is a strong beginner rule.

The beginner trap to avoid¶

People often treat profiles as a convenient place for "whatever I do not want in the Snakefile." That turns profiles into hidden workflow meaning.

Once that happens, two people can run the "same" workflow and silently compute different things.

Why this split matters¶

When the boundary is blurry, several bad things happen:

semantic choices drift into machine-local profile files
the same workflow means different things on different systems
debugging becomes harder because you cannot tell whether the issue is workflow logic or operating context
reviewers miss important meaning changes because they look like execution changes

Strong workflows keep that boundary teachable.

A simple healthy setup¶

Example config:

samples:
  - A
  - B
threshold: 10
reference: data/reference/ref.fa

Example profile:

cores: 4
printshellcmds: true
rerun-incomplete: true
latency-wait: 5

The config explains the workflow's semantic universe. The profile explains how Snakemake should behave in this operating context.

The most common beginner mistake¶

A beginner often stores semantic choices in a profile because it feels convenient:

cores: 4
samples:
  - A
  - B

This is a bad boundary.

Why:

samples changes the intended target set and output meaning
a profile should be swappable across machines or environments
semantic workflow meaning should not hide inside an execution-policy bundle

If a sample list changes, that should be a workflow-data discussion, not a machine-profile discussion.

Validation should happen early¶

Once config holds meaningful workflow data, the next responsibility is obvious:

validate it before jobs start.

Example:

from snakemake.utils import validate

configfile: "config/config.yaml"
validate(config, "config/schema.yaml")

This is a very strong beginner habit because it changes a vague late failure into an early explicit one.

Without early validation, people often discover mistakes too late:

missing keys
wrong shapes
invalid sample names
unsupported options

Those are easier to teach and repair at parse time than during job execution.

A concrete example of failing early¶

Suppose the schema requires samples, but the config says:

samplez:
  - A
  - B

Without validation, you may later hit:

a KeyError
a confusing expansion failure
an empty target list that feels mysterious

With validation, the workflow fails immediately and says the config shape is wrong.

That is much more humane.

A useful diagram¶

flowchart LR
  meaning["Workflow meaning"] --> config["config/config.yaml"]
  policy["Execution policy"] --> profile["profiles/default/config.yaml"]
  config --> outputs["Which outputs should exist"]
  profile --> runtime["How Snakemake runs to produce them"]

The diagram is simple on purpose. Module 01 does not need every Snakemake settings surface. It needs one clean mental split.

Profiles should not smuggle in workflow meaning¶

A profile can absolutely influence the run experience.

It can control:

concurrency
logging verbosity
retries
executor behavior

What it should not do is quietly choose:

which samples exist
what threshold defines success
which reference or panel is the scientific source of truth

If you change a profile and the outputs mean something different, the boundary has likely drifted.

Keep paths understandable¶

Config often carries paths. That is fine, but beginners need a rule:

paths in config are still semantic inputs if they determine what data or reference the rule uses.

That means they deserve:

clear naming
validation where possible
review attention when they change

They are not just operational details because they happen to be strings.

A useful review habit¶

When looking at a setting, try this short review:

if this value changes, should the result meaning change
if yes, can another reader find it in config easily
if no, does it belong in a profile or execution context instead
if it is in config, is its shape validated before any jobs start

Those questions keep the boundary stable.

What a good Module 01 answer sounds like¶

Strong:

Changing samples belongs in config because it changes the target surface. Changing cores belongs in the profile because it changes scheduling only.

Weak:

I put it in the profile because that file already existed.

A small example of the right explanation¶

Weak explanation:

the workflow behaves differently on my machine.

Stronger explanation:

the sample list was stored in a profile instead of config, so the workflow's semantic target surface changed with the execution context rather than with intentional workflow data.

Or:

the workflow failed late because the required config key was missing and the Snakefile did not validate config at parse time.

Those are repairable explanations.

Failure signatures worth recognizing¶

"It works with one profile but builds a different artifact set with another"¶

That often means semantic workflow data leaked into profiles.

"The workflow crashes halfway through because a config key is missing"¶

That usually means validation happened too late or not at all.

"We cannot tell whether this option belongs to science, workflow logic, or cluster policy"¶

That means the boundary between config and policy has not been written clearly enough.

"A machine-specific setting changed the meaning of results"¶

That is a strong sign the workflow meaning is not isolated cleanly.

What this page wants you to remember¶

Config is for meaning. Profiles are for operating policy.

If you keep that one split clear and validate config early, the workflow becomes easier to read, easier to review, and much less likely to surprise readers for the wrong reasons.