The best guardrail is the one you can delete

This series asks where design intent, the record of why a system is shaped the way it is, can live so an agent honours it. The last post put it in a document; this one puts it in the running code, and asks what happens to the guardrail once it’s there.

We had a pipeline where an LLM matched entities across several data providers. The same entity carries a different name and a different unique ID in each provider’s catalogue; the model decides two names are the same entity and records the mapping. When it’s wrong, nothing errors, the mapping just points at the wrong entity, quietly.

TL;DR:

About 10% of matches came back with the wrong ID: the model wrote the right names in its reasoning, then returned the ID of the adjacent entry. After the fix, effectively zero.
It was also expensive. The monthly model bill ran $100–200 in early 2026; the cleanup brought it under $50, more than half off.
The reflex fix was a guard that caught the bad IDs.
The real fix was to stop putting IDs in the prompt at all, so the model resolves names and code resolves IDs. That deleted the guard.
The rule of thumb: encode the invariant in the pipeline, not the prompt. A prompt asks nicely; the pipeline makes it true.

The wrong answer the model was sure about

The tell was almost funny. The model’s reasoning said, in effect, “same names, same entity”, then it returned the unique ID sitting one row above the right one. It understood the task and fumbled the clerical part, copying an ID off the wrong line.

So we added a guard: compare the returned ID against the names, reject the row when they disagree. It worked. It also meant every call produced output we didn’t trust, checked it, and threw some away. A load-bearing guard is a standing admission that the thing upstream is allowed to be wrong.

The fix that deleted the guard

The better move was to ask why the model could get the ID wrong: because we handed it IDs at all. So we stopped. The prompt now carries names and positions, no IDs; the model returns names and a positional reference, and ordinary deterministic code resolves that back to the real ID. The model never touches an ID, so it can’t transcribe one wrong. The failure mode was gone, so the guard had nothing left to catch, and we deleted it.

Before and after: on the left, the prompt includes IDs, the LLM returns an ID, it copies the adjacent entry about 10% of the time, and a guard has to check name-vs-ID and reject the bad rows; on the right, the prompt has names only, the LLM returns a name plus position, deterministic code resolves the ID, and the guard is struck out with nothing left to catch

Left: the model is handed IDs and can copy the wrong one, so a guard is mandatory. Right: the model never sees an ID, so the failure can't happen and the guard is deleted.

The guard described an invariant, the ID must match the names, and enforced it after the fact on every call. Moving IDs out of the prompt encoded it instead: IDs are assigned by code, never the model. This is an old idea with a name: make illegal states unrepresentable, or in manufacturing, poka-yoke, designing the process so the mistake can’t be made. A described invariant needs a guard on every run; an encoded one needs nothing, because the violation can’t be expressed.

A note on precision: what I measured, and what I didn't

Two numbers are from my notes at the time, not re-runnable today: the ~10% wrong-ID rate and the dollar amounts. The audit table that would recompute the rate landed in the same sprint as the fix, and the model and its pricing are anonymised, so treat both as honest recollection. The bill also fell for more than this one change: a cache for repeated calls and a learned name dictionary shipped alongside it, all the same lesson, the model was doing deterministic work that code should own.

The takeaway

A guardrail is design intent in the imperative: don’t let this be wrong. Easy to write, expensive to keep, because it runs forever and you stop reading it. On Monday:

Encode the invariant in the pipeline, not the prompt. “Use the correct ID” is advice the model can ignore; a pipeline where the model never sees an ID is a fact it can’t violate. It’s the same move that cut our change-failure rate: make it structural, not advisory.
For every guard you run, ask what design change would make it impossible to trip. If one exists, the guard is a patch; if none does, it’s genuinely load-bearing, keep it, and say so.
Count deletions as progress. A guard that can never be needed again is one less thing to watch, a better outcome than a guard that passes.

What would change my mind

If a matcher that never sees IDs still produces cross-wired mappings, a name that legitimately resolves to two entities, then removing IDs only relocated the failure. The check: audit the mappings written after the ID drop for wrong-entity matches that aren’t transcription errors. A meaningful rate there means the invariant wasn’t the whole story, and the guard comes back.

The reflex is to add a check. The senior move is to change the design until there’s nothing left to check. Encode the invariant in the pipeline, not the prompt.

Next in this series: your CLAUDE.md is code, and it rots like code. Some invariants can’t be made structural yet, so they live as a written rule the agent reads, and that rule decays the same way the code around it does.