The agent guessed a sensor's size. We built a mount around a part that doesn't exist.

Jerome Privott · · 4 min read

A small dashed rectangle labelled GUESSED, about 12 by 10, sitting inside a much larger solid rectangle labelled ACTUAL 25.4 by 17.8 millimetres with four M2.5 mounting holes.

The mount came off the printer looking exactly like the render. The sensor did not fit it. Not by a little — the pocket was roughly half the size of the board it was supposed to hold.

Nobody had measured anything. The design agent needed the dimensions of an SHT40 temperature and humidity breakout, did not have them, and produced a number that was the right order of magnitude and the right shape for the sentence it appeared in. Around 12 by 10 millimetres. Reasonable. A small sensor breakout is about that size.

The actual part is an Adafruit 4885: 25.4 by 17.8 mm, with four M2.5 mounting holes on a 20.32 by 12.70 mm pattern. Not a small discrepancy, and not a small design consequence — the mount was not a pocket that needed opening up, it was a mounting strategy for a part with no mounting holes, built around an object that does not exist.

The part that should worry you

Every gate downstream passed.

The geometry was valid. The tolerances were correct. The clearances cleared. The part fit the build volume, printed without supports, and the assembly check found no interference, because within the model everything was consistent. A wrong premise, carried through correctly, produces an internally flawless result.

This is the failure mode that survives review, and it is worth being precise about why. My gates were checking relationships between numbers. Does the pocket clear the part, does the wall meet minimum thickness, does the screw boss have enough meat. Not one of them was checking whether any number described a real object. That question had never been asked, so nothing could fail it.

You cannot close that with a better prompt. "Use accurate dimensions" is already implied by the task. "Do not guess" does not work, because from the inside a recalled number and an invented one feel identical — that is what makes it a hallucination rather than a lie.

The fix is a schema field, not a prompt

What worked was making the guess impossible to express.

Every dimension in a part definition now carries a required companion field: dimension_source. Where the number came from. A STEP file from the vendor. A datasheet, with the page. Calipers, with the reading.

The review engine enforces it. A dimension with no source is not a warning or a lower confidence score — it is a FAIL, and the pipeline stops.

The move is small and the effect is not. The agent can still be wrong; it can cite a datasheet and transcribe the number badly. What it can no longer do is quietly be wrong, because producing output now requires making a claim I can check in about fifteen seconds. Invention and citation used to be indistinguishable in the output. Now they are different shapes.

It also changes what the agent does when it doesn't know. Before, not knowing produced a plausible number. Now it produces a blocked pipeline and a request for the part. That is a vastly better failure: loud, early, and cheap.

The same bug wearing a different hat

The electrical side had the identical hole, and I only went looking because of the sensor.

An agent laying out a PCB needs a connector's pin order. Same situation: plausible, conventional, frequently right, and catastrophic when wrong in a way no geometric check can see. A mirrored pinout routes perfectly, passes DRC, manufactures beautifully and destroys whatever you plug into it.

So pinout_source is now required too, enforced the same way. Both fields exist for the same reason: the gate has to be able to see the difference between a number that was looked up and a number that was produced.

Generalising it

If you are running agents that emit facts — dimensions, prices, part numbers, API shapes, legal thresholds — the pattern transfers directly:

  1. Find the load-bearing facts. The ones where being wrong invalidates everything downstream rather than degrading it.
  2. Make provenance a required field next to each one. Not a confidence score — those are generated by the same process that generated the error, and a confident guess scores high.
  3. Fail closed on a missing source. If it is a warning, it will be ignored, including by you.
  4. Spot-check the citations. A required field invites a fabricated citation. Reading two per run is enough to keep it honest.

The cost is real: more scaffolding, more blocked runs, more moments where the pipeline demands something instead of proceeding. That is the feature. A pipeline that always produces output is not thereby producing correct output — it just has no way to tell you when it can't.

I have one scrapped mount that makes the argument better than this post does.


Written criteria before any code, and the engine that checks them. That is what we sell. How it works.

Related