← Index of work

Your AI agent is a factory

July 2026

The most useful way to think about agent reliability may come from the factory floor.

I used to think manufacturing and AI lived at opposite ends of the technological spectrum.

Manufacturing is physical. You can hold the thing you made, watch it move down a line, and point to the station where it went wrong. An AI agent is mostly invisible: a model moving through context, calling tools, reading documents, and leaving behind an answer or an action.

But I spent a stretch of my career in global R&D at Harry's, where the things we designed had to survive being manufactured at scale, and I have spent the last year building AI workflows. I keep seeing the same shape.

A customer request enters. A process transforms it. Tools and materials affect what happens along the way. An output comes out the other side. Sometimes it is excellent. Sometimes it is subtly wrong. And if the operation matters, "it usually works" is not a quality strategy.

To make the parallel precise, I think of every agent run as five connected things: a task, an environment, a trajectory, an output, and a set of verifiers that judge the work. The same verifier can grade live production, test proposed changes offline, compare models and prompts, or stop a bad output before delivery.

That is not testing software. That is running a quality system.

The agent is not the product. The run is.

The first shift is to stop treating "the agent" as the finished good.

The model, prompt, tools, skills, and orchestration are the production equipment. The individual run is the unit being produced. A strategic account brief, a resolved support ticket, a screened candidate, or an updated CRM record is what actually leaves the factory.

Drawn as a line, one run looks like this.

ONE RUN — from work order to accepted unit TASK the work order ENVIRONMENT docs · data · tool state = material EQUIPMENT model · prompt · skills · harness THE RUN — process steps, in order READ opens only what it is allowed to open REASON interprets, clusters, judges — the contestable part ACT writes something to the world TRAJECTORY the batch record — what it read, called, got back, changed OUTPUT the finished unit — the brief, the ticket, the decision STATE CHANGE what it wrote to the system of record TRAJECTORY VERIFIER made acceptably? OUTPUT VERIFIER meets specification? STATE VERIFIER world changed correctly? BLOCKING GATE — the andon cord critical defect stops the line · minor defect ships DELIVERED an accepted unit rework — with a reason attached FAILED RUN → THE EVAL SUITE the nonconformance becomes a permanent test change the process, not the unit WHAT MAKES IT A SYSTEM, NOT A DEMO: Every arrow that returns. Rework goes back with a reason. Failures go back as tests. Neither is optional — a line with no return path is not a factory, it is a conveyor belt pointed at your customers.

In manufacturing language, the mapping looks something like this:

Agent system Manufacturing system
User task Work order or customer specification
Documents, data, and tool state Incoming material
Model, prompt, skills, and harness Machine, tooling, and work instructions
Tool calls and intermediate reasoning Process steps
Trajectory Batch record or production traveler
Final artifact or state change Finished unit
Verifier Gauge, inspection, or test station
Blocking quality gate Line stop or quarantine
Offline task suite Pilot line and qualification run
Production monitoring Statistical process control
Failed run added to the eval set Nonconformance turned into corrective action

This is more than a neat analogy. It changes where you look for reliability.

If a finished component is out of tolerance, a good manufacturing team does not simply tell the operator to "be more careful." It asks whether the material was in spec, the work instruction was clear, the machine was calibrated, the process was stable, and the inspection method could reliably detect the defect.

Agent failures deserve the same treatment. A hallucinated number might come from a weak model. It might just as easily come from stale source data, an ambiguous task, a failed retrieval, a tool returning the wrong field, a prompt that rewarded fluency over uncertainty, or a verifier that mistook confidence for correctness. "The model got it wrong" is often as shallow a diagnosis as "the factory made a bad part."

A trajectory is a digital production record

The eval systems I trust inspect not only the final output but the full trajectory: what the agent read, which tools it called, what those tools returned, and what it changed.

Manufacturers learned long ago that final inspection is not enough. The FDA's process-validation guidance puts it more strongly than my analogy strictly needs: quality, safety, and efficacy are designed or built into the product, and quality "cannot be adequately assured merely by in-process and finished-product inspection or testing." Note that it rules out both. Checking more of the process helps, but it is never a substitute for designing the process. A polished final artifact can hide a broken one.

Say an agent produces a flawless-looking due-diligence report. A final-output grader may approve its structure, prose, and completeness. But the trajectory could reveal that the agent never opened two required documents, used a source outside the approved data room, or copied a number from an outdated filing. The report passed dimensional inspection while the wrong alloy went into the part.

This is why process checks and output checks answer different questions.

  • Output verifiers ask: Did the thing meet specification?
  • Trajectory verifiers ask: Was it made through an acceptable process?
  • State verifiers ask: Did the system change the outside world correctly?

For low-stakes work, the first is enough. For a candidate decision, a financial recommendation, or an agent that writes to a system of record, provenance and process become part of the product.

Runtime gates are the andon cord

Some verifiers have to block: they stop delivery, hand the agent an explanation, and require a revision before the work reaches anyone.

The manufacturing parallel is jidoka, one of the pillars of the Toyota Production System. Toyota describes it as automation with a human touch: when an abnormality appears, the machine or operator can stop production so defects are not passed downstream.

That is what a runtime gate does. It does not wait for a customer to discover that a deck lacks required citations or that an agent advanced the wrong record. It detects an abnormal condition at the point of work, stops the line, and routes the unit for rework or human review.

The important word is blocking. A dashboard that reports bad outputs after delivery is useful for learning, but it is not a quality gate. It is a returns report.

Not every defect should stop the line. Manufacturing teams distinguish critical, major, and minor defects. Agent systems need the same risk logic. An awkward sentence can ship. An unsupported legal conclusion should not. A weighted average alone can obscure this: five strong scores can mathematically cancel one catastrophic failure. Critical attributes therefore need hard thresholds, not just weights.

The verifier is a measurement system — and measurement systems can be bad

Once you see a verifier as a gauge, another problem becomes obvious: who verifies the verifier?

A measurement system has to be accurate enough for the tolerance it is judging. Teams test whether the same gauge produces consistent results under the same conditions — repeatability — and whether results remain consistent across operators, instruments, locations, or time — reproducibility. NIST treats both as properties of the measurement process, not of the part.

Model judges need the same discipline.

If a verifier gives the same artifact a 0.9 on Monday and a 0.6 on Tuesday, the production process may not have changed; the gauge may be noisy. If a generic judge consistently approves shallow market maps that domain experts reject, the gauge may be precise but wrong. If the agent and verifier share the same blind spot — especially when they use the same model family — the system can achieve internal agreement without external truth.

So verifiers need calibrating against expert judgment, with deterministic checks wherever they are possible. "The file exists in the required folder" should be code, not vibes. "The recommendation is commercially useful" may require a model judge, but that judge should be tested against people who actually do the work.

I drew this line hard in Henry, a vendor-operations agent I built: the model handles research and chat through tool calls, but the margin math runs in ordinary deterministic code. A number a database can compute is not a number worth asking a model to guess, and it is certainly not a number worth asking a second model to grade.

Before optimizing an agent against a verifier, I would want to know:

  • Does the verifier agree with itself across repeated judgments?
  • Does it agree with qualified humans, especially near the pass/fail boundary?
  • What are its false-accept and false-reject rates?
  • Can it be fooled by length, confidence, formatting, or copied rubric language?
  • Does its performance hold on unfamiliar cases?

A bad gauge does not merely misreport quality. In an optimization loop, it teaches the machine to manufacture the wrong thing more efficiently.

An eval suite is a control plan, not a pile of test cases

Manufacturing quality is not defined by inspecting a few pristine samples. A line has to perform across the real variation of raw materials, operators, equipment conditions, and customer requirements.

Agents are no different. A task suite full of clean prompts and complete data is qualifying a line on perfect material and calling it production-ready.

Real enterprise work arrives with missing context, stale documents, ambiguous requests, permissions failures, conflicting sources, and tools that time out. Eval teams call this coverage of the task distribution. A manufacturing engineer might call it understanding the operating range of the process.

So design the suite less like a trivia exam and more like a control plan:

  • What are the critical-to-quality attributes for each kind of work?
  • Which failure modes are most severe, most likely, and hardest to detect?
  • Where in the process can each failure be caught earliest?
  • Which checks can be deterministic, and which require judgment?
  • What production evidence should trigger tighter inspection or a new test?

It is also why one global score is not enough. Two agent versions can post the same average reward and carry completely different risk. One is consistently decent. The other is brilliant nine times out of ten and dangerously wrong on the tenth. In a factory, averages do not excuse a critical defect. In enterprise AI, they should not either.

Do not jump from the lab bench to full production

Before a manufacturer commits to full-scale production, it usually learns in stages. A formulation may begin on a lab bench, move to a pilot batch, then run through the actual line using commercial equipment, trained operators, real materials, target speeds, and an approved control plan. The point of a line trial is not simply to make a few good units. It is to learn whether the process can make good units repeatedly under production conditions.

Agent teams need the same discipline.

An offline eval is a pilot batch, not proof of commercial readiness. Mock tools are cleaner than production tools. Test documents are usually complete. Permissions behave. No one changes a record halfway through the run. Rate limits, concurrent users, long queues, partial outages, and human handoffs rarely appear with their full production messiness.

The gap between a good demo and a reliable deployment is the gap between a benchtop recipe and a line running at speed.

A sensible scale-up sequence, with the amount of real production conditions rising at every step:

SCALE-UP — a demo is stage one of six STAGE THE QUESTION IT ANSWERS · WHAT IT ADDS PRODUCTION REALITY 1 BENCH TEST mock everything Can it do the task at all? clean prompts, complete data, tools that never fail 2 PILOT BATCH the offline suite Consistent across normal, edge, and known-failure cases? variation arrives, but you still chose all of it 3 LINE TRIAL real conditions Does it hold against real data, permissions, latency, concurrency? production-shaped, but nothing reaches a customer 4 CONTROLLED a slice of live traffic Can it take real customers with the gates on? blocking verifiers, close monitoring, a rollback path 5 SCALE-UP full volume Does volume degrade yield, cycle time, or review capacity? ten sequential calls is not one hundred simultaneous ones 6 CONTINUED forever, not once Does it stay in control as inputs, tools, and models drift? every model swap and prompt edit is a new process The demo lives at stage one. The deployment lives at stage six. Almost everything expensive hides between three and five, where the tools are real, the volume is real, and nobody has checked whether the review queue can absorb the output.

This mirrors the lifecycle logic in manufacturing process validation: design the process, qualify it under commercial conditions, then continue verifying it during routine production. It also exposes failures that small tests conceal. A tool that works perfectly for ten sequential cases may collapse under a hundred simultaneous calls. An agent that saves one analyst time may create an impossible review queue at enterprise volume. A fast model may become slow once retries and verifier calls are included.

Scale is not just more volume. It introduces new process conditions.

The same reasoning applies after launch. Swapping the model, editing the system prompt, adding a tool, or changing a verifier is a process change. The larger the change — and the more severe the failures it could create — the more of the line should be requalified before rollout. "It passed last quarter" is not evidence about a materially different production process.

The real unit economics are cost per accepted output

Manufacturing teams do not optimize only for the cost of starting a unit. They care about yield, scrap, rework, inspection, downtime, throughput, and failures discovered after shipment. A cheaper input is not actually cheaper if it creates more defective product.

Agent economics work the same way.

The metric everyone reaches for first is cost per run: model tokens, tool calls, and compute. The one I care about is cost per accepted output:

(agent execution + tool usage + verification + retries + human review + expected failure cost) ÷ accepted outputs

The distinction reverses model decisions that look obvious. A small model can cost half as much on the first attempt, then require more retries, trigger more escalations, and eat more reviewer time. A larger model can be cheaper at the system level because more of its work is accepted the first time. Conversely, an expensive model may add no meaningful quality on routine work and should not be installed at every station merely because it performs best on the hardest cases.

First-pass yield is especially revealing: what percentage of runs clear every required quality gate without revision or human rework? An agent with a 95% eventual pass rate can still be operationally poor if only 50% pass the first time. The retries are the digital equivalent of a rework loop: they consume capacity, lengthen cycle time, and hide instability behind a respectable final number.

Manufacturing has a name for the whole picture: cost of quality. Spending falls into four buckets, and they are not equally priced.

COST OF QUALITY — the later you catch it, the more it costs cost of one defect, caught here ↓ PREVENTION design it out task + tool contracts source controls prompt + rubric design failure-mode analysis cheapest place to spend APPRAISAL look for it verifier calls audit samples red-team tests human review buys the other three down INTERNAL FAILURE caught before release retries and rework diagnosis time discarded outputs late delivery the rework loop, hidden EXTERNAL FAILURE caught by the customer bad decisions made unauthorized actions compliance incidents trust, and earning it back the only bucket with no cap every dollar you move left buys more than the one before it A verifier that costs cents and seconds is not overhead. It is the cheapest thing standing between an internal failure and an external one.

The goal is not to minimize appraisal in isolation. A verifier that costs a few cents and a few seconds is a bargain if it prevents one external failure. But running every possible verifier on every possible task is its own form of waste. Verifiers should have activation conditions for exactly this reason: apply the right inspection at the station where the relevant failure can occur.

The agent dashboard I would want therefore shows more than average quality and token spend. It shows first-pass yield, rework rate, cost per accepted output, cycle time, human minutes per unit, escape rate, and the cost and latency contribution of each verifier. That is the view needed to find the quality-cost frontier — not the cheapest model or the highest score in isolation, but the least expensive process that reliably meets the specification.

Put the human at the point of no return

I have argued elsewhere that the human belongs in the loop. The harder question is where to stand them.

Manufacturing does not spread inspection evenly along the line. It concentrates at the moments where being wrong stops costing one unit and starts costing the entire run. You approve a first article before the run, not during it. You sign off a sample before the tooling is cut, because once steel has been cut for a mold, the mistake is capital rather than scrap.

The economics on either side of those moments are not close. A defect caught on the first article costs one part and an afternoon. The same defect caught after five million units have shipped to a Tier 1 retailer costs the run, the freight, the rework or the write-off, the retailer's chargebacks, and a claims process that can take years to resolve. I have written before about watching that machinery from the Amazon side. It is slow in a way that appears in no unit-cost model.

Agents have the same asymmetry, and two questions decide where the human stands: how expensive is being wrong, and can you take it back?

The second matters more than teams expect, and it is not a restatement of the first. A wrong draft is expensive and free to fix. A wrong email is cheap and permanently sent. Reversibility, not cost, is what separates a rework loop from a remediation project.

WHERE TO STAND THE HUMAN COST OF BEING WRONG HARDER TO TAKE BACK VERIFY, THEN RELEASE expensive to get wrong · but you can fix it a triage that reorders a roadmap a brief someone will act on a draft recommendation blocking verifier + sampled human audit A NAMED HUMAN SIGNS expensive · and there is no undo money, or a promise to a customer a write other systems read as fact deletion, publication, a sent email every time — no sampling, no exceptions LET IT RUN cheap · and trivially fixable internal summaries and notes drafts nobody has acted on yet a ranked list a human reads next no gate — sample it for drift CONFIRM ANYWAY cheap · but you cannot unsend it a comment, a minor message a status the customer can see anything that leaves the building one click, not a full review SCALE MOVES WORK TOWARD THE TOP-RIGHT CORNER: One record is a unit. The same decision applied to forty thousand records overnight is a production run — and the prompt that made it is the tooling. Approving the policy is the irreversible step; the units are only what the tooling stamps.

The irreversible actions are a short and knowable list: anything that moves money, sends a message, publishes, deletes, promises something to a customer, or writes to a record other systems will read as fact. Everything that leaves the building. Before that line, an agent can iterate freely and cheaply, because retries are just rework. After it, there is no rework. There is only apology.

So put the human at the last reversible moment. Not at every step, which burns the review capacity you need at the step that matters. And not after delivery, which is a returns report.

There is one more thing the manufacturing frame makes obvious, and it is the part I think teams get wrong most often. The run is the unit — but the prompt, the policy, and the rubric are the tooling. One agent decision applied to one record is one part. The same decision applied to forty thousand records overnight is a production run, and the policy that produced it stamped every one of them. Sampling the output while nobody signs off on the policy change is inspecting parts while the mold goes uninspected.

This is also where the scale-up sequence bites. An agent that saves one analyst an afternoon can generate a review queue no team can absorb at full volume, and an overloaded reviewer is not a quality gate. They are a rubber stamp with a person's name on it — worse than no gate at all, because now the failure is documented as approved. A human gate has throughput, and throughput has to be planned like any other station on the line.

Production failures should become corrective action

The part of this framework I find most useful is the loop from production back into development.

When a non-blocking verifier catches a failure in a live run, that run becomes a new offline evaluation. The failure becomes reproducible. The team changes the prompt, skill, harness, tool, or other part of the system. The candidate runs against the suite. If quality improves without unacceptable cost or latency, the change can move forward.

This is corrective and preventive action, the same six steps a quality system would run:

  1. Contain the immediate issue.
  2. Record the nonconformance.
  3. Investigate the root cause.
  4. Change the process, not just the affected unit.
  5. Verify that the change worked.
  6. Monitor for recurrence.

The distinction between correction and corrective action matters. Asking an agent to redo one weak brief is correction. Changing the system so that the failure becomes less likely — and adding a test so it cannot quietly return — is corrective action.

This is how an agent stops being a demo and becomes an operation.

Where the manufacturing analogy breaks

The parallel is useful. It is not complete.

First, most enterprise agents are not mass-producing identical widgets. They are closer to a high-mix, make-to-order operation: every request is different, incoming information varies wildly, and the quality standard sometimes depends on context. That makes classical process capability hard to calculate. There may be no single tolerance band for "good strategy."

Second, agent behavior is probabilistic. The same task, environment, and configuration can produce a different trajectory on the next run. Variation is not just noise around the process; it is part of the process. A single passing run proves very little.

Third, many important specifications are contestable. A hole is 5.00 millimeters or it is not. Whether an account brief identifies the right commercial risk is a judgment call. Expert disagreement is not necessarily measurement error; sometimes it reveals that the quality standard itself is underspecified.

Fourth, the act of measuring can change the system. Once verifier feedback is used for optimization, the agent begins adapting to the gauge. This is Goodhart's law in factory clothing: when a measure becomes the target, it can stop being a good measure. Physical parts do not study the inspection rubric. Agents effectively can.

Finally, digital work makes 100% inspection economically possible in a way physical manufacturing often cannot. Every run can, in principle, be logged and graded. But total inspection does not create total assurance. If every unit is checked by the same weak verifier, you have scaled confidence, not quality.

The factory is the product

The biggest mistake in enterprise AI is grading the intelligence of the model while ignoring the reliability of the system around it.

The useful questions are more operational:

What exactly is the specification? What inputs are allowed? Which process steps must be observable? What can fail? Where should each failure be detected? Which defects stop delivery? Is the measurement system trustworthy? What happens to a failure after it is found? Can we prove that a change improved quality rather than merely moving the score?

That is why I think enterprise-agent reliability is fundamentally an evaluation-systems problem. The model matters. But durable advantage comes from defining quality, measuring it credibly, tracing defects to their source, and feeding what you learn back into the process.

In other words: the impressive thing is not an agent that can make one beautiful object.

It is a factory that can keep making good ones — and knows what to do when it does not.

Sources and further reading