Why the human stays in the loop

One question that comes up whenever you build an AI tool or workflow: does success mean full autonomy? Cherry, my feedback-triage tool, put the specific version of it in front of me — if the model is good enough to triage feedback, why should a person still review it? It's a fair question. The whole point of automating a first pass is to stop doing the first pass. Keeping a human in the loop looks, from a distance, like keeping the thing you were trying to remove.

For systems like this one, my answer is no — at least not yet. I built the correction step into Cherry from the beginning, on purpose, and the more I've worked on it, the more I've stopped thinking of it as a training wheel. It's a load-bearing wall. Here's why.

The model can't know what it was never told

Suppose Cherry reads this piece of feedback: "Checkout is terrible." A reasonable classification: bug, route to the checkout team. But the PM reading the triage knows something no model could infer: we redesigned checkout yesterday, on purpose — people are reacting to the redesign rather than reporting a defect. Or: this customer is on the mobile app; the web checkout is fine.

None of that lives in the feedback. It lives in the organization — in yesterday's launch, in last week's decision, in the tribal knowledge of who shipped what. A model can only reason over what it's given, and the freshest, most decision-relevant context is precisely the stuff nobody has written down yet. The human in the loop is how that context enters the system.

Someone has to decide what good looks like

My favorite question about any AI system: who decides when it's wrong? It can't be the model — that's circular. It has to be a person.

Say Cherry clusters forty comments into one issue: "customers hate pricing." A PM looks at it and says no — this is three separate problems wearing one trench coat: confusing pricing pages, unexpected fees, and enterprise billing. Three issues, three owners, three different fixes.

That correction does more than fix today's output. It's a definition of what better synthesis looks like, delivered by the only party qualified to define it. The system's standard of quality is human judgment, captured one correction at a time.

Sarcasm, and other human sports

"I LOVE waiting 45 minutes for support."

A person reads that and winces. A model might file it under positive sentiment. Models have gotten remarkably good at this kind of thing, but the tail of human expression is long — irony, in-jokes, domain shorthand, the customer who says "fine" in a way that means the opposite. The edge cases are exactly where automated confidence is least deserved, which is why Cherry surfaces its least confident calls for review first. Human attention is scarce; it should go where the model is shakiest. The obviously-right calls can wait.

Trust is built by showing your work

Imagine you own a roadmap, and someone hands you a priority list. Which sentence would you rather hear?

"The model says these are the top five issues."

Or: "The model proposed these five. Here's the source evidence for each. I reviewed them and made two corrections."

The second is the one a responsible decision-maker can act on. The two lists might even be identical. The difference is that judgment was visibly applied and someone put their name on it. That's what the human buys: it converts output into something a decision-maker will actually stake a quarter on.

The correction is the product

This is the part I find genuinely exciting. Without a correction step, the system is a line: feedback goes in, an answer comes out, and tomorrow's answer is exactly as good as today's. With one, it's a loop — and loops compound.

The box that most people skip is the third one: measure. A correction you don't measure is just an opinion with edit access.

The mechanics are simple to describe: compare the model's original output against the human-corrected version, graded by an independent evaluator — often another model — against explicit criteria. In Cherry's case the rubric is grounding, clustering, ranking, routing, and actionability, each scored separately. Nobody should blindly trust the evaluator — it's a model too, with its own failure modes. What it provides is a scalable measurement signal, kept honest by layering: deterministic checks catch what code can catch (valid schema, evidence present, ranking actually sorted), the evaluator grades the judgment calls, and humans periodically audit the evaluator against their own labels. I wrote up that whole stack in the eval cheat sheet; the short version is that no single grader is trustworthy, but the layers keep each other honest.

I learned the importance of this the embarrassing way. Early on, Cherry's quality score — an independent model grading each triage against a rubric — went down after I made a genuinely good correction. The problem wasn't the correction; it was the measurement. Applying a correction re-ran the entire triage from scratch, so the grader was comparing two different random drafts, and the noise swamped the signal. The fix was to apply corrections surgically — change only what the human's judgment requires, leave every other byte identical — so the score's movement reflects the correction and nothing else. Once that was in place, the number became trustworthy: it rises when a correction genuinely helps, and a falling correction rate over time means the system is learning. "It gets better" stopped being a claim and became a measurement.

Which is the real answer to how these systems improve. The system doesn't get better because a human made an edit. It gets better because the edit was measured — compared, scored, and folded back into prompts, retrieval, and evaluation criteria. The edit is a data point. The measurement is what turns it into learning.

The point

The goal is for the model to eliminate the repetitive first-pass work — the reading, clustering, and sorting that consumed hours — so that human attention lands where it's irreplaceable: the calls that require context the model was never given, and the standards only a person can set. Replacing judgment was never on the table. The model proposes. A person decides. The system measures whether the person's refinements made it better, and tomorrow starts ahead of today.

That's not a human in the way of the loop. That's the human as the loop.