“Human in the loop” sounds like a safety strategy. But I’m starting to think it’s too vague to be useful.
Post from 7 October 2026
Imagine a coding agent makes 20 changes:
- formats a file
- adds test scaffolding
- updates documentation
- refactors some repetitive code
- changes an authorization rule
- modifies a database schema
If a human has to carefully approve every one of those changes, we’ve created a review bottleneck and may have erased much of the productivity gain.
But letting the agent make all the decisions autonomously doesn’t seem right either. I’m starting to see human-in-the-loop less as a review strategy and more as a risk-allocation decision.
I’d put more human judgement around a change when:
- getting it wrong has meaningful consequences,
- the change is difficult to reverse,
- or deciding what’s correct depends on context the AI may not have.
Most decisions then land in one of the following zones:
- Low stakes, clear tasks: let the agent act. For example, code formatting, boilerplate, test scaffolding, mechanical refactors, docs.
- Low stakes, fuzzy tasks: let the agent propose, and spot-check a sample instead of reviewing everything.
- High stakes: a human approves at minimum. Auth, schema migrations, concurrency, public API contracts, destructive operations. If the requirements are also unclear, the human makes the decision instead of just signing off on the agent’s.
Even “high consequence” doesn’t always mean a human must inspect every individual action. Sometimes the better answer is a deterministic control.
For example, instead of asking someone to review every AI-generated refund, the system might simply enforce a rule: “Refunds over $500 require human approval.”
The goal isn’t to put a human between the AI and everything it does. It’s to decide where human judgement actually adds value, and where software can enforce the boundary instead.
Loading comments…
Comments aren’t loading right now.
The post is unaffected; the comment service didn’t respond. Try again in a moment, or come back later.