← 博客

Human-in-the-Loop for AI Agents: Patterns That Scale (2026)

Human-in-the-loop (HITL) is the practice of keeping a person able to approve, correct or stop an AI agent at the moments that matter. Done badly, it makes you the bottleneck of your own agents. Done well, it lets you supervise ten agents with less effort than one used to take.

Where to put the approval gates

  • Irreversible actions. Deploying, deleting, sending, paying: always behind explicit approval.
  • Cheap-to-review outputs. A 5-line plan takes seconds to review and saves hours of misdirected work — approve plans, not just results.
  • Spend and time thresholds. If an agent exceeds X tokens or Y minutes on a task, pause and ask.
  • Trust by track record. Tasks the agent has done well 20 times can graduate to auto-approval; novel ones can't.

The classic failure: rubber-stamping

If every approval reaches you without context — without seeing what the agent did before or will do next — you end up approving on autopilot. A good HITL interface shows the agent's state, the step it proposes and its reasoning in the same view. The quality of your decisions is capped by the quality of that view.

Approval latency = system speed

An agent blocked waiting for your approval is wasted capacity. If you can only approve from your desk, your agents work at the speed of your chair. Being able to review and approve from your phone, in the moment, turns HITL from a brake into an advantage: the agents keep moving while you live your life.

Scaling from 1 to N agents

With several agents, the pattern that works is a single decision queue: every approval request from every agent in one place, ordered by urgency. Your job stops being watching terminals and becomes dispatching decisions — which is exactly the work a human does better than any agent.

The patterns, named

Most HITL designs are one of seven shapes. Knowing which one you are using is what stops a supervision layer from drifting into either a rubber stamp or a bottleneck.

  • Approval gate. The agent stops before a specific action and waits. Correct for anything irreversible; expensive if applied to everything.
  • Plan-then-execute. The human approves a plan once, and the agent runs it without further interruption. One decision buys a lot of autonomy, and it is the highest-leverage gate of the seven.
  • Autonomy budget. The agent acts freely within limits — spend, time, files it may touch — and escalates only when it would exceed one. Supervision becomes a boundary rather than an interruption.
  • Escalation on uncertainty. The agent asks when its own confidence is low. Cheap and useful, with the caveat that models are poorly calibrated: an agent that is confidently wrong will not raise its hand.
  • Shadow mode. The agent proposes and a human executes, until the proposals are good enough to trust. The standard way to earn autonomy for a new task.
  • Post-hoc sampling. Nothing blocks; you review a percentage afterwards. The right pattern for high-volume, low-stakes work, where an approval gate would cost more than the mistakes.
  • Circuit breaker. No human in the moment at all — the system halts itself on an anomaly (spend spike, repeated failures, a loop) and asks afterwards. This is the pattern that saves you at 3am.

Mature setups combine them: plan-then-execute for the shape of the work, an autonomy budget inside it, approval gates on the handful of irreversible steps, and a circuit breaker underneath the whole thing.

Choosing between them

Two questions decide it. How reversible is the action? Sending an email, deploying, paying, deleting — these do not get sampled after the fact, they get a gate. Drafting, summarising, searching — these do not get a gate, they get sampled.

And how often does it happen? A gate on something that occurs twice a day is supervision. The same gate on something that occurs two hundred times a day is a queue you will start clearing without reading, which is worse than no gate at all — it manufactures the appearance of oversight.

What breaks as you add agents

The pattern that works at one agent usually fails at ten, and it fails quietly. Approvals that arrived one at a time start arriving in bursts. Requests lose their context because you no longer remember which agent was doing what. The queue grows faster than you clear it, and the response is to approve faster rather than to change the design.

The fixes are structural, not personal: raise the autonomy budget for tasks with a track record, move from per-step gates to per-plan gates, and make the queue show the agent's state next to its request so a decision takes ten seconds instead of two minutes of reconstruction.

Common questions

What is human-in-the-loop for AI agents? Keeping a person able to approve, correct or stop an agent at the moments that matter, without watching it the rest of the time. The goal is not to review everything — it is to spend your attention only where it changes the outcome.

Does HITL slow agents down? Badly designed, yes: an agent blocked on your approval is idle capacity, and if you can only approve from your desk, your agents run at the speed of your chair. Designed well, it speeds the system up, because agents can act with more autonomy when there is a reliable way to stop them.

Which pattern should I start with? Plan-then-execute plus a circuit breaker. One approval at the start where you can still cheaply change direction, and an automatic halt for when something goes wrong while you are asleep. Add narrower gates only for irreversible actions.

How do I stop rubber-stamping? Reduce the number of approvals until each one deserves reading. If you are approving more than a handful of things an hour, the answer is not more discipline — it is a wider autonomy budget and gates on fewer, more consequential steps.

Orchestrate your AI agents from your phone

ConductLoop lets you monitor and steer the AI agents running on your computer from your phone — local-first, no accounts. Join the waitlist for early access.

Join the waitlist →
在 WhatsApp 上聊天