Human-in-the-Loop Design for AI Agents: Approvals That People Actually Read

Requiring human approval for risky agent actions is one of the most effective safety controls available, and one of the most commonly done badly. A dialog that says "Allow this action? Yes / No" appears dozens of times a day, and people soon click Yes without reading. This article explains how to choose what needs approval and how to design the approval itself so it genuinely catches errors.

What human-in-the-loop is for

Approval addresses two different problems:

• Mistakes: the model misunderstood and is about to do the wrong thing.
• Attacks: a prompt injection has redirected the agent toward something harmful.

In both cases the human is a check that does not share the model's blind spots. This only works if the human sees accurate information and has time to evaluate it.

Step 1: Classify actions by impact

Build a short table of every tool and what happens if it is misused.

The question is not "is the agent probably right?" but "how bad is it if it is wrong, and can we undo it?"

Step 2: Show the real action, not a summary

The most important design rule: display the concrete operation and its parameters, generated by your code, not a description written by the model. A model that has been manipulated can describe a harmful action in harmless words.

Poor prompt: "The assistant wants to send a message to complete your request. Allow?"

Better prompt:

Send email
To:      alice@external-example.com   (external domain)
Subject: Q3 customer list
Attachment: customers.csv (14,203 rows)
[Approve]  [Edit]  [Reject]

Highlight what is unusual: external recipients, large data volumes, new destinations, amounts above a threshold.

Step 3: Reduce approval volume

Every unnecessary prompt trains users to ignore the necessary ones. To keep prompts rare and meaningful:

• Automate low-risk actions rather than asking.
• Allow users to pre-approve narrow, specific rules ("always allow reading files in this project folder"), with scopes that are visible and revocable.
• Batch related low-risk actions into one review.
• Remember decisions for identical actions within a session.
• Never offer a blanket "always allow everything" option for high-impact tools.

Step 4: Make rejection and editing easy

Users should be able to reject an action and say why, and to edit parameters before approving. Rejection feedback goes back to the agent so it can try another approach. A flow where "No" simply stops the whole task discourages saying no.

Step 5: Use friction proportionally

For the highest-impact actions, add deliberate friction: type the target name to confirm, wait a few seconds, require a second approver, or require re-authentication. Use this sparingly so that it keeps its meaning.

Step 6: Handle asynchronous work

Agents that run in the background cannot wait at a dialog. Options include pausing at a checkpoint and notifying the user, queuing actions for batch review, or restricting unattended agents to low-impact tools. Make sure notifications contain the same concrete details as an interactive prompt.

Measuring whether oversight works

Track:

• Approval rate. If 99.9% of prompts are approved, either the prompts are unnecessary or people are not reading.
• Time to decision. Decisions in under a second suggest reflex clicking.
• Edit and rejection rate. Some rejections are healthy.
• Seeded tests. Periodically inject a known-bad action in a test environment and see whether reviewers catch it.
• Incidents traced to approved actions, as a signal that prompts lack information.

Common mistakes

1. Showing the model's description instead of the actual parameters.
2. Asking for approval on everything, causing fatigue.
3. Placing the only safeguard in the prompt ("ask before deleting") rather than in code that enforces it.
4. Letting the agent request approval in a channel the user does not see.
5. Treating approval as the only control. It should sit on top of least privilege, not replace it (secure agentic workflows).

Good human oversight is a design problem: show the right information, ask rarely, and make the safe choice easy.