Agent demos look great. In production, what matters is what happens when the model is confidently wrong.
The problem is not intelligence
The problem is authority. An agent that can open tickets, move stock or email a customer is not a chatbot: it is an actor with side effects. Without a human (or a hard policy) on the critical path, you are automating risk.
Three layers that work
- Classify before acting. Separate “inform”, “propose” and “execute”. Only the last one touches systems.
- Threshold-based approval. Cheap or reversible actions can run alone; expensive, irreversible or ambiguous ones wait for a person.
- Readable audit. Who approved what, with which context, and what the agent did. Without this there is no improvement and no compliance story.
Human-in-the-loop is not “slowness”
Done well, humans do not review everything: they review exceptions. The agent absorbs volume; the person absorbs responsibility. That split is what scales in small teams.
What to avoid
- Endless prompts instead of permissions.
- Model “confidence” as the only brake (it is a score, not a guarantee).
- Flows where a human only sees the outcome when it is already too late.
If you are wiring agents into ERP, support or logistics, start with the action map and who can sign off each one. The model is the engine; the human loop is the brake and the steering wheel.