Home / AI & GenAI

AI & GenAIHow-toIntermediate

Advisory: designing human-in-the-loop for AI actions - approvals, confidence thresholds and review queues

Human review is the main safety control for AI that takes actions, but badly designed reviews turn into rubber-stamping. Decide which actions need approval by risk, show reviewers the evidence they need, route by confidence, measure reviewer agreement and overrides, and reduce review only when data shows it's safe.

Deepak Bhardwaj2 min read

Applies toAI agents and LLM-driven workflows that change systems, send communications or make decisions (IT operations, ticket handling, finance, customer communication)

Decide what needs approval

Risk Example action Review
Low, reversible Tagging a ticket, drafting a reply for a human to send None or sampled
Medium Sending a customer email, closing a ticket, restarting a non-prod service Approval or confidence-based
High, irreversible or costly Changing production config, issuing refunds, deleting data, access grants Always approval (sometimes two people)

Patterns

Pattern How
Approve before action Agent proposes; human approves/edits/rejects; system executes
Confidence routing High confidence → auto; low → human queue
Sampled audit Auto-execute low-risk actions; humans review a sample afterwards
Escalation Agent hands off to a human when stuck or out of policy

Give reviewers what they need

  • The proposed action in plain language and exact parameters.
  • Evidence: sources, data used, reasoning summary.
  • Impact: what will change, who is affected, how to undo.
  • One-click approve/edit/reject with a reason field.
Proposed action: Restart service "payments-worker" on node aks-pool2-03
Why: Health check failing for 12 min; memory 98%; runbook RB-114 step 3
Impact: ~30 s of reduced worker capacity; no data loss expected
Rollback: N/A (restart). If it fails, page on-call.
[Approve] [Edit] [Reject: reason]

Avoid rubber-stamping

Problem Countermeasure
Too many low-value approvals Remove review for low-risk actions; batch similar ones
Reviewers approve without reading Highlight what's unusual; occasional known-bad test items
Time pressure SLAs and queue sizing; auto-expire stale proposals rather than auto-approve

Measure

Metric Use
Approval / edit / reject rates Quality of proposals
Override reasons What to fix in prompts/tools
Time to decision Queue health
Incidents after approved actions Review effectiveness

Reduce review (e.g. move from approval to sampled audit) only when reject/edit rates stay low over a meaningful period, and record that decision.

Warning

A proposal that times out must not execute by default. Expired approvals should cancel the action and notify someone.

Checklist

  • Actions classified by risk
  • Review pattern per risk level
  • Reviewer view shows evidence, impact and rollback
  • Rubber-stamping countermeasures in place
  • Approval metrics tracked
  • Changes to review levels decided on data

Related fixes