Applies toLLM agents built on any framework (custom loops, LangGraph, Semantic Kernel, OpenAI/Azure AI Agents, Bedrock Agents) with tools that change data or systems
Symptom
- Users or monitoring report unexpected changes: tickets closed, records updated, emails or messages sent, cloud resources modified.
- Traces show an agent calling write tools repeatedly or with odd arguments.
- Possibly a prompt injection: the agent read a document, email or web page that told it to do something.
Priority order
- Stop further actions (minutes).
- Contain: cut credentials, isolate affected systems.
- Assess damage and restore.
- Investigate: understand the cause before re-enabling.
Step 1: stop new runs (first 5 minutes)
Pull the highest switch that is safe:
| Switch | Command / action |
|---|---|
| Agent feature flag | az appconfig feature disable --name appcs-prod --feature agent-ticket-triage --yes |
| Gateway switch for the agent's app | Set its APIM named value to on, or suspend its subscription |
| Queue that feeds the agent | Disable the trigger (Function app setting, Logic App disable, pause consumer) |
| Schedule | Disable the job or workflow that starts runs |
Step 2: halt runs already in progress
A flag that's only checked at start-up won't stop a run that's 40 steps into a loop.
- If your runtime checks a stop flag before each step and each tool call, set it now (per run, per agent or global).
- Otherwise stop the workers: scale the deployment to zero, stop the container app or function app.
kubectl -n agents scale deployment ticket-agent --replicas=0
az functionapp stop --name func-agent-prod --resource-group rg-agents
az containerapp revision deactivate --name ca-agent --resource-group rg-agents --revision <rev>
Step 3: cut the agent's ability to act
Even if a process survives, it must not be able to use tools:
- Disable the agent's identity or remove its role assignments on target systems:
az role assignment list --assignee <agent-managed-identity-object-id> -o table
az role assignment delete --assignee <object-id> --role "Contributor" --scope <scope>
- Rotate or revoke API tokens the agent uses for SaaS tools (ticketing, email, CRM).
- Disable write tools in the tool registry or MCP server, leaving read-only tools if investigation needs them.
Step 4: contain and assess
- Freeze the affected systems if needed: put the ticket queue in maintenance mode, pause outbound email.
- Use the agent's traces (OpenTelemetry spans, tool call logs) to list every write action in the incident window: tool, arguments, target, result.
// Example if tool calls are logged to Application Insights as dependencies
dependencies
| where timestamp > ago(6h) and cloud_RoleName == "ticket-agent"
| where name startswith "tool."
| project timestamp, operation_Id, name, data, resultCode
| order by timestamp asc
- Build the reverse list: what to restore, re-open or recall, and who to notify (customers emailed by mistake, owners of changed records).
Step 5: restore
| Damage | Restore from |
|---|---|
| Records changed or deleted | Database PITR or the app's audit/history tables |
| Tickets closed | Ticketing system audit log; bulk re-open |
| Emails sent | Usually can't be unsent. Send a correction; recall only works in limited cases |
| Cloud resources changed | Activity log + IaC to redeploy; resource locks for next time |
Step 6: find the cause before re-enabling
| Question | Evidence |
|---|---|
| Was the input malicious (prompt injection)? | Content of documents/emails the agent read just before the actions |
| Did a prompt or model change cause it? | Deployment history of prompts, model versions |
| Did the tool allow too much? | Tool permissions vs. the task (e.g. "close ticket" had no limits) |
| Why didn't guardrails stop it? | Missing approval step, no rate limit on write tools |
Step 7: re-enable in stages
- Read-only mode (write tools disabled), observe.
- Write tools with human approval for every action.
- Limits per run (max write actions, max targets) before removing approval.
Caution
Don't just flip the switch back once the queue is cleared. Without a fix, the same input or loop repeats the damage.
Prevent it next time
- Stop flag checked before every step and tool call; tested monthly.
- Least-privilege tool credentials per agent; no shared admin tokens.
- Human approval for irreversible actions; per-run caps on write actions.
- Treat retrieved content as untrusted input (see the prompt injection checklist).
- Rate limits and anomaly alerts on write-tool calls.