Home / AI & GenAI

AI & GenAIChecklistAdvanced

Checklist: giving an AI agent its own identity - scoped, short-lived credentials and an audit trail for every tool call

Agents that run on a shared service account or a user's long-lived token are hard to contain and impossible to audit. A checklist for per-agent identities, delegation, session-scoped credentials and traceable tool calls on Azure and AWS.

Deepak Bhardwaj7 min read

Applies toTool-calling agents and MCP servers on Azure (Entra ID managed identities, Azure RBAC) and AWS (IAM roles, STS); any LLM provider

Why this matters

Most agent incidents get harder to handle because of identity, not the model. The agent ran as a shared automation account, so disabling it broke three other jobs. Or it held a developer's personal token, so the audit log shows a human doing things nobody remembers doing. Or the credential was valid for 90 days and sitting in an environment variable that tool output could leak.

Use this checklist before an agent gets any tool that reads sensitive data or changes state.

1. Decide whose authority each tool uses

Every tool runs under one of two models. Write down which, per tool.

Model When to use Risk
Agent's own identity Background agents, ops automation, shared read-only data Agent can reach data the requesting user cannot
On behalf of the user (delegated) User-facing assistants touching per-user data (mail, tickets, documents) Token handling in the agent; consent and scope design
  • Each tool is labelled "agent identity" or "delegated" in the tool registry.
  • Delegated tools use the user's token exchanged for a downstream token (for example the Entra ID on-behalf-of flow), never a stored refresh token in the prompt or memory.
  • No tool falls back to the agent identity when a delegated token is missing. It fails.

2. One identity per agent, per environment

  • Each agent has its own identity in each environment: agent-triage-prod, agent-triage-staging. No sharing with CI pipelines, other agents or humans.
  • Read and write tools use different identities where the platform allows, so you can revoke write without stopping diagnosis.
  • Identity names and tags record owner, agent name and environment.

Azure, with a user-assigned managed identity:

az identity create -g rg-agents-prod -n id-agent-triage-prod \
  --tags owner=platform-sre agent=triage env=prod

PRINCIPAL=$(az identity show -g rg-agents-prod -n id-agent-triage-prod --query principalId -o tsv)

az role assignment create --assignee-object-id "$PRINCIPAL" \
  --assignee-principal-type ServicePrincipal \
  --role "Monitoring Reader" \
  --scope /subscriptions/<sub-id>/resourceGroups/rg-checkout-prod

AWS, with a dedicated role:

aws iam create-role --role-name agent-triage-prod \
  --assume-role-policy-document file://trust-agent-runtime.json \
  --max-session-duration 3600 \
  --tags Key=owner,Value=platform-sre Key=agent,Value=triage Key=env,Value=prod

3. Scope permissions to the tools, not to the agent's ambitions

  • Permissions are derived from the tool list. If no tool deletes resources, the identity cannot delete resources.
  • Scope is the narrowest resource group, account, namespace or table set the tools need.
  • Built-in broad roles (Contributor, AdministratorAccess, PowerUserAccess) are not assigned.
  • Write permissions are conditional where possible: Azure ABAC conditions on storage, IAM condition keys on tags (aws:ResourceTag/...).
  • Kubernetes tools use a dedicated ServiceAccount with a namespaced Role, not cluster-admin.

Review what an Azure identity actually holds:

az role assignment list --assignee "$PRINCIPAL" --all \
  --query "[].{role:roleDefinitionName, scope:scope}" -o table

4. Make credentials short-lived and session-scoped

  • No static secrets for the agent's own identity. Use managed identity, workload identity federation or role assumption.
  • Credential lifetime is one hour or less, and the agent runtime refreshes it. It never asks the model to handle tokens.
  • On AWS, each agent session assumes the role with a session policy that narrows permissions further for that task, and a source identity that ties it to the request.
aws sts assume-role \
  --role-arn arn:aws:iam::<acct>:role/agent-triage-prod \
  --role-session-name "triage-$(date +%s)" \
  --source-identity "triage-req-7f3a" \
  --duration-seconds 900 \
  --policy file://session-readonly-checkout.json

The effective permissions are the intersection of the role policy and the session policy, so a session for a read-only question cannot write even though the role can.

Warning

Never place tokens, keys or connection strings in the system prompt, conversation memory or tool results. Anything in the context window can be echoed back, logged in traces or extracted by prompt injection. Credentials belong in the tool runtime only.

5. Keep secrets out of the model's reach

  • Tool implementations fetch secrets from Key Vault or Secrets Manager at call time using the agent identity.
  • Tool results are filtered before they enter the context: strip Authorization headers, connection strings and key-like values.
  • MCP servers run with their own identity and do not forward the client's credentials to downstream systems unless the tool is explicitly delegated.
  • Traces and prompt logs go through the same redaction as application logs.

6. Make every tool call traceable to a request

You should be able to answer "which user request caused this API call?" in minutes.

  • The agent generates a request ID per user turn and passes it to every tool.
  • Tools put the ID where the cloud audit log records it: the STS session name or source identity on AWS, a client request ID header or resource tag on Azure where supported, and always in the agent's own tool-call log.
  • The agent's tool-call log records request ID, user, tool, arguments (redacted), identity used, result status and duration.

Find what the agent identity did in Azure:

AzureActivity
| where TimeGenerated > ago(24h)
| where Caller in ("<managed-identity-client-id>", "<managed-identity-principal-id>")
| project TimeGenerated, OperationNameValue, ActivityStatusValue, ResourceGroup, _ResourceId
| order by TimeGenerated desc

The Caller value for a managed identity can be its client ID or object ID depending on the operation; include both.

And in AWS:

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=Username,AttributeValue=triage-1731150000 \
  --max-results 50 \
  --query "Events[].{t:EventTime,e:EventName,src:EventSource}" --output table

7. Plan revocation before you need it

  • You can disable one agent's identity without affecting other workloads: remove role assignments in Azure, attach a deny-all policy to the AWS role.
  • You know how long existing tokens stay valid after revocation. Azure access tokens and STS sessions remain usable until expiry unless you take further action, which is why short lifetimes matter.
  • For AWS, you have tested revoking active sessions (an inline deny on aws:TokenIssueTime older than now, which the console's "Revoke active sessions" action creates).
  • The revocation steps are in the agent's emergency stop runbook and have been drilled.

Caution

Revoking an identity mid-task can leave half-finished changes: a ticket updated but not assigned, a scale-up without the matching config. Pair revocation with a check of the agent's tool-call log for writes made in the last session.

8. Review on a schedule

  • Quarterly: compare granted permissions with permissions actually used (Azure activity logs, IAM Access Analyzer last-accessed data) and remove the unused.
  • On every new tool: permission change reviewed by someone other than the agent's developer.
  • On model or prompt change: no change to identity, but rerun the safety tests that check forbidden actions.

Quick reference

Do Don't
One identity per agent per environment Share an automation account
Managed identity or role assumption Static keys in env vars
Session policy per task One broad role for everything
Request ID in every tool call Rely on the model's own log of what it did
Delegated tokens for per-user data Agent identity reading every user's mailbox