Sev1: passwords, keys or personal data found in LLM prompts, logs and traces

#Typical ways it happens

• Users paste credentials or customer records into a chat assistant.
• An agent's tool returns secrets (environment dumps, config files) into the context.
• Full prompt/response logging turned on for debugging and never turned off.
• Traces exported to a third-party observability SaaS with long retention.
• Production conversations copied into evaluation datasets.

#Step 1: stop new capture (first hour)

• Turn off full prompt/response logging, or switch to redacted/metadata-only logging, in the app and gateway.
• Disable export to external tracing tools until redaction is in place.
• Check provider-side logging you enabled: Bedrock model invocation logging (S3/CloudWatch), Azure diagnostic settings that include request/response content.

#Step 2: rotate secrets first

Any credential seen in logs is compromised: anyone with log access could have read it.

• List the secret types found (API keys, DB passwords, tokens, connection strings).
• Rotate each one; revoke old values; check for use of the old values in the period they were exposed (sign-in logs, CloudTrail, provider usage).

Rotation goes first because purging logs takes longer, and copies may exist that you never find.

#Step 3: find every copy

Example KQL to locate likely secrets:

union AppTraces, AppDependencies
| where TimeGenerated > ago(90d)
| where tostring(Properties) matches regex @"(?i)(api[_-]?key|password|secret)\s*[:=]\s*\S{8,}"
   or tostring(Properties) matches regex @"AKIA[0-9A-Z]{16}"
| summarize count() by bin(TimeGenerated, 1d), AppRoleName

#Step 4: purge

• Log Analytics: the Purge API deletes data matching a filter (requires the Data Purger role; it's slow and meant for compliance cases). Alternatively, if the table holds only this data, reduce retention, but that removes everything.

az rest --method post \
  --url "https://management.azure.com/subscriptions/<sub>/resourceGroups/rg-obs/providers/Microsoft.OperationalInsights/workspaces/law-prod/purge?api-version=2020-08-01" \
  --body '{"table":"AppTraces","filters":[{"column":"TimeGenerated","operator":">","value":"2026-10-01T00:00:00"},{"column":"AppRoleName","operator":"==","value":"chat-api"}]}'

• CloudWatch: delete affected log streams/groups (no per-record delete); S3 exports: delete objects (and old versions).
• Observability SaaS: delete traces via their API/UI; ask their support about backups.
• Caches and conversation stores: delete affected entries.
• Evaluation sets: replace with redacted versions.

Record what was purged and when. Data protection teams will need it.

#Step 5: involve the right people

Personal data in logs may be a reportable breach depending on who could access it and where it went. Security and data protection decide. Give them: data types, number of records/users, storage locations, who had access, retention, and purge status.

#Step 6: fix the pipeline

• Redact before writing: run PII/secret detection on prompts and responses before logging (Presidio, cloud DLP services, regexes for key formats). See the Presidio redaction post.
• Log metadata by default (token counts, latency, model, prompt version, document IDs); full text only with sampling, short retention and restricted access.
• Secret scanning on tool outputs before they enter the model context.
• Tell users not to paste credentials, and detect it in the UI.

Warning

Debug logging "for one day" is how most of these incidents start. Put an expiry on any full-content logging switch.

#Prevent it next time

• Logging policy for AI apps: what may be stored, where, for how long, and who can read it.
• Redaction in the logging pipeline, tested with sample secrets.
• Short default retention for content logs; access reviews.
• Periodic scans of log stores for secret patterns.