Sev1: an LLM API key leaked to a public repo and someone is burning your quota

#Symptoms

• GitHub secret scanning, a partner or the provider emails you about an exposed key.
• Token usage or cost jumps far above normal, often at night, from models or regions you do not use.
• 429 errors for your real application because someone else is consuming the quota.

#Step 1: revoke or rotate now (5-10 minutes)

#Azure OpenAI

Resources have two keys so you can rotate without downtime. Find which key leaked (compare the last characters), switch apps to the other key if possible, then regenerate the leaked one:

az cognitiveservices account keys list -g rg-ai -n aoai-prod --query "{key1:key1, key2:key2}" -o table   # compare suffixes only
az cognitiveservices account keys regenerate -g rg-ai -n aoai-prod --key-name key1

If you cannot tell which key leaked or both are in the same config, regenerate both and accept a short outage while apps pick up the new key.

Stronger containment: disable key auth entirely if your apps can use Entra ID:

az resource update -g rg-ai -n aoai-prod --resource-type Microsoft.CognitiveServices/accounts \
  --set properties.disableLocalAuth=true

#OpenAI API

Delete the key in the platform dashboard (Project > API keys). Create a new project-scoped key with only the permissions the app needs, store it in your secrets manager, and redeploy.

#Amazon Bedrock (IAM keys)

aws iam update-access-key --user-name svc-genai --access-key-id AKIA... --status Inactive
aws iam create-access-key --user-name svc-genai      # only if the app still needs keys
aws iam delete-access-key --user-name svc-genai --access-key-id AKIA...

Check for anything the attacker created with that identity (other access keys, roles, users) via CloudTrail.

Caution

Deleting a key only stops new requests. Assume everything the key could access was accessed: for AWS, that may include more than Bedrock if the IAM user had broad permissions.

#Step 2: confirm abuse and scope

Azure: metrics by deployment and time.

az monitor metrics list --resource $(az cognitiveservices account show -g rg-ai -n aoai-prod --query id -o tsv) \
  --metric ProcessedPromptTokens GeneratedTokens --interval PT1H --aggregation Total \
  --dimension ModelDeploymentName --offset 3d -o table

With diagnostic logs enabled (RequestResponse category), find caller IPs:

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES" and TimeGenerated > ago(3d)
| summarize calls = count() by CallerIPAddress, OperationName, bin(TimeGenerated, 1h)
| order by calls desc

AWS: CloudTrail and Bedrock invocation logs.

aws cloudtrail lookup-events --lookup-attributes AttributeKey=AccessKeyId,AttributeValue=AKIA... \
  --start-time $(date -u -d '-7 days' +%FT%TZ) \
  --query "Events[].{t:EventTime, name:EventName, src:CloudTrailEvent}" --output json | jq -r '.[] | [.t, .name, (.src|fromjson|.sourceIPAddress), (.src|fromjson|.awsRegion)] | @tsv' | sort | uniq -c | head -40

Calls in regions you never use (attackers often spread across regions to get more quota) are a clear sign.

Record: first abusive call, total tokens/cost, regions, IPs, whether any data (files, assistants, fine-tuning jobs, knowledge bases) was accessed.

#Step 3: contain cost

• Lower quota (TPM) on deployments temporarily if abuse continues through another path.
• AWS: SCP denying bedrock:* in regions you do not use; Azure: policy restricting allowed locations for AI resources.
• Check for resources the attacker may have created (new deployments, fine-tuning jobs, other accounts' resources) and remove them.
• Contact provider support about the fraudulent usage; have timestamps and evidence ready.

#Step 4: remove the secret from code

Rotating makes the leaked key useless, but remove it anyway so scanners stop alerting and nobody copies the pattern:

git log -S "aoai-prod" --all --oneline          # find commits containing it (search for a distinctive part)
pip install git-filter-repo
git filter-repo --replace-text <(echo 'sk-proj-REDACTED_FULL_KEY==>REMOVED')

Force-pushing rewritten history affects every clone; coordinate with the team. Forks and cached copies may still have the key, which is why rotation (step 1) is the real fix.

#Step 5: move to keyless authentication

Azure example:

from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import AzureOpenAI
client = AzureOpenAI(azure_endpoint=ENDPOINT, api_version=API_VERSION,
    azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"))

#Step 6: confirm

• Old key rejected (test returns 401).
• Token usage back to the normal profile; no calls from unknown IPs or regions.
• Application healthy with the new credential or keyless auth.

#Prevent it next time

• Secret scanning with push protection on all repositories.
• No API keys in code, notebooks or .env files committed to Git; pre-commit hooks (gitleaks, detect-secrets).
• Keyless auth wherever the platform supports it.
• Budget and anomaly alerts on AI spend, per resource, with a low threshold for out-of-hours spikes.
• Network restrictions (private endpoints, IP rules) so a leaked key alone is not enough.