Application Gateway WAF blocks legitimate requests with 403: find the rule and add a scoped exclusion

#Symptom

A form submit, API call or file upload that worked yesterday now returns 403 Forbidden. The response comes from Application Gateway, not from your app, and the backend logs show no request. It typically starts after one of these:

• A release that adds a new field, header or JSON payload (rich text, SQL-like search strings, base64 blobs, long cookies).
• The WAF policy moved from Detection to Prevention mode.
• The managed rule set was upgraded, for example CRS 3.1 to 3.2 or CRS to DRS 2.x.

If the backend logs do show the request, the 403 comes from the app, not the WAF. Stop here and debug the app.

#How blocking works (30 seconds you need)

CRS 3.2 and DRS 2.x use anomaly scoring. Every rule that matches adds to a score. The request isn't blocked by the rule that matched. It's blocked when the total crosses the threshold, and rule 949110 (inbound anomaly score exceeded) does the blocking.

So in the firewall log, one blocked request usually shows:

• One or more rows with action Matched. These are the rules you need to fix.
• One row with action Blocked for rule 949110. This only tells you the score was too high.

Group the rows by transaction ID to see every rule that fired for one request.

#Step 1: Find the blocked request in the firewall log

You need the ApplicationGatewayFirewallLog diagnostic category sent to Log Analytics. If it isn't enabled, turn it on and reproduce the problem. Logs that were never collected can't be searched.

If diagnostics write to the AzureDiagnostics table:

AzureDiagnostics
| where TimeGenerated > ago(2h)
| where Category == "ApplicationGatewayFirewallLog"
| where requestUri_s has "/api/orders"          // the failing path
| project TimeGenerated, transactionId_g, ruleId_s, action_s, Message,
          details_message_s, details_data_s, clientIp_s, hostname_s, requestUri_s
| order by TimeGenerated desc

If diagnostics write to the resource-specific AGWFirewallLogs table:

AGWFirewallLogs
| where TimeGenerated > ago(2h)
| where RequestUri has "/api/orders"
| project TimeGenerated, TransactionId, RuleId, Action, Message, DetailedMessage,
          DetailedData, ClientIp, Hostname, RequestUri, PolicyScopeName
| order by TimeGenerated desc

Note

Column names vary between schema versions and between the two tables. If a column doesn't resolve, run AGWFirewallLogs | getschema (or check the AzureDiagnostics columns) and adjust.

Then list every rule that fired for each blocked transaction:

AzureDiagnostics
| where TimeGenerated > ago(2h) and Category == "ApplicationGatewayFirewallLog"
| summarize Rules = make_set(ruleId_s), Actions = make_set(action_s),
            Data = make_set(details_data_s, 5) by transactionId_g, requestUri_s
| where Actions has "Blocked"
| take 50

The details_data_s / DetailedData value is what you're after. It shows which part of the request matched, for example Matched Data: ' OR found within ARGS:comment or a cookie or header name.

#Step 2: Read the rule ID

The first three digits of a CRS rule ID tell you the rule family:

To list the groups and rules your rule set version has (group names differ between CRS and DRS):

az network application-gateway waf-config list-rule-sets \
  --type OWASP --version 3.2 --group "*" -o table

#Step 3: Pick the fix

#Per-rule exclusion (preferred)

The example excludes the request argument comment from two SQLi rules. The rest of the request, and every other rule, still runs against comment.

RG=rg-prod-network
POLICY=wafpol-prod-web

az network application-gateway waf-policy managed-rule exclusion rule-set add \
  -g $RG --policy-name $POLICY \
  --match-variable RequestArgNames \
  --match-operator Equals \
  --selector comment \
  --type OWASP --version 3.2 \
  --group-name REQUEST-942-APPLICATION-ATTACK-SQLI \
  --rule-ids 942130 942440

Valid --match-variable values include RequestArgNames, RequestHeaderNames, RequestCookieNames and the ...Keys / ...Values variants. Operators are Equals, StartsWith, EndsWith, Contains and EqualsAny. For DRS, use --type Microsoft_DefaultRuleSet with your version (for example 2.1) and the DRS group name from the list-rule-sets output. Run --help on your CLI version to confirm.

Warning

--match-operator EqualsAny excludes every field of that type. With RequestHeaderNames that means every header is skipped by the listed rules. Use it only when you've checked that's what you want.

#Global exclusion (all rules)

Use this only for values the WAF can never usefully inspect, such as an opaque, signed session cookie:

az network application-gateway waf-policy managed-rule exclusion add \
  -g $RG --policy-name $POLICY \
  --match-variable RequestCookieNames \
  --selector-match-operator Equals \
  --selector "__Host-session"

#Disabling a rule

Check what's already overridden first. Updating a group with a rule list replaces the existing overrides for that group.

az network application-gateway waf-policy managed-rule rule-set list \
  -g $RG --policy-name $POLICY -o json

If a rule has to go, put it in a WAF policy that's associated with only that listener (per-site policy). That way the other sites behind the same gateway keep full protection.

#Step 4: Verify

1. Repeat the original request. It should return the backend's response.
2. Run the transaction query again. The excluded rule should no longer appear for that field, and no row should show 949110 Blocked.
3. Check that other paths still get blocked. Send a harmless test string like ?q=1' OR '1'='1 to a non-production listener and confirm the 403 still comes back.

Caution

Don't switch the policy to Detection mode to "unblock users" on a public site, except for a short, approved window. Detection mode logs attacks and lets them all through. If you must, set a reminder to switch back and record it in the incident: az network application-gateway waf-policy policy-setting update -g $RG --policy-name $POLICY --mode Prevention --state Enabled

#Prevent it next time

• Run new rule set versions and new apps in Detection first on a staging listener with production-like traffic. Review Matched rows for a week, then add exclusions before you switch to Prevention.
• Put WAF tests in the release pipeline. Replay real API payloads (rich text, search strings, uploads) against staging behind the WAF, and fail the build on any 403.
• Manage the WAF policy as code (Bicep or Terraform). Every exclusion then carries a comment with the ticket number, the field and the rule IDs, and gets reviewed like any other change.
• Alert on spikes in blocks per host. A sudden rise right after a deploy is nearly always a false positive. A slow rise from many client IPs is more likely a real attack:

AzureDiagnostics
| where Category == "ApplicationGatewayFirewallLog" and action_s == "Blocked"
| summarize Blocks = count() by bin(TimeGenerated, 5m), hostname_s

• Review exclusions every quarter. Remove the ones whose field or endpoint no longer exists.