Sev1: private endpoints suddenly resolve to public IPs — private DNS zone link or record removed

#Symptoms

• Many services fail together: SQL Cannot open server, Storage 403 This request is not authorized, Key Vault Public network access is disabled.
• nslookup <account>.blob.core.windows.net from inside Azure now returns a public IP.
• The change log shows a DNS zone, VNet link or a resource group cleanup in the last few hours.

#1. Confirm it is DNS

From an affected VM or pod:

nslookup stproddata.blob.core.windows.net
nslookup sql-prod.database.windows.net

#2. Check the zone and its links

az network private-dns zone list -o table
az network private-dns link vnet list -g rg-dns -z privatelink.blob.core.windows.net -o table
az network private-dns record-set a list -g rg-dns -z privatelink.blob.core.windows.net -o table

Restore a missing link:

az network private-dns link vnet create -g rg-dns -z privatelink.blob.core.windows.net \
  -n link-hub --virtual-network /subscriptions/<sub>/resourceGroups/rg-net/providers/Microsoft.Network/virtualNetworks/vnet-hub \
  --registration-enabled false

#3. Restore missing A records

Get the private IP from the endpoint's NIC and re-create the record, or re-attach the DNS zone group so Azure manages it:

az network private-endpoint show -g rg-data -n pe-stproddata-blob --query "customDnsConfigs" -o json

az network private-endpoint dns-zone-group create -g rg-data --endpoint-name pe-stproddata-blob \
  -n default --private-dns-zone privatelink.blob.core.windows.net --zone-name blob

#4. Custom DNS servers and hybrid

If VNets use custom DNS (domain controllers, Infoblox) the zone must be reachable from that DNS:

• Conditional forwarders for blob.core.windows.net, database.windows.net and so on must point to an Azure DNS Private Resolver inbound endpoint or a forwarder in a linked VNet.
• The forwarder's VNet must be linked to every privatelink.* zone.
• Flush caches after the fix: ipconfig /flushdns, Clear-DnsServerCache on Windows DNS, resolvectl flush-caches on Linux.

#5. Who changed it

az monitor activity-log list -g rg-dns --offset 24h \
  --query "[?contains(operationName.value,'privateDnsZones')].{when:eventTimestamp, op:operationName.value, who:caller, status:status.value}" -o table

#Prevent it next time

• Keep all privatelink.* zones in one platform-owned resource group with a CanNotDelete lock.
• Use Azure Policy (DeployIfNotExists) to create DNS zone groups for new private endpoints automatically.
• Add a synthetic check that resolves a few critical FQDNs from inside the VNet and alerts on a public answer.