Symptoms
- AG dashboard shows the new primary as healthy and synchronized.
- Applications fail with
Login timeout expired,A network-related or instance-specific error, or connect to the old primary and getThe target database is in an availability group and is currently accessible for connections when the application intent is set to read onlyor read-only errors. - Some app servers work, others do not.
Step 1: is the listener online? (2 minutes)
SELECT ag.name AS ag, l.dns_name, l.port, ip.ip_address, ip.state_desc
FROM sys.availability_group_listeners l
JOIN sys.availability_groups ag ON ag.group_id = l.group_id
JOIN sys.availability_group_listener_ip_addresses ip ON ip.listener_id = l.listener_id;
Get-ClusterResource | Where-Object ResourceType -in 'Network Name','IP Address','SQL Server Availability Group' |
Format-Table Name, State, OwnerNode, ResourceType -AutoSize
| Finding | Go to |
|---|---|
Network Name or IP resource Failed | Step 2 |
| All online, clients still fail | Step 3 (DNS / client) |
| Azure/AWS VMs, all online | Step 4 (load balancer) |
Step 2: listener resource failed
Check the cluster log for the reason:
Get-ClusterLog -Node SQL02 -TimeSpan 30 -Destination C:\Temp
Select-String -Path C:\Temp\*cluster.log -Pattern "Network Name|DNS|0x" | Select-Object -Last 40
Common causes:
- Cluster name object (CNO) lacks rights to update the DNS record or the listener computer object in AD (errors about access denied when registering DNS). Grant the CNO "Full control" on the listener's computer object and on the DNS record, or have DNS admins fix record ownership.
- IP address not valid on the new subnet (multi-subnet AG without an IP for that subnet). Add an IP for each subnet:
ALTER AVAILABILITY GROUP [AG1] MODIFY LISTENER N'AG1-LSN' (ADD IP ((N'10.20.2.50', N'255.255.255.0')));
Bring it online:
Start-ClusterResource -Name "AG1_AG1-LSN"
Step 3: DNS and client settings
In a multi-subnet AG the listener has one IP per subnet. Only the IP in the active subnet is online.
Resolve-DnsName AG1-LSN.corp.local
Get-ClusterResource "AG1_AG1-LSN" | Get-ClusterParameter RegisterAllProvidersIP, HostRecordTTL
| Setting | Effect |
|---|---|
RegisterAllProvidersIP = 1 | DNS returns all IPs; clients must use MultiSubnetFailover=True to try them in parallel |
RegisterAllProvidersIP = 0 | DNS returns only the online IP; clients wait for TTL (default 1200 s) after failover |
HostRecordTTL | How long clients cache the old IP |
Clients that lack MultiSubnetFailover=True with RegisterAllProvidersIP=1 try IPs one at a time and time out. Fix in the connection string:
Server=tcp:AG1-LSN,1433;Database=Sales;MultiSubnetFailover=True;ApplicationIntent=ReadWrite;Encrypt=True
If you cannot change all clients now (legacy apps), reduce stale caching:
Get-ClusterResource "AG1_AG1-LSN" | Set-ClusterParameter RegisterAllProvidersIP 0
Get-ClusterResource "AG1_AG1-LSN" | Set-ClusterParameter HostRecordTTL 300
Stop-ClusterResource "AG1_AG1-LSN"; Start-ClusterResource "AG1_AG1-LSN"
Warning
Restarting the listener resource drops all current connections. Do it in a short, announced window, and restart the AG resource dependency chain correctly (the AG resource depends on the listener).
On client machines, flush caches: ipconfig /flushdns. Application connection pools may hold dead connections; recycle app pools or restart services.
Step 4: cloud VMs with a load balancer listener
On Azure VMs in a single subnet (and some AWS designs), the listener IP is on an internal load balancer that sends traffic to whichever node passes the health probe.
Get-ClusterResource "AG1_IP" | Get-ClusterParameter ProbePort, Address, SubnetMask
Test-NetConnection SQL02 -Port 59999 # probe port from a peer
Check:
- Probe port set on the cluster IP resource and matches the load balancer probe.
- Windows Firewall on every node allows the probe port and 1433.
- Load balancer rule has floating IP (direct server return) enabled.
az network lb probe list -g rg-sql --lb-name ilb-sql -o table
az network lb rule list -g rg-sql --lb-name ilb-sql --query "[].{name:name, floatingIp:enableFloatingIp, port:frontendPort}" -o table
Azure VMs on Windows Server 2016+ can avoid the load balancer entirely with multi-subnet AGs or a distributed network name (DNN) listener on SQL Server 2019 CU8+.
Step 5: confirm
SELECT @@SERVERNAME AS connected_to, DATABASEPROPERTYEX('Sales','Updateability') AS mode;
Run from each app server through the listener. It should return the new primary and READ_WRITE.
Prevent it next time
- All connection strings: listener name,
MultiSubnetFailover=True; no node names. - Test a planned failover quarterly, including every application tier.
- Pre-stage CNO permissions and DNS record ownership; document probe ports in cloud setups.
- Alert on cluster resource state changes for the listener.