Patching EC2 fleets with SSM Patch Manager

#Prerequisites (where most failures come from)

An instance only appears in Systems Manager when all three of these are true:

1. The SSM Agent is installed and running (it's preinstalled on most AWS AMIs).
2. The instance profile includes the AmazonSSMManagedInstanceCore policy, or Default Host Management Configuration is turned on.
3. The instance can reach the ssm, ssmmessages and ec2messages endpoints, either through NAT/internet or through VPC interface endpoints.

aws ssm describe-instance-information \
  --query "InstanceInformationList[].{id:InstanceId,ping:PingStatus,agent:AgentVersion,os:PlatformName}" --output table

If an instance is missing or shows ConnectionLost, check the agent log on the instance: /var/log/amazon/ssm/amazon-ssm-agent.log on Linux, or %PROGRAMDATA%\Amazon\SSM\Logs on Windows.

#Step 1: Scan

Scanning installs nothing and shows you what's missing:

aws ssm send-command --document-name "AWS-RunPatchBaseline" \
  --targets "Key=tag:PatchGroup,Values=prod-web" \
  --parameters "Operation=Scan" \
  --max-concurrency "20%" --max-errors "10%" \
  --comment "Patch scan $(date +%F)"

aws ssm describe-instance-patch-states --instance-ids i-0abc123 \
  --query "InstancePatchStates[].{missing:MissingCount,failed:FailedCount,installed:InstalledCount,pendingReboot:InstalledPendingRebootCount}" \
  --output table

aws ssm describe-instance-patches --instance-id i-0abc123 \
  --filters "Key=State,Values=Missing" --query "Patches[].{kb:KBId,title:Title,severity:Severity}" --output table

#Step 2: Install in waves

aws ssm send-command --document-name "AWS-RunPatchBaseline" \
  --targets "Key=tag:PatchGroup,Values=prod-web" \
  --parameters "Operation=Install,RebootOption=RebootIfNeeded" \
  --max-concurrency "1" --max-errors "0" \
  --cloud-watch-output-config "CloudWatchOutputEnabled=true"

• Set max-concurrency to 1 or a small percentage for fleets behind a load balancer, so capacity is never all down at once.
• Set max-errors 0 so the rollout stops at the first failure.
• Use RebootOption=NoReboot only if you've scheduled a reboot separately. Otherwise patches stay pending and the instance reports non-compliant.

Tip

For recurring patching, use a Patch Policy in Quick Setup or a Maintenance Window instead of ad-hoc send-command. Both handle scheduling, waves and logging for you.

#Step 3: Report compliance

aws ssm list-compliance-summaries --filters "Key=ComplianceType,Values=Patch,Type=EQUAL" --output table
aws ssm list-resource-compliance-summaries \
  --filters "Key=ComplianceType,Values=Patch,Type=EQUAL" "Key=OverallSeverity,Values=CRITICAL,Type=EQUAL" \
  --query "ResourceComplianceSummaryItems[?Status=='NON_COMPLIANT'].ResourceId" --output text

#Patch baselines

• The AWS default baselines approve Critical/Important security patches 7 days after release. Use a custom baseline if you need a different delay or extra classifications.
• Set ApproveAfterDays to something like 7 for production and 0 for dev, so dev catches problems first.
• Register a baseline to a patch group: aws ssm register-patch-baseline-for-patch-group --baseline-id pb-xxx --patch-group prod-web.

#When an install fails

1. Get the command output: aws ssm list-command-invocations --command-id <id> --details.
2. On Linux, the usual causes are a repository that can't be reached, a full /var or /boot, or a lock held by another package manager process.
3. On Windows, check that the Windows Update service is enabled and look in C:\ProgramData\Amazon\PatchBaselineOperations\Logs.