Locked out of an S3 bucket by its own bucket policy: recover access when even admins get AccessDenied

#Symptoms

• aws s3 ls s3://my-bucket returns AccessDenied for every user and role, including ones with AdministratorAccess.
• aws s3api get-bucket-policy and put-bucket-policy also return AccessDenied.
• The problem started right after someone applied a bucket policy with a Deny statement, typically:
  • aws:SourceVpce / aws:SourceVpc ("only allow access from our VPC endpoint")
  • aws:SourceIp ("only from the office")
  • aws:PrincipalArn with NotPrincipal or StringNotEquals ("only this role")

An explicit Deny in a bucket policy beats any Allow in IAM. If the Deny also covers s3:GetBucketPolicy, s3:PutBucketPolicy and s3:DeleteBucketPolicy, nobody can fix it through normal IAM roles.

#Step 1: Confirm it's the bucket policy

From a principal that should have access:

aws sts get-caller-identity
aws s3api get-bucket-policy --bucket my-bucket

The error message usually says "with an explicit deny in a resource-based policy". If it says "in a service control policy", the problem is an SCP, not the bucket (fix that in the management account).

Check CloudTrail for who changed the policy and what it was:

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=PutBucketPolicy \
  --max-results 5 --query 'Events[].CloudTrailEvent' --output text | head -c 4000

The requestParameters.bucketPolicy field contains the policy that was applied. Save it; you'll want to fix it, not just delete it.

#Step 2: Recover with the account root user

The root user of the account that owns the bucket can always delete a bucket policy, even when the policy denies everyone. (An SCP can still block root in a member account; see Step 4.)

1. Sign in to the console as the root user (email + password + MFA) of the bucket's account.
2. S3 > the bucket > Permissions > Bucket policy > Delete.

Or, if you have root credentials configured in a temporary CLI profile (not recommended long-term):

aws s3api delete-bucket-policy --bucket my-bucket --profile root-breakglass

Caution

Deleting the bucket policy removes all statements, including ones that gave other accounts or services (CloudFront OAC, log delivery, replication) access. Have the corrected policy ready to apply straight after, and expect a short gap for those integrations.

Remove any root access keys you created for this immediately afterwards.

#Step 3: Rewrite the policy so it can't lock you out again

The pattern that bites people:

{
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:*",
  "Resource": ["arn:aws:s3:::my-bucket", "arn:aws:s3:::my-bucket/*"],
  "Condition": { "StringNotEquals": { "aws:SourceVpce": "vpce-0abc123" } }
}

That denies everything from outside the endpoint, including the console and your admin's CLI from a laptop. Safer version: scope the Deny to data actions and exempt a break-glass role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyDataAccessOutsideVpce",
      "Effect": "Deny",
      "Principal": "*",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket"],
      "Resource": ["arn:aws:s3:::my-bucket", "arn:aws:s3:::my-bucket/*"],
      "Condition": {
        "StringNotEquals": { "aws:SourceVpce": "vpce-0abc123" },
        "ArnNotLike": { "aws:PrincipalArn": "arn:aws:iam::111122223333:role/S3BreakGlass" }
      }
    },
    {
      "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": ["arn:aws:s3:::my-bucket", "arn:aws:s3:::my-bucket/*"],
      "Condition": { "Bool": { "aws:SecureTransport": "false" } }
    }
  ]
}

Why this is safer:

• Bucket configuration actions (GetBucketPolicy, PutBucketPolicy, DeleteBucketPolicy) aren't in the Deny, so IAM admins can still repair the policy.
• A named break-glass role is exempt, so you don't need root next time.
• Multiple condition operators in one statement are ANDed: the Deny applies only when the request is not from the endpoint and not from the break-glass role.

Apply it from inside the VPC or as the break-glass role:

aws s3api put-bucket-policy --bucket my-bucket --policy file://bucket-policy.json

#Condition key gotchas

#Step 4: When even root is blocked

• SCP: in a member account, an SCP that denies s3:DeleteBucketPolicy blocks root too. Detach or change it from the management account.
• Root user unavailable: no password, lost MFA. Use the account recovery process with AWS Support, or, in Organizations with centralised root access enabled, perform a privileged root task from the management account (check AWS Organizations documentation for "centralized root access" and the sts:AssumeRoot task policies).

#Prevent it next time

• Validate bucket policies before applying: aws accessanalyzer validate-policy --policy-type RESOURCE_POLICY --policy-document file://bucket-policy.json, and use IAM Access Analyzer's policy checks in CI.
• Always exempt a break-glass role and never deny the bucket-policy management actions.
• Apply S3 policy changes through IaC with a review step; block console PutBucketPolicy on production buckets with an SCP that exempts the pipeline role.
• Alert on PutBucketPolicy and DeleteBucketPolicy via EventBridge so a bad change is noticed in minutes.