Home / AWS

AWSChecklistAdvanced

S3 lifecycle rules not expiring or transitioning objects: a checklist

A lifecycle rule exists but the bucket keeps growing. Check filters, versioning, delete markers, minimum size and day rules, Object Lock and how to prove what lifecycle actually did.

Deepak Bhardwaj8 min read

Applies toAmazon S3 general purpose buckets, S3 Lifecycle configuration, AWS CLI v2

Symptom

You added a lifecycle rule weeks ago. Storage Lens or the bill says the bucket is the same size or bigger. Objects that should be in Glacier are still in Standard, or "expired" data is still there.

Lifecycle does work, but it is quiet: it runs asynchronously, does not report errors, and does exactly what the rule says rather than what you meant. Work through this checklist in order.

Quick map: symptom, cause, fix

Symptom Likely cause Fix
Objects disappear from listings but storage doesn't drop Versioned bucket: expiration only added delete markers Add NoncurrentVersionExpiration
Millions of delete markers, slow listings No rule removes expired object delete markers Add ExpiredObjectDeleteMarker
Rule never matches anything Prefix or tag filter doesn't match real keys Compare filter to actual keys and tags
Small objects never transition Default 128 KB minimum size for transitions Accept it, or change the default / add a size filter
Storage grows with no visible objects Incomplete multipart uploads Add AbortIncompleteMultipartUpload
Rule disappeared Another tool rewrote the whole lifecycle config Manage the full config in one place
Versions can't be removed Object Lock retention or legal hold Expected; wait for retention to end
Objects eligible but still there today Lifecycle is asynchronous Wait; billing changes when eligible

1. Read the configuration that is actually on the bucket

aws s3api get-bucket-lifecycle-configuration --bucket my-bucket

NoSuchLifecycleConfiguration means there is none. Then check:

  • Status is Enabled on every rule you expect to run.
  • The rule you remember is still there. put-bucket-lifecycle-configuration replaces the entire configuration. A Terraform module, a CloudFormation stack and a console edit fighting over one bucket is a common reason a rule vanishes.

Find who last changed it:

aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=PutBucketLifecycle \
  --max-results 10 \
  --query 'Events[].{Time:EventTime,User:Username,Resource:Resources[0].ResourceName}'

The CloudTrail event name is PutBucketLifecycle even though the API call is PutBucketLifecycleConfiguration.

2. Does the filter match real keys?

A rule with a filter only acts on objects that match all of its conditions.

  • Prefix: it's a literal string match from the start of the key, case-sensitive, no wildcards. logs/ does not match Logs/2026/... or /logs/....
  • Tags: the key and value must match exactly. Objects uploaded before a tagging change keep their old tags.
  • And: with several conditions, an object must match every one of them.
  • Size: ObjectSizeGreaterThan and ObjectSizeLessThan are in bytes.

Check a real object against the rule:

aws s3api list-objects-v2 --bucket my-bucket --prefix logs/ --max-items 5 \
  --query 'Contents[].{Key:Key,Size:Size,Modified:LastModified,Class:StorageClass}'

aws s3api get-object-tagging --bucket my-bucket --key logs/2026/10/01/app.log

head-object tells you whether S3 has already matched the object to an expiration rule:

aws s3api head-object --bucket my-bucket --key logs/2026/10/01/app.log \
  --query '{Expiration:Expiration,Class:StorageClass,Modified:LastModified}'

An Expiration value such as expiry-date="...", rule-id="expire-logs" confirms the match and gives the date. A null value means no expiration rule applies to that object.

3. Versioned buckets: current, noncurrent and delete markers

This is the most common reason storage doesn't drop.

aws s3api get-bucket-versioning --bucket my-bucket

If the status is Enabled or Suspended:

  • Expiration with Days acts on current versions only. It adds a delete marker; the data becomes a noncurrent version and is still billed.
  • NoncurrentVersionExpiration is what permanently deletes old versions. NewerNoncurrentVersions lets you keep the last N.
  • ExpiredObjectDeleteMarker: true removes delete markers that no longer have any versions behind them. It can't sit in the same Expiration element as Days or Date, so use a separate rule if needed.
  • AbortIncompleteMultipartUpload cleans up parts from failed uploads, which are billed but don't appear in normal listings.

A configuration that covers all four:

{
  "Rules": [
    {
      "ID": "logs-tier-and-expire",
      "Status": "Enabled",
      "Filter": { "Prefix": "logs/" },
      "Transitions": [ { "Days": 30, "StorageClass": "STANDARD_IA" },
                       { "Days": 90, "StorageClass": "GLACIER_IR" } ],
      "Expiration": { "Days": 365 },
      "NoncurrentVersionExpiration": { "NoncurrentDays": 30, "NewerNoncurrentVersions": 3 },
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    },
    {
      "ID": "remove-expired-delete-markers",
      "Status": "Enabled",
      "Filter": {},
      "Expiration": { "ExpiredObjectDeleteMarker": true }
    }
  ]
}
aws s3api put-bucket-lifecycle-configuration --bucket my-bucket \
  --lifecycle-configuration file://lifecycle.json

Caution

This call replaces every existing rule on the bucket. Export the current config first with get-bucket-lifecycle-configuration > lifecycle-backup.json, merge your change, and review the diff. A rule with an empty filter and a short Expiration applies to the whole bucket and can delete production data.

Measure what is noncurrent before and after:

aws s3api list-object-versions --bucket my-bucket --prefix logs/ \
  --query '{Noncurrent:length(Versions[?IsLatest==`false`]),DeleteMarkers:length(DeleteMarkers)}'

For large buckets, use S3 Inventory with versions included instead of listing.

4. Transition rules that S3 won't apply

  • Minimum object size: by default, lifecycle doesn't transition objects smaller than 128 KB. Check the bucket's setting:
aws s3api get-bucket-lifecycle-configuration --bucket my-bucket \
  --query 'TransitionDefaultMinimumObjectSize'

You can set --transition-default-minimum-object-size varies_by_storage_class on put-bucket-lifecycle-configuration, but transitioning tiny objects usually costs more in transition requests and per-object overhead than it saves. Check current pricing before changing it.

  • Minimum days: transitions from Standard to STANDARD_IA or ONEZONE_IA need the object to be at least 30 days old. The API rejects shorter values.
  • Direction: lifecycle only moves objects "down" the storage class order. It never moves Glacier objects back to Standard; that needs a restore and copy.
  • Day order: each later transition in a rule must have a larger Days value, and the gap rules between classes apply. If the put call fails, read the error; it names the conflict.
  • Minimum storage duration charges: expiring or transitioning an object early from IA or Glacier classes still bills the minimum duration. This is a cost surprise, not a failure.

5. Things that legitimately block deletion

  • Object Lock: versions under retention or legal hold can't be deleted by lifecycle. Expiration still adds delete markers on current versions.
aws s3api get-object-lock-configuration --bucket my-bucket
aws s3api get-object-retention --bucket my-bucket --key path/file --version-id <id>
  • Replication: lifecycle actions on the source bucket aren't replicated. The destination bucket needs its own rules.
  • MFA delete enabled on the bucket: lifecycle can't be configured. Check MFADelete in get-bucket-versioning.

6. Timing: eligible is not the same as done

  • Days are counted from object creation (or from when a version became noncurrent) and rounded up to the next midnight UTC.
  • Lifecycle evaluates and queues work asynchronously. On large buckets, actions can take days to finish after objects become eligible.
  • Billing for expiration and transition changes when the object becomes eligible, even if the action hasn't run yet. So the bill can drop before the object count does.
  • Storage Lens and the BucketSizeBytes CloudWatch metric update daily. Don't judge a rule on the same day you set it.

7. Prove what lifecycle did

S3 server access logs record lifecycle actions under the S3 requester with operations such as S3.EXPIRE.OBJECT, S3.CREATE.DELETEMARKER and S3.TRANSITION_SIA.OBJECT (other S3.TRANSITION_* values per storage class). If you query access logs in Athena:

SELECT operation, count(*) AS actions
FROM s3_access_logs_db.my_bucket_logs
WHERE operation LIKE 'S3.EXPIRE%'
   OR operation LIKE 'S3.TRANSITION%'
   OR operation = 'S3.CREATE.DELETEMARKER'
GROUP BY operation
ORDER BY actions DESC;

You can also subscribe to s3:LifecycleExpiration:* and s3:LifecycleTransition event notifications to count actions as they happen.

Prevent it next time

  • Own each bucket's lifecycle configuration in one tool and one place. Never mix console edits with infrastructure code on the same bucket.
  • For every versioned bucket, include noncurrent version expiration, expired delete marker clean-up and multipart abort in the baseline.
  • After adding a rule, check head-object on a sample key for the Expiration header, then check storage metrics a week later.
  • Use S3 Inventory or Storage Lens to track noncurrent bytes and incomplete multipart upload bytes per bucket, and review them monthly.