Home / Data Platform / Warehouses

Data PlatformTroubleshootingIntermediate

Sev1: Snowpipe stopped loading — files land in storage but tables stay empty

New files arrive in S3, Azure Blob or GCS but Snowflake tables stop updating. Check pipe status, event notifications, integration permissions and load errors, then backfill missed files.

Deepak Bhardwaj3 min read

Applies toSnowflake Snowpipe with auto-ingest (S3/SQS, Azure Event Grid, GCS Pub/Sub)

Symptoms

  • Dashboards fed by Snowpipe stop updating; files keep arriving in the stage location.
  • No errors in the pipeline that writes files, because the failure is on the load side.

1. Check the pipe status

SELECT SYSTEM$PIPE_STATUS('raw.events.pipe_events');

Key fields in the JSON:

Field What to look for
executionState RUNNING is good; PAUSED, STOPPED_* or similar is not
pendingFileCount Growing means loads are queued or stuck
lastReceivedMessageTimestamp Old timestamp = notifications not arriving
lastForwardedMessageTimestamp Messages received but not forwarded
error / lastPulledFromChannelTimestamp Integration or channel problems

If paused:

ALTER PIPE raw.events.pipe_events SET PIPE_EXECUTION_PAUSED = FALSE;

2. Check load history for errors

SELECT file_name, status, first_error_message, row_count, last_load_time
FROM TABLE(INFORMATION_SCHEMA.COPY_HISTORY(
  TABLE_NAME => 'RAW.EVENTS.EVENTS',
  START_TIME => DATEADD(hour, -24, CURRENT_TIMESTAMP())))
ORDER BY last_load_time DESC;
Error Cause Fix
File format or parse errors Producer changed format/columns Fix file format or COPY options
Access denied on the stage Storage integration role/permissions changed Step 4
No rows at all Notifications not arriving Step 3

3. Notifications not arriving

S3: the bucket event notification must send ObjectCreated events to the pipe's SQS queue (notification_channel in SHOW PIPES):

aws s3api get-bucket-notification-configuration --bucket my-landing-bucket

Someone adding another notification with Terraform can overwrite the existing configuration, which silently removes Snowpipe's.

Azure: Event Grid subscription to the storage queue and the notification integration must be enabled:

SHOW NOTIFICATION INTEGRATIONS;
DESC NOTIFICATION INTEGRATION ni_events;
az eventgrid event-subscription list --source-resource-id <storage-account-id> -o table

Check the Snowflake service principal still has the Storage Queue Data Contributor role.

Path filters: if producers changed the folder path, the pipe's COPY (stage path) or the notification prefix filter may no longer match.

4. Storage integration permissions

DESC STORAGE INTEGRATION si_landing;
LIST @raw.events.stage_events;

If LIST fails, the cloud role/consent behind the integration changed (AWS IAM role trust policy, Azure consent or role assignment).

5. Backfill missed files

After fixing, Snowpipe does not automatically pick up files whose notifications were lost. Refresh loads files staged in the last 7 days that haven't been loaded:

ALTER PIPE raw.events.pipe_events REFRESH;
-- or a narrower path
ALTER PIPE raw.events.pipe_events REFRESH PREFIX = '2026/10/31/';

For older files, use COPY INTO directly. Snowflake's load metadata prevents loading the same file twice for 64 days for COPY; for Snowpipe, check the documentation for its load history window.

Warning

Don't use FORCE = TRUE on COPY INTO during backfill unless you have deduplication downstream. It reloads files already loaded and creates duplicates.

Prevent it next time

  • Alert when lastReceivedMessageTimestamp or table row counts stop moving for N minutes.
  • Manage bucket notifications in one place in IaC so they aren't overwritten.
  • Validate file format changes with producers through data contracts.
  • Monitor COPY_HISTORY for failed files daily.