Sev1: a dbt --full-refresh wiped history from an incremental model

#How it happens

An incremental model accumulates data from a source that keeps only, say, 30 days (an API extract, a staging table with retention). dbt run --full-refresh drops and rebuilds the model from the source — so everything older than 30 days is gone.

#1. Stop further runs

Pause the schedule (dbt Cloud job, Airflow DAG) so the next run doesn't overwrite again, and tell consumers.

#2. Find when and how

# dbt Cloud: job run history shows the command; dbt Core: check run logs / orchestrator logs
grep -R "full-refresh" logs/ | tail

Warehouse-side query history gives the exact time of the CREATE OR REPLACE.

#3. Restore from the warehouse

Snowflake (Time Travel within retention):

CREATE TABLE analytics.fct_events_restore CLONE analytics.fct_events
  BEFORE (STATEMENT => '<query_id_of_create_or_replace>');

CREATE OR REPLACE drops the old table, so you may need UNDROP instead: rename the new table out of the way, then UNDROP TABLE analytics.fct_events;

Databricks / Delta:

DESCRIBE HISTORY analytics.fct_events LIMIT 10;
RESTORE TABLE analytics.fct_events TO VERSION AS OF 2210;

If dbt used CREATE OR REPLACE TABLE, Delta keeps history across the replace, so RESTORE works within retention.

BigQuery:

CREATE TABLE analytics.fct_events_restore AS
SELECT * FROM analytics.fct_events FOR SYSTEM_TIME AS OF TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 6 HOUR);

For tables that were dropped and recreated, check BigQuery's table snapshot/undelete options within the time-travel window.

#4. Re-apply recent data

The restored table is missing whatever the normal incremental runs would have added since the restore point. Run the model normally (not full refresh) — the incremental logic fills the gap from the source:

dbt run --select fct_events

Then rebuild downstream models: dbt run --select fct_events+.

#5. Protect the model

{{ config(
    materialized='incremental',
    unique_key='event_id',
    incremental_strategy='merge',
    full_refresh=false
) }}

With full_refresh=false, dbt ignores --full-refresh for this model. Rebuilds then need a deliberate config change in a pull request.

Tip

Mark models whose history can't be rebuilt from sources with a tag (tags=['irreplaceable']) and back them up or snapshot them regularly.

#Prevent it next time

• full_refresh=false on every model whose source doesn't keep full history.
• Production jobs in dbt Cloud/orchestrator only; no personal dbt run --target prod.
• Longer time-travel retention on irreplaceable tables.
• A row-count drop test (dbt_utils or custom test) that fails the run if a table loses more than X% of rows.