Delta table features broke external readers: deletion vectors, column mapping and protocol versions that Synapse, Power BI or older Spark can't read

#Symptoms

• A report or external query that read the table yesterday fails today with messages such as:
  • Delta protocol version (3,7) is too new for this version of Delta
  • Unsupported reader features: deletionVectors
  • Table features configured ... are not supported / column mapping errors
• Databricks itself reads the table fine.
• Nobody changed the table deliberately, but a runtime upgrade, a MERGE, or a new workspace default happened.

#Why it happens

Delta tables declare the minimum reader and writer protocol versions and, in newer versions, a list of table features. A reader that doesn't support a feature must refuse the table, or it would return wrong results.

Features that commonly raise the protocol:

Recent Databricks runtimes enable some of these by default for new tables (for example deletion vectors), and a workspace setting can auto-enable them. One ALTER TABLE or OPTIMIZE on a new runtime may upgrade an existing table.

#Step 1: see the protocol and features

DESCRIBE DETAIL main.sales.orders;
-- minReaderVersion, minWriterVersion, tableFeatures

SHOW TBLPROPERTIES main.sales.orders;
-- delta.enableDeletionVectors, delta.columnMapping.mode, delta.feature.* entries

DESCRIBE HISTORY main.sales.orders;
-- find the commit that upgraded the protocol (operation, operationParameters, user, timestamp)

#Step 2: decide: downgrade the table or change the reader?

Check the current feature support of each reader in its documentation; support changes often.

#Step 3: remove a feature safely

Databricks supports dropping some table features. Example for deletion vectors:

-- 1. Stop new deletion vectors being written
ALTER TABLE main.sales.orders SET TBLPROPERTIES ('delta.enableDeletionVectors' = false);

-- 2. Rewrite files to apply existing deletion vectors and drop the feature
ALTER TABLE main.sales.orders DROP FEATURE deletionVectors;

Dropping a reader feature is a two-step process: the first DROP FEATURE removes its use from the current version, but older versions in the history still use it. After the retention period, you complete it by truncating history:

ALTER TABLE main.sales.orders DROP FEATURE deletionVectors TRUNCATE HISTORY;

Requirements and wait times depend on the runtime; read the "Drop a Delta table feature" page before running this in production.

Warning

TRUNCATE HISTORY removes time travel to versions before the drop. Make sure no process relies on older versions (audits, RESTORE plans, CDF consumers).

Column mapping is harder to remove: it requires rewriting the table. Often a better option is a compatible copy:

CREATE OR REPLACE TABLE main.export.orders_compat
TBLPROPERTIES ('delta.enableDeletionVectors' = false)
AS SELECT * FROM main.sales.orders;

#Step 4: stop it happening again

Disable automatic enablement where external readers matter:

• Workspace setting: check the admin setting that auto-enables deletion vectors for new tables, and turn it off for workspaces that produce tables for external engines.
• Session or cluster default for jobs that create tables:

spark.databricks.delta.properties.defaults.enableDeletionVectors false

• Table properties at creation in your DDL templates:

CREATE TABLE main.export.customers (...)
TBLPROPERTIES ('delta.enableDeletionVectors' = false);

#Step 5: better ways to share with other engines

#Troubleshooting table

#Prevent it next time

• Keep a list of external readers per table, with the Delta features each supports.
• Pin table properties in DDL for tables consumed outside Databricks.
• Test runtime upgrades against external readers, not just Databricks jobs.
• Prefer Delta Sharing or UniForm over direct file access for external consumers.