Home / Data Platform / Warehouses
Cloud data warehouses: Snowflake, Synapse and serverless SQL sizing, cost and recovery. 19 posts.
An UPDATE without the right WHERE clause or a DELETE wiped production data. Find the query ID, clone the table as it was just before that statement, compare, and restore only the damaged rows (or swap the whole table) before the Time Travel window closes.
After activating or editing an account-level network policy, logins fail with "IP is not allowed to access Snowflake" for users, BI tools and pipelines. Find a path back in, unset or fix the policy, then roll network rules out safely with user-level testing.
Find out why a Snowflake dynamic table is stale - auto-suspension, upstream failures, lost change tracking, skipped refreshes or a FULL refresh mode - and fix it safely.
Read the Snowflake Query Profile to find where time goes, measure partition pruning, decide when a clustering key is worth its cost, and use search optimization for selective lookups instead of over-sizing warehouses.
A scheduled Snowflake task stopped loading data. Read SHOW TASKS and TASK_HISTORY, find the suspended, skipped or failing run, fix privileges or a stale stream, and restart the graph safely.
Files land in the bucket but never reach the table. Read SYSTEM$PIPE_STATUS and COPY_HISTORY, fix the pipe, notification or file error, then reload the gap without duplicates.
Serverless SQL engines bill by data scanned, so file layout decides cost. Find the most expensive queries, convert to partitioned Parquet, make queries prune partitions and columns, materialise repeated results with CTAS/CETAS and put per-query and per-day limits in place.
Snowflake storage includes much more than active data. Time Travel, Fail-safe, data retained for clones and churn from frequent rewrites can multiply the bill. Measure each component per table, set retention per data class, use transient tables for rebuildable data, review automatic clustering spend and clean up stages.
Warehouses are where most Snowflake credits go. Separate workloads into purpose-built warehouses, size them by query behaviour rather than habit, set auto-suspend tight, use multi-cluster for concurrency instead of bigger sizes, try query acceleration for outlier scans, and guard everything with resource monitors.
Dedicated SQL pools bill per DWU-hour whether busy or idle. Pause and scale on a schedule, isolate loads from reporting with workload groups and classifiers, enable result set caching and materialized views for repeated queries, and keep columnstore and statistics healthy so a smaller DWU does the job.
Recover Snowflake data after a DROP, TRUNCATE, bad UPDATE or a pipeline that overwrote a table. Use UNDROP, Time Travel AT/BEFORE queries and zero-copy clones, then swap safely and understand Fail-safe limits.
Clear a Synapse dedicated SQL pool where queries pile up in the queue. Find what holds concurrency slots, kill or isolate runaway loads, use workload groups and classifiers to protect BI, and scale if you must.
A practical migration plan from an appliance data warehouse to Snowflake. Inventory and usage analysis, schema and SQL conversion, historical data transfer, rebuilding loads, testing with reconciliation, cutover of BI tools and decommissioning.
Split a Snowflake credit spike by service, warehouse, query and serverless feature with ACCOUNT_USAGE views, stop the burn, and add guardrails so it doesn't repeat.
Work through the active role, the USAGE chain on database, schema and warehouse, future grants and ownership when Snowflake users can't see objects.
The first-week setup for a production Snowflake account, covering role hierarchy, warehouse sizing and timeouts, SSO and MFA, network policies, resource monitors, Time Travel and audit.
Replicate Snowflake databases and account objects to another region or cloud with failover groups, refresh them on a schedule, redirect clients with connection URLs, and run a failover drill.
Use query history and the query profile to tell warehouse queuing from spilling and poor partition pruning, then fix it with multi-cluster scaling, sizing or clustering.
Find skewed tables and expensive shuffle and broadcast steps in a Synapse dedicated SQL pool, and fix them with the right distribution, statistics and columnstore health.
No posts in this category yet.