Saturday, 15 November 2025

Snowflake 05: What Is Snowflake Adaptive Compute?

 

Adaptive Compute is a new compute model in Snowflake (currently in private preview) that automates many of the resource-management decisions for your virtual warehouses.

Key Features / What It Does

  1. Automatic Sizing
    • Snowflake decides the cluster size, how many clusters to run, and when to scale up/down
    • You no longer need to manually pick “XS, S, M, …” warehouse sizes or configure min/max clusters.
  2. Smart Auto-Suspend / Resume
    • It picks optimal idle times for suspending and resuming warehouses to save credits.
    • Reduces unnecessary cost without hurting performance.
  3. Intelligent Query Routing
    • Queries are routed “behind the scenes” to the right-sized clusters
    • This means your workloads don’t need to know which warehouse size they’re hitting — Snowflake handles it.
  4. Shared Resource Pools
    • All “Adaptive Warehouses” in your account share a pool of compute.
    • This helps maximize utilization and reduces wasted compute.
  5. Better Price-Performance
    • Leverages next-gen hardware and performance improvements.
    • Because resources are shared and auto-optimized, you potentially save money while getting good performance.
  6. Seamless Migration
    • You can convert a standard warehouse to an “Adaptive Warehouse” with a simple ALTER command — without downtime.
    • Existing policies, permissions, names, and billing structures remain intact.
  7. FinOps Compatibility
    • Adaptive Compute works with Snowflake’s cost control tools (like budgets, resource monitors).
    • You can still monitor costs in ACCOUNT_USAGE, use budgeting, and even do chargebacks / showbacks.

Why It’s a Big Deal / Use-Case Benefits

  • Operational Simplicity: You don’t need to think about infrastructure sizing; Snowflake handles it — less DevOps work.
  • Cost Efficiency: Since compute is shared and dynamically allocated, you’re less likely to over-provision.
  • Better Performance: Queries get routed intelligently, minimizing queuing and using “just enough” resources.
  • Scalability: Ideal for mixed workloads (BI, analytics, ad-hoc, batch) — you don’t need separate warehouses for different jobs.
  • FinOps Friendly: Maintains visibility and financial controls — no black box.

Risks / Things to Watch Out For

  • Private Preview: Since it’s in private preview, behavior, performance, and pricing may change.
  • Less Control: Teams that like tuning warehouse size, cluster counts, or scaling policy in fine detail may feel limited.
  • Cost Spikes Risk: If many heavy queries come in, Snowflake may scale aggressively — potentially increasing cost. Keebo (an external cost-management tool) warns that without careful limits, you could pay more.  
  • Monitoring Changes: Traditional warehouse metrics (size, clusters) are abstracted away, so you need to rely on new or different observability tools.

 

No comments:

Post a Comment

Data Engineering - Client Interview question regarding data collection.

What is the source of data How the data will be extracted from the source What will the data format be? How often should data be collected? ...