Keebo | 8 Snowflake Metrics to Help You Optimize Your Spend Now

8 Snowflake Metrics to Help You Optimize Your Spend Now

Snowflake monitoring is critical if you want to stay on top of cloud costs. The faster you can retrieve relevant Snowflake metrics, the faster you can adjust and optimize your performance. 

While there’s a range of tools for Snowflake monitoring, each has its own price tag. Rather than spend money to save money, here are eight metrics to monitor for free with Snowflake Workload Intelligence from Keebo.

How to monitor Snowflake performance at no additional cost

The status quo for Snowflake cost optimization is: spend money to save money. Yes, Snowflake provides their own optimization tools, but they’re limited in what they can do. Beyond basic diagnostics, you have to pay. 

Alternatively, you could build your own in-house solution. But those are expensive to build. You’re back to—you guessed it—spending money to save money. 

Plus, even if you spend on cost optimization advice, that’s all it is: advice. Unless you act on it quickly, your costs won’t go down. And once you optimize for today, time marches on, and you’ll need to re-optimize based on shifts in query volume, complexity, user engagement, etc. 

This status quo is broken. Fortunately, we’re here to fix it. 

Snowflake Workload Intelligence is a free app you can download from the Snowflake Marketplace that surfaces insights from your data to identify top opportunities for cost reduction. It runs natively in your Snowflake instances: no plugins, setup, or back-and-forth data exchange. 

8 metrics to monitor with Snowflake Workload Intelligence

Once you install Snowflake Workload Intelligence, you can start monitoring and analyzing a range of metrics: eight of the most critical are listed here. Because Snowflake pricing uses a complex structure, no one metric can tell you exactly whether your costs are going up.

For example, let’s say you’re running queries on an inexpensive, extra-small warehouse. You may have a query that executes for ten hours straight. Sure, that’s a lot of warehouse time. Upgrading to a larger warehouse will certainly reduce querying time, but one hour on a 5XL warehouse is more expensive than ten on an XS. While there may be plenty of other reasons to upgrade—user expectations around querying times—looking at query run time alone doesn’t give you the full picture. 

So you need to look at Snowflake usage from different angles. Here are eight our customers have found to be the most important. 

1. Credit usage by warehouse

The best place to start is with credit usage, preferably on a daily basis. First, look at the aggregate. Then, break it down by warehouse. With either view, you can quickly establish your daily baseline, then easily identify abnormal spikes. 

2. Query execution time

Query execution time helps indicate where you’re struggling to provision resources appropriately. This typically happens to longer and more complex (and, thus, more expensive) queries. This is a quick way to identify resource drains and take steps to optimize query writing, provision more resources, or both. 

3. Queuing execution time

If you don’t have enough compute resources to execute all your queries, Snowflake will form a queue and execute them when resources come available. Snowflake Workload Intelligence enables you to analyze queuing time to determine where potential bottlenecks exist. 

4. Query success/failure rate

If you’re looking to reduce Snowflake costs, a place to find low-hanging fruit is your query success/failure rates. Too many failed queries means you’re wasting valuable compute on activity that drives no actual outcome. As with any of our other query-based analyses, you can break this down by query types, templates, databases queried, schemas, and more.

5. Query types

Different types of queries run faster or slower depending on how they’re worded or structured. If certain query types are consuming more resources, you can look at those specific SQL commands and find a way to arrive at the same information more efficiently. 

6. Spillage

Spillage occurs when data moves from one warehouse to another due to under-provisioned resources, and Snowflake starts saving the data remotely. If you see queries with high levels of spillage, they’re probably running on the wrong warehouses. This means you need more resources—i.e. bigger warehouses.

7. Heavy hitting queries

Snowflake Workload Intelligence quickly identifies queries that are consuming the most resources. These “heavy hitters” include what we call query templates. This is where the tool sees that only one value (e.g. the date) is different, and analyzes the recurring series as a single query template. This helps to keep your data from being clogged up by noise. 

8. Heavy hitting users

Finally, you can use Snowflake Workload Intelligence to identify and analyze the users who cost the most money to operate. These are typically running the most queries and using the most total data. This is critical information that enables you to make necessary organizational changes to keep costs under control. 

Start from the top, then drill down

Snowflake monitoring—whether you’re using Snowflake Workload Intelligence or another tool—happens in two ways. The first is a more proactive approach, where you log on to see how things are running, and notice something out of the ordinary. No one’s complaining yet, so you have enough lead time to resolve the issue before it balloons into a bigger problem.

Then, of course, there’s the more reactive approach, where you get a complaint—either in terms of unexpected costs or lags in performance—and you have to diagnose the issue. 

In either case, these eight metrics are critical to not only finding the issue, but deciding on the best course of action moving forward. The visibility Snowflake Workload Intelligence provides is an essential part of optimizing your Snowflake warehouses, whether you want to do it yourself, or alongside our fully-automated Snowflake Cost Optimization solution.

Add Snowflake Workload Intelligence to your Snowflake instance and start monitoring your cloud performance for free today. 

Author

Keebo | 8 Snowflake Metrics to Help You Optimize Your Spend Now
Skye Callan
Articles: 8