The Snowflake Savings Machine: Why Keebo is different from other optimization products

Many companies promise to lower your Snowflake bill through optimization. But what are they really offering to do for you? The answer is…not enough.

Introduction

As I walked around the show floor at the recent Snowflake Summit in Las Vegas, I noticed a plethora of companies promising to save you money on Snowflake. Some said they would create savings through “continuous optimization.” Others said they were “automated.” One went so far as to claim they will save 40% “with the click of a button.”

I’m not sure those words mean what they think they mean. In this article I’d like to explain how Keebo is different from the sudden surplus of optimization products, and the critical capability you should make sure to get.

Observe and Report

There are at least six products out there now that I classify as “observe and report” for Snowflake optimization. Some are backed by famous names. Others appear to be passion projects from Snowflake enthusiasts. Still others are “products” from consulting companies that are likely looking for longer-term, billable engagements. Regardless, these products all do essentially the same thing: provide dashboards and reports on Snowflake performance and utilization. Take a glance at any of their websites and you’ll find they all begin to sound alike. In this illustration I’ve assembled a bunch of screen captures of their key messages:

They might describe it with terms like insights, but the reality is that they are simply observing your Snowflake usage and then providing reports about it back to you. Sure, some of those reports are beautiful. But you need more than a nice-looking report. “This warehouse over here is really expensive…someone oughta do something about that!” This doesn’t strike me as helpful. Even if they can provide a specific recommendation, who is going to implement it? A consultant? One of your own data engineers? Will that person be available if things change or go wrong?

Some of these products can assist you in scheduling an optimization, such as reducing the size of a warehouse at specific times of the week and then increasing it later–basically a glorified chron job. Even still, there are problems with the observe and report method because it is not dynamic:

• Can’t respond to problems. A recommendation implemented today might not work, or even cause harm, if there is a spike in the workload tomorrow. Observe and report products need human intervention and so cannot adjust automatically.
• Missing out on savings. Conversely, there may arise opportunities to save even more money when workloads change, but an observe and report product cannot adjust to this either.
• Doesn’t help overworked data teams. While pointing someone in the right direction is a nice thing to do, they still have to go do it. If the actions later need to be undone or changed, they have to do it again–but hopefully it isn’t at some strange hour when nobody is available. I’d rather have my valuable team of data engineers doing more valuable work, like building data pipelines and helping users with analytics.

Observe and Act

If you want Snowflake to run as optimally as possible, and thus as inexpensively as possible, you need something to take action and do real-time optimization for you. That’s Keebo. Yes, we can give you nice-looking charts and graphs, too. But we go further and actually perform the optimizations. We are a true savings machine.

For warehouse optimizations, we automatically adjust auto-suspend, warehouse size, and multi-cluster parameters. It takes 30 minutes to set up and requires no changes to your data or applications. We give you a friendly settings screen to help guide our machine learning technology to best meet your goals. We monitor 67 Snowflake usage stats (we don’t access user data) in real-time to do this, making the right optimization at the right time. This adds up to thousands of optimizations every month for each of our customers. This screenshot shows the optimizations we are doing in real time of just the auto-suspend settings for one week of a customer’s warehouse:

For query acceleration, we don’t merely report on the biggest and baddest queries. You probably know what those are anyway. We solve the performance problems of thousands of queries in real time by rewriting SQL with correctness guarantees, and building smart models (similar to materialized views) to make queries run in a second or two. For this, our machine learning robots run within budget and data freshness parameters set by you. This screenshot shows customer queries being dynamically re-written if it will help them run faster within the budget allocated, and bypassing those that don’t need rewriting. This is a big step beyond a performance insight to performance improvement:

For more on our technology, see our Warehouse Optimization evaluation guide and Query Acceleration overview and demo.

Aligned Goals

We are so serious about prioritizing action over reporting, we even price Warehouse Optimization based on your savings. We charge you a percentage of what we save you. This way you can be assured that our goals and your goals are aligned. Simply put, we have to get actual results for you or we won’t be able to keep you as a customer. This pricing model is unique to Keebo.

What About Slingshot?

One “observe and report” product in particular had a huge presence at the Snowflake Summit: Capital One Slingshot. They were everywhere you looked. Despite Slingshot’s prominence and household name sponsor, Keebo has actually been leading this space three years longer than Slingshot has been available. Perhaps you are thinking of using Slingshot for approval workflows and now wonder if it can handle optimization too. My concern would be that Slingshot cannot respond in real time, is not automatic, and thus doesn’t do enough to save you money and time. Keebo is the true Snowflake savings machine, optimizing warehouses and accelerating queries in real time without intervention needed from your data team. Slingshot simply cannot do that. You can take the burden off your data team with Keebo instead of merely alerting them to issues and providing suggestions.

Side-by-Side Comparison

Finally, here is a summary of how Keebo is different from the observe and report products like Slingshot:

	Keebo: Observe and Act	Others: Observe and Report
Real-time optimizations	Yes. Our robots will happily optimize 24/7/365.	No. Only periodic recommendations that constantly need to be verified by a human expert before deploying.
Real-time performance protection	Yes. Our robots will back-off in real time if we detect issues.	No. Slowdowns can happen with dynamic workloads and need human intervention at unpredictable hours.
Query acceleration	Automates query rewriting with correctness guarantees, handles arbitrary complex queries, builds smart models.	High-level suggestions only (no actual rewrites), support for simple queries only.
User control	Can provide guidelines for real-time AI optimizations or choose a “set and forget” approach.	Route all optimization recommendations through human expert for vetting and approval.
Warehouse Optimization pricing	Percentage of actual savings for aligned incentives. The more savings, the better for both parties.	Percentage of Snowflake spend or various subscription fees which lead to misaligned incentives. More savings mean the vendor makes less.
Monitoring and visibility	Performance, usage, cost.	Performance, usage, cost.
Approval workflows	N/A	Yes (Slingshot)