Keebo: Revolutionizing Warehouse Optimization with a Technical Lens
Data teams face the challenge of delivering fast insights while keeping costs under control. The traditional approach of selecting a fixed set of configuration parameters for Snowflake compute resources, such as warehouses, often necessitates a trade-off between performance and cost savings.
But at Keebo, we’ve developed a solution that aims to balance both. Our six-layer Snowflake cost optimization framework is designed to help organizations make the most of their data infrastructure without the usual trade-offs. Let’s explore how we can support your team in overcoming these challenges.
How to address modern data warehouse challenges—with no compromises
Snowflake data warehouses are essential for analytics, but they’re notoriously tricky to manage. Need fast query speeds for real-time decisions? That often means scaling up and watching costs soar. Want to keep expenses in check? You risk sluggish performance and a frustrating user experience.
At Keebo, we’ve seen this struggle firsthand, so we’ve built a solution that refuses to settle for trade-offs. Our solution dynamically adapts to your workloads, delivering top-tier performance and cost efficiency.
Our six-layer framework isn’t just a toolset, where you pick and choose which layer you need and ignore the ones you don’t. It’s a holistic strategy designed to tackle inefficiency at its core. Each layer builds on the last, delivering a tailored solution that fits your business perfectly.
Layer 1: Configuration Schedules – Aligning Resources with Needs
At Keebo, our Configuration Schedules feature is designed to seamlessly align your data warehouse resources with predictable workloads, such as weekly ETL jobs. For example, let’s say you’re running a resource-intensive ETL job every Sunday night, consolidating large datasets like sales figures, customer interactions, or inventory updates from the past week.
With Keebo, you can not only automate the scaling process for this job, but also configure specific optimization parameters for each scheduled interval. You can set a larger warehouse size, adjust the minimum and maximum number of clusters for efficient scaling based on workload demands, and even enable or disable specific optimization algorithms to tailor Keebo’s behavior precisely to your needs. This ensures the ETL job runs smoothly with the heavy data load. Once the job completes, Keebo automatically reverts to your default settings to conserve resources.
For other recurring tasks like daily reporting, you can define different parameters—such as a smaller warehouse size with minimal clusters—to prioritize cost efficiency. Across all schedules, you maintain precise control over which algorithms to enable or disable, ensuring Keebo’s optimization mechanisms align perfectly with your goals. This blend of automation and detailed customization helps your warehouse perform at its best for every workload.
Layer 2: Sliders – Empowering You to Define Your Goals
At Keebo, we believe optimization should reflect your priorities, and our Sliders feature is the way for end users to inform the system of their goals. These sliders give you fine-grained control over your optimization priorities, whether you want to maximize Best Performance, Lowest Cost, or a healthy balance of the two.
Keebo’s algorithms then automatically self-tune to deliver the best possible savings while adhering to your expressed desires. This mechanism allows you to regulate the aggressiveness of Keebo’s optimization algorithms with ease, ensuring that our system aligns perfectly with your business needs. We’ve designed this feature to give you intuitive control, so you can focus on your outcomes while we handle the complexity of optimization.
Layer 3: Performance Guardrails – Protecting Your SLAs
Meeting SLAs is critical, so Keebo offers Performance Guardrails to ensure you never fail to meet your users’ expectations . Keebo continuously monitors key metrics in real time to safeguard performance, automatically reverting to a safe configuration if any threshold is breached during the evaluation window.
You can define your SLAs with precision using metrics like maximum latency – where we perform a backoff if any single query exceeds the latency threshold, ensuring your workloads do not experience unacceptable delays. Similarly, we track maximum queue size, which, when triggered, executes a backoff if the number of queries waiting in the queue exceeds your specified limit at any point. Additionally, our maximum queue time metric ensures that if any query waits in the queue longer than your defined threshold, we take action to restore performance.
These customizable guardrails allow you to set the exact standards that matter to your business for each individual warehouse. Keebo works diligently to maintain them.
Layer 4: Condition-Based Upsizing – Scaling with Intelligence
Unexpected workload spikes can strain even the most carefully planned systems. That’s why we’ve built our Condition-Based Upsizing feature to help you manage these challenges proactively. This layer enhances your warehouse’s adaptability by enabling you to define dynamic rules based on real-time performance metrics related to query latency or queue behavior.
For instance, if query latency exceeds a threshold you’ve set—say, 100 seconds during a sudden surge in dashboard usage—Keebo will automatically increase the warehouse size to meet the demand, ensuring smooth performance for your users. During each optimization cycle, Keebo evaluates your rules and upsizes the warehouse, up to a maximum number of levels you specify, whenever the condition is met. When the spike subsides and the condition no longer applies, Keebo scales the warehouse back down—either by one level or to your default size, based on your preference.
This means you can confidently set warehouse parameters for regular workloads, knowing Keebo will dynamically adjust resources during unexpected spikes. By offering this flexibility to tailor scaling rules to your unique workload patterns and business needs, we aim to improve data processing efficiency while avoiding overprovisioning and keeping costs in check.
Layer 5: Optimization Algorithms – Stability with Precision
We’ve already discussed how our scheduling features let you align warehouse resources with your workload demands, offering tailored control over optimization settings. The fifth layer takes this a step further by enabling you to fine-tune Keebo’s behavior across all schedules.
You can enable or disable specific optimization algorithms, like Automated Downsizing or Multi-Cluster Optimization, to ensure Keebo’s mechanisms align perfectly with your operational goals.
For critical operations, like a monthly ETL pipeline that processes large datasets to update your financial reporting systems, this layer provides unparalleled control. You may choose to disable Automated Downsizing during this pipeline to prevent any risk of resource reduction, guaranteeing stability when reliability is non-negotiable. Once the pipeline completes, Keebo automatically re-enables the algorithm for regular operations, maintaining efficiency without manual intervention. This precise risk management blends automation with detailed customization, helping your warehouse deliver consistent performance across diverse workloads.
Layer 6: Smart Query Routing – Efficiency and Performance in Harmony
Finally, smart Query Routing dynamically directs each query to the right-sized warehouse on the fly. Our routing algorithms autonomously account for warehouse size, current load, and the query’s own characteristics when making decisions. This ensures both cost efficiency and optimal performance without any manual intervention.
A common use case for query routing occurs when lightweight queries run on oversized, expensive warehouses. At the same time, heavier queries are routed to appropriately sized warehouses to avoid delays.
By grouping similar queries together, each warehouse handles a more consistent workload, making it easier for Keebo’s AI to apply real-time optimizations effectively.
This approach delivers multiple benefits:
- Reduce costs by preventing overprovisioning
- Boost query performance by automatically routing high-priority or heavy queries to larger or underutilized warehouses, preventing SLA violations and ensuring light queries aren’t delayed by more demanding ones
- Free your data team by decoupling application logic from performance concerns, letting them focus on writing accurate queries while Keebo handles selections of the best warehouses for query processing.
Why Keebo Wins: Built for Data Teams, by Data Experts
Keebo isn’t just a product. It’s a platform crafted to solve the full range of warehouse headaches. Automation, customization, and real-time insights come together so your team can focus on analytics, not infrastructure.
The proof? 95% of our customers achieve their goals with minimal setup, and our full framework scales to handle even the toughest workloads.
At Keebo, we’ve distilled deep technical know-how into a platform that’s both powerful and practical. We’re not just optimizing—we’re evolving with your business, adapting as your needs grow.
Ready to see the difference? Explore how we can transform your data strategy today!
Author
