Snowflake Cortex vs. External LLMs: Pricing, Governance, and When to Use Each

Most Snowflake customers exploring large language models (LLMs) face a practical fork in the road: Should we start with Snowflake Cortex, adopt a hybrid pattern, or build on an external LLM platform?

The short answer: if your data and controls live in Snowflake, start with Cortex. Shift to hybrid when you need model flexibility without wholesale data movement. Go external when specialization or application scale make the economics or capabilities compelling.

The best answer for your team depends on five variables you can evaluate quickly: data gravity and governance, model breadth and control, performance envelope, operational tolerance, and total cost of ownership. This guide covers each one.

What Is Snowflake Cortex?

Cortex provides LLM and AI capabilities within Snowflake’s existing governance framework. Instead of moving information into a separate application stack, you invoke language and semantic functions where your access controls, masking policies, lineage, and budgets already apply.

That proximity reduces surface area and lets you reuse the operational habits you’ve built for the warehouse. You still need evaluation and cost discipline, and the rest of this guide covers both.

Cortex isn’t a single tool. It’s a set of primitives you can compose. The pattern to keep in mind is simple: bring questions to the data, and keep sensitive rows and columns under existing controls.

These features share a common approach: data remains in Snowflake with controlled access through views or indexes.

Cortex vs. Hybrid RAG vs. External LLM Platforms

With Snowflake as your system of record, architecture choices are mostly about where the logic runs and how much crosses the governance boundary. In practice you’ll pick from three patterns.

Cortex-first (all in Snowflake) is the most conservative path: retrieval, semantic search, and generation happen inside Snowflake’s RBAC, masking, and audit. Teams tend to start here because it minimizes moving parts and shortens the path from idea to value.

Hybrid RAG splits the world into a governed data plane (Snowflake) and a logic plane (an external model). You keep embeddings, indexes, and retrieval close to source tables, you assemble a minimal, redacted context, and you send only that across the boundary for generation. The payoff is model flexibility without duplicating sources of truth.

External platform treats the model layer as a product surface of its own. An external ML platform (or a self-hosted stack) handles model selection, routing, fine-tunes, tool and agent orchestration, A/B testing, and evaluation gates. Snowflake remains the data backbone.

How Is Snowflake Cortex Priced?

Because Cortex operates inside Snowflake, usage is metered the way you’re used to: consumption rolls up to credits under your account and edition. Cortex AI Functions bill credits per million input and output tokens, at rates set per function and model in the Snowflake Service Consumption Table.

Inside Snowflake, spend usually flows through four doors:

• Retrieval and preparation. Warehouses execute the views that power prompts: filtering, joining, pre-aggregating. This is classic query optimization territory. Project fewer columns, trim rows aggressively, and pre-aggregate so your context starts small.
• Embeddings and vector search. You’ll pay to generate embeddings, build indexes, and refresh them as content changes. Cortex Search also bills serving compute per GB per month of indexed data while the service is available. Index cadence (hourly vs. nightly) is the cost lever most teams tune last.
• LLM functions. When you call Cortex functions for generation, summarization, translation, extraction, or classification, credits scale with the tokens in each request and response and with how often you call them. That circles back to retrieval shape.
• Fine-tuning. Training runs bill by the tokens used in training. Running the fine-tuned model then bills per token like any other model.

Two implications follow. First, the cheapest tokens are the ones you never send, so fix upstream SQL before you tinker with prompts. Second, treat LLM usage like any other Snowflake workload: tag queries by project, watch Cortex and warehouse credit consumption against budgets, and attribute costs so teams see their own impact.

See Where Your Cortex Credits Go

Snowflake records AI Function usage per query in CORTEX_AISQL_USAGE_HISTORY, including the query tag. Tag the session before the pipeline runs:

ALTER SESSION SET QUERY_TAG = 'llm:support_ticket_triage';

Then attribution is a GROUP BY:

-- Cortex AI Function credits by project tag, function, and model, last 30 days SELECT     query_tag,     function_name,     model_name,     SUM(tokens)        AS tokens,     SUM(token_credits) AS token_credits FROM SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AISQL_USAGE_HISTORY WHERE usage_time >= DATEADD(day, -30, CURRENT_TIMESTAMP()) GROUP BY query_tag, function_name, model_name ORDER BY token_credits DESC;

Two limits to know. The view covers usage from November 17, 2025 onward, and token credits exclude warehouse credits, so attribute retrieval compute separately. To include AI_PARSE_DOCUMENT, use CORTEX_AI_FUNCTIONS_USAGE_HISTORY instead.

Size the Prompt Before You Send It

AI_COUNT_TOKENS returns the token count for a given function and model, without calling the model:

SELECT     ticket_id,     AI_COUNT_TOKENS(         'ai_complete',         'mistral-large2',         CONCAT('Summarize this support ticket: ', ticket_body)     ) AS prompt_tokens FROM support_tickets ORDER BY prompt_tokens DESC LIMIT 20;

The rows at the top are where truncation will save the most.

How Are External LLMs Priced?

For Snowflake customers, the cleaner view is a two-ledger bill. Inside Snowflake, Cortex AI Functions bill credits per million tokens processed, and the retrieval queries that build the prompt bill warehouse credits. Outside Snowflake, you pay the provider per token, and sometimes for hosting.

In Hybrid or External patterns, you add the provider’s ledger and, sometimes, a hosting bill:

• Tokens in and out. Providers charge for context and for completions. Long inputs and verbose outputs both push this number up.
• Reserved or self-hosted capacity. If you rent an inference endpoint or operate your own GPUs, you pay even when traffic is low. That can be efficient at steady, high volume, but wasteful for spiky loads without autoscaling.
• Fine-tuning and storage. Where supported, you’ll see charges for training runs and for storing adapters and checkpoints. Worth it only if they reduce tokens or errors enough to move cost per successful task.
• Observability overhead. Whether bundled or separate, collecting model monitoring signals (latency, failures, token counts, refusals) is part of the true cost because it prevents surprises.

Network egress exists, but when you send only minimal, redacted context across the boundary, it should be a small share of the total.

A design note: the easiest way to keep this side predictable is to treat the external stack as a logic plane. Keep embeddings and retrieval in Snowflake, send only a small, redacted packet to the model, and write outputs and feedback back to Snowflake for lineage and allocation.

How Do You Compare Cortex and External LLM Costs?

Pricing gets simpler once you stop comparing rate cards line by line. The decision you’re making isn’t “which price is lower.” It’s where each dollar actually buys a successful task: a grounded answer, a correct SQL suggestion, a classified document.

Start by defining success in the language of the use case: SQL correctness for generation, grounded-answer rate for RAG, summary fidelity for summarization. Then compute:

Cost per successful task = (Snowflake credits × price per credit + external tokens × price per token + any hosting) ÷ number of successful tasks

This number does three useful things. First, it forces you to account for retrieval shape, since wide queries inflate both credits and tokens. Second, it highlights where optimizations matter most (retrieval vs. inference vs. network). Third, it lets you compare Cortex, Hybrid, and External apples-to-apples without arguing about line items.

Making the Comparison Fair

Compare options using these metrics over a representative time period:

• Requests per day per use case
• Average context tokens after truncation
• Average output tokens per answer
• Cache hit rate (for safe, repeatable answers)
• Concurrency at peak (drives retries and tail latency)

Hold retrieval constant while you swap generation options. If you change prompt templates or ranking rules, re-run the whole set, because even small changes to context size will invalidate yesterday’s numbers.

Forecasting that People Trust

Keep forecasting lightweight but live. A small sheet with those five inputs, multiplied by your credit and token rates, yields a monthly envelope that you can sanity-check at a glance. When any input changes materially (a new team, a longer context cap, a lower cache hit rate), re-run the envelope.

When Should You Use an External LLM Instead of Cortex?

Think in this order: data gravity, capability, ops tolerance, and economics. If sensitive data must stay in Snowflake, start in-platform. If a specific model outside the Cortex catalog improves results on your evaluation set, use a hybrid setup. If you are building a user-facing product that needs deep orchestration, use a cloud ML platform or self-hosted endpoint as your logic plane, and keep Snowflake as the data plane.

Before you leave Cortex, check Cortex Fine-tuning and Cortex Agents first, since both run inside the same governance boundary. Where a dedicated ML platform gives you more control is deep orchestration and experimentation tooling. If a specific route needs that, the fix is architectural, not political: introduce Hybrid for that route while keeping embeddings and retrieval in Snowflake.

A Decision Checklist You Can Run In One Meeting

Use this checklist to prevent over-engineering and keep your architecture following evidence:

• Data gravity. Must the relevant data remain inside Snowflake’s RBAC and masking boundary? If yes, start Cortex-first and treat cross-boundary calls as exceptions.
• Capability gap. Does a specific external model move the needle on your evaluation set? If yes, try Hybrid for that one route. If not, optimize retrieval and prompts first.
• Interaction model. Are users interacting live (strict p95) or consuming batch or internal outputs? Live experiences justify Hybrid sooner. Batch and internal usually don’t.
• Ops tolerance. Do you have the SRE and ML-ops capacity to run and observe another estate? If not, stay in-platform until you do, or until evidence makes the trade-off undeniable.
• Unit economics. Can you compute cost per successful task with a single trace? If you can’t measure it, you can’t control it. Pause until that harness exists.
• Exit and rollback. What would make you revert a change? Define rollback triggers (quality drops, p95 spikes, budget breaches) before you flip traffic.

A Staged Rollout That Avoids Rearchitecture

The incremental path works because it earns each step.

1. Prove value in place: one Cortex-first use case with clear success criteria, narrow views, masked columns, templated prompts, and a tiny offline evaluation set.
2. Add grounding: embeddings and managed search over curated corpora, with an index refresh cadence that matches business freshness.
3. Introduce Hybrid selectively: route a small fraction of requests to an external model where your evaluation set shows a real gain. Keep retrieval in Snowflake and carry correlation IDs across the boundary.
4. Adopt a platform only if the application demands orchestration and A/B discipline you can’t get otherwise.

Just as important, give yourself permission to revert. If p95 blows past SLOs, cache hit rate collapses, or cost per successful task creeps up over two consecutive reviews, route traffic back to the simpler path while you fix retrieval, truncation, or caching. Simplicity is not a step backward. It’s how you preserve momentum while you keep risk and spend where you want them.

How to Keep Data Governed When You Call an External Model

When Snowflake is your system of record, governance isn’t an add-on. It’s the frame that determines every other choice. The safest posture is to keep Snowflake as the governed data plane and treat LLMs (Cortex or external) as the logic plane. That boundary gives you a simple rule: data may be read under existing policies, only minimal, redacted context crosses the line, and every hop is logged and traceable.

Put controls in the data, not in the prompt

Governance is sturdier when it’s enforced at the view that feeds your prompts, not as a last-mile redaction script. Tag sensitive columns. Attach masking and row-access policies that apply automatically when a service role reads those views. If a column shouldn’t appear in an answer, it shouldn’t be available to the retrieval query in the first place.

Minimize and sanitize before anything leaves Snowflake

Language models get more expensive and riskier the more you send them. Assemble context with chunk → rank → truncate, deduplicate near-matches, and strip nonessential metadata. Replace obvious identifiers with placeholders (<customer_email>, <account_id>) so the model has the signals it needs without the secrets it doesn’t.

Identity, secrets, and network posture

Production traffic should flow through service principals with the narrowest rights necessary. Keep keys in a secrets manager, and never embed them in SQL, configs, or prompt templates. For any external endpoint, prefer private connectivity, co-locate it in the same region as your Snowflake account, and operate on allowlists rather than broad egress. Finally, set providers to not retain or train on your data unless you have explicit contractual approval and a documented reason.

One trace or it didn’t happen

Security without visibility is guesswork. Generate a correlation ID at the edge, include it in Snowflake query tags for retrieval, carry it inside prompt metadata, and log it with the model response and any write-backs. Now you can separate retrieval latency from generation latency, tie quality signals to cost, and reconstruct incidents without hunting across systems. No new observability religion required.

Bottom line: Keep sensitive data where Snowflake already protects it. Send out only what’s necessary, scrubbed and small. Make every request traceable end-to-end so cost, latency, and quality read as one story.

How to Benchmark Cortex Against an External Model

Benchmarking only helps if it looks like production. For Snowflake customers, that means governed retrieval inside Snowflake, templated prompts with explicit parameters, and one trace that follows every request from query to answer. The goal isn’t to crown a universal “best” model. It’s to decide what works for your data, under your controls, at a cost you can live with.

What to measure

Start with task quality, defined the way you’ll judge success in the real system. Add latency, both end-to-end and by stage, because p95 is what users feel and what drives retries. Fold in cost per successful task so you’re comparing dollars for outcomes, not dollars for tokens or credits. If a metric won’t influence a go/no-go decision, don’t measure it.

• Grounded-answer rate: (answers where all claims can be traced to a retrieved context snippet) ÷ (total answers).
• SQL correctness: (generated queries that execute without error AND return a result set matching the gold standard’s schema and values) ÷ (total queries).
• Latency: p50/p95 end-to-end and by stage. Note the concurrency during the run so results are interpretable.
• Cost per successful task: inside Snowflake, outside Snowflake, and the total.
• Stability: variance across N re-runs for the same prompts. High variance usually signals temperature and decoding settings or retrieval instability.

Build an evaluation set that mirrors reality

An evaluation is only as good as its inputs. Sample real prompts and questions from your teams (with PII safely masked), and pair each with the retrieval view that will power the production prompt. Balance easy, typical, and edge cases. Otherwise you’ll overfit to the happy path and be surprised in week two. You can keep the whole set in Snowflake tables so it’s easy to version and re-run when prompts or models change.

Control the variables when you run

Most “benchmark drama” comes from hidden differences. Fix the retrieval path first: use the exact views, filters, and ranking logic you’ll ship. Template prompts and pass model parameters (temperature, top_p, max_tokens) explicitly. AI_COMPLETE accepts all three. Decide how you’ll treat caching before you start: either warm caches deliberately and log hit rate, or disable caching for apples-to-apples runs. Then change one dimension at a time. If you’re comparing Cortex to an external model, keep retrieval constant and swap only the generation step.

Snowflake’s AI Observability runs batch evaluations for Cortex Agents, and its TruLens integration can stream telemetry from custom apps, including ones hosted outside Snowflake, into your account. That keeps the harness where your teams already work.

Report like a decision-maker, not a lab report

A good benchmark readout fits on one page. Show distributions, not just single numbers: p50 and p95 for latency, interquartile ranges for quality scores. Compare results by task type, since a model that wins at summarization may lose at SQL generation. And say the quiet part out loud. If Model X is cheaper but fails on edge cases, use routing: start with a fast, low-cost model, then escalate on low confidence.

Common Pitfalls and How to Avoid Them

Most LLM projects don’t fail because a model can’t reason. They fail because everyday engineering habits don’t carry over to language-driven workloads. The antidotes are boring by design: narrow the data surface before you prompt, cap what can grow, and keep one trace from query to answer.

Wide queries that bloat prompts. Most LLM cost overruns start as ordinary SQL. Curate narrow views that project only essential fields, push predicates down, and pre-aggregate where possible. Upstream query optimization reduces tokens and latency more reliably than downstream prompt tweaks.

Unbounded context windows. Packing every “possibly relevant” paragraph into a prompt feels safe but inflates tokens, slows p95, and often hurts answer quality. Treat context as a scarce resource: chunk → rank → deduplicate → truncate.

Refreshing indexes more often than anyone reads them. Refreshing on every change feels “fresh” but adds credit consumption. Cortex Search refreshes on the TARGET_LAG you set, so align it with business freshness. Defining a primary key lets refreshes take Snowflake’s faster incremental path:

ALTER CORTEX SEARCH SERVICE support_docs_search SET TARGET_LAG = '1 hour'; ALTER CORTEX SEARCH SERVICE support_docs_search SET PRIMARY KEY = (doc_id);

Snowflake also notes that most search services don’t index faster beyond a LARGE warehouse, and many need only MEDIUM.

Shadow vector stores and split-brain retrieval. Standing up a second vector store outside Snowflake “just in case” doubles indexing and storage costs and fragments relevance tuning. Keep retrieval in Snowflake, and send only a minimal, redacted context to any external model.

Unlogged caches that distort tests. Provider-side caches can make pilots look cheaper than production. If a benchmark seems too good to be true, you probably measured a warm cache, not the real system.

Over-rotating to bigger models. If retrieval is noisy or duplicative, you’ll pay more to process the same clutter. Establish routing: small, fast model first, and escalate only on low-confidence cases. Improve ranking and truncation before you reach for more parameters.

Failing to close the feedback loop. Instrument your UI for thumbs up and thumbs down, and log these signals in Snowflake using the correlation ID. Use this data to identify failure patterns, enrich your evaluation set, and build datasets for fine-tuning.

Practical takeaway: Most pitfalls are the absence of ordinary discipline. Narrow the views that feed prompts, cap context and deduplicate aggressively, refresh indexes on a sensible cadence, keep retrieval in Snowflake, and make every request traceable end-to-end.

Conclusion: Start Where Your Data and Controls Live

If Snowflake is already your system of record, the surest way to get dependable value from large language models is to begin where your governance works today. Treat Snowflake as the governed data plane, and whichever model you choose (Cortex, a managed API, a cloud ML platform, or a self-hosted endpoint) as the logic plane. Keep the bridge between them narrow, explicit, and observable.

There’s one more habit that matters as usage grows. Every pattern in this guide runs its retrieval on Snowflake warehouses, and LLM traffic changes how those warehouses are loaded. The warehouse settings that fit last quarter’s dashboards may not fit this quarter’s prompt traffic. Keeping warehouse size, suspend settings, and concurrency matched to the workload as it shifts is the layer Keebo adds on top of what Snowflake provides.

The final, practical test of any design here is whether you can explain it on one page: where a request starts, what data is read under which policies, what small context (if any) crosses the boundary, which model answers, how you know the answer met the bar, and what you paid for that success. If your team can tell that story clearly, you’re ready to scale. If not, simplify until you can.

FAQ

What is Snowflake Cortex and when should you use it?

Cortex is Snowflake’s built-in AI service, ideal for teams prioritizing governed data, minimal egress, and rapid internal prototyping. It suits summarization, classification, extraction, governed agents, and SQL-assist use cases, and it keeps LLM processing inside the RBAC, masking, and audit you already run.

Is Snowflake Cortex an LLM?

No. Cortex provides LLM and AI capabilities within Snowflake’s governance framework. As of October 2026, Cortex AI Functions run models from OpenAI, Anthropic, Meta, Mistral AI, DeepSeek, and xAI. You choose the model per call, for example as the first argument to AI_COMPLETE.

How does Cortex compare to hybrid and external LLM platforms?

Cortex minimizes data movement and governance risk. Hybrid adds access to models outside the Cortex catalog with modest plumbing cost. External platforms support deep orchestration for advanced, low-latency apps, but add operational complexity and a second ledger of token costs.

How is Snowflake Cortex priced?

Cortex AI Functions bill credits per million input and output tokens, at rates set per function and model in Snowflake’s Service Consumption Table. Warehouse credits for retrieval queries bill separately, and Cortex Search adds refresh, embedding, serving, and storage costs. Prompt and context size drive token cost directly.

Can you fine-tune models in Snowflake Cortex?

Yes. Cortex Fine-tuning is generally available in select regions. It customizes supported base models on your own examples, billed by training tokens, and you call the fine-tuned model through AI_COMPLETE like any other. Check it before moving a route to an external platform.

Does Snowflake Cortex support AI agents?

Yes. Cortex Agents plans multi-step requests, calls tools, and combines Cortex Analyst for structured data with Cortex Search for unstructured data, all inside Snowflake’s governed environment. It also supports custom tools, code execution, and MCP connectors, without you running your own orchestration loop.

When does a hybrid pattern make sense?

When teams want to keep embeddings and retrieval in Snowflake but call a specific external model for generation, balancing flexibility and control. Choose it when a model outside the Cortex catalog measurably improves results on your evaluation set and cost per successful task stays within your envelope.

Why are economics critical when scaling LLM workloads?

As concurrency and context sizes grow, token and credit costs compound. True efficiency is measured in dollars per successful task, not per token. A cheaper model that fails more often can cost more per useful answer than a pricier one that succeeds.

How do you track Cortex costs?

Query SNOWFLAKE.ACCOUNT_USAGE.CORTEX_AISQL_USAGE_HISTORY, which records tokens and token credits per query, function, model, and query tag. Set a QUERY_TAG per project so you can group spend. Token credits exclude warehouse credits, so attribute retrieval compute separately.

What’s a simple rule of thumb for adoption?

Start with Cortex if your data and controls live in Snowflake, move to hybrid when you need a model outside the Cortex catalog, and go external when specialization justifies it. Reassess quarterly, since models, token prices, and Snowflake features keep evolving.