dbt-Cloud agents — when warehouse-side LLM calls beat application-side

The pattern is not new. SQL-side functions that call language models have been a thing since the major warehouses added them. What is new is that the dbt community has organized around a workflow — dbt-Cloud agents, the dbt-LLM macros, the warehouse-native UDFs that wrap the major providers — that makes warehouse-side LLM work approachable for the data engineer rather than the AI engineer.

I have spent the last quarter watching three data teams move pieces of their LLM stack from application-side to warehouse-side. The decisions were not unanimous, the migrations were not painless, and the architectural lessons were not what either side expected at the start. This piece is the digest.

The two sides

The application-side pattern: a Python or TypeScript service receives a request, calls an LLM, gets a response, returns it. The data warehouse, if it is involved at all, is a destination — write the request and the response to a table for later analysis.

The warehouse-side pattern: a SQL query (or a dbt model) calls an LLM as part of its execution. The query reads from a table, passes rows through a UDF that hits the LLM, writes the result back to a table. The data never leaves the warehouse.

Both work. The arguments for one over the other are about specificity, not absolutes.

Where warehouse-side wins

Three workloads where warehouse-side is the right call:

Batch enrichment of structured data

You have a table of customer feedback. You want to add a sentiment column, a category column, and a “should this be escalated” column. Application-side, you query the rows, send each one through the LLM, write the results back. Warehouse-side, you write a dbt model that selects the rows, runs them through the warehouse’s LLM UDF, and materializes the enriched table.

The warehouse-side version is shorter (often by an order of magnitude in lines), runs as part of your existing scheduled dbt job, and respects your warehouse’s existing security model. The data does not leave the perimeter. The audit log is the same audit log you already trust.

The cost-per-row is often roughly the same as application-side, because both ultimately call the same LLM API; the warehouse adds a thin markup. Where the cost diverges is in the operational overhead: the warehouse-side version eliminates the request-routing layer, the error-handling layer, and the orchestration layer. For batch work where latency is not the constraint, that’s a meaningful simplification.

Joining LLM output with structured data

You have an enriched table from a previous step. You want to filter to customers whose feedback was negative, join against your CRM, and email the relevant account manager. Application-side, you do this in code. Warehouse-side, you do it in SQL.

The warehouse-side version composes naturally with the rest of your data work. The LLM-derived columns are just columns. They join, they filter, they aggregate. The application-side version requires marshaling data out of the warehouse, into the application, and back. The marshaling is where bugs live.

Long-running classification jobs

You have a million rows. You want to classify each one. Application-side, you build a job runner, handle retries, manage parallelism, watch for rate limits. Warehouse-side, you let the warehouse handle it. Snowflake, BigQuery, and Databricks have all added some flavor of batch LLM execution; the warehouse handles the rate limits, the retries, the parallelism. It is slower than a well-tuned application-side job, but it is much less code.

Where application-side wins

Three workloads where staying application-side is still right:

Latency-sensitive user-facing calls

A user typed a message and is waiting for a response. The warehouse-side path is too slow. Even with the warehouse’s lowest-latency tier, the query-planning overhead alone is hundreds of milliseconds before the LLM call starts. For user-facing requests, the application-side direct API call is the right call.

Agentic loops with state

The LLM call is part of an agentic loop. The output of one call drives the input of the next. The state is too rich and too session-bound to be modeled cleanly in tables. The application-side runtime is the right substrate for state machines and graph-shaped orchestration. The warehouse is the wrong shape.

Calls that need access to non-warehouse tools

The LLM call needs to hit a Stripe API, a Slack webhook, an internal microservice. The warehouse cannot make outbound HTTP calls (in most configurations, by design). The application-side path is the only path.

What dbt-Cloud agents change

The dbt-Cloud agents pattern (the macros and UDFs that have organized in the dbt ecosystem over the last two quarters) does not change the math on what belongs warehouse-side. It changes the math on what is approachable from the warehouse side for the average data team.

A year ago, calling an LLM from inside a dbt model required custom UDF infrastructure, careful concurrency management, and a willingness to debug stack traces inside the warehouse. The macros and Cloud-side primitives have collapsed most of that. Calling {{ llm_classify(column='feedback_text', categories=['positive', 'negative', 'neutral']) }} inside a model is now a one-line thing.

The effect on a data team’s posture is meaningful. Teams that would have routed every LLM workload through an application-side service because the warehouse-side path was unfamiliar now route the batch workloads through the warehouse. The application-side service shrinks to the latency-sensitive and state-rich calls, which is the work it is best at anyway.

Migration patterns

Three patterns I have watched work:

1. Start with one batch job. Pick the enrichment job whose application-side code is the most painful to maintain. Rewrite as a dbt model. The first one takes a week; the second takes a day; the third takes an afternoon.
2. Keep the per-row cost visible. Warehouses are good at making costs invisible. LLM calls in UDFs accumulate quickly. Tag every warehouse-side LLM job with a cost-center label. Track the per-job cost weekly. If a job’s cost suddenly triples, you want to know.
3. Run an audit cadence. The warehouse-side path is so easy that teams will route too much through it. Quarterly, audit the warehouse-side LLM jobs against the question “is the latency profile really batch?” The jobs that should not be there will become obvious.

What I would change about my own advice

The dbt-Cloud agents space is moving. Six months ago, the warehouse-side path was a real engineering project. Today, it is a one-line macro for the common cases. Six months from now, the picture probably changes again — more primitives, smoother orchestration, better cost visibility.

The architectural question is the same regardless: is this call batch or latency-sensitive, structured or stateful, warehouse-resident or cross-system? Pick the side that matches the call. The tooling is now good enough on the warehouse side that the answer is no longer “application-side by default.”

— Ginger Wolfe-Suarez