dbt Labs BlogAI & LLMs

dbt State targets wasted compute with model-level freshness and selective builds

dbt spent most of its history effectively stateless, with no memory of earlier runs and no record of what had actually been applied to the warehouse. At dbt Summit, dbt Labs product manager Reuben McCreanor and Fivetran principal software engineer Iaroslav Zeigerman - a lead developer of dbt State and co-founder of Tobiko, the team behind SQLMesh - presented dbt State as the fix, built around a single principle: each run should build only what changed.

In production, that means freshness logic moves from job schedules into model-level code. Teams set a lag_tolerance for how stale a model may get and define what counts as new data for a source, down to a custom SQL condition, at folder, subfolder or individual model level. McCreanor cited dbt Labs' own project, where 19 jobs on often clashing schedules became 2.

In development, auto-cloning from production and auto-deferral let an analytics engineer or an AI agent build and test one model without rebuilding everything upstream or hand-managing manifest.json files. Fanatics senior analytics engineer Alvin Chai said declaring SLAs at the model and letting it decide what needs to run had already delivered 25-30% model reuse and roughly 15% in Snowflake savings on a pilot project.

Zeigerman demoed the Jaffle Shop project: a second dbt build skipping models with no upstream changes, a state:modified selector that needs no manifest file, a zero-copy clone pulling an already-built table from another environment, and a dbt explain command answering why a model did or didn't build. He also showed rebasing a teammate's already-run model from their dev environment without re-execution.

Planned work includes lag-tolerance tuning recommendations from warehouse query history, an experimental column-aware mode that skips a rebuild when upstream code changed but the columns a model depends on didn't, and a compare-changes view in the platform UI.

The team drew boundaries in Q&A: dbt State does not replace source freshness alerting, since state concerns execution efficiency rather than monitoring, and alerting remains on the roadmap. Views are still rebuilt when their underlying query changes, and snapshots are exempt from state entirely. Pricing is usage-based in Daily Active Target Tables, charging only for the first reuse of a table each day, with views not billed at all.

dbt State is available now via pip for dbt 1.7 through 1.12, is built into dbt v2, and is a single checkbox on the dbt platform.

Source: dbt Labs Blog

← Back to News