Analytics engineers Maksim Romanchuk and Paul Pedrozo of WMCHealth presented a design pattern at dbt Summit for serving near real-time operational reporting and full historical analytics from the same business logic, toggled by one environment variable. They described the session as a technical walkthrough of a publicly available pattern, not a case study of confidential data, and said tool names reflect their own environment rather than an endorsement.
WMCHealth runs near real-time operations across nine hospitals on fragmented legacy systems, creating two competing needs from one set of logic: five-minute refreshes over active records for dashboards, alerts and notifications, and nightly full-dataset runs for trend analysis and reporting.
The team ruled out several fixes. Scaling the warehouse didn't hold up on cost at a five-minute cadence. Two separate dbt projects risked logic drift, since a metric change in one copy could be forgotten in the other. Incremental models alone were still too slow against very large tables.
Their answer: one model, one codebase, two execution modes, switched by an env_var('ENV_NRT') check. With ENV_NRT set to '1', the model builds against active records only as a table on a five-minute schedule, with a median runtime of 278 seconds. With the default '0', the same model builds against the full dataset overnight as an incremental materialization, median 4,340 seconds.
Downstream, models that inner join to the narrowed anchor inherit the smaller scope with no extra code, while models that don't join add an explicit WHERE filter checking ENV_NRT through an EXISTS clause. Both pipelines share the same codebase and CI/CD setup; the environment variable on each job is the only difference.
The results: a 93.6% runtime reduction, from a median of 4,340 seconds to 278, and more than 99% less data processed per run in near real-time mode. That took live refresh from a 72-minute starting point to five minutes. Several staging models showed reductions in the 99.3% to 99.9% range.
Their takeaways: the fix was smarter decisions about what gets processed and when, not a bigger warehouse. Environment variables are underused in dbt beyond simple feature flags because they can change runtime behavior. And inner-join propagation is the cheapest win in the pattern, since a well-designed upstream anchor lets narrower scope flow through the DAG for free.
Source: dbt Labs Blog