An AI agent querying production data on its own can produce answers that look right and are badly wrong. At dbt Summit 2026, AWS solutions architects Neela Kulkarni and Venkatesh Aravamudan walked through a fix built on the dbt Semantic Layer and the Amazon Redshift MCP Server, designed so the agent never writes its own SQL.
They illustrated the risk with a simple query: summing gross_amount by region on fct_orders returned $14.86 million. Actual revenue was $11.36 million - a 30.8% overstatement. The figure was pre-discount and pre-refund, refunds lived in an unjoined table, cancelled and pending orders were counted, and internal test accounts were flagged but not excluded. All four rules exist only in a semantic layer, not the schema.
The presenters cited Gartner's $12.9 million average annual cost of poor data quality, the 73% of organizations naming data quality as their top barrier to AI, and a projection that over 40% of agentic AI projects will be canceled by 2027, largely over governance gaps. They also pointed to the BEAVER benchmark, co-authored by AWS researchers, where state-of-the-art agents reached only 10.8% accuracy on real enterprise warehouse queries.
Their architecture splits responsibilities. The dbt MCP Server governs what gets queried, resolving business questions against certified metrics via tools like list_metrics and query_metrics, with MetricFlow compiling the SQL rather than the language model. The Redshift MCP Server governs execution, enforcing read-only, single-statement queries at three layers: the server, IAM-scoped credentials, and database least-privilege grants. Because agents pick from an enumerated metric list, the presenters called fabricated metrics structurally impossible. Each answer carries a receipt: metric name, compiled SQL, source model, dbt test results, and a Redshift query ID traceable through SYS_QUERY_HISTORY.
A live demo served a developer who could approve compiled SQL and a business user who received a cited plain-language answer, across a native Redshift query, a blended query joining Redshift with Apache Iceberg data in S3 Tables, and a data lake query run on dbt's Athena and DuckDB adapters - with identical governance throughout. Their takeaway: governance must be architectural, not aspirational.
Source: dbt Labs Blog