Sweetwater Sound, the country's largest online music gear retailer, has built a proactive data observability practice around Datadog as it scaled its dbt-based data platform, according to Daniel Gonzalez and Derek Andres, analytics engineer tech leads at the Fort Wayne, Indiana company.
The retailer has expanded rapidly since 2020 into lines of business including Gear Exchange and marketplaces on Amazon, Reverb and eBay, while running a 600-plus person sales engineering team. Because all of Sweetwater's developers are in-house, the data team can talk directly to whoever built a source system when it needs a change.
The data stack moved through several eras. Domo arrived in the 2010s as a drag-and-drop dashboarding tool treated like a data warehouse without really being one. By 2020, rapid growth had produced thousands of reports, duplicated transformations and no formal governance. In 2023 the company began migrating to a governed infrastructure built on dbt, touching 26 departments and more than 100 stakeholders. By 2025 Domo was fully deprecated and the new stack fully integrated; the dbt project now carries more than 11,000 column descriptions.
Observability became necessary as the data team integrated more closely with IT, where DBAs and data engineers were already using Datadog to monitor source systems. Extending the same tool upstream was a natural fit rather than a new tool to sell internally.
The clearest example: a user reported that the hourly sales dashboard showed invoices with no attributed sales channel. Records were still arriving in source tables, so the data looked fresh, and the issue recurred on a fairly stable weekly cadence. Using existing dashboards for Kafka inserts by topic, the team traced a spike in transactions to one websql source and found a legacy script that had been incorrectly scheduled. They rescheduled it and began discussing moving that workload into the cloud warehouse. A related challenge came from compute management: as their DAG expanded and batch frequency needs grew, they risked job timeouts when the warehouse ran hot, and Datadog's native Google Cloud integrations and prebuilt metrics helped smooth out that risk.
Sweetwater's current setup includes monitors configured automatically across entire schemas and databases, row count monitors that adjust to each table's expected volume, freshness monitors tuned to tolerate planned nightly downtime, customizable alert and recovery messaging routed to different targets for different teams, and machine-learning-driven alerting thresholds with a simple annotation framework for manual correction.
Next, the team is expanding into lineage and query monitoring beyond dbt itself, using PupCLI and the Datadog MCP server to make observability accessible from the terminal or an AI assistant, and adopting CI/CD observability features that automate impact analysis and data profiling for changed models in their git provider.
Source: dbt Labs Blog