On a recent podcast, dbt Labs designer Sam Ferguson discussed a shift in data work, moving from a focus on query writing to orchestrating outcomes. The technical implications are significant. Ferguson’s work on embedded natural language in SQL at Mode Analytics—where prompts sit directly within code comments—suggests a system design where context is king. The quality of an AI’s output depends heavily on the proximity and clarity of its instructions, making a well-commented codebase a strategic asset for both human and machine readability.

This evolution mirrors a historical pattern Ferguson cited: the disappearance of the typist role when word processors removed the constraint of expensive editing. For data engineers, the constraint of manually translating business logic into code is softening. The new bottleneck is human attention and prioritization. When an agent can generate a hundred pipeline options, the engineering challenge becomes defining the correct one and establishing the governance for it to run reliably.

The conversation highlighted a practical trade-off: velocity versus coherence. A single engineer using AI can move faster, but without disciplined documentation and shared context, team velocity becomes uneven. The accountability principle remains unchanged; the engineer who prompts and ships a model owns its output. Furthermore, codifying context for an agent—defining canonical models, business logic, and legacy exceptions—forces a rigor in documentation that many teams have deferred. The technical work of building a robust agent context is, in effect, the work of building a more maintainable data platform. The tools are changing, but the engineering fundamentals of clarity, collaboration, and intentional design are now more critical than ever.

Source: dbt Labs Blog