DataHub is an open-source metadata platform, originally built at LinkedIn, for data discovery, lineage, and governance. It models an organization's data as a real-time metadata graph - datasets, dashboards, pipelines, ML features, and the people and teams around them - and makes it searchable, documented, and traceable end to end.

The metadata graph

Rather than a static catalog, DataHub represents entities (a table, a dashboard, a Kafka topic) and the relationships between them (this dashboard reads that table, which is produced by this Airflow job). That graph powers column-level lineage, impact analysis ("what breaks if I drop this column?"), and rich search across the whole estate.

How metadata gets in

DataHub ingests metadata through a large library of connectors - Snowflake, BigQuery, dbt, Airflow, Kafka, Looker, and many more - run as scheduled jobs or pushed via its API. It captures schemas, lineage, usage statistics, and ownership. Because ingestion is pluggable and the model is extensible, teams can add custom entities and sources.

# DataHub ingestion recipe (YAML) — pull metadata from Snowflake
source:
  type: snowflake
  config:
    account_id: myacct
    warehouse: COMPUTE_WH
    include_table_lineage: true
sink:
  type: datahub-rest
  config: { server: "http://datahub-gms:8080" }

What it provides

  • Discovery - search and browse datasets with schemas, descriptions, and popularity.
  • Lineage - table- and column-level, across ingestion, transformation, and BI.
  • Governance - ownership, glossary terms, tags, and domains; data classification and access context.
  • Observability hooks - freshness and assertion signals integrated into the asset view.

DataHub vs the alternatives

Among catalogs, DataHub competes with open-source Amundsen and OpenMetadata and with commercial platforms like Collibra, Alation, and Atlan. DataHub's strengths are its extensible real-time metadata model, strong lineage, and an active open-source community (a managed version, Acryl, exists). Amundsen is simpler and discovery-focused; OpenMetadata is a newer all-in-one open-source option; the commercial tools add polished governance workflows and enterprise support. Choose DataHub when you want a powerful, extensible, self-hosted metadata platform and have the engineering capacity to run it.