Monitoring & Logging

Monitoring and logging are the two halves of operational visibility, and confusing them is the fastest way to build an observability stack that costs a fortune and answers nothing. Monitoring watches numeric signals over time - request rate, error count, queue depth, p99 latency - and alerts when they cross a threshold. Logging records discrete events with full context - a stack trace, a request body, the exact SQL that timed out - so that once an alert fires, you can find out why. Metrics tell you that something broke; logs tell you what broke. A working stack needs both, plus a way to trace a single request across the services it touches.

Metrics, logs, and traces

The three pillars of observability map to different questions and different storage economics. Metrics are cheap: a counter or a histogram bucket is a few bytes, aggregated at write time, and a time-series database can hold years of them. Logs are expensive: every event is a full line of text, and a busy service emits gigabytes a day, which is why log platforms charge by ingest volume and why retention policies matter so much. Traces sit in between, stitching the spans of one request together so you can see where the 800ms went. Most teams over-log and under-trace, then discover at renewal time that their Datadog or Splunk bill is dominated by debug lines nobody reads.

The platforms

The commercial all-in-one observability vendors bundle metrics, logs, traces, and dashboards behind one agent. Datadog is the breadth leader; New Relic pioneered APM and now prices by ingested data; Dynatrace and AppDynamics compete on automatic dependency mapping and AI-driven root-cause analysis for large enterprise estates. Splunk built the category for log search and SIEM and remains the heavyweight for security and compliance use cases.

On the open-source side, the log-collection and log-management layer is a separate concern from the alerting layer. Fluentd is the CNCF log collector that routes events from anywhere to anywhere. Grafana Loki indexes only labels rather than full log text, which makes it dramatically cheaper to run than Splunk for high-volume logs. Graylog is the self-hosted log-management platform built on Elasticsearch/OpenSearch.

Infrastructure monitoring

Before APM and structured logs, there was host and service monitoring: is the disk full, is the daemon up, did the cron job exit non-zero. Nagios defined this category with its check-plugin model and remains in service across thousands of data centers. The check-and-alert pattern it established underlies almost every monitoring tool that followed, even the ones that replaced its config-file approach with auto-discovery.

In this section

AppDynamics
A Cisco-owned APM platform for large enterprise Java and .NET estates, organized around business transactions tracked end to end across tiers. Alerts via health rules that compare behavior against dynamic baselines learned by time of day rather than fixed thresholds.
Datadog
A SaaS observability platform that collects metrics, logs, and traces through a single host agent with 800+ integrations, tying everything together with tags. Known for breadth and for ingest-volume pricing that rewards indexing logs selectively.
Dynatrace
An enterprise observability platform whose OneAgent auto-discovers and instruments every process, while Smartscape maps service dependencies and the Davis engine correlates anomalies into ranked root-cause problems instead of alert storms. Queried with DQL over the Grail store.
Fluentd
A CNCF-graduated log collector that gathers tagged events from many sources, transforms them through filters, and routes them to many backends by tag - a unified logging layer that decouples applications from any one log store. Buffers and retries to avoid dropping logs.
Grafana Loki
A log aggregation system that indexes only stream labels, not log content, storing raw lines as compressed chunks in object storage - far cheaper than full-text engines. Queried with LogQL, modeled on PromQL, and dependent on keeping label cardinality low.
Graylog
An open-source, self-hosted log management platform built on Elasticsearch or OpenSearch, adding inputs, rule-based streams, parsing pipelines, dashboards, alerting, and access control. The open alternative for teams wanting a self-contained log platform without per-gigabyte SaaS pricing.
Nagios
The tool that defined infrastructure monitoring: a scheduler that runs check plugins on an interval, reads their exit codes as OK/WARNING/CRITICAL/UNKNOWN, and notifies on state change. Its language-agnostic plugin contract underlies a large share of later monitoring tools.
New Relic
The APM pioneer, now a full observability platform storing metrics, logs, traces, and events in one place queried with NRQL. Prices by data ingested plus user seats, and ingests OpenTelemetry directly alongside its own language agents.
Splunk
The platform that built the log-search category: index any text data and query it with the piped Search Processing Language. Extracts fields at search time for schema flexibility, dominates SIEM through Splunk Enterprise Security, and is priced largely by indexed volume.