OpenTelemetry reached general availability for traces and metrics in 2021. The project, created from the merger of OpenTracing and OpenCensus, provides a vendor-neutral observability instrumentation standard.
Why standardisation matters
Before OpenTelemetry, each observability vendor (Datadog, New Relic, Honeycomb, Jaeger, Zipkin) had its own instrumentation SDK. Changing observability vendors meant re-instrumenting your codebase. OpenTelemetry separates instrumentation from the backend: applications are instrumented once with the OTEL SDK, and the OTEL Collector routes telemetry to any backend that supports the OTLP protocol. Vendor lock-in is eliminated at the instrumentation layer.
The OTEL Collector
The OpenTelemetry Collector is a vendor-agnostic agent that receives telemetry from applications, processes it (filtering, sampling, attribute enrichment), and exports it to one or more backends. Running a Collector sidecar (in Kubernetes: as a daemonset or deployment) centralises telemetry routing configuration without changing application code. The Collector's pipeline model (receiver, processor, exporter) makes it composable for complex telemetry routing requirements.
Auto-instrumentation
OTEL auto-instrumentation agents (Java and .NET use Java agent / CLR profiler techniques; Node.js and Python patch popular frameworks at import time) instrument HTTP clients, database drivers, message queue clients, and web framework request handlers without any code changes. The auto-instrumentation quality varies by language and framework but provides 80% of production observability value with zero code changes, a compelling starting point.
The baggage concept
OTEL baggage allows key-value pairs to be propagated across process boundaries in distributed traces. A user ID, a feature flag value, or a deployment version can be set at the entry point of a request and read by any service in the call chain. Combined with span attributes, baggage enables high-cardinality analysis: trace all requests from a specific user, compare performance between feature flag variants, isolate errors to a specific deployment version.