Go Observability with OpenTelemetry

Overview

Modern Go applications require comprehensive observability through the three pillars: traces, metrics, and logs. OpenTelemetry provides vendor-neutral instrumentation for distributed tracing, Prometheus offers powerful metrics collection, and Go's slog package (1.21+) delivers structured logging with minimal overhead.

Key Features:

🔍 OpenTelemetry: Distributed tracing with context propagation
📊 Prometheus: Metrics collection with /metrics endpoint
📝 Structured Logging: slog with JSON formatting and correlation IDs
🎯 Auto-Instrumentation: HTTP/gRPC middleware patterns
💚 Health Checks: Kubernetes-ready readiness/liveness probes
🔄 Graceful Shutdown: Clean exporter shutdown and signal handling

When to Use This Skill

Activate this skill when:

Instrumenting microservices for production observability
Setting up distributed tracing across service boundaries
Creating operational dashboards with Prometheus/Grafana
Debugging production performance issues or bottlenecks
Implementing SLOs and monitoring SLIs
Adding observability to existing Go applications
Correlating logs, traces, and metrics for debugging

Core Observability Principles

The Three Pillars

Traces: Understand request flow across distributed systems
Metrics: Measure system behavior and performance over time
Logs: Record discrete events for debugging and audit

Correlation Strategy

All three pillars must share common identifiers:

Trace ID: Links all operations in a request
Span ID: Identifies specific operation within trace
Request ID: Correlates logs with traces and metrics

OpenTelemetry Integration

Installation

bash

go get go.opentelemetry.io/otel
go get go.opentelemetry.io/otel/sdk
go get go.opentelemetry.io/otel/exporters/jaeger
go get go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp

Basic Setup

package main

import (
    "context"
    "log"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/exporters/jaeger"
    "go.opentelemetry.io/otel/sdk/resource"
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
    semconv "go.opentelemetry.io/otel/semconv/v1.21.0"
)

func initTracer(serviceName string) (*sdktrace.TracerProvider, error) {
    // Create Jaeger exporter
    exporter, err := jaeger.New(jaeger.WithCollectorEndpoint(
        jaeger.WithEndpoint("http://localhost:14268/api/traces"),
    ))
    if err != nil {
        return nil, err
    }

    // Create resource with service name
    res, err := resource.Merge(
        resource.Default(),
        resource.NewWithAttributes(
            semconv.SchemaURL,
            semconv.ServiceName(serviceName),
            semconv.ServiceVersion("1.0.0"),
        ),
    )
    if err != nil {
        return nil, err
    }

    // Create tracer provider
    tp := sdktrace.NewTracerProvider(
        sdktrace.WithBatcher(exporter),
        sdktrace.WithResource(res),
        sdktrace.WithSampler(sdktrace.AlwaysSample()), // Use probability sampler in production
    )

    otel.SetTracerProvider(tp)
    return tp, nil
}

func main() {
    tp, err := initTracer("order-service")
    if err != nil {
        log.Fatal(err)
    }
    defer func() {
        if err := tp.Shutdown(context.Background()); err != nil {
            log.Printf("Error shutting down tracer: %v", err)
        }
    }()

    // Application code...
}

Creating Spans

import (
    "context"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/attribute"
    "go.opentelemetry.io/otel/codes"
    "go.opentelemetry.io/otel/trace"
)

func ProcessOrder(ctx context.Context, order Order) error {
    tracer := otel.Tracer("order-service")
    ctx, span := tracer.Start(ctx, "ProcessOrder")
    defer span.End()

    // Add attributes
    span.SetAttributes(
        attribute.String("order.id", order.ID),
        attribute.Int("order.items", len(order.Items)),
        attribute.Float64("order.total", order.Total),
    )

    // Validate order (creates child span)
    if err := validateOrder(ctx, order); err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, "validation failed")
        return err
    }

    // Fulfill order
    if err := fulfillOrder(ctx, order); err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, "fulfillment failed")
        return err
    }

    span.SetStatus(codes.Ok, "order processed successfully")
    return nil
}

func validateOrder(ctx context.Context, order Order) error {
    _, span := otel.Tracer("order-service").Start(ctx, "validateOrder")
    defer span.End()

    // Validation logic...
    return nil
}

HTTP Middleware Instrumentation

import (
    "net/http"

    "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)

func main() {
    // Wrap handler with automatic tracing
    handler := http.HandlerFunc(orderHandler)
    wrappedHandler := otelhttp.NewHandler(handler, "order-handler")

    http.Handle("/orders", wrappedHandler)
    http.ListenAndServe(":8080", nil)
}

// Manual instrumentation for more control
func orderHandler(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    tracer := otel.Tracer("order-service")

    ctx, span := tracer.Start(ctx, "orderHandler")
    defer span.End()

    // Extract order ID from request
    orderID := r.URL.Query().Get("id")
    span.SetAttributes(attribute.String("order.id", orderID))

    // Process order with propagated context
    order, err := fetchOrder(ctx, orderID)
    if err != nil {
        span.RecordError(err)
        http.Error(w, "Order not found", http.StatusNotFound)
        return
    }

    // ... handle response
}

Prometheus Metrics

Installation

bash

go get github.com/prometheus/client_golang/prometheus
go get github.com/prometheus/client_golang/prometheus/promhttp

Metric Types and Patterns

package metrics

import (
    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promauto"
)

var (
    // Counter: Monotonically increasing value
    httpRequestsTotal = promauto.NewCounterVec(
        prometheus.CounterOpts{
            Name: "http_requests_total",
            Help: "Total number of HTTP requests",
        },
        []string{"method", "path", "status"},
    )

    // Gauge: Value that can go up or down
    activeConnections = promauto.NewGauge(
        prometheus.GaugeOpts{
            Name: "active_connections",
            Help: "Number of active connections",
        },
    )

    // Histogram: Observations bucketed by value
    httpRequestDuration = promauto.NewHistogramVec(
        prometheus.HistogramOpts{
            Name:    "http_request_duration_seconds",
            Help:    "HTTP request duration in seconds",
            Buckets: prometheus.DefBuckets, // [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]
        },
        []string{"method", "path"},
    )

    // Summary: Similar to histogram but calculates quantiles
    dbQueryDuration = promauto.NewSummaryVec(
        prometheus.SummaryOpts{
            Name:       "db_query_duration_seconds",
            Help:       "Database query duration",
            Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001},
        },
        []string{"query_type"},
    )
)

Metrics Middleware

import (
    "net/http"
    "strconv"
    "time"

    "github.com/prometheus/client_golang/prometheus/promhttp"
)

// Metrics middleware that instruments all HTTP handlers
func MetricsMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()

        // Track active connections
        activeConnections.Inc()
        defer activeConnections.Dec()

        // Wrap response writer to capture status code
        rw := &responseWriter{ResponseWriter: w, statusCode: http.StatusOK}

        // Call next handler
        next.ServeHTTP(rw, r)

        // Record metrics
        duration := time.Since(start).Seconds()
        httpRequestDuration.WithLabelValues(r.Method, r.URL.Path).Observe(duration)
        httpRequestsTotal.WithLabelValues(r.Method, r.URL.Path, strconv.Itoa(rw.statusCode)).Inc()
    })
}

type responseWriter struct {
    http.ResponseWriter
    statusCode int
}

func (rw *responseWriter) WriteHeader(code int) {
    rw.statusCode = code
    rw.ResponseWriter.WriteHeader(code)
}

// Expose metrics endpoint
func main() {
    http.Handle("/metrics", promhttp.Handler())

    handler := MetricsMiddleware(http.HandlerFunc(orderHandler))
    http.Handle("/orders", handler)

    http.ListenAndServe(":8080", nil)
}

Custom Metrics Example

func ProcessPayment(ctx context.Context, payment Payment) error {
    timer := prometheus.NewTimer(dbQueryDuration.WithLabelValues("payment_insert"))
    defer timer.ObserveDuration()

    // Process payment
    if err := db.InsertPayment(payment); err != nil {
        httpRequestsTotal.WithLabelValues("POST", "/payments", "500").Inc()
        return err
    }

    httpRequestsTotal.WithLabelValues("POST", "/payments", "200").Inc()
    return nil
}

Structured Logging with slog

Basic Setup (Go 1.21+)

package main

import (
    "context"
    "log/slog"
    "os"
)

func initLogger() *slog.Logger {
    // JSON logger for production
    handler := slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
        Level: slog.LevelInfo,
        AddSource: true, // Include file:line information
    })

    logger := slog.New(handler)
    slog.SetDefault(logger) // Set as default logger
    return logger
}

func main() {
    logger := initLogger()

    logger.Info("service starting",
        "service", "order-service",
        "version", "1.0.0",
        "port", 8080,
    )
}

Context-Aware Logging

import (
    "context"
    "log/slog"

    "go.opentelemetry.io/otel/trace"
)

// Add trace context to logger
func LoggerWithTrace(ctx context.Context) *slog.Logger {
    span := trace.SpanFromContext(ctx)
    spanCtx := span.SpanContext()

    return slog.With(
        "trace_id", spanCtx.TraceID().String(),
        "span_id", spanCtx.SpanID().String(),
    )
}

func HandleRequest(ctx context.Context, req Request) error {
    logger := LoggerWithTrace(ctx)

    logger.Info("processing request",
        "request_id", req.ID,
        "method", req.Method,
        "path", req.Path,
    )

    if err := processRequest(ctx, req); err != nil {
        logger.Error("request failed",
            "error", err,
            "duration_ms", time.Since(req.StartTime).Milliseconds(),
        )
        return err
    }

    logger.Info("request completed successfully",
        "duration_ms", time.Since(req.StartTime).Milliseconds(),
    )
    return nil
}

Log Levels and Structured Fields

func ProcessOrder(ctx context.Context, order Order) error {
    logger := LoggerWithTrace(ctx).With(
        "order_id", order.ID,
        "user_id", order.UserID,
    )

    logger.Debug("validating order", "items", len(order.Items))

    if len(order.Items) == 0 {
        logger.Warn("empty order received")
        return ErrEmptyOrder
    }

    logger.Info("order validation passed")

    if err := fulfillOrder(ctx, order); err != nil {
        logger.Error("fulfillment failed",
            "error", err,
            slog.Group("order_details",
                "total", order.Total,
                "items", len(order.Items),
            ),
        )
        return err
    }

    logger.Info("order processed successfully",
        "total", order.Total,
    )
    return nil
}

Health Checks and Graceful Shutdown

Health Check Endpoints

import (
    "context"
    "database/sql"
    "encoding/json"
    "net/http"
    "time"
)

type HealthChecker struct {
    db *sql.DB
    // Add other dependencies
}

type HealthStatus struct {
    Status      string            `json:"status"`
    Version     string            `json:"version"`
    Checks      map[string]string `json:"checks"`
    Timestamp   time.Time         `json:"timestamp"`
}

// Liveness probe - is the app running?
func (hc *HealthChecker) LivenessHandler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    json.NewEncoder(w).Encode(map[string]string{
        "status": "alive",
    })
}

// Readiness probe - is the app ready to serve traffic?
func (hc *HealthChecker) ReadinessHandler(w http.ResponseWriter, r *http.Request) {
    ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
    defer cancel()

    status := HealthStatus{
        Status:    "ready",
        Version:   "1.0.0",
        Checks:    make(map[string]string),
        Timestamp: time.Now(),
    }

    // Check database
    if err := hc.db.PingContext(ctx); err != nil {
        status.Status = "not_ready"
        status.Checks["database"] = "unhealthy: " + err.Error()
        w.WriteHeader(http.StatusServiceUnavailable)
    } else {
        status.Checks["database"] = "healthy"
    }

    // Add more dependency checks (Redis, external APIs, etc.)

    w.Header().Set("Content-Type", "application/json")
    if status.Status == "ready" {
        w.WriteHeader(http.StatusOK)
    }
    json.NewEncoder(w).Encode(status)
}

Graceful Shutdown

import (
    "context"
    "net/http"
    "os"
    "os/signal"
    "syscall"
    "time"
)

func main() {
    // Initialize tracer
    tp, err := initTracer("order-service")
    if err != nil {
        log.Fatal(err)
    }

    // Setup HTTP server
    server := &http.Server{
        Addr:    ":8080",
        Handler: setupRoutes(),
    }

    // Channel for shutdown signals
    shutdown := make(chan os.Signal, 1)
    signal.Notify(shutdown, os.Interrupt, syscall.SIGTERM)

    // Start server in goroutine
    go func() {
        slog.Info("server starting", "port", 8080)
        if err := server.ListenAndServe(); err != http.ErrServerClosed {
            log.Fatal(err)
        }
    }()

    // Wait for shutdown signal
    <-shutdown
    slog.Info("shutdown signal received")

    // Create shutdown context with timeout
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()

    // Shutdown HTTP server
    slog.Info("shutting down HTTP server")
    if err := server.Shutdown(ctx); err != nil {
        slog.Error("HTTP server shutdown error", "error", err)
    }

    // Shutdown tracer provider (flush spans)
    slog.Info("shutting down tracer")
    if err := tp.Shutdown(ctx); err != nil {
        slog.Error("tracer shutdown error", "error", err)
    }

    slog.Info("shutdown complete")
}

Complete Instrumentation Example

package main

import (
    "context"
    "database/sql"
    "log/slog"
    "net/http"
    "os"
    "time"

    "github.com/prometheus/client_golang/prometheus/promhttp"
    "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/attribute"
)

type Server struct {
    db     *sql.DB
    logger *slog.Logger
}

func (s *Server) orderHandler(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()

    // Get tracer and create span
    tracer := otel.Tracer("order-service")
    ctx, span := tracer.Start(ctx, "orderHandler")
    defer span.End()

    // Create context-aware logger with trace ID
    logger := s.logger.With(
        "trace_id", span.SpanContext().TraceID().String(),
        "request_id", r.Header.Get("X-Request-ID"),
    )

    orderID := r.URL.Query().Get("id")
    span.SetAttributes(attribute.String("order.id", orderID))

    logger.Info("fetching order", "order_id", orderID)

    // Fetch order from database
    order, err := s.fetchOrder(ctx, orderID)
    if err != nil {
        span.RecordError(err)
        logger.Error("failed to fetch order", "error", err)
        http.Error(w, "Order not found", http.StatusNotFound)
        return
    }

    logger.Info("order fetched successfully",
        "order_id", orderID,
        "items", len(order.Items),
    )

    // Return order as JSON
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(order)
}

func (s *Server) fetchOrder(ctx context.Context, orderID string) (*Order, error) {
    _, span := otel.Tracer("order-service").Start(ctx, "fetchOrder")
    defer span.End()

    // Time database query
    start := time.Now()

    var order Order
    err := s.db.QueryRowContext(ctx, "SELECT * FROM orders WHERE id = ?", orderID).Scan(&order)

    duration := time.Since(start).Seconds()
    dbQueryDuration.WithLabelValues("select_order").Observe(duration)

    return &order, err
}

func setupRoutes(s *Server, hc *HealthChecker) http.Handler {
    mux := http.NewServeMux()

    // Health endpoints (no tracing needed)
    mux.HandleFunc("/health", hc.LivenessHandler)
    mux.HandleFunc("/ready", hc.ReadinessHandler)
    mux.Handle("/metrics", promhttp.Handler())

    // Business endpoints (with tracing)
    orderHandler := http.HandlerFunc(s.orderHandler)
    mux.Handle("/orders", otelhttp.NewHandler(orderHandler, "orders"))

    // Wrap everything with metrics middleware
    return MetricsMiddleware(mux)
}

Decision Trees

When to Use OpenTelemetry

Use OpenTelemetry When:

Building distributed systems with multiple services
Need to trace requests across service boundaries
Debugging performance issues in microservices
Want vendor-neutral observability (switch backends easily)
Require correlation between traces, metrics, and logs

Don't Use OpenTelemetry When:

Building simple monolithic applications
Performance overhead is critical (consider sampling)
Team lacks observability infrastructure (Jaeger, Zipkin)

When to Use Prometheus

Use Prometheus When:

Need time-series metrics for monitoring and alerting
Building operational dashboards (Grafana)
Measuring SLIs for SLO compliance
Tracking business metrics (requests/sec, conversion rates)
Kubernetes/containerized environments

Don't Use Prometheus When:

Need high-cardinality metrics (Prometheus has limits)
Require long-term metric storage (use Thanos/Cortex)
Need push-based metrics (Prometheus is pull-based)

When to Use slog

Use slog When:

Go 1.21+ projects (standard library, zero dependencies)
Need structured logging with JSON output
Want high-performance logging with minimal allocations
Integrating with log aggregation systems (Loki, ELK)

Don't Use slog When:

Go < 1.21 (use zap or zerolog instead)
Need complex log routing or filtering (use zap)
Require very specific features (audit trails, etc.)

Sampling Strategy Decision

Always Sample When:

Development/staging environments
Total traffic < 100 requests/sec
Debugging specific issues

Probabilistic Sampling When:

Production with moderate traffic (100-10K req/sec)
Sample rate: 1-10% typically

Tail-Based Sampling When:

High traffic production (>10K req/sec)
Only sample errors and slow requests
Requires tail-sampling processor (OpenTelemetry Collector)

Anti-Patterns to Avoid

❌ Not Propagating Context

WRONG: Breaking trace context

func processOrder(order Order) error {
    // Creates new context, loses trace!
    ctx := context.Background()
    return validateOrder(ctx, order)
}

CORRECT: Propagate context through call chain

func processOrder(ctx context.Context, order Order) error {
    // Propagates trace context
    return validateOrder(ctx, order)
}

❌ Cardinality Explosion

WRONG: Unbounded label values

// user_id can have millions of values!
httpRequests.WithLabelValues(r.Method, r.URL.Path, userID).Inc()

CORRECT: Use bounded labels

// Only method and path (bounded values)
httpRequests.WithLabelValues(r.Method, r.URL.Path).Inc()
// Track user-specific metrics separately if needed

❌ Logging Sensitive Data

WRONG: Exposing PII and secrets

logger.Info("user login",
    "email", user.Email,        // PII!
    "password", user.Password,  // CRITICAL!
    "token", authToken,         // SECRET!
)

CORRECT: Redact sensitive information

logger.Info("user login",
    "user_id", user.ID,  // Safe identifier
    "method", "password",
)

❌ Not Closing Spans

WRONG: Span leaks memory

func processOrder(ctx context.Context) error {
    ctx, span := tracer.Start(ctx, "processOrder")
    // Missing defer span.End()!

    if err := validate(); err != nil {
        return err  // Span never closed!
    }

    return nil
}

CORRECT: Always defer span.End()

func processOrder(ctx context.Context) error {
    ctx, span := tracer.Start(ctx, "processOrder")
    defer span.End()  // Always runs

    if err := validate(); err != nil {
        span.RecordError(err)
        return err
    }

    return nil
}

❌ Synchronous Metric Export

WRONG: Blocking requests with metric export

// Synchronous export blocks HTTP handler
exporter := jaeger.New(jaeger.WithCollectorEndpoint(...))
tp := sdktrace.NewTracerProvider(
    sdktrace.WithSyncer(exporter),  // BAD: Synchronous!
)

CORRECT: Use batching for async export

// Batching exports asynchronously
tp := sdktrace.NewTracerProvider(
    sdktrace.WithBatcher(exporter),  // GOOD: Async batching
)

❌ Missing Graceful Shutdown

WRONG: Losing traces on shutdown

func main() {
    tp, _ := initTracer("service")
    // Missing shutdown - spans lost!
    http.ListenAndServe(":8080", nil)
}

CORRECT: Shutdown exporters properly

func main() {
    tp, _ := initTracer("service")
    defer tp.Shutdown(context.Background())

    // Handle signals and graceful shutdown
    server.ListenAndServe()
}

Best Practices

Context Propagation: Always pass
```
context.Context
```
through call chains
Bounded Labels: Keep metric label cardinality under 1000 combinations
Sampling: Use probabilistic sampling in high-traffic production
Correlation IDs: Include trace_id in logs for correlation
Health Checks: Implement both
```
/health
```
(liveness) and
```
/ready
```
(readiness)
Graceful Shutdown: Flush traces and metrics before exit
Error Recording: Use
```
span.RecordError()
```
for automatic error tracking
Metric Naming: Follow Prometheus naming conventions (
```
_total
```
,
```
_seconds
```
)
Log Levels: Use appropriate levels (Debug, Info, Warn, Error)
Auto-Instrumentation: Use middleware for HTTP/gRPC when possible

Metric Naming Conventions

Follow Prometheus best practices:

Counter Metrics (always increasing):

```
http_requests_total
```
(not
```
http_requests
```
)
```
payment_transactions_total
```
```
errors_total
```

Gauge Metrics (can go up or down):

```
active_connections
```
```
queue_size
```
```
memory_usage_bytes
```

Histogram/Summary Metrics (observations):

http_request_duration_seconds

(not

_milliseconds

)

```
db_query_duration_seconds
```
```
response_size_bytes
```

Label Naming:

Use
```
method
```
, not
```
http_method
```
Use
```
status
```
, not
```
status_code
```
or
```
http_status
```
Use snake_case, not camelCase

Resources

Official Documentation:

OpenTelemetry Go: https://opentelemetry.io/docs/instrumentation/go/
Prometheus Client Library: https://github.com/prometheus/client_golang
Go slog Package: https://pkg.go.dev/log/slog

Recent Guides (2025):

"Observability in Go: What Real Engineers Are Saying in 2025" (Quesma Blog)
"Monitoring Go Apps with OpenTelemetry Metrics" (Better Stack, 2025)
Prometheus Best Practices: https://prometheus.io/docs/practices/naming/

Related Skills:

golang-web-frameworks: HTTP server patterns and middleware
golang-testing-strategies: Testing instrumented code
verification-before-completion: Validating observability setup

Quick Reference

Initialize OpenTelemetry

tp, _ := initTracer("service-name")
defer tp.Shutdown(context.Background())

Create Spans

ctx, span := otel.Tracer("name").Start(ctx, "operation")
defer span.End()
span.SetAttributes(attribute.String("key", "value"))

Define Metrics

counter := promauto.NewCounterVec(opts, []string{"label"})
histogram := promauto.NewHistogramVec(opts, []string{"label"})

Structured Logging

logger := slog.With("trace_id", traceID)
logger.Info("message", "key", value)

Health Checks

http.HandleFunc("/health", livenessHandler)
http.HandleFunc("/ready", readinessHandler)

Token Estimate: ~5,000 tokens (entry point + full content) Version: 1.0.0 Last Updated: 2025-12-03

golang-observability-opentelemetry

NPX Install

Tags

SKILL.md Content

Go Observability with OpenTelemetry

Overview

When to Use This Skill

Core Observability Principles

The Three Pillars

Correlation Strategy

OpenTelemetry Integration

Installation

Basic Setup

Creating Spans

HTTP Middleware Instrumentation

Prometheus Metrics

Installation

Metric Types and Patterns

Metrics Middleware

Custom Metrics Example

Structured Logging with slog

Basic Setup (Go 1.21+)

Context-Aware Logging

Log Levels and Structured Fields

Health Checks and Graceful Shutdown

Health Check Endpoints

Graceful Shutdown

Complete Instrumentation Example

Decision Trees

When to Use OpenTelemetry

When to Use Prometheus

When to Use slog

Sampling Strategy Decision

Anti-Patterns to Avoid

❌ Not Propagating Context

❌ Cardinality Explosion

❌ Logging Sensitive Data

❌ Not Closing Spans

❌ Synchronous Metric Export

❌ Missing Graceful Shutdown

Best Practices

Metric Naming Conventions

Resources

Quick Reference

Initialize OpenTelemetry

Create Spans

Define Metrics

Structured Logging

Health Checks