opentelemetry

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

OpenTelemetry

OpenTelemetry

Adopt vendor-neutral telemetry with consistent instrumentation across services.
采用厂商中立的遥测方案,在所有服务中实现统一的可观测性埋点。

When to Use This Skill

适用场景

  • Debugging latency across microservices
  • Standardizing observability data model and naming
  • Sending telemetry to Prometheus, Grafana, Datadog, or OTLP backends
  • Building SLO dashboards with trace-to-log correlation
  • Instrumenting Python or Node.js applications with tracing and metrics
  • Setting up auto-instrumentation for existing services without code changes
  • 排查微服务间的延迟问题
  • 标准化可观测性数据模型与命名规范
  • 将遥测数据发送至Prometheus、Grafana、Datadog或OTLP后端
  • 构建支持链路-日志关联的SLO仪表盘
  • 为Python或Node.js应用添加链路追踪与指标埋点
  • 为现有服务设置自动埋点,无需修改代码

Prerequisites

前置条件

  • Application services running in containers or on VMs
  • Backend for traces (Jaeger, Tempo, Datadog, or any OTLP receiver)
  • Backend for metrics (Prometheus, Mimir, or OTLP receiver)
  • Kubernetes cluster (for collector deployment) or VM with systemd
  • Network access from services to collector, and collector to backends
  • 运行在容器或VM中的应用服务
  • 链路追踪后端(Jaeger、Tempo、Datadog或任意OTLP接收器)
  • 指标采集后端(Prometheus、Mimir或OTLP接收器)
  • Kubernetes集群(用于Collector部署)或带有systemd的VM
  • 服务到Collector、Collector到后端的网络访问权限

Core Workflow

核心工作流程

  1. Define semantic conventions for services, environments, and versions.
  2. Add SDK or auto-instrumentation in each service.
  3. Run an OpenTelemetry Collector to receive, transform, and export telemetry.
  4. Validate cardinality and sampling to control cost.
  5. Create golden signals dashboards and alerting from collected data.
  1. 定义服务、环境和版本的语义规范
  2. 在每个服务中添加SDK或启用自动埋点
  3. 运行OpenTelemetry Collector以接收、转换并导出遥测数据
  4. 验证基数与采样策略,控制成本
  5. 基于采集的数据创建黄金信号仪表盘与告警规则

Collector Production Configuration

生产环境Collector配置

yaml
undefined
yaml
undefined

otel-collector-config.yaml

otel-collector-config.yaml

receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318

Scrape Prometheus endpoints

prometheus: config: scrape_configs: - job_name: "kubernetes-pods" kubernetes_sd_configs: - role: pod relabel_configs: - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape] action: keep regex: "true" - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_port] action: replace target_label: address regex: (.+) replacement: $$1

Host metrics for infrastructure monitoring

hostmetrics: collection_interval: 30s scrapers: cpu: {} memory: {} disk: {} network: {} load: {}
processors: batch: send_batch_size: 1024 timeout: 5s
memory_limiter: check_interval: 1s limit_mib: 512 spike_limit_mib: 128
attributes: actions: - key: deployment.environment value: production action: upsert

Drop high-cardinality attributes to control cost

filter/drop-debug: traces: span: - 'attributes["http.request.header.x-debug"] == "true"'

Reduce cardinality on URL paths

transform/normalize-routes: trace_statements: - context: span statements: - replace_pattern(attributes["url.path"], "/users/[0-9]+", "/users/{id}") - replace_pattern(attributes["url.path"], "/orders/[0-9]+", "/orders/{id}")

Resource detection for cloud environments

resourcedetection: detectors: [env, system, gcp, aws, azure] timeout: 5s
exporters:

Send traces to Tempo/Jaeger

otlp/traces: endpoint: tempo:4317 tls: insecure: true

Send metrics to Prometheus via remote write

prometheusremotewrite: endpoint: http://mimir:9009/api/v1/push tls: insecure: true

Send logs to Loki

otlp/logs: endpoint: loki:4317 tls: insecure: true

Debug exporter for development

debug: verbosity: basic
service: telemetry: logs: level: info metrics: address: 0.0.0.0:8888
pipelines: traces: receivers: [otlp] processors: [memory_limiter, resourcedetection, transform/normalize-routes, batch, attributes] exporters: [otlp/traces] metrics: receivers: [otlp, prometheus, hostmetrics] processors: [memory_limiter, resourcedetection, batch, attributes] exporters: [prometheusremotewrite] logs: receivers: [otlp] processors: [memory_limiter, resourcedetection, batch, attributes] exporters: [otlp/logs]
undefined
receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318

Scrape Prometheus endpoints

prometheus: config: scrape_configs: - job_name: "kubernetes-pods" kubernetes_sd_configs: - role: pod relabel_configs: - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape] action: keep regex: "true" - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_port] action: replace target_label: address regex: (.+) replacement: $$1

Host metrics for infrastructure monitoring

hostmetrics: collection_interval: 30s scrapers: cpu: {} memory: {} disk: {} network: {} load: {}
processors: batch: send_batch_size: 1024 timeout: 5s
memory_limiter: check_interval: 1s limit_mib: 512 spike_limit_mib: 128
attributes: actions: - key: deployment.environment value: production action: upsert

Drop high-cardinality attributes to control cost

filter/drop-debug: traces: span: - 'attributes["http.request.header.x-debug"] == "true"'

Reduce cardinality on URL paths

transform/normalize-routes: trace_statements: - context: span statements: - replace_pattern(attributes["url.path"], "/users/[0-9]+", "/users/{id}") - replace_pattern(attributes["url.path"], "/orders/[0-9]+", "/orders/{id}")

Resource detection for cloud environments

resourcedetection: detectors: [env, system, gcp, aws, azure] timeout: 5s
exporters:

Send traces to Tempo/Jaeger

otlp/traces: endpoint: tempo:4317 tls: insecure: true

Send metrics to Prometheus via remote write

prometheusremotewrite: endpoint: http://mimir:9009/api/v1/push tls: insecure: true

Send logs to Loki

otlp/logs: endpoint: loki:4317 tls: insecure: true

Debug exporter for development

debug: verbosity: basic
service: telemetry: logs: level: info metrics: address: 0.0.0.0:8888
pipelines: traces: receivers: [otlp] processors: [memory_limiter, resourcedetection, transform/normalize-routes, batch, attributes] exporters: [otlp/traces] metrics: receivers: [otlp, prometheus, hostmetrics] processors: [memory_limiter, resourcedetection, batch, attributes] exporters: [prometheusremotewrite] logs: receivers: [otlp] processors: [memory_limiter, resourcedetection, batch, attributes] exporters: [otlp/logs]
undefined

Collector Kubernetes Deployment

Kubernetes环境Collector部署

yaml
undefined
yaml
undefined

otel-collector-deployment.yaml

otel-collector-deployment.yaml

apiVersion: apps/v1 kind: Deployment metadata: name: otel-collector namespace: observability spec: replicas: 2 selector: matchLabels: app: otel-collector template: metadata: labels: app: otel-collector spec: containers: - name: collector image: otel/opentelemetry-collector-contrib:0.98.0 args: ["--config=/etc/otel/config.yaml"] ports: - containerPort: 4317 name: otlp-grpc - containerPort: 4318 name: otlp-http - containerPort: 8888 name: metrics resources: requests: cpu: 200m memory: 256Mi limits: cpu: "1" memory: 512Mi volumeMounts: - name: config mountPath: /etc/otel livenessProbe: httpGet: path: / port: 13133 readinessProbe: httpGet: path: / port: 13133 volumes: - name: config configMap: name: otel-collector-config

apiVersion: v1 kind: Service metadata: name: otel-collector namespace: observability spec: selector: app: otel-collector ports: - name: otlp-grpc port: 4317 targetPort: 4317 - name: otlp-http port: 4318 targetPort: 4318 - name: metrics port: 8888 targetPort: 8888
undefined

apiVersion: apps/v1 kind: Deployment metadata: name: otel-collector namespace: observability spec: replicas: 2 selector: matchLabels: app: otel-collector template: metadata: labels: app: otel-collector spec: containers: - name: collector image: otel/opentelemetry-collector-contrib:0.98.0 args: ["--config=/etc/otel/config.yaml"] ports: - containerPort: 4317 name: otlp-grpc - containerPort: 4318 name: otlp-http - containerPort: 8888 name: metrics resources: requests: cpu: 200m memory: 256Mi limits: cpu: "1" memory: 512Mi volumeMounts: - name: config mountPath: /etc/otel livenessProbe: httpGet: path: / port: 13133 readinessProbe: httpGet: path: / port: 13133 volumes: - name: config configMap: name: otel-collector-config

apiVersion: v1 kind: Service metadata: name: otel-collector namespace: observability spec: selector: app: otel-collector ports: - name: otlp-grpc port: 4317 targetPort: 4317 - name: otlp-http port: 4318 targetPort: 4318 - name: metrics port: 8888 targetPort: 8888
undefined

Python SDK Instrumentation

Python SDK埋点实现

python
undefined
python
undefined

tracing_setup.py

tracing_setup.py

"""Initialize OpenTelemetry tracing and metrics for a Python service.""" from opentelemetry import trace, metrics from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.sdk.metrics import MeterProvider from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader from opentelemetry.sdk.resources import Resource from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter from opentelemetry.instrumentation.requests import RequestsInstrumentor from opentelemetry.instrumentation.flask import FlaskInstrumentor from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor import os
def init_telemetry(service_name: str, service_version: str): """Initialize OTel SDK with traces and metrics.""" resource = Resource.create({ "service.name": service_name, "service.version": service_version, "deployment.environment": os.getenv("DEPLOY_ENV", "development"), })
# Traces
trace_exporter = OTLPSpanExporter(
    endpoint=os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://otel-collector:4317"),
    insecure=True,
)
tracer_provider = TracerProvider(resource=resource)
tracer_provider.add_span_processor(BatchSpanProcessor(trace_exporter))
trace.set_tracer_provider(tracer_provider)

# Metrics
metric_exporter = OTLPMetricExporter(
    endpoint=os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://otel-collector:4317"),
    insecure=True,
)
metric_reader = PeriodicExportingMetricReader(metric_exporter, export_interval_millis=15000)
meter_provider = MeterProvider(resource=resource, metric_readers=[metric_reader])
metrics.set_meter_provider(meter_provider)

# Auto-instrument common libraries
RequestsInstrumentor().instrument()
SQLAlchemyInstrumentor().instrument()

return trace.get_tracer(service_name), metrics.get_meter(service_name)
"""Initialize OpenTelemetry tracing and metrics for a Python service.""" from opentelemetry import trace, metrics from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.sdk.metrics import MeterProvider from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader from opentelemetry.sdk.resources import Resource from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter from opentelemetry.instrumentation.requests import RequestsInstrumentor from opentelemetry.instrumentation.flask import FlaskInstrumentor from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor import os
def init_telemetry(service_name: str, service_version: str): """Initialize OTel SDK with traces and metrics.""" resource = Resource.create({ "service.name": service_name, "service.version": service_version, "deployment.environment": os.getenv("DEPLOY_ENV", "development"), })
# Traces
trace_exporter = OTLPSpanExporter(
    endpoint=os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://otel-collector:4317"),
    insecure=True,
)
tracer_provider = TracerProvider(resource=resource)
tracer_provider.add_span_processor(BatchSpanProcessor(trace_exporter))
trace.set_tracer_provider(tracer_provider)

# Metrics
metric_exporter = OTLPMetricExporter(
    endpoint=os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://otel-collector:4317"),
    insecure=True,
)
metric_reader = PeriodicExportingMetricReader(metric_exporter, export_interval_millis=15000)
meter_provider = MeterProvider(resource=resource, metric_readers=[metric_reader])
metrics.set_meter_provider(meter_provider)

# Auto-instrument common libraries
RequestsInstrumentor().instrument()
SQLAlchemyInstrumentor().instrument()

return trace.get_tracer(service_name), metrics.get_meter(service_name)

Usage example

Usage example

tracer, meter = init_telemetry("order-service", "1.2.0")
tracer, meter = init_telemetry("order-service", "1.2.0")

Custom span

Custom span

with tracer.start_as_current_span("process_order") as span: span.set_attribute("order.id", order_id) span.set_attribute("order.total", total) # ... business logic ...
with tracer.start_as_current_span("process_order") as span: span.set_attribute("order.id", order_id) span.set_attribute("order.total", total) # ... business logic ...

Custom metric

Custom metric

request_counter = meter.create_counter( "app.requests", description="Total application requests", ) request_counter.add(1, {"route": "/api/orders", "method": "POST"})
undefined
request_counter = meter.create_counter( "app.requests", description="Total application requests", ) request_counter.add(1, {"route": "/api/orders", "method": "POST"})
undefined

Node.js SDK Instrumentation

Node.js SDK埋点实现

javascript
// tracing.js
// Initialize OpenTelemetry for a Node.js service.
// Load this file BEFORE any other imports: node -r ./tracing.js app.js
const { NodeSDK } = require("@opentelemetry/sdk-node");
const { OTLPTraceExporter } = require("@opentelemetry/exporter-trace-otlp-grpc");
const { OTLPMetricExporter } = require("@opentelemetry/exporter-metrics-otlp-grpc");
const { PeriodicExportingMetricReader } = require("@opentelemetry/sdk-metrics");
const { getNodeAutoInstrumentations } = require("@opentelemetry/auto-instrumentations-node");
const { Resource } = require("@opentelemetry/resources");
const { ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION } = require("@opentelemetry/semantic-conventions");

const resource = new Resource({
  [ATTR_SERVICE_NAME]: process.env.SERVICE_NAME || "node-service",
  [ATTR_SERVICE_VERSION]: process.env.SERVICE_VERSION || "1.0.0",
  "deployment.environment": process.env.DEPLOY_ENV || "development",
});

const sdk = new NodeSDK({
  resource,
  traceExporter: new OTLPTraceExporter({
    url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || "http://otel-collector:4317",
  }),
  metricReader: new PeriodicExportingMetricReader({
    exporter: new OTLPMetricExporter({
      url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || "http://otel-collector:4317",
    }),
    exportIntervalMillis: 15000,
  }),
  instrumentations: [
    getNodeAutoInstrumentations({
      "@opentelemetry/instrumentation-http": {
        ignoreIncomingPaths: ["/health", "/ready"],
      },
      "@opentelemetry/instrumentation-express": { enabled: true },
      "@opentelemetry/instrumentation-pg": { enabled: true },
      "@opentelemetry/instrumentation-redis": { enabled: true },
    }),
  ],
});

sdk.start();
process.on("SIGTERM", () => sdk.shutdown());
javascript
// tracing.js
// Initialize OpenTelemetry for a Node.js service.
// Load this file BEFORE any other imports: node -r ./tracing.js app.js
const { NodeSDK } = require("@opentelemetry/sdk-node");
const { OTLPTraceExporter } = require("@opentelemetry/exporter-trace-otlp-grpc");
const { OTLPMetricExporter } = require("@opentelemetry/exporter-metrics-otlp-grpc");
const { PeriodicExportingMetricReader } = require("@opentelemetry/sdk-metrics");
const { getNodeAutoInstrumentations } = require("@opentelemetry/auto-instrumentations-node");
const { Resource } = require("@opentelemetry/resources");
const { ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION } = require("@opentelemetry/semantic-conventions");

const resource = new Resource({
  [ATTR_SERVICE_NAME]: process.env.SERVICE_NAME || "node-service",
  [ATTR_SERVICE_VERSION]: process.env.SERVICE_VERSION || "1.0.0",
  "deployment.environment": process.env.DEPLOY_ENV || "development",
});

const sdk = new NodeSDK({
  resource,
  traceExporter: new OTLPTraceExporter({
    url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || "http://otel-collector:4317",
  }),
  metricReader: new PeriodicExportingMetricReader({
    exporter: new OTLPMetricExporter({
      url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || "http://otel-collector:4317",
    }),
    exportIntervalMillis: 15000,
  }),
  instrumentations: [
    getNodeAutoInstrumentations({
      "@opentelemetry/instrumentation-http": {
        ignoreIncomingPaths: ["/health", "/ready"],
      },
      "@opentelemetry/instrumentation-express": { enabled: true },
      "@opentelemetry/instrumentation-pg": { enabled: true },
      "@opentelemetry/instrumentation-redis": { enabled: true },
    }),
  ],
});

sdk.start();
process.on("SIGTERM", () => sdk.shutdown());

Auto-Instrumentation with Kubernetes Operator

使用Kubernetes Operator实现自动埋点

yaml
undefined
yaml
undefined

otel-auto-instrumentation.yaml

otel-auto-instrumentation.yaml

Install the OTel Operator first:

Install the OTel Operator first:

helm install opentelemetry-operator open-telemetry/opentelemetry-operator \

helm install opentelemetry-operator open-telemetry/opentelemetry-operator \

--namespace observability --create-namespace

--namespace observability --create-namespace

Define instrumentation for Python services

Define instrumentation for Python services

apiVersion: opentelemetry.io/v1alpha1 kind: Instrumentation metadata: name: python-instrumentation namespace: default spec: exporter: endpoint: http://otel-collector.observability:4317 propagators: - tracecontext - baggage sampler: type: parentbased_traceidratio argument: "0.25" python: image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-python:0.44b0 env: - name: OTEL_PYTHON_LOG_CORRELATION value: "true"

apiVersion: opentelemetry.io/v1alpha1 kind: Instrumentation metadata: name: python-instrumentation namespace: default spec: exporter: endpoint: http://otel-collector.observability:4317 propagators: - tracecontext - baggage sampler: type: parentbased_traceidratio argument: "0.25" python: image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-python:0.44b0 env: - name: OTEL_PYTHON_LOG_CORRELATION value: "true"

Define instrumentation for Node.js services

Define instrumentation for Node.js services

apiVersion: opentelemetry.io/v1alpha1 kind: Instrumentation metadata: name: nodejs-instrumentation namespace: default spec: exporter: endpoint: http://otel-collector.observability:4317 propagators: - tracecontext - baggage sampler: type: parentbased_traceidratio argument: "0.25" nodejs: image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-nodejs:0.49.1

To instrument a pod, add the annotation:

```yaml
apiVersion: opentelemetry.io/v1alpha1 kind: Instrumentation metadata: name: nodejs-instrumentation namespace: default spec: exporter: endpoint: http://otel-collector.observability:4317 propagators: - tracecontext - baggage sampler: type: parentbased_traceidratio argument: "0.25" nodejs: image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-nodejs:0.49.1

要为Pod添加埋点,添加以下注解:

```yaml

For Python:

For Python:

metadata: annotations: instrumentation.opentelemetry.io/inject-python: "true"
metadata: annotations: instrumentation.opentelemetry.io/inject-python: "true"

For Node.js:

For Node.js:

metadata: annotations: instrumentation.opentelemetry.io/inject-nodejs: "true"
undefined
metadata: annotations: instrumentation.opentelemetry.io/inject-nodejs: "true"
undefined

Sampling Strategies

采样策略

yaml
undefined
yaml
undefined

Tail-based sampling config (in collector)

Tail-based sampling config (in collector)

processors: tail_sampling: decision_wait: 10s num_traces: 100000 policies: # Always keep error traces - name: errors type: status_code status_code: status_codes: [ERROR]
  # Always keep slow traces (> 2s)
  - name: slow-traces
    type: latency
    latency:
      threshold_ms: 2000

  # Sample 10% of successful traces
  - name: normal-traffic
    type: probabilistic
    probabilistic:
      sampling_percentage: 10

  # Always keep traces with specific attributes
  - name: important-users
    type: string_attribute
    string_attribute:
      key: user.tier
      values: [enterprise, premium]

  # Rate limit per service to prevent one service from dominating
  - name: rate-limit
    type: rate_limiting
    rate_limiting:
      spans_per_second: 500
undefined
processors: tail_sampling: decision_wait: 10s num_traces: 100000 policies: # Always keep error traces - name: errors type: status_code status_code: status_codes: [ERROR]
  # Always keep slow traces (> 2s)
  - name: slow-traces
    type: latency
    latency:
      threshold_ms: 2000

  # Sample 10% of successful traces
  - name: normal-traffic
    type: probabilistic
    probabilistic:
      sampling_percentage: 10

  # Always keep traces with specific attributes
  - name: important-users
    type: string_attribute
    string_attribute:
      key: user.tier
      values: [enterprise, premium]

  # Rate limit per service to prevent one service from dominating
  - name: rate-limit
    type: rate_limiting
    rate_limiting:
      spans_per_second: 500
undefined

Best Practices

最佳实践

  • Use tail-based sampling for high-volume production traces.
  • Tag telemetry with
    service.name
    ,
    service.version
    , and
    deployment.environment
    .
  • Drop noisy attributes early in the collector.
  • Keep metric label cardinality low for stable query performance.
  • Use resource detectors to automatically populate cloud metadata.
  • Separate collector pools for traces vs metrics if volume requires it.
  • Set memory_limiter on every collector pipeline to prevent OOM.
  • Use the contrib collector image for production (includes more receivers/exporters).
  • 针对高流量生产环境使用基于尾部的采样策略
  • 为遥测数据添加
    service.name
    、
    service.version
    和
    deployment.environment
    标签
  • 在Collector中尽早过滤冗余属性
  • 降低指标标签基数以保证查询性能稳定
  • 使用资源探测器自动填充云环境元数据
  • 若数据量较大,为链路追踪和指标分别设置独立的Collector池
  • 为每个Collector流水线设置memory_limiter以防止内存溢出
  • 生产环境使用contrib Collector镜像(包含更多接收器/导出器)

Troubleshooting

故障排查

SymptomCheckFix
No traces arriving at backendCollector logs for export errorsVerify endpoint URL and network policy
Missing spans in a tracePropagation headers stripped by proxyConfigure proxy to pass
traceparent
header
High memory on collectorToo many in-flight traces for tail samplingReduce
num_traces
or increase memory limit
Metric cardinality explosionUnbounded label values (user IDs, URLs)Add transform processor to normalize values
Auto-instrumentation not workingPod annotation missing or operator not runningVerify operator is healthy and annotation is correct
Duplicate metricsBoth SDK and auto-instrumentation activeUse only one instrumentation method per signal
症状检查项修复方案
后端未收到链路数据Collector导出错误日志验证端点URL与网络策略
链路中缺少Span代理剥离了传播头配置代理允许传递
traceparent
头
Collector内存占用过高尾部采样的在途链路过多减少
num_traces
或提高内存限制
指标基数爆炸存在无界标签值(如用户ID、URL)添加transform处理器标准化值
自动埋点不生效Pod注解缺失或Operator未运行验证Operator健康状态与注解正确性
指标重复SDK与自动埋点同时启用每个信号仅使用一种埋点方式

Related Skills

相关技能

  • prometheus-grafana - Dashboarding and alerting
  • datadog - Managed observability backend
  • alerting-oncall - On-call routing and escalation
  • rag-observability-evals - RAG-specific observability
  • agent-observability - AI agent tracing
  • prometheus-grafana - 仪表盘与告警
  • datadog - 托管式可观测性后端
  • alerting-oncall - 值班路由与升级
  • rag-observability-evals - RAG专属可观测性
  • agent-observability - AI Agent链路追踪