Loading...
Loading...
Compare original and translation side by side
| Component | Purpose |
|---|---|
| Distributor | Validates requests, preprocesses labels, routes to ingesters |
| Ingester | Buffers logs in memory, compresses into chunks, writes to storage |
| Querier | Executes LogQL queries from ingesters and storage |
| Query Frontend | Accelerates queries via splitting, caching, scheduling |
| Query Scheduler | Manages per-tenant query queues for fairness |
| Index Gateway | Serves index queries for TSDB stores |
| Compactor | Merges index files, manages retention, handles deletion |
| Ruler | Evaluates alerting and recording rules |
| 组件 | 用途 |
|---|---|
| Distributor | 验证请求、预处理标签、将请求路由至Ingester |
| Ingester | 在内存中缓冲日志,压缩为块,写入存储 |
| Querier | 从Ingester和存储中执行LogQL查询 |
| Query Frontend | 通过拆分、缓存、调度加速查询 |
| Query Scheduler | 管理每个租户的查询队列,确保公平性 |
| Index Gateway | 为TSDB存储提供索引查询服务 |
| Compactor | 合并索引文件、管理日志保留、处理删除请求 |
| Ruler | 评估告警与记录规则 |
Log Source → Distributor → Ingester → Object Storage
↓
Chunks + IndexesQuery → Query Frontend → Query Scheduler → Querier
↓
Ingesters + Storage日志源 → Distributor → Ingester → 对象存储
↓
日志块 + 索引查询 → Query Frontend → Query Scheduler → Querier
↓
Ingesters + 存储-target=all-target=alldeploymentMode: SimpleScalable
write:
replicas: 3 # Distributor + Ingester
read:
replicas: 2 # Query Frontend + Querier
backend:
replicas: 2 # Compactor + Index Gateway + Query Scheduler + RulerdeploymentMode: SimpleScalable
write:
replicas: 3 # Distributor + Ingester
read:
replicas: 2 # Query Frontend + Querier
backend:
replicas: 2 # Compactor + Index Gateway + Query Scheduler + RulerdeploymentMode: Distributed
ingester:
replicas: 3
zoneAwareReplication:
enabled: true
distributor:
replicas: 3
querier:
replicas: 3
queryFrontend:
replicas: 2
queryScheduler:
replicas: 2
compactor:
replicas: 1
indexGateway:
replicas: 2deploymentMode: Distributed
ingester:
replicas: 3
zoneAwareReplication:
enabled: true
distributor:
replicas: 3
querier:
replicas: 3
queryFrontend:
replicas: 2
queryScheduler:
replicas: 2
compactor:
replicas: 1
indexGateway:
replicas: 2loki:
schemaConfig:
configs:
- from: "2024-04-01"
store: tsdb
object_store: azure # or s3, gcs
schema: v13
index:
prefix: loki_index_
period: 24hloki:
schemaConfig:
configs:
- from: "2024-04-01"
store: tsdb
object_store: azure # 或 s3, gcs
schema: v13
index:
prefix: loki_index_
period: 24hloki:
storage:
type: azure
bucketNames:
chunks: loki-chunks
ruler: loki-ruler
admin: loki-admin
azure:
accountName: <storage-account-name>
# Option 1: User-Assigned Managed Identity (Recommended)
useManagedIdentity: true
useFederatedToken: false
userAssignedId: <identity-client-id>
# Option 2: Account Key (Dev only)
# accountKey: <account-key>
requestTimeout: 30sloki:
storage:
type: azure
bucketNames:
chunks: loki-chunks
ruler: loki-ruler
admin: loki-admin
azure:
accountName: <storage-account-name>
# 选项1:用户分配的托管标识(推荐)
useManagedIdentity: true
useFederatedToken: false
userAssignedId: <identity-client-id>
# 选项2:账户密钥(仅开发环境使用)
# accountKey: <account-key>
requestTimeout: 30sloki:
storage:
type: s3
bucketNames:
chunks: my-loki-chunks-2024
ruler: my-loki-ruler-2024
admin: my-loki-admin-2024
s3:
endpoint: s3.us-east-1.amazonaws.com
region: us-east-1
# Use IAM roles or access keys
accessKeyId: <access-key>
secretAccessKey: <secret-key>
s3ForcePathStyle: falseloki:
storage:
type: s3
bucketNames:
chunks: my-loki-chunks-2024
ruler: my-loki-ruler-2024
admin: my-loki-admin-2024
s3:
endpoint: s3.us-east-1.amazonaws.com
region: us-east-1
# 使用IAM角色或访问密钥
accessKeyId: <access-key>
secretAccessKey: <secret-key>
s3ForcePathStyle: falseloki:
storage:
type: gcs
bucketNames:
chunks: my-loki-gcs-bucket
gcs:
bucketName: my-loki-gcs-bucket
# Uses Workload Identity or service accountloki:
storage:
type: gcs
bucketNames:
chunks: my-loki-gcs-bucket
gcs:
bucketName: my-loki-gcs-bucket
# 使用工作负载身份或服务账户loki:
ingester:
chunk_encoding: snappy # Recommended (fast + efficient)
chunk_target_size: 1572864 # ~1.5MB compressed
max_chunk_age: 2h # Max time before flush
chunk_idle_period: 30m # Flush idle chunks
flush_check_period: 30s
flush_op_timeout: 10m| Setting | Recommended | Purpose |
|---|---|---|
| snappy | Best speed-to-compression balance |
| 1.5MB | Target compressed chunk size |
| 2h | Limits memory and data loss exposure |
| 30m | Flushes inactive streams |
loki:
ingester:
chunk_encoding: snappy # 推荐(速度与效率平衡)
chunk_target_size: 1572864 # ~1.5MB 压缩后大小
max_chunk_age: 2h # 刷新前的最长时间
chunk_idle_period: 30m # 刷新闲置日志块
flush_check_period: 30s
flush_op_timeout: 10m| 设置项 | 推荐值 | 用途 |
|---|---|---|
| snappy | 最佳的速度与压缩比平衡 |
| 1.5MB | 压缩后日志块的目标大小 |
| 2h | 限制内存占用和数据丢失风险 |
| 30m | 刷新不活跃的日志流 |
loki:
limits_config:
# Retention
retention_period: 744h # 31 days
# Ingestion limits
ingestion_rate_mb: 50
ingestion_burst_size_mb: 100
per_stream_rate_limit: 3MB
per_stream_rate_limit_burst: 15MB
# Query limits
max_query_series: 10000
max_query_lookback: 720h
max_entries_limit_per_query: 10000
# Required for OTLP
allow_structured_metadata: true
volume_enabled: true
# Sample rejection
reject_old_samples: true
reject_old_samples_max_age: 168h # 7 days
max_label_names_per_series: 25loki:
limits_config:
# 日志保留
retention_period: 744h # 31天
# 写入限制
ingestion_rate_mb: 50
ingestion_burst_size_mb: 100
per_stream_rate_limit: 3MB
per_stream_rate_limit_burst: 15MB
# 查询限制
max_query_series: 10000
max_query_lookback: 720h
max_entries_limit_per_query: 10000
# OTLP 所需配置
allow_structured_metadata: true
volume_enabled: true
# 样本拒绝
reject_old_samples: true
reject_old_samples_max_age: 168h # 7天
max_label_names_per_series: 25loki:
compactor:
retention_enabled: true
retention_delete_delay: 2h
retention_delete_worker_count: 50
compaction_interval: 10m
delete_request_store: azure # Match your storage typeloki:
compactor:
retention_enabled: true
retention_delete_delay: 2h
retention_delete_worker_count: 50
compaction_interval: 10m
delete_request_store: azure # 匹配你的存储类型undefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedexporters:
otlphttp:
endpoint: http://loki-gateway:3100/otlp
headers:
X-Scope-OrgID: "my-tenant"
service:
pipelines:
logs:
receivers: [otlp]
exporters: [otlphttp]loki:
limits_config:
allow_structured_metadata: true # Required for OTLPexporters:
otlphttp:
endpoint: http://loki-gateway:3100/otlp
headers:
X-Scope-OrgID: "my-tenant"
service:
pipelines:
logs:
receivers: [otlp]
exporters: [otlphttp]loki:
limits_config:
allow_structured_metadata: true # OTLP 所需配置| OTLP Attribute | Loki Label |
|---|---|
| |
| |
| |
| |
| |
| OTLP 属性 | Loki 标签 |
|---|---|
| |
| |
| |
| |
| |
helm repo add grafana https://grafana.github.io/helm-charts
helm repo updatehelm repo add grafana https://grafana.github.io/helm-charts
helm repo updatehelm install loki grafana/loki \
--namespace monitoring \
--values values.yamlhelm install loki grafana/loki \
--namespace monitoring \
--values values.yamldeploymentMode: Distributed
loki:
auth_enabled: true
schemaConfig:
configs:
- from: "2024-04-01"
store: tsdb
object_store: azure
schema: v13
index:
prefix: loki_index_
period: 24h
storage:
type: azure
azure:
accountName: mystorageaccount
useManagedIdentity: true
userAssignedId: <client-id>
bucketNames:
chunks: loki-chunks
ruler: loki-ruler
admin: loki-admin
limits_config:
retention_period: 2160h # 90 days
allow_structured_metadata: true
ingester:
replicas: 3
zoneAwareReplication:
enabled: true
resources:
requests:
cpu: 2
memory: 8Gi
limits:
cpu: 4
memory: 16Gi
querier:
replicas: 3
maxUnavailable: 2
queryFrontend:
replicas: 2
distributor:
replicas: 3
compactor:
replicas: 1
indexGateway:
replicas: 2
maxUnavailable: 1deploymentMode: Distributed
loki:
auth_enabled: true
schemaConfig:
configs:
- from: "2024-04-01"
store: tsdb
object_store: azure
schema: v13
index:
prefix: loki_index_
period: 24h
storage:
type: azure
azure:
accountName: mystorageaccount
useManagedIdentity: true
userAssignedId: <client-id>
bucketNames:
chunks: loki-chunks
ruler: loki-ruler
admin: loki-admin
limits_config:
retention_period: 2160h # 90天
allow_structured_metadata: true
ingester:
replicas: 3
zoneAwareReplication:
enabled: true
resources:
requests:
cpu: 2
memory: 8Gi
limits:
cpu: 4
memory: 16Gi
querier:
replicas: 3
maxUnavailable: 2
queryFrontend:
replicas: 2
distributor:
replicas: 3
compactor:
replicas: 1
indexGateway:
replicas: 2
maxUnavailable: 1undefinedundefinedaz identity create \
--name loki-identity \
--resource-group <rg>
IDENTITY_CLIENT_ID=$(az identity show --name loki-identity --resource-group <rg> --query clientId -o tsv)
IDENTITY_PRINCIPAL_ID=$(az identity show --name loki-identity --resource-group <rg> --query principalId -o tsv)az vmss identity assign \
--resource-group <aks-node-rg> \
--name <vmss-name> \
--identities /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/loki-identityaz role assignment create \
--role "Storage Blob Data Contributor" \
--assignee-object-id $IDENTITY_PRINCIPAL_ID \
--scope /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<storage>loki:
storage:
azure:
useManagedIdentity: true
userAssignedId: <IDENTITY_CLIENT_ID>az identity create \
--name loki-identity \
--resource-group <rg>
IDENTITY_CLIENT_ID=$(az identity show --name loki-identity --resource-group <rg> --query clientId -o tsv)
IDENTITY_PRINCIPAL_ID=$(az identity show --name loki-identity --resource-group <rg> --query principalId -o tsv)az vmss identity assign \
--resource-group <aks-node-rg> \
--name <vmss-name> \
--identities /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/loki-identityaz role assignment create \
--role "Storage Blob Data Contributor" \
--assignee-object-id $IDENTITY_PRINCIPAL_ID \
--scope /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<storage>loki:
storage:
azure:
useManagedIdentity: true
userAssignedId: <IDENTITY_CLIENT_ID>loki:
auth_enabled: trueloki:
auth_enabled: trueundefinedundefinedundefinedundefined
**2. Authorization Failure (Azure)**
```bash
**2. 授权失败(Azure)**
```bash
**3. Ingester OOM**
```yaml
**3. Ingester 内存溢出**
```yaml
**4. Query Timeout**
```yaml
loki:
querier:
query_timeout: 5m
max_concurrent: 8
query_scheduler:
max_outstanding_requests_per_tenant: 2048
**4. 查询超时**
```yaml
loki:
querier:
query_timeout: 5m
max_concurrent: 8
query_scheduler:
max_outstanding_requests_per_tenant: 2048undefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedGET /ready
GET /metricsGET /ready
GET /metrics