Back to Details

ebpf-observability

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

eBPF Observability

eBPF可观测性

eBPF (extended Berkeley Packet Filter) allows you to run sandboxed programs in the Linux kernel without modifying kernel source code or loading kernel modules. This skill covers using eBPF for deep observability, network monitoring, and security enforcement across cloud-native infrastructure.
eBPF（extended Berkeley Packet Filter，扩展伯克利数据包过滤器）允许你在Linux内核中运行沙箱化程序，无需修改内核源代码或加载内核模块。本技能内容涵盖如何在云原生基础设施中使用eBPF实现深度可观测性、网络监控和安全管控。

1. When to Use

1. 适用场景

Use eBPF-based observability when you need:
  • Deep performance debugging -- trace kernel-level latency, syscall overhead, and scheduling delays that application-level metrics cannot reveal.
  • Network observability without sidecars -- capture L3/L4/L7 flows, DNS queries, and TCP state transitions directly from the kernel, eliminating the CPU and memory overhead of sidecar proxies.
  • Security monitoring at the kernel boundary -- detect container escapes, unexpected process execution, sensitive file access, and anomalous syscall patterns in real time.
  • Continuous profiling in production -- generate CPU flame graphs and memory allocation profiles with negligible overhead (typically under 1% CPU).
  • Service mesh replacement or augmentation -- Cilium can replace kube-proxy and provide identity-aware network policies enforced at the kernel level.
Avoid eBPF when your kernel version is below 4.19, when you are running on managed platforms that restrict BPF capabilities, or when your debugging needs are fully met by application-level tracing.
在以下场景中适合使用基于eBPF的可观测性：
  • 深度性能调试——追踪应用级指标无法揭示的内核级延迟、系统调用开销和调度延迟。
  • 无需Sidecar的网络可观测性——直接从内核捕获L3/L4/L7流量、DNS查询和TCP状态转换，消除Sidecar代理带来的CPU和内存开销。
  • 内核边界的安全监控——实时检测容器逃逸、意外进程执行、敏感文件访问和异常系统调用模式。
  • 生产环境持续剖析——以极低的开销（通常CPU占用低于1%）生成CPU火焰图和内存分配剖析报告。
  • 服务网格替代或增强——Cilium可以替代kube-proxy，并在内核层面执行基于身份的网络策略。
当你的内核版本低于4.19、运行在限制BPF功能的托管平台上，或者你的调试需求可完全通过应用级追踪满足时，避免使用eBPF。

2. Prerequisites

2. 前置条件

Kernel Version Requirements

内核版本要求

FeatureMinimum KernelRecommended Kernel
Basic BPF maps & probes4.95.10+
BPF CO-RE (BTF support)5.25.10+
BPF ring buffer5.85.10+
BPF LSM hooks5.75.15+
Cilium full features4.195.10+
Tetragon4.195.13+
功能最低内核版本推荐内核版本
基础BPF映射与探针4.95.10+
BPF CO-RE（支持BTF）5.25.10+
BPF环形缓冲区5.85.10+
BPF LSM钩子5.75.15+
Cilium完整功能4.195.10+
Tetragon4.195.13+

Verify Kernel Support

验证内核支持

bash
undefined
bash
undefined

Check kernel version

检查内核版本

uname -r
uname -r

Verify BTF (BPF Type Format) is enabled -- required for CO-RE

验证BTF（BPF类型格式）是否启用——CO-RE必需

ls /sys/kernel/btf/vmlinux
ls /sys/kernel/btf/vmlinux

Check BPF filesystem is mounted

检查BPF文件系统是否已挂载

mount | grep bpf
mount | grep bpf

If not mounted, mount it

若未挂载，执行挂载

sudo mount -t bpf bpf /sys/fs/bpf
sudo mount -t bpf bpf /sys/fs/bpf

Verify BPF JIT is enabled

验证BPF JIT是否启用

cat /proc/sys/net/core/bpf_jit_enable
cat /proc/sys/net/core/bpf_jit_enable

Should return 1; if not:

应返回1；若未返回：

sudo sysctl net.core.bpf_jit_enable=1
undefined
sudo sysctl net.core.bpf_jit_enable=1
undefined

Install Toolchain

安装工具链

bash
undefined
bash
undefined

Ubuntu/Debian -- install bpftrace, bcc tools, and libbpf

Ubuntu/Debian —— 安装bpftrace、bcc工具和libbpf

sudo apt-get update sudo apt-get install -y bpftrace bpfcc-tools libbpf-dev linux-headers-$(uname -r)
sudo apt-get update sudo apt-get install -y bpftrace bpfcc-tools libbpf-dev linux-headers-$(uname -r)

Fedora/RHEL

Fedora/RHEL

sudo dnf install -y bpftrace bcc-tools libbpf-devel kernel-devel
sudo dnf install -y bpftrace bcc-tools libbpf-devel kernel-devel

Verify bpftrace works

验证bpftrace可用

sudo bpftrace -e 'BEGIN { printf("eBPF is working\n"); exit(); }'

---
sudo bpftrace -e 'BEGIN { printf("eBPF is working\n"); exit(); }'

---

3. Cilium Setup

3. Cilium部署

Cilium replaces kube-proxy with eBPF-based networking, providing identity-aware security and deep network observability via Hubble.
Cilium使用基于eBPF的网络替代kube-proxy，通过Hubble提供基于身份的安全和深度网络可观测性。

Install Cilium on Kubernetes

在Kubernetes上安装Cilium

bash
undefined
bash
undefined

Add the Cilium Helm repo

添加Cilium Helm仓库

helm repo add cilium https://helm.cilium.io/ helm repo update
helm repo add cilium https://helm.cilium.io/ helm repo update

Install Cilium with Hubble enabled

安装启用Hubble的Cilium

helm install cilium cilium/cilium --version 1.16.4
--namespace kube-system
--set kubeProxyReplacement=true
--set k8sServiceHost="${API_SERVER_IP}"
--set k8sServicePort="${API_SERVER_PORT}"
--set hubble.enabled=true
--set hubble.relay.enabled=true
--set hubble.ui.enabled=true
--set hubble.metrics.enableOpenMetrics=true
--set hubble.metrics.enabled="{dns,drop,tcp,flow,port-distribution,icmp,httpV2:exemplars=true;labelsContext=source_ip,source_namespace,source_workload,destination_ip,destination_namespace,destination_workload}"
helm install cilium cilium/cilium --version 1.16.4
--namespace kube-system
--set kubeProxyReplacement=true
--set k8sServiceHost="${API_SERVER_IP}"
--set k8sServicePort="${API_SERVER_PORT}"
--set hubble.enabled=true
--set hubble.relay.enabled=true
--set hubble.ui.enabled=true
--set hubble.metrics.enableOpenMetrics=true
--set hubble.metrics.enabled="{dns,drop,tcp,flow,port-distribution,icmp,httpV2:exemplars=true;labelsContext=source_ip,source_namespace,source_workload,destination_ip,destination_namespace,destination_workload}"

Wait for Cilium to be ready

等待Cilium就绪

cilium status --wait
undefined
cilium status --wait
undefined

Install the Cilium CLI and Hubble CLI

安装Cilium CLI和Hubble CLI

bash
undefined
bash
undefined

Cilium CLI

Cilium CLI

CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt) curl -L --remote-name "https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-amd64.tar.gz" sudo tar xzvf cilium-linux-amd64.tar.gz -C /usr/local/bin rm cilium-linux-amd64.tar.gz
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt) curl -L --remote-name "https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-amd64.tar.gz" sudo tar xzvf cilium-linux-amd64.tar.gz -C /usr/local/bin rm cilium-linux-amd64.tar.gz

Hubble CLI

Hubble CLI

HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt) curl -L --remote-name "https://github.com/cilium/hubble/releases/download/${HUBBLE_VERSION}/hubble-linux-amd64.tar.gz" sudo tar xzvf hubble-linux-amd64.tar.gz -C /usr/local/bin rm hubble-linux-amd64.tar.gz
undefined
HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt) curl -L --remote-name "https://github.com/cilium/hubble/releases/download/${HUBBLE_VERSION}/hubble-linux-amd64.tar.gz" sudo tar xzvf hubble-linux-amd64.tar.gz -C /usr/local/bin rm hubble-linux-amd64.tar.gz
undefined

Hubble Network Observability

Hubble网络可观测性

bash
undefined
bash
undefined

Port-forward the Hubble Relay

端口转发Hubble Relay

cilium hubble port-forward &
cilium hubble port-forward &

Observe all flows in real time

实时观测所有流量

hubble observe --follow
hubble observe --follow

Filter flows by namespace

按命名空间过滤流量

hubble observe --namespace production --follow
hubble observe --namespace production --follow

Filter by verdict (dropped traffic)

按裁决结果过滤（被丢弃的流量）

hubble observe --verdict DROPPED --follow
hubble observe --verdict DROPPED --follow

Filter by DNS queries

按DNS查询过滤

hubble observe --protocol DNS --follow
hubble observe --protocol DNS --follow

Filter HTTP traffic to a specific service

过滤流向特定服务的HTTP流量

hubble observe --to-label "app=api-server" --protocol HTTP --follow
hubble observe --to-label "app=api-server" --protocol HTTP --follow

Export flows as JSON for ingestion into SIEM

将流量导出为JSON格式以导入SIEM系统

hubble observe --output json --last 1000 > flows.json
undefined
hubble observe --output json --last 1000 > flows.json
undefined

Hubble UI Access

访问Hubble UI

bash
undefined
bash
undefined

Port-forward the Hubble UI

端口转发Hubble UI

kubectl port-forward -n kube-system svc/hubble-ui 12000:80
kubectl port-forward -n kube-system svc/hubble-ui 12000:80

Access at http://localhost:12000 -- provides a real-time service dependency map

访问地址：http://localhost:12000 —— 提供实时服务依赖关系图


---

---

4. Tetragon for Security

4. Tetragon安全管控

Tetragon is Cilium's runtime security enforcement engine. It uses eBPF to observe and enforce security policies at the kernel level with zero application changes.
Tetragon是Cilium的运行时安全执行引擎。它使用eBPF在内核层面观测并执行安全策略，无需修改应用代码。

Install Tetragon

安装Tetragon

bash
helm repo add cilium https://helm.cilium.io/
helm repo update

helm install tetragon cilium/tetragon \
  --namespace kube-system \
  --set tetragon.grpc.enabled=true \
  --set tetragon.exportFilename=/var/run/cilium/tetragon/tetragon.log
bash
helm repo add cilium https://helm.cilium.io/
helm repo update

helm install tetragon cilium/tetragon \
  --namespace kube-system \
  --set tetragon.grpc.enabled=true \
  --set tetragon.exportFilename=/var/run/cilium/tetragon/tetragon.log

Install the tetra CLI

安装tetra CLI

curl -LO "https://github.com/cilium/tetragon/releases/latest/download/tetra-linux-amd64.tar.gz" sudo tar xzvf tetra-linux-amd64.tar.gz -C /usr/local/bin rm tetra-linux-amd64.tar.gz
undefined
curl -LO "https://github.com/cilium/tetragon/releases/latest/download/tetra-linux-amd64.tar.gz" sudo tar xzvf tetra-linux-amd64.tar.gz -C /usr/local/bin rm tetra-linux-amd64.tar.gz
undefined

Process Execution Monitoring

进程执行监控

yaml
undefined
yaml
undefined

process-monitor.yaml -- TracingPolicy to monitor all process executions

process-monitor.yaml —— 监控所有进程执行的TracingPolicy

apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: process-execution-monitor spec: kprobes: [] tracepoints: [] uprobes: [] enforcers: []

process_exec and process_exit events are always emitted by default

Use tetra CLI to observe them:


```bash
apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: process-execution-monitor spec: kprobes: [] tracepoints: [] uprobes: [] enforcers: []

process_exec和process_exit事件默认始终会被触发

使用tetra CLI观测这些事件：


```bash

Watch all process executions cluster-wide

监控集群内所有进程执行情况

kubectl exec -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact --process-exec
kubectl exec -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact --process-exec

Filter to a specific namespace

过滤特定命名空间的进程

kubectl exec -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact
--namespace production
undefined
kubectl exec -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact
--namespace production
undefined

File Access Tracking

文件访问追踪

yaml
undefined
yaml
undefined

file-access-policy.yaml -- detect reads/writes to sensitive files

file-access-policy.yaml —— 检测对敏感文件的读/写操作

apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: sensitive-file-access spec: kprobes: - call: "security_file_open" syscall: false args: - index: 0 type: "file" selectors: - matchArgs: - index: 0 operator: "Prefix" values: - "/etc/shadow" - "/etc/passwd" - "/etc/kubernetes/pki" - "/var/run/secrets/kubernetes.io" - "/root/.ssh"

```bash
kubectl apply -f file-access-policy.yaml
apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: sensitive-file-access spec: kprobes: - call: "security_file_open" syscall: false args: - index: 0 type: "file" selectors: - matchArgs: - index: 0 operator: "Prefix" values: - "/etc/shadow" - "/etc/passwd" - "/etc/kubernetes/pki" - "/var/run/secrets/kubernetes.io" - "/root/.ssh"

```bash
kubectl apply -f file-access-policy.yaml

Observe file access events

观测文件访问事件

kubectl exec -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact
| grep "sensitive-file-access"
undefined
kubectl exec -n kube-system ds/tetragon -c tetragon -- tetra getevents -o compact
| grep "sensitive-file-access"
undefined

Network Connection Enforcement

网络连接管控

yaml
undefined
yaml
undefined

restrict-egress.yaml -- block unexpected outbound connections

restrict-egress.yaml —— 阻止意外的出站连接

apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: restrict-egress-connections spec: kprobes: - call: "tcp_connect" syscall: false args: - index: 0 type: "sock" selectors: - matchArgs: - index: 0 operator: "DAddr" values: - "169.254.169.254" # Block IMDS access matchActions: - action: Sigkill - matchNamespaces: - namespace: Mnt operator: NotIn values: - "host_mnt"

```bash
kubectl apply -f restrict-egress.yaml
apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: restrict-egress-connections spec: kprobes: - call: "tcp_connect" syscall: false args: - index: 0 type: "sock" selectors: - matchArgs: - index: 0 operator: "DAddr" values: - "169.254.169.254" # 阻止IMDS访问 matchActions: - action: Sigkill - matchNamespaces: - namespace: Mnt operator: NotIn values: - "host_mnt"

```bash
kubectl apply -f restrict-egress.yaml

Privileged Escalation Detection

特权升级检测

yaml
undefined
yaml
undefined

detect-privilege-escalation.yaml

detect-privilege-escalation.yaml

apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: detect-privilege-escalation spec: kprobes: - call: "__x64_sys_setuid" syscall: true args: - index: 0 type: "int" selectors: - matchArgs: - index: 0 operator: "Equal" values: - "0" matchActions: - action: Post rateLimit: "1m" - call: "__x64_sys_setns" syscall: true args: - index: 1 type: "int" selectors: - matchActions: - action: Post

```bash
kubectl apply -f detect-privilege-escalation.yaml
apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: detect-privilege-escalation spec: kprobes: - call: "__x64_sys_setuid" syscall: true args: - index: 0 type: "int" selectors: - matchArgs: - index: 0 operator: "Equal" values: - "0" matchActions: - action: Post rateLimit: "1m" - call: "__x64_sys_setns" syscall: true args: - index: 1 type: "int" selectors: - matchActions: - action: Post

```bash
kubectl apply -f detect-privilege-escalation.yaml

5. bpftrace One-Liners

5. bpftrace单行命令

These are practical bpftrace commands you can run directly in production for targeted debugging.
这些是可直接在生产环境运行的实用bpftrace命令，用于针对性调试。

Syscall Latency

系统调用延迟

bash
undefined
bash
undefined

Trace read() syscall latency distribution (microseconds)

追踪read()系统调用延迟分布（微秒）

sudo bpftrace -e 'tracepoint:syscalls:sys_enter_read { @start[tid] = nsecs; } tracepoint:syscalls:sys_exit_read /@start[tid]/ { @usecs = hist((nsecs - @start[tid]) / 1000); delete(@start[tid]); }'
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_read { @start[tid] = nsecs; } tracepoint:syscalls:sys_exit_read /@start[tid]/ { @usecs = hist((nsecs - @start[tid]) / 1000); delete(@start[tid]); }'

Top 10 slowest syscalls by total time

按总耗时统计Top 10慢系统调用

sudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @start[tid] = nsecs; } tracepoint:raw_syscalls:sys_exit /@start[tid]/ { @ns[probe] = sum(nsecs - @start[tid]); delete(@start[tid]); } END { print(@ns, 10); }'
undefined
sudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @start[tid] = nsecs; } tracepoint:raw_syscalls:sys_exit /@start[tid]/ { @ns[probe] = sum(nsecs - @start[tid]); delete(@start[tid]); } END { print(@ns, 10); }'
undefined

DNS Tracing

DNS追踪

bash
undefined
bash
undefined

Trace DNS queries via UDP port 53 sends

通过UDP端口53发送追踪DNS查询

sudo bpftrace -e 'kprobe:udp_sendmsg { $sk = (struct sock *)arg0; $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) << 8); if ($dport == 53) { printf("%-8d %-16s DNS query to %s\n", pid, comm, ntop($sk->__sk_common.skc_daddr)); } }'
sudo bpftrace -e 'kprobe:udp_sendmsg { $sk = (struct sock *)arg0; $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) << 8); if ($dport == 53) { printf("%-8d %-16s DNS query to %s\n", pid, comm, ntop($sk->__sk_common.skc_daddr)); } }'

Count DNS queries by source process

按源进程统计DNS查询次数

sudo bpftrace -e 'kprobe:udp_sendmsg { $sk = (struct sock *)arg0; $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) << 8); if ($dport == 53) { @dns[comm] = count(); } }'
undefined
sudo bpftrace -e 'kprobe:udp_sendmsg { $sk = (struct sock *)arg0; $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) << 8); if ($dport == 53) { @dns[comm] = count(); } }'
undefined

TCP Retransmits

TCP重传

bash
undefined
bash
undefined

Trace TCP retransmits with source/destination

追踪带有源/目标信息的TCP重传

sudo bpftrace -e 'kprobe:tcp_retransmit_skb { $sk = (struct sock *)arg0; $daddr = ntop($sk->__sk_common.skc_daddr); $saddr = ntop($sk->__sk_common.skc_rcv_saddr); $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) << 8); $sport = $sk->__sk_common.skc_num; printf("%-20s %-6d -> %-20s %-6d (%s)\n", $saddr, $sport, $daddr, $dport, comm); }'
undefined
sudo bpftrace -e 'kprobe:tcp_retransmit_skb { $sk = (struct sock *)arg0; $daddr = ntop($sk->__sk_common.skc_daddr); $saddr = ntop($sk->__sk_common.skc_rcv_saddr); $dport = ($sk->__sk_common.skc_dport >> 8) | (($sk->__sk_common.skc_dport & 0xff) << 8); $sport = $sk->__sk_common.skc_num; printf("%-20s %-6d -> %-20s %-6d (%s)\n", $saddr, $sport, $daddr, $dport, comm); }'
undefined

Disk I/O Latency

磁盘I/O延迟

bash
undefined
bash
undefined

Block I/O latency histogram by device

按设备统计块I/O延迟直方图

sudo bpftrace -e 'tracepoint:block:block_rq_issue { @start[args->dev, args->sector] = nsecs; } tracepoint:block:block_rq_complete /@start[args->dev, args->sector]/ { @usecs[args->dev] = hist((nsecs - @start[args->dev, args->sector]) / 1000); delete(@start[args->dev, args->sector]); }'
sudo bpftrace -e 'tracepoint:block:block_rq_issue { @start[args->dev, args->sector] = nsecs; } tracepoint:block:block_rq_complete /@start[args->dev, args->sector]/ { @usecs[args->dev] = hist((nsecs - @start[args->dev, args->sector]) / 1000); delete(@start[args->dev, args->sector]); }'

Top processes by disk I/O bytes

按进程统计Top磁盘I/O字节数

sudo bpftrace -e 'tracepoint:block:block_rq_issue { @bytes[comm] = sum(args->bytes); } interval:s:5 { print(@bytes, 10); clear(@bytes); }'
undefined
sudo bpftrace -e 'tracepoint:block:block_rq_issue { @bytes[comm] = sum(args->bytes); } interval:s:5 { print(@bytes, 10); clear(@bytes); }'
undefined

Container-Aware Tracing

容器感知追踪

bash
undefined
bash
undefined

Trace process exec inside containers (cgroup-filtered)

追踪容器内的进程执行（按cgroup过滤）

sudo bpftrace -e 'tracepoint:syscalls:sys_enter_execve { printf("%-8d %-8d %-16s %s\n", pid, cgroup, comm, str(args->filename)); }'
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_execve { printf("%-8d %-8d %-16s %s\n", pid, cgroup, comm, str(args->filename)); }'

Memory allocation hotspots per container

按容器统计内存分配热点

sudo bpftrace -e 'kprobe:__alloc_pages { @pages[cgroup] = count(); } interval:s:10 { print(@pages, 10); clear(@pages); }'

---
sudo bpftrace -e 'kprobe:__alloc_pages { @pages[cgroup] = count(); } interval:s:10 { print(@pages, 10); clear(@pages); }'

---

6. Prometheus Integration

6. Prometheus集成

Hubble Metrics for Prometheus

Hubble指标对接Prometheus

Hubble automatically exposes Prometheus metrics when configured in the Cilium Helm install. Verify the metrics endpoint:
bash
undefined
在Cilium Helm安装配置中启用后，Hubble会自动暴露Prometheus指标。验证指标端点：
bash
undefined

Check that Hubble metrics are being served

检查Hubble指标是否已提供

kubectl exec -n kube-system ds/cilium -- curl -s http://localhost:9965/metrics | head -50

Create a ServiceMonitor for Prometheus Operator:

```yaml
kubectl exec -n kube-system ds/cilium -- curl -s http://localhost:9965/metrics | head -50

为Prometheus Operator创建ServiceMonitor：

```yaml

hubble-servicemonitor.yaml

hubble-servicemonitor.yaml

apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: hubble-metrics namespace: kube-system labels: app: cilium spec: selector: matchLabels: k8s-app: cilium endpoints: - port: hubble-metrics interval: 15s path: /metrics
undefined
apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: hubble-metrics namespace: kube-system labels: app: cilium spec: selector: matchLabels: k8s-app: cilium endpoints: - port: hubble-metrics interval: 15s path: /metrics
undefined

eBPF Exporter for Custom Kernel Metrics

eBPF Exporter自定义内核指标

bash
undefined
bash
undefined

Deploy cloudflare/ebpf_exporter for custom kernel metrics

部署cloudflare/ebpf_exporter以获取自定义内核指标

helm repo add ebpf-exporter https://cloudflare.github.io/ebpf_exporter helm install ebpf-exporter ebpf-exporter/ebpf-exporter
--namespace monitoring
--set config.programs[0].name=oom_kills
--set config.programs[0].metrics.counters[0].name=oom_kill_total
--set config.programs[0].metrics.counters[0].help="Total number of OOM kills"

Example ebpf_exporter config for tracking OOM kills and run queue latency:

```yaml
helm repo add ebpf-exporter https://cloudflare.github.io/ebpf_exporter helm install ebpf-exporter ebpf-exporter/ebpf-exporter
--namespace monitoring
--set config.programs[0].name=oom_kills
--set config.programs[0].metrics.counters[0].name=oom_kill_total
--set config.programs[0].metrics.counters[0].help="Total number of OOM kills"

用于追踪OOM杀死和运行队列延迟的ebpf_exporter配置示例：

```yaml

ebpf-exporter-config.yaml

ebpf-exporter-config.yaml

programs:
  • name: oom_kills metrics: counters: - name: oom_kill_total help: "Total number of OOM kills" labels: - name: cgroup size: 128 decoders: - name: string kprobes: oom_kill_process: count_oom
  • name: runqlat metrics: histograms: - name: run_queue_latency_seconds help: "Run queue latency histogram in seconds" bucket_type: exp2 bucket_min: 0 bucket_max: 26 bucket_multiplier: 0.000000001 tracepoints: sched:sched_wakeup: trace_wakeup sched:sched_switch: trace_switch
undefined
programs:
  • name: oom_kills metrics: counters: - name: oom_kill_total help: "Total number of OOM kills" labels: - name: cgroup size: 128 decoders: - name: string kprobes: oom_kill_process: count_oom
  • name: runqlat metrics: histograms: - name: run_queue_latency_seconds help: "Run queue latency histogram in seconds" bucket_type: exp2 bucket_min: 0 bucket_max: 26 bucket_multiplier: 0.000000001 tracepoints: sched:sched_wakeup: trace_wakeup sched:sched_switch: trace_switch
undefined

Grafana Dashboard

Grafana仪表盘

Import these community dashboards for eBPF metrics:
bash
undefined
导入这些社区仪表盘用于eBPF指标展示：
bash
undefined

Hubble dashboard -- Grafana dashboard ID 16611

Hubble仪表盘——Grafana仪表盘ID 16611

Cilium Agent dashboard -- Grafana dashboard ID 16612

Cilium Agent仪表盘——Grafana仪表盘ID 16612

Cilium Operator dashboard -- Grafana dashboard ID 16613

Cilium Operator仪表盘——Grafana仪表盘ID 16613

Or create a ConfigMap for automatic provisioning

或创建ConfigMap实现自动配置

kubectl create configmap grafana-cilium-dashboard
--from-file=cilium-dashboard.json
--namespace monitoring
-o yaml --dry-run=client |
kubectl label --local -f - grafana_dashboard=1 -o yaml |
kubectl apply -f -

Key Prometheus queries for eBPF-sourced metrics:

```promql
kubectl create configmap grafana-cilium-dashboard
--from-file=cilium-dashboard.json
--namespace monitoring
-o yaml --dry-run=client |
kubectl label --local -f - grafana_dashboard=1 -o yaml |
kubectl apply -f -

基于eBPF指标的关键Prometheus查询：

```promql

Dropped packets rate by reason

按原因统计丢包率

rate(hubble_drop_total[5m])
rate(hubble_drop_total[5m])

DNS error rate by query type

按查询类型统计DNS错误率

sum(rate(hubble_dns_responses_total{rcode!="No Error"}[5m])) by (rcode, qtypes)
sum(rate(hubble_dns_responses_total{rcode!="No Error"}[5m])) by (rcode, qtypes)

HTTP request latency (p99) from Hubble L7 visibility

来自Hubble L7可见性的HTTP请求延迟（p99）

histogram_quantile(0.99, sum(rate(hubble_http_request_duration_seconds_bucket[5m])) by (le, destination))
histogram_quantile(0.99, sum(rate(hubble_http_request_duration_seconds_bucket[5m])) by (le, destination))

TCP retransmit rate from eBPF exporter

来自eBPF exporter的TCP重传率

rate(tcp_retransmits_total[5m])
rate(tcp_retransmits_total[5m])

Run queue latency p99

运行队列延迟p99

histogram_quantile(0.99, sum(rate(run_queue_latency_seconds_bucket[5m])) by (le))

---
histogram_quantile(0.99, sum(rate(run_queue_latency_seconds_bucket[5m])) by (le))

---

7. Network Observability

7. 网络可观测性

L3/L4 Flow Logging

L3/L4流量日志

bash
undefined
bash
undefined

Log all TCP connections with Hubble

使用Hubble记录所有TCP连接

hubble observe --type l3/l4 --protocol TCP --follow
hubble observe --type l3/l4 --protocol TCP --follow

Filter SYN packets only (new connections)

仅过滤SYN包（新连接）

hubble observe --type trace:to-endpoint --tcp-flags SYN --follow
hubble observe --type trace:to-endpoint --tcp-flags SYN --follow

Export flows to a file for batch analysis

将流量导出到文件用于批量分析

hubble observe --output json --since 1h > network-flows.json
hubble observe --output json --since 1h > network-flows.json

Count flows by destination service over the last hour

统计过去一小时内流向目标服务的流量数

hubble observe --output json --since 1h |
jq -r '.destination.labels[] | select(startswith("k8s:app="))' |
sort | uniq -c | sort -rn | head -20
undefined
hubble observe --output json --since 1h |
jq -r '.destination.labels[] | select(startswith("k8s:app="))' |
sort | uniq -c | sort -rn | head -20
undefined

L7 Protocol Visibility

L7协议可见性

Enable L7 visibility with Cilium annotations on target pods:
yaml
undefined
通过在目标Pod上添加Cilium注解启用L7可见性：
yaml
undefined

Annotate a namespace for HTTP visibility

为命名空间添加注解以启用HTTP可见性

apiVersion: v1 kind: Namespace metadata: name: production annotations: policy.cilium.io/proxy-visibility: "<Egress/53/UDP/DNS>,<Ingress/80/TCP/HTTP>,<Ingress/443/TCP/HTTP>"

```bash
apiVersion: v1 kind: Namespace metadata: name: production annotations: policy.cilium.io/proxy-visibility: "<Egress/53/UDP/DNS>,<Ingress/80/TCP/HTTP>,<Ingress/443/TCP/HTTP>"

```bash

Observe L7 HTTP flows

观测L7 HTTP流量

hubble observe --type l7 --protocol HTTP --follow
hubble observe --type l7 --protocol HTTP --follow

Filter by HTTP status code (5xx errors)

按HTTP状态码过滤（5xx错误）

hubble observe --type l7 --http-status "500+" --follow
hubble observe --type l7 --http-status "500+" --follow

Filter by HTTP method and path

按HTTP方法和路径过滤

hubble observe --type l7 --http-method GET --http-path "/api/v1/.*" --follow
undefined
hubble observe --type l7 --http-method GET --http-path "/api/v1/.*" --follow
undefined

DNS Monitoring

DNS监控

bash
undefined
bash
undefined

All DNS queries and responses

所有DNS查询和响应

hubble observe --type l7 --protocol DNS --follow
hubble observe --type l7 --protocol DNS --follow

DNS queries that returned NXDOMAIN

返回NXDOMAIN的DNS查询

hubble observe --type l7 --protocol DNS --dns-rcode NXDOMAIN --follow
hubble observe --type l7 --protocol DNS --dns-rcode NXDOMAIN --follow

DNS latency analysis with bpftrace

使用bpftrace分析DNS延迟

sudo bpftrace -e 'kprobe:dns_resolve { @start[tid] = nsecs; } kretprobe:dns_resolve /@start[tid]/ { @dns_latency_us = hist((nsecs - @start[tid]) / 1000); delete(@start[tid]); }'
undefined
sudo bpftrace -e 'kprobe:dns_resolve { @start[tid] = nsecs; } kretprobe:dns_resolve /@start[tid]/ { @dns_latency_us = hist((nsecs - @start[tid]) / 1000); delete(@start[tid]); }'
undefined

Service Dependency Map Generation

服务依赖关系图生成

Hubble UI automatically generates service maps. For programmatic access:
bash
undefined
Hubble UI会自动生成服务关系图。如需程序化访问：
bash
undefined

Get a service map via Hubble Relay API

通过Hubble Relay API获取服务关系图

hubble observe --output json --since 24h |
jq '{src: .source.labels, dst: .destination.labels, verdict: .verdict}' |
jq -s 'group_by(.src, .dst) | map({ source: .[0].src, destination: .[0].dst, flow_count: length, verdicts: [.[].verdict] | group_by(.) | map({(.[0]): length}) | add })' > service-map.json

---
hubble observe --output json --since 24h |
jq '{src: .source.labels, dst: .destination.labels, verdict: .verdict}' |
jq -s 'group_by(.src, .dst) | map({ source: .[0].src, destination: .[0].dst, flow_count: length, verdicts: [.[].verdict] | group_by(.) | map({(.[0]): length}) | add })' > service-map.json

---

8. Security Monitoring

8. 安全监控

Detect Container Escapes

检测容器逃逸

yaml
undefined
yaml
undefined

container-escape-detection.yaml

container-escape-detection.yaml

apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: detect-container-escape spec: kprobes: - call: "__x64_sys_unshare" syscall: true args: - index: 0 type: "int" selectors: - matchActions: - action: Post - call: "__x64_sys_mount" syscall: true args: - index: 0 type: "string" - index: 1 type: "string" - index: 2 type: "string" selectors: - matchArgs: - index: 2 operator: "Equal" values: - "proc" - "sysfs" - "cgroup" matchActions: - action: Post - call: "__x64_sys_ptrace" syscall: true args: - index: 0 type: "int" selectors: - matchActions: - action: Post

```bash
kubectl apply -f container-escape-detection.yaml
apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: detect-container-escape spec: kprobes: - call: "__x64_sys_unshare" syscall: true args: - index: 0 type: "int" selectors: - matchActions: - action: Post - call: "__x64_sys_mount" syscall: true args: - index: 0 type: "string" - index: 1 type: "string" - index: 2 type: "string" selectors: - matchArgs: - index: 2 operator: "Equal" values: - "proc" - "sysfs" - "cgroup" matchActions: - action: Post - call: "__x64_sys_ptrace" syscall: true args: - index: 0 type: "int" selectors: - matchActions: - action: Post

```bash
kubectl apply -f container-escape-detection.yaml

Unexpected Syscall Detection

意外系统调用检测

yaml
undefined
yaml
undefined

unexpected-syscalls.yaml -- alert on dangerous syscalls

unexpected-syscalls.yaml —— 对危险系统调用发出警报

apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: unexpected-syscalls spec: kprobes: - call: "__x64_sys_bpf" syscall: true args: - index: 0 type: "int" selectors: - matchNamespaces: - namespace: Pid operator: NotIn values: - "host_ns" matchActions: - action: Post - call: "__x64_sys_perf_event_open" syscall: true selectors: - matchNamespaces: - namespace: Pid operator: NotIn values: - "host_ns" matchActions: - action: Post - call: "__x64_sys_init_module" syscall: true selectors: - matchActions: - action: Sigkill
undefined
apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: unexpected-syscalls spec: kprobes: - call: "__x64_sys_bpf" syscall: true args: - index: 0 type: "int" selectors: - matchNamespaces: - namespace: Pid operator: NotIn values: - "host_ns" matchActions: - action: Post - call: "__x64_sys_perf_event_open" syscall: true selectors: - matchNamespaces: - namespace: Pid operator: NotIn values: - "host_ns" matchActions: - action: Post - call: "__x64_sys_init_module" syscall: true selectors: - matchActions: - action: Sigkill
undefined

File Integrity Monitoring

文件完整性监控

yaml
undefined
yaml
undefined

file-integrity-monitor.yaml

file-integrity-monitor.yaml

apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: file-integrity-monitor spec: kprobes: - call: "security_file_open" syscall: false args: - index: 0 type: "file" selectors: - matchArgs: - index: 0 operator: "Prefix" values: - "/etc/" - "/usr/bin/" - "/usr/sbin/" - "/usr/lib/" matchActions: - action: Post rateLimit: "1m" - call: "security_inode_rename" syscall: false args: - index: 0 type: "path" - index: 1 type: "path" selectors: - matchActions: - action: Post

```bash
kubectl apply -f file-integrity-monitor.yaml
apiVersion: cilium.io/v1alpha1 kind: TracingPolicy metadata: name: file-integrity-monitor spec: kprobes: - call: "security_file_open" syscall: false args: - index: 0 type: "file" selectors: - matchArgs: - index: 0 operator: "Prefix" values: - "/etc/" - "/usr/bin/" - "/usr/sbin/" - "/usr/lib/" matchActions: - action: Post rateLimit: "1m" - call: "security_inode_rename" syscall: false args: - index: 0 type: "path" - index: 1 type: "path" selectors: - matchActions: - action: Post

```bash
kubectl apply -f file-integrity-monitor.yaml

Stream events to your SIEM

将事件流式传输到SIEM系统

kubectl logs -n kube-system ds/tetragon -c export-stdout -f |
jq 'select(.process_kprobe.policy_name == "file-integrity-monitor")' |
tee /dev/stderr |
curl -X POST -H "Content-Type: application/json" -d @- https://siem.internal/api/events

---
kubectl logs -n kube-system ds/tetragon -c export-stdout -f |
jq 'select(.process_kprobe.policy_name == "file-integrity-monitor")' |
tee /dev/stderr |
curl -X POST -H "Content-Type: application/json" -d @- https://siem.internal/api/events

---

9. Performance Profiling

9. 性能剖析

Continuous Profiling with Parca

使用Parca进行持续剖析

Parca uses eBPF to collect CPU profiles continuously with minimal overhead.
bash
undefined
Parca使用eBPF以极低开销持续收集CPU剖析数据。
bash
undefined

Install Parca Agent via Helm

通过Helm安装Parca Agent

helm repo add parca https://parca-dev.github.io/helm-charts helm repo update
helm install parca-agent parca/parca-agent
--namespace parca
--create-namespace
--set config.node=true
--set config.store.address="parca-server.parca.svc:7070"
--set config.store.insecure=true
--set config.debuginfo.strip=true
--set config.debuginfo.upload.enabled=true
undefined
helm repo add parca https://parca-dev.github.io/helm-charts helm repo update
helm install parca-agent parca/parca-agent
--namespace parca
--create-namespace
--set config.node=true
--set config.store.address="parca-server.parca.svc:7070"
--set config.store.insecure=true
--set config.debuginfo.strip=true
--set config.debuginfo.upload.enabled=true
undefined

Continuous Profiling with Pyroscope

使用Pyroscope进行持续剖析

bash
undefined
bash
undefined

Install Grafana Pyroscope with eBPF profiling

安装支持eBPF剖析的Grafana Pyroscope

helm repo add grafana https://grafana.github.io/helm-charts helm repo update
helm install pyroscope grafana/pyroscope
--namespace pyroscope
--create-namespace
--set ebpf.enabled=true
--set agent.mode=ebpf
undefined
helm repo add grafana https://grafana.github.io/helm-charts helm repo update
helm install pyroscope grafana/pyroscope
--namespace pyroscope
--create-namespace
--set ebpf.enabled=true
--set agent.mode=ebpf
undefined

CPU Flame Graphs with bpftrace

使用bpftrace生成CPU火焰图

bash
undefined
bash
undefined

Sample kernel and user stacks at 99Hz for 30 seconds

以99Hz采样内核和用户栈，持续30秒

sudo bpftrace -e 'profile:hz:99 { @[kstack, ustack, comm] = count(); }'
-d 30 > stacks.out
sudo bpftrace -e 'profile:hz:99 { @[kstack, ustack, comm] = count(); }'
-d 30 > stacks.out

Using perf with BPF for flame graphs

使用perf结合BPF生成火焰图

sudo perf record -F 99 -a -g -- sleep 30 sudo perf script > perf.stacks
sudo perf record -F 99 -a -g -- sleep 30 sudo perf script > perf.stacks

Convert to flame graph (using Brendan Gregg's tools)

转换为火焰图（使用Brendan Gregg的工具）

git clone https://github.com/brendangregg/FlameGraph.git ./FlameGraph/stackcollapse-perf.pl perf.stacks |
./FlameGraph/flamegraph.pl > flamegraph.svg
undefined
git clone https://github.com/brendangregg/FlameGraph.git ./FlameGraph/stackcollapse-perf.pl perf.stacks |
./FlameGraph/flamegraph.pl > flamegraph.svg
undefined

Off-CPU Analysis

离线CPU分析

bash
undefined
bash
undefined

Trace off-CPU time to find where threads are blocked

追踪离线CPU时间以找出线程阻塞位置

sudo bpftrace -e ' kprobe:finish_task_switch { $prev = (struct task_struct *)arg0; if ($prev->__state != 0) { @block_start[$prev->pid] = nsecs; } if (@block_start[tid]) { @off_cpu_us[kstack, comm] = sum((nsecs - @block_start[tid]) / 1000); delete(@block_start[tid]); } } END { print(@off_cpu_us, 20); }'
undefined
sudo bpftrace -e ' kprobe:finish_task_switch { $prev = (struct task_struct *)arg0; if ($prev->__state != 0) { @block_start[$prev->pid] = nsecs; } if (@block_start[tid]) { @off_cpu_us[kstack, comm] = sum((nsecs - @block_start[tid]) / 1000); delete(@block_start[tid]); } } END { print(@off_cpu_us, 20); }'
undefined

Memory Leak Detection

内存泄漏检测

bash
undefined
bash
undefined

Track memory allocations not freed

追踪未释放的内存分配

sudo bpftrace -e ' kprobe:kmalloc { @allocs[kstack] = count(); @bytes[kstack] = sum(arg0); } kprobe:kfree { @frees = count(); } interval:s:10 { print(@bytes, 10); } '
sudo bpftrace -e ' kprobe:kmalloc { @allocs[kstack] = count(); @bytes[kstack] = sum(arg0); } kprobe:kfree { @frees = count(); } interval:s:10 { print(@bytes, 10); }'

Per-process heap growth tracking

按进程追踪堆内存增长

sudo bpftrace -e ' uprobe:/lib/x86_64-linux-gnu/libc.so.6:malloc { @size[comm, tid] = sum(arg0); } interval:s:5 { print(@size, 10); clear(@size); } '

---
sudo bpftrace -e ' uprobe:/lib/x86_64-linux-gnu/libc.so.6:malloc { @size[comm, tid] = sum(arg0); } interval:s:5 { print(@size, 10); clear(@size); }'

---

10. Troubleshooting

10. 故障排查

Common eBPF Issues

常见eBPF问题

BPF verifier rejects program:
bash
undefined
BPF验证器拒绝程序：
bash
undefined

Get verbose verifier output

获取详细验证器输出

sudo bpftrace -d -e 'your_program_here' 2>&1 | tail -50
sudo bpftrace -d -e 'your_program_here' 2>&1 | tail -50

Common causes:

常见原因：

- Unbounded loops (BPF requires bounded loops or unrolled iterations)

- 无限循环（BPF要求循环有界或展开迭代）

- Stack size exceeds 512 bytes

- 栈大小超过512字节

- Accessing memory without null checks

- 访问内存未做空检查

- Back-edges in control flow (pre-5.3 kernels)

- 控制流存在回边（5.3之前的内核）


**BTF not available:**

```bash

**BTF不可用：**

```bash

Check if BTF is compiled into the kernel

检查内核是否编译了BTF

cat /boot/config-$(uname -r) | grep CONFIG_DEBUG_INFO_BTF
cat /boot/config-$(uname -r) | grep CONFIG_DEBUG_INFO_BTF

If not, install BTF data from btfhub

若未编译，从btfhub安装BTF数据

https://github.com/aquasecurity/btfhub

https://github.com/aquasecurity/btfhub

wget "https://github.com/aquasecurity/btfhub-archive/raw/main/ubuntu/22.04/x86_64/$(uname -r).btf.tar.xz" tar xvf "$(uname -r).btf.tar.xz"

**Permission denied:**

```bash
wget "https://github.com/aquasecurity/btfhub-archive/raw/main/ubuntu/22.04/x86_64/$(uname -r).btf.tar.xz" tar xvf "$(uname -r).btf.tar.xz"

**权限被拒绝：**

```bash

BPF requires CAP_BPF (or CAP_SYS_ADMIN on older kernels)

BPF需要CAP_BPF权限（旧内核需要CAP_SYS_ADMIN）

For containers, add to securityContext:

对于容器，添加到securityContext：

securityContext:

securityContext:

capabilities:

capabilities:

add: ["BPF", "PERFMON", "SYS_RESOURCE"]

add: ["BPF", "PERFMON", "SYS_RESOURCE"]

Check current capabilities

检查当前权限

cat /proc/self/status | grep Cap capsh --decode=$(cat /proc/self/status | grep CapEff | awk '{print $2}')

**Cilium pods not starting:**

```bash
cat /proc/self/status | grep Cap capsh --decode=$(cat /proc/self/status | grep CapEff | awk '{print $2}')

**Cilium Pod无法启动：**

```bash

Check Cilium agent logs

检查Cilium Agent日志

kubectl logs -n kube-system -l k8s-app=cilium --tail=100
kubectl logs -n kube-system -l k8s-app=cilium --tail=100

Verify BPF filesystem

验证BPF文件系统

kubectl exec -n kube-system ds/cilium -- mount | grep bpf
kubectl exec -n kube-system ds/cilium -- mount | grep bpf

Check for conflicting CNIs

检查是否存在冲突的CNI

ls /etc/cni/net.d/
ls /etc/cni/net.d/

Run Cilium connectivity test

运行Cilium连通性测试

cilium connectivity test

**Tetragon events missing:**

```bash
cilium connectivity test

**Tetragon事件缺失：**

```bash

Verify TracingPolicy is loaded

验证TracingPolicy已加载

kubectl get tracingpolicies
kubectl get tracingpolicies

Check Tetragon agent logs for verifier errors

检查Tetragon Agent日志中的验证器错误

kubectl logs -n kube-system ds/tetragon -c tetragon --tail=200 | grep -i error
kubectl logs -n kube-system ds/tetragon -c tetragon --tail=200 | grep -i error

Verify the kprobe is attached

验证kprobe已附加

kubectl exec -n kube-system ds/tetragon -c tetragon --
cat /sys/kernel/debug/kprobes/list | grep your_function

**High overhead from eBPF programs:**

```bash
kubectl exec -n kube-system ds/tetragon -c tetragon --
cat /sys/kernel/debug/kprobes/list | grep your_function

**eBPF程序开销过高：**

```bash

List all loaded BPF programs and their run time

列出所有已加载的BPF程序及其运行时间

sudo bpftool prog show sudo bpftool prog profile id <PROG_ID> duration 5
sudo bpftool prog show sudo bpftool prog profile id <PROG_ID> duration 5

Check map memory usage

检查映射内存使用情况

sudo bpftool map show sudo bpftool map dump id <MAP_ID> | wc -l
sudo bpftool map show sudo bpftool map dump id <MAP_ID> | wc -l

If a program is consuming too much CPU, check its run count and time

若某个程序CPU占用过高，检查其运行次数和时间

sudo bpftool prog show id <PROG_ID> --json | jq '{run_cnt, run_time_ns}'
sudo bpftool prog show id <PROG_ID> --json | jq '{run_cnt, run_time_ns}'

Detach a misbehaving program

分离行为异常的程序

sudo bpftool prog detach id <PROG_ID> type <ATTACH_TYPE>
undefined
sudo bpftool prog detach id <PROG_ID> type <ATTACH_TYPE>
undefined

Kernel Compatibility Matrix

内核兼容性矩阵

bash
undefined
bash
undefined

Quick check: which eBPF features your kernel supports

快速检查：你的内核支持哪些eBPF功能

sudo bpftool feature probe kernel
sudo bpftool feature probe kernel

Check specific program types

检查特定程序类型

sudo bpftool feature probe kernel | grep program_type
sudo bpftool feature probe kernel | grep program_type

Check available map types

检查可用映射类型

sudo bpftool feature probe kernel | grep map_type
sudo bpftool feature probe kernel | grep map_type

Check available helper functions

检查可用辅助函数

sudo bpftool feature probe kernel | grep helper
undefined
sudo bpftool feature probe kernel | grep helper
undefined