Source Code Study:
https://github.com/cilium/cilium/blob/master/bpf/bpf_lxc.c#L1223
int handle_policy(struct __ctx_buff *ctx)
|
-> tail_ipv4_policy()
|
-> ipv4_policy()
|
V
https://github.com/cilium/cilium/blob/master/bpf/lib/policy_log.h#L46
send_policy_verdict_notify()
在此function放入 struct policy_verdict_notify 格式化的資料
|
-> ctx_event_output()
(這function是被remap來自skb_event_output()
會將event送到perf's ring buffer in Kernel)
#Cilium 有 perf reader 可以從 Kernel 讀 perf's ring buffer 然後解開msg
https://github.com/cilium/ebpf/blob/master/perf/reader.go
#另外一個會送send_trace_notify()的來源 (有很多bpf的點都會送)
https://github.com/cilium/cilium/blob/master/bpf/bpf_network.c
int from_network(struct __ctx_buff *ctx)
|
V
https://github.com/cilium/cilium/blob/master/bpf/lib/trace.h
send_trace_notify()
這個function會發送packet tracing events是很重要的flow來源
#上面兩個例子的notify type 是 CILIUM_NOTIFY_POLICY_VERDICT & CILIUM_NOTIFY_TRACE
#到了Hubble就會是Flow (Layer3/4)主要來源
case monitorAPI.MessageTypeDrop,
monitorAPI.MessageTypeTrace,
monitorAPI.MessageTypePolicyVerdict:
return p.l34.Decode(payload, decoded)
case monitorAPI.MessageTypeAccessLog:
return p.l7.Decode(payload, decoded)
BPF datapath events 定義在這
https://github.com/cilium/cilium/blob/master/pkg/monitor/api/types.go
#跟container有關的send_trace_notify()的來源
https://github.com/cilium/cilium/blob/master/bpf/bpf_lxc.c
tail_handle_ipv4()
|
-> handle_ipv4_from_lxc()
|
V
https://github.com/cilium/cilium/blob/master/bpf/lib/trace.h
send_trace_notify()
測試:
kubectl get pods -n kube-system
kubectl exec -n kube-system ds/cilium -- cilium status
hubble observe --verdict DROPPED --pod XXX
hubble observe --since 3m --protocol http --output=json | \
jq -sr 'map(select(.l7.type=="RESPONSE" and .l7.http.method=="GET")) |
group_by(.l7.http.url) |
map({url: .[0].l7.http.url, max_latency_ms: ((map(.l7.latency_ns | tonumber) | max) / 1000000 }) |
sort_by(.max_latency_ms)'
hubble observe --since 3m --protocol tcp --output=json | \
jq -sr 'map(select(.l4.TCP.destination_port==8080)) |
group_by(.l4.TCP.flags)'
hubble observe --since=10m -t l7 -j \
| jq 'select(.l7.dns.rcode==3) | .destination.namespace + "/" + .destination.pod_name' \
| sort | uniq -c | sort -r
hubble-relay serve --peer-service=unix:///var/run/cilium/hubble.sock --listen-address=:4245
curl \
--silent \
--unix-socket /var/run/cilium/hubble.sock \
--request GET \
http://localhost/ \
| jq .
hubble serve --listen-client-urls=0.0.0.0:50051 --listen-client-urls=unix:///var/run/hubble.sock --metrics-server :6943 --metric=dns:query --metric=drop --metric=tcp --metric=flow --
