fix(observability): stop throttling node-exporter
node-exporter averages about 2m CPU, but each scrape runs its collectors in parallel. With a 250m limit (25ms per 100ms CFS period) 26% of its CFS periods were throttled over 30 minutes on 2026-11-11 (container_cpu_cfs_throttled_periods_total / cfs_periods_total), which delays scrapes and can make them time out. Raise the CPU limit to one core. The request stays 100m, so scheduling does not change, and a runaway collector is still capped. Removing the limit was rejected because every other workload in this repository sets one. Risk: none beyond one core of headroom on a 12-core node. The live DaemonSet changes only when the observability core is applied again. Verification: the throttling ratio and usage came from live Prometheus queries. helm template of prometheus-node-exporter 4.56.1 with these values renders limits.cpu "1". The effect after rollout is not measured yet.
This commit is contained in:
@@ -32,7 +32,11 @@ resources:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
cpu: 250m
|
||||
# Average use is about 2m, but each scrape runs the collectors in
|
||||
# parallel. At 250m (25ms per 100ms CFS period) 26% of periods were
|
||||
# throttled on 2026-11-11. One core removes the throttling and still caps
|
||||
# a runaway collector.
|
||||
cpu: "1"
|
||||
memory: 256Mi
|
||||
|
||||
serviceAccount:
|
||||
|
||||
Reference in New Issue
Block a user