fix(observability): stop throttling node-exporter

node-exporter averages about 2m CPU, but each scrape runs its
collectors in parallel. With a 250m limit (25ms per 100ms CFS period)
26% of its CFS periods were throttled over 30 minutes on 2026-11-11
(container_cpu_cfs_throttled_periods_total / cfs_periods_total), which
delays scrapes and can make them time out.

Raise the CPU limit to one core. The request stays 100m, so
scheduling does not change, and a runaway collector is still capped.
Removing the limit was rejected because every other workload in this
repository sets one.

Risk: none beyond one core of headroom on a 12-core node. The live
DaemonSet changes only when the observability core is applied again.

Verification: the throttling ratio and usage came from live
Prometheus queries. helm template of prometheus-node-exporter 4.56.1
with these values renders limits.cpu "1". The effect after rollout is
not measured yet.
This commit is contained in:
DongHyeonka
2026-11-12 07:28:03 +09:00
parent 619b90552f
commit bed20b456b
@@ -32,7 +32,11 @@ resources:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
# Average use is about 2m, but each scrape runs the collectors in
# parallel. At 250m (25ms per 100ms CFS period) 26% of periods were
# throttled on 2026-11-11. One core removes the throttling and still caps
# a runaway collector.
cpu: "1"
memory: 256Mi
serviceAccount: