docs: replace prose placeholders in reproduction steps with executable commands

The audit found ~80 placeholders, and the damaging ones were where the
measuring apparatus itself was prose rather than a command:

  a6  "( curl ... ) & 를 20개 띄우고 wait"  — the 22.2s headline came from this
  a3  "<로그인 반복, sid 를 /tmp/sids 에>"  — the whole RPO measurement
  a3  "<sid 목록>"                          — the control it is compared against
  a5  "<수신 파드IP>"                       — the injection
  a8  writes /tmp/tok, reads /tmp/rt        — self-inconsistent, sent an empty token
  b3  $KC / $RT / $NEW never assigned
  c2  bare kcadm.sh with no kubectl exec
  a1  conntrack tuples written by hand, though the direction flips per restart

Each is now a shell-expandable form: pod IPs from jsonpath, the admin password
from the secret, ids from kcadm --format csv, conntrack tuples derived from
"conntrack -L" with awk rather than transcribed.

Then the rewritten commands were executed against the live cluster, and one
of them failed — the 20-way load generator, written as "kubectl run --rm -i",
lost its output stream twice in a row. That is a trap this series already hit
once, and the rewrite reintroduced it. A-6 now uses a resident probe pod that
collects into a file and is cat-ed once; verified 20/20 lines.

Evidence: docs/evidence/followup/05-command-reproducibility.txt

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
DongHyeonka
2026-09-04 17:01:54 +09:00
co-authored by Claude Opus 5
parent f3f3a8da46
commit 74c9b3cea7
20 changed files with 259 additions and 64 deletions
+43 -7
View File
@@ -337,15 +337,51 @@ ssh kc-lab-2 '
# 3. 걸렸는지 카운터로 확인 — Sent 가 0 이면 해석 금지
ssh kc-lab-2 'sudo tc -s qdisc show dev flannel.1 | grep -A2 netem'
# 4. 단일 요청 지연 (대조군은 같은 노드의 keycloak-0)
kubectl -n keycloak-lab run t --rm -i --restart=Never --image=curlimages/curl:8.11.1 \
--command -- curl -s -o /dev/null -w '%{time_total}\n' -X POST http://<pod>:8080/realms/master/protocol/openid-connect/token ...
# 변수 준비 — 아래 명령들이 이 값을 쓴다
K0=$(kubectl -n keycloak-lab get pod keycloak-0 -o jsonpath='{.status.podIP}')
K1=$(kubectl -n keycloak-lab get pod keycloak-1 -o jsonpath='{.status.podIP}')
PW=$(kubectl -n keycloak-lab get secret keycloak-lab-secrets \
-o jsonpath='{.data.KC_BOOTSTRAP_ADMIN_PASSWORD}' | base64 -d)
TE=/realms/master/protocol/openid-connect/token
# 5. 동시 부하로 풀 고갈 재현
# ( curl ... ) & 를 20개 띄우고 wait
# 4. 상주 탐침을 먼저 띄운다.
# ★ kubectl run --rm -i 로 20건을 동시에 돌리면 출력이 유실된다(실측: 20줄 중
# 일부만 도착하거나 아예 끊긴다). 결과는 파드 안 파일에 모으고 한 번에 cat 한다.
kubectl -n keycloak-lab run a6-probe --image=curlimages/curl:8.11.1 \
--restart=Never --command -- sleep 1800
kubectl -n keycloak-lab wait --for=condition=Ready pod/a6-probe --timeout=120s
# 6. 풀 지표
curl -s http://<pod>:9000/metrics | grep -E '^agroal_(blocking_time|max_used|awaiting)'
# 5. 단일 요청 지연 — 대조군(keycloak-0, DB 와 같은 노드) 대 시험군(keycloak-1)
kubectl -n keycloak-lab exec a6-probe -- sh -c "
for t in keycloak-0:$K0 keycloak-1:$K1; do
n=\${t%%:*}; ip=\${t#*:}; T=0; i=0
while [ \$i -lt 15 ]; do
D=\$(curl -s -o /dev/null -w %{time_total} -X POST http://\$ip:8080$TE \
-d grant_type=password -d client_id=admin-cli -d username=admin -d password=$PW)
T=\$(echo \"\$T \$D\" | awk '{print \$1+\$2}'); i=\$((i+1))
done
echo \"\$n 평균 \$(echo \$T | awk '{printf \"%.0f\", \$1*1000/15}') ms\"
done"
# 6. 동시 부하 20건 — & 로 띄우고 wait. 순차로 돌리면 풀 경합이 재현되지 않는다.
kubectl -n keycloak-lab exec a6-probe -- sh -c "
rm -f /tmp/load; i=0
while [ \$i -lt 20 ]; do
( curl -s -o /dev/null -w '%{http_code} %{time_total}\n' --max-time 60 \
-X POST http://$K1:8080$TE -d grant_type=password -d client_id=admin-cli \
-d username=admin -d password=$PW >> /tmp/load ) &
i=\$((i+1))
done
wait"
kubectl -n keycloak-lab exec a6-probe -- cat /tmp/load > /tmp/load.txt
wc -l /tmp/load.txt # 20 이 아니면 수집이 샌 것이다
awk '{print $1}' /tmp/load.txt | sort | uniq -c
awk '{print $2}' /tmp/load.txt | sort -g | tail -1 # 최대 지연
# 7. 풀 지표 — 부하 직후에 읽어야 blocking_time 이 남아 있다
kubectl -n keycloak-lab exec a6-probe -- sh -c \
"curl -s http://$K1:9000/metrics \
| grep -E '^agroal_(blocking_time_average|max_used_count|awaiting_count|active_count)'"
# 7. 해제
ssh kc-lab-2 'sudo tc qdisc del dev flannel.1 root'