docs: A-4 — a dead pod reports healthier than a live one

Kubernetes keeps calling the node Ready for forty seconds while users already see failures, and the pod on the powered-off machine stays ready=true because its kubelet can no longer contradict itself. Eviction waits another five minutes, then the StatefulSet refuses to recreate its pod and the replacement Deployment pod cannot schedule because the local-path volume is pinned to the dead node. Killing the server node instead shows the opposite shape: containerd keeps the workload running while the API server, Traefik and the observability stack disappear, so the outage is the missing path rather than the missing application. Traefik at one replica is the ingress single point of failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
docs: A-3 — four logins returned tokens for sessions the crash erased
2026-09-04 12:25:30 +09:00 · 2026-09-04 12:05:08 +09:00 · 2026-09-04 11:57:55 +09:00 · 2026-09-04 11:52:49 +09:00 · 2026-09-04 11:16:59 +09:00 · 2026-09-04 10:50:23 +09:00
159 changed files with 15281 additions and 10 deletions
@@ -6,5 +6,13 @@
 backend/target/
 build/
 e2e/node_modules/
+google-e2e/node_modules/
 frontend/node_modules/
 frontend/dist/
+
+bff/target
+token-mediator/target
+
+# lab cloud-init contains a console password; keep the filled copy local
+deploy/lab/cloud-init/kc-lab.yaml
+deploy/lab/cloud-init/kc-lab-*.yaml
@@ -0,0 +1,22 @@
+[    1289ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/login:0
+[    1417ms] [VERBOSE] [DOM] Input elements should have autocomplete attributes (suggested: "username"): (More info: https://goo.gl/9p2vKq) %o @ https://app2.hyeonworks.com/login:0
+[    9024ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    9130ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/:0
+[    9456ms] [WARNING] Deprecation warning: value provided is not in a recognized RFC2822 or ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions. Non RFC2822/ISO date formats are discouraged. Please refer to http://momentjs.com/guides/#/warnings/js-date/ for more info.
+Arguments: 
+[0] _isAMomentObject: true, _isUTC: false, _useUTC: false, _l: undefined, _i: Thu, 27 Aug 2026 13:03:49, _f: undefined, _strict: undefined, _locale: [object Object]
+Error
+    at a.createFromInputFallback (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:3)
+    at an (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:25647)
+    at un (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:29355)
+    at aa (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:29221)
+    at on (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:28938)
+    at sa (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:29715)
+    at A (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:29748)
+    at a (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:621:89)
+    at f (https://app2.hyeonworks.com/public/build/3719.c065b2e146c4c8347d51.js:1:4635)
+    at u (https://app2.hyeonworks.com/public/build/322.177b4bb01c5d74f9b28f.js:2473:47448) @ https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:620
+[    9605ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   10620ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   11527ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   13875ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
@@ -0,0 +1,7 @@
+[     144ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[     153ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore:0
+[    1077ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    2101ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    2922ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    7323ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    9370ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
@@ -0,0 +1,8 @@
+[     271ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&orgId=1&panes=%7B%22a%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_statistics_approximate_entries_unique%7Bcache%3D%5C%22sessions%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bpod%7D%7D%20on%20%7B%7Bnode%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-15m%22%2C%22to%22%3A%22now%22%7D%7D%7D:0
+[     346ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    1512ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    2433ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    6941ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   13188ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   21578ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   25998ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
@@ -0,0 +1,7 @@
+[     766ms] [WARNING] An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing. @ https://auth.hyeonworks.com/realms/master/protocol/openid-connect/3p-cookies/step1.html:0
+[     781ms] [WARNING] An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing. @ https://auth.hyeonworks.com/realms/master/protocol/openid-connect/3p-cookies/step2.html:0
+[   17929ms] [WARNING] An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing. @ https://auth.hyeonworks.com/realms/master/protocol/openid-connect/3p-cookies/step1.html:0
+[   17949ms] [WARNING] An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing. @ https://auth.hyeonworks.com/realms/master/protocol/openid-connect/3p-cookies/step2.html:0
+[   17981ms] [WARNING] An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing. @ https://auth.hyeonworks.com/realms/master/protocol/openid-connect/login-status-iframe.html:0
+[   18447ms] [WARNING] For accessibility reasons an aria-label should be specified on nav groups if a title isn't @ https://auth.hyeonworks.com/resources/9v5yc/admin/keycloak.v2/assets/main-BbID33M6.js:7
+[   18462ms] [WARNING] For accessibility reasons an aria-label should be specified on nav groups if a title isn't @ https://auth.hyeonworks.com/resources/9v5yc/admin/keycloak.v2/assets/main-BbID33M6.js:7
@@ -0,0 +1,2 @@
+[      75ms] [ERROR] Failed to load resource: the server responded with a status of 404 (Not Found) @ https://hyeonworks.com/questions:0
+[     100ms] [ERROR] Failed to load resource: the server responded with a status of 404 (Not Found) @ https://hyeonworks.com/favicon.ico:0
@@ -0,0 +1 @@
+[     149ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
@@ -0,0 +1 @@
+[     103ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
@@ -0,0 +1 @@
+[      95ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
@@ -0,0 +1 @@
+[     132ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
@@ -0,0 +1 @@
+[     239ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
@@ -0,0 +1 @@
+[     105ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
@@ -0,0 +1 @@
+[     108ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
@@ -0,0 +1,41 @@
+[     344ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[     869ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&panes=%7B%22cf1%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_cluster_size%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bpod%7D%7D+on+%7B%7Bnode%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-45m%22%2C%22to%22%3A%22now%22%7D%7D%7D&orgId=1:0
+[    1014ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    2039ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    4085ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    7289ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   15556ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   26818ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   32659ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   35930ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   53849ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   61722ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   79450ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   84571ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   87233ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   92665ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  113244ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  119181ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  137390ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  157071ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  166091ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  182011ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  188613ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  206228ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  218561ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  229607ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  235818ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  237158ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  252095ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  261118ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  273809ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  280227ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  292650ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  306477ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  309957ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  311984ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  325683ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  344586ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  357576ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  359294ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  364740ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
@@ -0,0 +1,129 @@
+[     615ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&panes=%7B%22te0%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22up%7Bjob%3D%5C%22keycloak%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22up+%E2%80%94+%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-20m%22%2C%22to%22%3A%22now%22%7D%7D%7D&orgId=1:0
+[     929ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    2566ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    5541ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[    7990ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   12402ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   17640ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   20285ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   28277ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   34341ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   34985ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   50081ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   65649ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   75046ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   75890ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   96067ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  113467ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  121972ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  139681ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  150971ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  161211ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  180744ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  191401ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  210668ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  229082ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  239345ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  252119ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  266045ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  276705ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  289911ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  296875ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  315099ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  325431ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  328418ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  343774ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  347054ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  352405ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  361291ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  370639ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  374286ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  377413ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  394362ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  405423ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  425476ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  434299ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  441775ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  450786ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  456829ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  468812ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  478913ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  491241ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  495228ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  502295ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  511625ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  524619ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  533050ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  553189ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  571699ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  575413ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  588005ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  599161ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  608816ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  619031ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  626612ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  639003ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  655077ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  674532ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  677197ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  696027ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  709512ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  727505ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  747139ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  754565ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  773682ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  784674ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  787399ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  802845ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  825589ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  832768ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  842043ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  852717ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  871748ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  876571ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  894491ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  895515ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  904310ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  916681ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  934752ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  941175ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  946004ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  960849ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  962180ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  984446ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  997579ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1010105ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1027190ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1043388ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1061311ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1066040ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1080349ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1092336ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1104312ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1109362ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1127047ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1134728ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1154899ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1161253ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1174971ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1184491ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1185340ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1189111ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1196984ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1204794ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1247961ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1286322ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1319329ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1362771ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1396881ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1434184ms] [ERROR] Failed to load resource: the server responded with a status of 502 () @ https://app2.hyeonworks.com/api/user/auth-tokens/rotate:0
+[ 1443506ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1465016ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 502 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1513337ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1556906ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1562963ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 502 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1572896ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 502 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1576390ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 503 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1593794ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1600651ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[ 1616424ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
@@ -0,0 +1,20 @@
+[     268ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/login:0
+[     379ms] [VERBOSE] [DOM] Input elements should have autocomplete attributes (suggested: "username"): (More info: https://goo.gl/9p2vKq) %o @ https://app2.hyeonworks.com/login:0
+[   10453ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   10532ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&panes=%7B%22ich%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22up%7Bjob%3D%7E%5C%22keycloak%7Cnode-exporter%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bjob%7D%7D+%E2%80%94+%7B%7Bpod%7D%7D%7B%7Bnode%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-55m%22%2C%22to%22%3A%22now%22%7D%7D%7D&orgId=1:0
+[   11687ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   13737ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   15113ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   20185ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   27259ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   40154ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   41870ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   50493ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   54082ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   60121ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   74362ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   83269ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   95349ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[   98008ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  104458ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
+[  123816ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
@@ -0,0 +1,2 @@
+- main [ref=e7]:
+  - status "Loading" [ref=e10]
@@ -0,0 +1 @@
+- main [ref=f3e7]
@@ -0,0 +1,2 @@
+- main [ref=f6e7]:
+  - status "Loading" [ref=f6e10]
@@ -0,0 +1,104 @@
+- generic [ref=f9e4]:
+  - link "Skip to main content" [ref=f9e5] [cursor=pointer]:
+    - /url: "#pageContent"
+  - banner [ref=f9e7]:
+    - generic [ref=f9e8]:
+      - link [ref=f9e10] [cursor=pointer]:
+        - /url: /
+        - img "Grafana" [ref=f9e11]
+      - generic [ref=f9e14]:
+        - button "Search or jump to..." [ref=f9e18] [cursor=pointer]
+        - generic [ref=f9e19]: ctrl+k
+      - generic [ref=f9e23]:
+        - button "New" [ref=f9e24] [cursor=pointer]
+        - button "Help" [ref=f9e30] [cursor=pointer]
+        - button "News" [ref=f9e33] [cursor=pointer]
+        - button "Profile" [ref=f9e36] [cursor=pointer]:
+          - img "User avatar" [ref=f9e37]
+    - generic [ref=f9e38]:
+      - button "Open menu" [ref=f9e40] [cursor=pointer]
+      - navigation "Breadcrumbs" [ref=f9e43]:
+        - list [ref=f9e44]:
+          - listitem [ref=f9e45]:
+            - link "Home" [ref=f9e46] [cursor=pointer]:
+              - /url: /
+          - listitem [ref=f9e50]:
+            - link "Explore" [ref=f9e51] [cursor=pointer]:
+              - /url: /explore
+          - listitem [ref=f9e55]:
+            - generic "Prometheus" [ref=f9e56]
+      - generic [ref=f9e57]:
+        - button "Show more items" [ref=f9e60] [cursor=pointer]
+        - button "Toggle top search bar" [ref=f9e64] [cursor=pointer]
+  - main [ref=f9e70]:
+    - generic [ref=f9e72]:
+      - heading "Explore" [level=1] [ref=f9e73]
+      - generic [ref=f9e78]:
+        - navigation "Explore toolbar" [ref=f9e80]:
+          - navigation "Search links" [ref=f9e82]:
+            - generic [ref=f9e83]:
+              - button "Content outline" [expanded] [ref=f9e85] [cursor=pointer]:
+                - generic [ref=f9e88]: Outline
+              - generic [ref=f9e93] [cursor=pointer]:
+                - img "Prometheus logo" [ref=f9e95]
+                - textbox "Select a data source" [ref=f9e96]:
+                  - /placeholder: ""
+          - button "Show more items" [ref=f9e102] [cursor=pointer]
+        - generic [ref=f9e106]:
+          - generic [ref=f9e110]:
+            - button "Collapse outline" [expanded] [ref=f9e112] [cursor=pointer]:
+              - img "arrow-from-right" [ref=f9e113]
+            - button "Queries" [ref=f9e116] [cursor=pointer]:
+              - img "arrow" [ref=f9e117]
+          - generic [ref=f9e124]:
+            - generic [ref=f9e126]:
+              - generic "Query editor row" [ref=f9e129]:
+                - generic [ref=f9e130]:
+                  - generic [ref=f9e132]:
+                    - generic [ref=f9e133]:
+                      - button "Collapse query row" [expanded] [ref=f9e134] [cursor=pointer]
+                      - generic [ref=f9e137]:
+                        - button "Query editor row title A" [ref=f9e138] [cursor=pointer]:
+                          - generic [ref=f9e139]: A
+                        - emphasis [ref=f9e140]: (Prometheus)
+                    - generic [ref=f9e141]:
+                      - button "Show data source help" [ref=f9e143] [cursor=pointer]
+                      - button "Duplicate query" [ref=f9e147] [cursor=pointer]
+                      - button "Hide response" [ref=f9e151] [cursor=pointer]
+                      - button "Remove query" [ref=f9e155] [cursor=pointer]
+                      - button "Drag and drop to reorder" [ref=f9e158]:
+                        - img "Drag and drop to reorder" [ref=f9e159]
+                  - generic [ref=f9e162]:
+                    - generic [ref=f9e163]:
+                      - button "Kick start your query" [ref=f9e164] [cursor=pointer]
+                      - generic [ref=f9e167]:
+                        - generic [ref=f9e168] [cursor=pointer]: Explain
+                        - generic [ref=f9e169]:
+                          - checkbox "Explain Toggle switch" [ref=f9e170]
+                          - generic "Toggle switch" [ref=f9e171] [cursor=pointer]
+                      - radiogroup [ref=f9e176]:
+                        - generic [ref=f9e177]:
+                          - radio "Builder" [ref=f9e178] [cursor=pointer]
+                          - generic [ref=f9e179] [cursor=pointer]: Builder
+                        - generic [ref=f9e180]:
+                          - radio "Code" [checked] [ref=f9e181] [cursor=pointer]
+                          - generic [ref=f9e182] [cursor=pointer]: Code
+                    - generic [ref=f9e184]:
+                      - generic [ref=f9e186]:
+                        - button "Loading metrics..." [disabled] [ref=f9e187] [cursor=pointer]
+                        - generic [ref=f9e190]: Loading editor
+                      - 'button "Options Legend: {{pod}} on {{node}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f9e198] [cursor=pointer]':
+                        - generic [ref=f9e202]:
+                          - heading "Options" [level=6] [ref=f9e203]
+                          - generic [ref=f9e204]:
+                            - generic [ref=f9e205]: "Legend: {{pod}} on {{node}}"
+                            - generic [ref=f9e206]: "Format: Time series"
+                            - generic [ref=f9e207]: "Step: auto"
+                            - generic [ref=f9e208]: "Type: Range"
+                            - generic [ref=f9e209]: "Exemplars: false"
+              - generic [ref=f9e210]:
+                - button "Add query" [ref=f9e211] [cursor=pointer]
+                - button "Query history" [ref=f9e215] [cursor=pointer]
+                - button "Query inspector" [ref=f9e219] [cursor=pointer]
+            - generic:
+              - main
@@ -0,0 +1,4 @@
+- main [ref=f12e3]:
+  - generic [ref=f12e4]:
+    - progressbar "Contents" [ref=f12e5]
+    - paragraph [ref=f12e8]: Loading the Administration Console
@@ -0,0 +1,4 @@
+- generic [active] [ref=f15e1]:
+  - progressbar "Loading" [ref=f15e4]
+  - generic:
+    - list
@@ -0,0 +1 @@
+- generic [active] [ref=f18e1]: Not Found
@@ -0,0 +1,151 @@
+- generic [active] [ref=e1]:
+  - generic [ref=e4]:
+    - link "Skip to main content" [ref=e5] [cursor=pointer]:
+      - /url: "#pageContent"
+    - banner [ref=e7]:
+      - generic [ref=e8]:
+        - link [ref=e10] [cursor=pointer]:
+          - /url: /
+          - img "Grafana" [ref=e11]
+        - generic [ref=e14]:
+          - button "Search or jump to..." [ref=e18] [cursor=pointer]
+          - generic [ref=e19]: ctrl+k
+        - generic [ref=e23]:
+          - button "New" [ref=e24] [cursor=pointer]
+          - button "Help" [ref=e30] [cursor=pointer]
+          - button "News" [ref=e33] [cursor=pointer]
+          - button "Profile" [ref=e36] [cursor=pointer]:
+            - img "User avatar" [ref=e37]
+      - generic [ref=e38]:
+        - button "Open menu" [ref=e40] [cursor=pointer]
+        - navigation "Breadcrumbs" [ref=e43]:
+          - list [ref=e44]:
+            - listitem [ref=e45]:
+              - link "Home" [ref=e46] [cursor=pointer]:
+                - /url: /
+            - listitem [ref=e50]:
+              - link "Explore" [ref=e51] [cursor=pointer]:
+                - /url: /explore
+            - listitem [ref=e55]:
+              - generic "Prometheus" [ref=e56]
+        - generic [ref=e57]:
+          - generic [ref=e60]:
+            - button "Copy shortened URL" [ref=e61] [cursor=pointer]
+            - button "Open copy link options" [ref=e64] [cursor=pointer]
+          - button "Toggle top search bar" [ref=e68] [cursor=pointer]
+    - main [ref=e74]:
+      - generic [ref=e76]:
+        - heading "Explore" [level=1] [ref=e77]
+        - generic [ref=e82]:
+          - navigation "Explore toolbar" [ref=e84]:
+            - navigation "Search links" [ref=e86]:
+              - generic [ref=e87]:
+                - button "Content outline" [expanded] [ref=e89] [cursor=pointer]:
+                  - generic [ref=e92]: Outline
+                - generic [ref=e97] [cursor=pointer]:
+                  - img "Prometheus logo" [ref=e99]
+                  - textbox "Select a data source" [ref=e100]:
+                    - /placeholder: ""
+            - generic [ref=e104]:
+              - button "Split the pane" [ref=e106] [cursor=pointer]:
+                - generic [ref=e109]: Split
+              - button "Add" [ref=e111] [cursor=pointer]
+              - generic [ref=e116]:
+                - 'button "Time range selected: Last 45 minutes" [ref=e117] [cursor=pointer]'
+                - button "Zoom out time range" [ref=e122] [cursor=pointer]
+              - generic [ref=e126]:
+                - button "Run query" [ref=e127] [cursor=pointer]
+                - button "Auto refresh turned off. Choose refresh time interval" [ref=e131] [cursor=pointer]
+          - generic [ref=e135]:
+            - generic [ref=e139]:
+              - button "Collapse outline" [expanded] [ref=e141] [cursor=pointer]:
+                - img "arrow-from-right" [ref=e142]
+              - button "Queries" [ref=e145] [cursor=pointer]:
+                - img "arrow" [ref=e146]
+              - button "Graph" [ref=e150] [cursor=pointer]:
+                - img "graph-bar" [ref=e151]
+            - generic [ref=e158]:
+              - generic [ref=e160]:
+                - generic "Query editor row" [ref=e163]:
+                  - generic [ref=e164]:
+                    - generic [ref=e166]:
+                      - generic [ref=e167]:
+                        - button "Collapse query row" [expanded] [ref=e168] [cursor=pointer]
+                        - generic [ref=e171]:
+                          - button "Query editor row title A" [ref=e172] [cursor=pointer]:
+                            - generic [ref=e173]: A
+                          - emphasis [ref=e174]: (Prometheus)
+                      - generic [ref=e175]:
+                        - button "Show data source help" [ref=e177] [cursor=pointer]
+                        - button "Duplicate query" [ref=e181] [cursor=pointer]
+                        - button "Hide response" [ref=e185] [cursor=pointer]
+                        - button "Remove query" [ref=e189] [cursor=pointer]
+                        - button "Drag and drop to reorder" [ref=e192]:
+                          - img "Drag and drop to reorder" [ref=e193]
+                    - generic [ref=e196]:
+                      - generic [ref=e197]:
+                        - button "Kick start your query" [ref=e198] [cursor=pointer]
+                        - generic [ref=e201]:
+                          - generic [ref=e202] [cursor=pointer]: Explain
+                          - generic [ref=e203]:
+                            - checkbox "Explain Toggle switch" [ref=e204]
+                            - generic "Toggle switch" [ref=e205] [cursor=pointer]
+                        - radiogroup [ref=e210]:
+                          - generic [ref=e211]:
+                            - radio "Builder" [ref=e212] [cursor=pointer]
+                            - generic [ref=e213] [cursor=pointer]: Builder
+                          - generic [ref=e214]:
+                            - radio "Code" [checked] [ref=e215] [cursor=pointer]
+                            - generic [ref=e216] [cursor=pointer]: Code
+                      - generic [ref=e218]:
+                        - generic [ref=e220]:
+                          - button "Metrics browser" [ref=e221] [cursor=pointer]
+                          - code [ref=e228]:
+                            - generic [ref=e229]:
+                              - generic [ref=e234]: vendor_cluster_size
+                              - textbox "Editor content;Press Alt+F1 for Accessibility Options." [ref=e239]: vendor_cluster_size
+                        - 'button "Options Legend: {{pod}} on {{node}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=e245] [cursor=pointer]':
+                          - generic [ref=e249]:
+                            - heading "Options" [level=6] [ref=e250]
+                            - generic [ref=e251]:
+                              - generic [ref=e252]: "Legend: {{pod}} on {{node}}"
+                              - generic [ref=e253]: "Format: Time series"
+                              - generic [ref=e254]: "Step: auto"
+                              - generic [ref=e255]: "Type: Range"
+                              - generic [ref=e256]: "Exemplars: false"
+                - generic [ref=e257]:
+                  - button "Add query" [ref=e258] [cursor=pointer]
+                  - button "Query history" [ref=e262] [cursor=pointer]
+                  - button "Query inspector" [ref=e266] [cursor=pointer]
+              - main [ref=e270]:
+                - region [ref=e272]:
+                  - generic [ref=e273]:
+                    - heading "Graph" [level=2] [ref=e275]
+                    - radiogroup [ref=e278]:
+                      - generic [ref=e279]:
+                        - radio "Lines" [checked] [ref=e280] [cursor=pointer]
+                        - generic [ref=e281] [cursor=pointer]: Lines
+                      - generic [ref=e282]:
+                        - radio "Bars" [ref=e283] [cursor=pointer]
+                        - generic [ref=e284] [cursor=pointer]: Bars
+                      - generic [ref=e285]:
+                        - radio "Points" [ref=e286] [cursor=pointer]
+                        - generic [ref=e287] [cursor=pointer]: Points
+                      - generic [ref=e288]:
+                        - radio "Stacked lines" [ref=e289] [cursor=pointer]
+                        - generic [ref=e290] [cursor=pointer]: Stacked lines
+                      - generic [ref=e291]:
+                        - radio "Stacked bars" [ref=e292] [cursor=pointer]
+                        - generic [ref=e293] [cursor=pointer]: Stacked bars
+                  - list [ref=e302]:
+                    - listitem [ref=e303]:
+                      - button "keycloak-0 on kc-lab-2" [ref=e307] [cursor=pointer]
+                    - listitem [ref=e308]:
+                      - button "keycloak-0 on kc-lab-2" [ref=e312] [cursor=pointer]
+                    - listitem [ref=e313]:
+                      - button "keycloak-1 on kc-lab-1" [ref=e317] [cursor=pointer]
+  - generic [ref=e322]:
+    - alert
+    - alert
+    - complementary
+    - complementary
@@ -0,0 +1,145 @@
+- generic [active] [ref=f3e1]:
+  - generic [ref=f3e4]:
+    - link "Skip to main content" [ref=f3e5] [cursor=pointer]:
+      - /url: "#pageContent"
+    - banner [ref=f3e7]:
+      - generic [ref=f3e8]:
+        - link [ref=f3e10] [cursor=pointer]:
+          - /url: /
+          - img "Grafana" [ref=f3e11]
+        - generic [ref=f3e14]:
+          - button "Search or jump to..." [ref=f3e18] [cursor=pointer]
+          - generic [ref=f3e19]: ctrl+k
+        - generic [ref=f3e23]:
+          - button "New" [ref=f3e24] [cursor=pointer]
+          - button "Help" [ref=f3e30] [cursor=pointer]
+          - button "News" [ref=f3e33] [cursor=pointer]
+          - button "Profile" [ref=f3e36] [cursor=pointer]:
+            - img "User avatar" [ref=f3e37]
+      - generic [ref=f3e38]:
+        - button "Open menu" [ref=f3e40] [cursor=pointer]
+        - navigation "Breadcrumbs" [ref=f3e43]:
+          - list [ref=f3e44]:
+            - listitem [ref=f3e45]:
+              - link "Home" [ref=f3e46] [cursor=pointer]:
+                - /url: /
+            - listitem [ref=f3e50]:
+              - link "Explore" [ref=f3e51] [cursor=pointer]:
+                - /url: /explore
+            - listitem [ref=f3e55]:
+              - generic "Prometheus" [ref=f3e56]
+        - generic [ref=f3e57]:
+          - generic [ref=f3e60]:
+            - button "Copy shortened URL" [ref=f3e61] [cursor=pointer]
+            - button "Open copy link options" [ref=f3e64] [cursor=pointer]
+          - button "Toggle top search bar" [ref=f3e68] [cursor=pointer]
+    - main [ref=f3e74]:
+      - generic [ref=f3e76]:
+        - heading "Explore" [level=1] [ref=f3e77]
+        - generic [ref=f3e82]:
+          - navigation "Explore toolbar" [ref=f3e84]:
+            - navigation "Search links" [ref=f3e86]:
+              - generic [ref=f3e87]:
+                - button "Content outline" [expanded] [ref=f3e89] [cursor=pointer]:
+                  - generic [ref=f3e92]: Outline
+                - generic [ref=f3e97] [cursor=pointer]:
+                  - img "Prometheus logo" [ref=f3e99]
+                  - textbox "Select a data source" [ref=f3e100]:
+                    - /placeholder: ""
+            - generic [ref=f3e104]:
+              - button "Split the pane" [ref=f3e106] [cursor=pointer]:
+                - generic [ref=f3e109]: Split
+              - button "Add" [ref=f3e111] [cursor=pointer]
+              - generic [ref=f3e116]:
+                - 'button "Time range selected: Last 20 minutes" [ref=f3e117] [cursor=pointer]'
+                - button "Zoom out time range" [ref=f3e122] [cursor=pointer]
+              - generic [ref=f3e126]:
+                - button "Run query" [ref=f3e127] [cursor=pointer]
+                - button "Auto refresh turned off. Choose refresh time interval" [ref=f3e131] [cursor=pointer]
+          - generic [ref=f3e135]:
+            - generic [ref=f3e139]:
+              - button "Collapse outline" [expanded] [ref=f3e141] [cursor=pointer]:
+                - img "arrow-from-right" [ref=f3e142]
+              - button "Queries" [ref=f3e145] [cursor=pointer]:
+                - img "arrow" [ref=f3e146]
+              - button "Graph" [ref=f3e150] [cursor=pointer]:
+                - img "graph-bar" [ref=f3e151]
+            - generic [ref=f3e158]:
+              - generic [ref=f3e160]:
+                - generic "Query editor row" [ref=f3e163]:
+                  - generic [ref=f3e164]:
+                    - generic [ref=f3e166]:
+                      - generic [ref=f3e167]:
+                        - button "Collapse query row" [expanded] [ref=f3e168] [cursor=pointer]
+                        - generic [ref=f3e171]:
+                          - button "Query editor row title A" [ref=f3e172] [cursor=pointer]:
+                            - generic [ref=f3e173]: A
+                          - emphasis [ref=f3e174]: (Prometheus)
+                      - generic [ref=f3e175]:
+                        - button "Show data source help" [ref=f3e177] [cursor=pointer]
+                        - button "Duplicate query" [ref=f3e181] [cursor=pointer]
+                        - button "Hide response" [ref=f3e185] [cursor=pointer]
+                        - button "Remove query" [ref=f3e189] [cursor=pointer]
+                        - button "Drag and drop to reorder" [ref=f3e192]:
+                          - img "Drag and drop to reorder" [ref=f3e193]
+                    - generic [ref=f3e196]:
+                      - generic [ref=f3e197]:
+                        - button "Kick start your query" [ref=f3e198] [cursor=pointer]
+                        - generic [ref=f3e201]:
+                          - generic [ref=f3e202] [cursor=pointer]: Explain
+                          - generic [ref=f3e203]:
+                            - checkbox "Explain Toggle switch" [ref=f3e204]
+                            - generic "Toggle switch" [ref=f3e205] [cursor=pointer]
+                        - radiogroup [ref=f3e210]:
+                          - generic [ref=f3e211]:
+                            - radio "Builder" [ref=f3e212] [cursor=pointer]
+                            - generic [ref=f3e213] [cursor=pointer]: Builder
+                          - generic [ref=f3e214]:
+                            - radio "Code" [checked] [ref=f3e215] [cursor=pointer]
+                            - generic [ref=f3e216] [cursor=pointer]: Code
+                      - generic [ref=f3e218]:
+                        - generic [ref=f3e220]:
+                          - button "Loading metrics..." [disabled] [ref=f3e221] [cursor=pointer]
+                          - code [ref=f3e228]:
+                            - generic [ref=f3e229]:
+                              - generic [ref=f3e234]: "up{job=\"keycloak\"}"
+                              - textbox "Editor content;Press Alt+F1 for Accessibility Options." [ref=f3e239]: "up{job=\"keycloak\"}"
+                        - 'button "Options Legend: up — {{pod}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f3e245] [cursor=pointer]':
+                          - generic [ref=f3e249]:
+                            - heading "Options" [level=6] [ref=f3e250]
+                            - generic [ref=f3e251]:
+                              - generic [ref=f3e252]: "Legend: up — {{pod}}"
+                              - generic [ref=f3e253]: "Format: Time series"
+                              - generic [ref=f3e254]: "Step: auto"
+                              - generic [ref=f3e255]: "Type: Range"
+                              - generic [ref=f3e256]: "Exemplars: false"
+                - generic [ref=f3e257]:
+                  - button "Add query" [ref=f3e258] [cursor=pointer]
+                  - button "Query history" [ref=f3e262] [cursor=pointer]
+                  - button "Query inspector" [ref=f3e266] [cursor=pointer]
+              - main [ref=f3e270]:
+                - region [ref=f3e272]:
+                  - generic [ref=f3e273]:
+                    - heading "Graph" [level=2] [ref=f3e275]
+                    - radiogroup [ref=f3e278]:
+                      - generic [ref=f3e279]:
+                        - radio "Lines" [checked] [ref=f3e280] [cursor=pointer]
+                        - generic [ref=f3e281] [cursor=pointer]: Lines
+                      - generic [ref=f3e282]:
+                        - radio "Bars" [ref=f3e283] [cursor=pointer]
+                        - generic [ref=f3e284] [cursor=pointer]: Bars
+                      - generic [ref=f3e285]:
+                        - radio "Points" [ref=f3e286] [cursor=pointer]
+                        - generic [ref=f3e287] [cursor=pointer]: Points
+                      - generic [ref=f3e288]:
+                        - radio "Stacked lines" [ref=f3e289] [cursor=pointer]
+                        - generic [ref=f3e290] [cursor=pointer]: Stacked lines
+                      - generic [ref=f3e291]:
+                        - radio "Stacked bars" [ref=f3e292] [cursor=pointer]
+                        - generic [ref=f3e293] [cursor=pointer]: Stacked bars
+                  - generic [ref=f3e294]: Loading plugin panel...
+  - generic [ref=f3e299]:
+    - alert
+    - alert
+    - complementary
+    - complementary
@@ -0,0 +1,44 @@
+- main [ref=f6e7]:
+  - generic [ref=f6e9]:
+    - generic [ref=f6e11]:
+      - generic [ref=f6e12]:
+        - img "Grafana" [ref=f6e13]
+        - heading "Welcome to Grafana" [level=1] [ref=f6e15]
+      - generic [ref=f6e19]:
+        - generic [ref=f6e20]:
+          - generic [ref=f6e21]: Email or username
+          - textbox "Email or username" [active] [ref=f6e28]:
+            - /placeholder: email or username
+        - generic [ref=f6e29]:
+          - generic [ref=f6e30]: Password
+          - generic [ref=f6e36]:
+            - textbox "Password" [ref=f6e37]:
+              - /placeholder: password
+            - switch "Show password" [ref=f6e39] [cursor=pointer]
+        - button "Log in" [ref=f6e42] [cursor=pointer]
+        - link "Forgot your password?" [ref=f6e45] [cursor=pointer]:
+          - /url: /user/password/send-reset-email
+    - list [ref=f6e49]:
+      - listitem [ref=f6e50]:
+        - link "Documentation" [ref=f6e53] [cursor=pointer]:
+          - /url: https://grafana.com/docs/grafana/latest/?utm_source=grafana_footer
+        - text: "|"
+      - listitem [ref=f6e54]:
+        - link "Support" [ref=f6e57] [cursor=pointer]:
+          - /url: https://grafana.com/products/enterprise/?utm_source=grafana_footer
+        - text: "|"
+      - listitem [ref=f6e58]:
+        - link "Community" [ref=f6e61] [cursor=pointer]:
+          - /url: https://community.grafana.com/?utm_source=grafana_footer
+        - text: "|"
+      - listitem [ref=f6e62]:
+        - link "Open Source" [ref=f6e63] [cursor=pointer]:
+          - /url: https://grafana.com/oss/grafana?utm_source=grafana_footer
+        - text: "|"
+      - listitem [ref=f6e64]:
+        - link "Grafana v11.4.0 (b58701869e)" [ref=f6e65] [cursor=pointer]:
+          - /url: https://github.com/grafana/grafana/blob/main/CHANGELOG.md
+        - text: "|"
+      - listitem [ref=f6e66]:
+        - link "New version available!" [ref=f6e69] [cursor=pointer]:
+          - /url: https://grafana.com/grafana/download?utm_source=grafana_footer
@@ -0,0 +1,159 @@
+- generic [active] [ref=f9e1]:
+  - generic [ref=f9e4]:
+    - link "Skip to main content" [ref=f9e5] [cursor=pointer]:
+      - /url: "#pageContent"
+    - banner [ref=f9e7]:
+      - generic [ref=f9e8]:
+        - link [ref=f9e10] [cursor=pointer]:
+          - /url: /
+          - img "Grafana" [ref=f9e11]
+        - generic [ref=f9e14]:
+          - button "Search or jump to..." [ref=f9e18] [cursor=pointer]
+          - generic [ref=f9e19]: ctrl+k
+        - generic [ref=f9e23]:
+          - button "New" [ref=f9e24] [cursor=pointer]
+          - button "Help" [ref=f9e30] [cursor=pointer]
+          - button "News" [ref=f9e33] [cursor=pointer]
+          - button "Profile" [ref=f9e36] [cursor=pointer]:
+            - img "User avatar" [ref=f9e37]
+      - generic [ref=f9e38]:
+        - button "Open menu" [ref=f9e40] [cursor=pointer]
+        - navigation "Breadcrumbs" [ref=f9e43]:
+          - list [ref=f9e44]:
+            - listitem [ref=f9e45]:
+              - link "Home" [ref=f9e46] [cursor=pointer]:
+                - /url: /
+            - listitem [ref=f9e50]:
+              - link "Explore" [ref=f9e51] [cursor=pointer]:
+                - /url: /explore
+            - listitem [ref=f9e55]:
+              - generic "Prometheus" [ref=f9e56]
+        - generic [ref=f9e57]:
+          - generic [ref=f9e60]:
+            - button "Copy shortened URL" [ref=f9e61] [cursor=pointer]
+            - button "Open copy link options" [ref=f9e64] [cursor=pointer]
+          - button "Toggle top search bar" [ref=f9e68] [cursor=pointer]
+    - main [ref=f9e74]:
+      - generic [ref=f9e76]:
+        - heading "Explore" [level=1] [ref=f9e77]
+        - generic [ref=f9e82]:
+          - navigation "Explore toolbar" [ref=f9e84]:
+            - navigation "Search links" [ref=f9e86]:
+              - generic [ref=f9e87]:
+                - button "Content outline" [expanded] [ref=f9e89] [cursor=pointer]:
+                  - generic [ref=f9e92]: Outline
+                - generic [ref=f9e97] [cursor=pointer]:
+                  - img "Prometheus logo" [ref=f9e99]
+                  - textbox "Select a data source" [ref=f9e100]:
+                    - /placeholder: ""
+            - generic [ref=f9e104]:
+              - button "Split the pane" [ref=f9e106] [cursor=pointer]:
+                - generic [ref=f9e109]: Split
+              - button "Add" [ref=f9e111] [cursor=pointer]
+              - generic [ref=f9e116]:
+                - 'button "Time range selected: Last 55 minutes" [ref=f9e117] [cursor=pointer]'
+                - button "Zoom out time range" [ref=f9e122] [cursor=pointer]
+              - generic [ref=f9e126]:
+                - button "Run query" [ref=f9e127] [cursor=pointer]
+                - button "Auto refresh turned off. Choose refresh time interval" [ref=f9e131] [cursor=pointer]
+          - generic [ref=f9e135]:
+            - generic [ref=f9e139]:
+              - button "Collapse outline" [expanded] [ref=f9e141] [cursor=pointer]:
+                - img "arrow-from-right" [ref=f9e142]
+              - button "Queries" [ref=f9e145] [cursor=pointer]:
+                - img "arrow" [ref=f9e146]
+              - button "Graph" [ref=f9e150] [cursor=pointer]:
+                - img "graph-bar" [ref=f9e151]
+            - generic [ref=f9e158]:
+              - generic [ref=f9e160]:
+                - generic "Query editor row" [ref=f9e163]:
+                  - generic [ref=f9e164]:
+                    - generic [ref=f9e166]:
+                      - generic [ref=f9e167]:
+                        - button "Collapse query row" [expanded] [ref=f9e168] [cursor=pointer]
+                        - generic [ref=f9e171]:
+                          - button "Query editor row title A" [ref=f9e172] [cursor=pointer]:
+                            - generic [ref=f9e173]: A
+                          - emphasis [ref=f9e174]: (Prometheus)
+                      - generic [ref=f9e175]:
+                        - button "Show data source help" [ref=f9e177] [cursor=pointer]
+                        - button "Duplicate query" [ref=f9e181] [cursor=pointer]
+                        - button "Hide response" [ref=f9e185] [cursor=pointer]
+                        - button "Remove query" [ref=f9e189] [cursor=pointer]
+                        - button "Drag and drop to reorder" [ref=f9e192]:
+                          - img "Drag and drop to reorder" [ref=f9e193]
+                    - generic [ref=f9e196]:
+                      - generic [ref=f9e197]:
+                        - button "Kick start your query" [ref=f9e198] [cursor=pointer]
+                        - generic [ref=f9e201]:
+                          - generic [ref=f9e202] [cursor=pointer]: Explain
+                          - generic [ref=f9e203]:
+                            - checkbox "Explain Toggle switch" [ref=f9e204]
+                            - generic "Toggle switch" [ref=f9e205] [cursor=pointer]
+                        - radiogroup [ref=f9e210]:
+                          - generic [ref=f9e211]:
+                            - radio "Builder" [ref=f9e212] [cursor=pointer]
+                            - generic [ref=f9e213] [cursor=pointer]: Builder
+                          - generic [ref=f9e214]:
+                            - radio "Code" [checked] [ref=f9e215] [cursor=pointer]
+                            - generic [ref=f9e216] [cursor=pointer]: Code
+                      - generic [ref=f9e218]:
+                        - generic [ref=f9e220]:
+                          - button "Metrics browser" [ref=f9e221] [cursor=pointer]
+                          - code [ref=f9e228]:
+                            - generic [ref=f9e229]:
+                              - generic [ref=f9e234]: "up{job=~\"keycloak|node-exporter\"}"
+                              - textbox "Editor content;Press Alt+F1 for Accessibility Options." [ref=f9e239]: "up{job=~\"keycloak|node-exporter\"}"
+                        - 'button "Options Legend: {{job}} — {{pod}}{{node}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f9e245] [cursor=pointer]':
+                          - generic [ref=f9e249]:
+                            - heading "Options" [level=6] [ref=f9e250]
+                            - generic [ref=f9e251]:
+                              - generic [ref=f9e252]: "Legend: {{job}} — {{pod}}{{node}}"
+                              - generic [ref=f9e253]: "Format: Time series"
+                              - generic [ref=f9e254]: "Step: auto"
+                              - generic [ref=f9e255]: "Type: Range"
+                              - generic [ref=f9e256]: "Exemplars: false"
+                - generic [ref=f9e257]:
+                  - button "Add query" [ref=f9e258] [cursor=pointer]
+                  - button "Query history" [ref=f9e262] [cursor=pointer]
+                  - button "Query inspector" [ref=f9e266] [cursor=pointer]
+              - main [ref=f9e270]:
+                - region [ref=f9e272]:
+                  - generic [ref=f9e273]:
+                    - heading "Graph" [level=2] [ref=f9e275]
+                    - radiogroup [ref=f9e278]:
+                      - generic [ref=f9e279]:
+                        - radio "Lines" [checked] [ref=f9e280] [cursor=pointer]
+                        - generic [ref=f9e281] [cursor=pointer]: Lines
+                      - generic [ref=f9e282]:
+                        - radio "Bars" [ref=f9e283] [cursor=pointer]
+                        - generic [ref=f9e284] [cursor=pointer]: Bars
+                      - generic [ref=f9e285]:
+                        - radio "Points" [ref=f9e286] [cursor=pointer]
+                        - generic [ref=f9e287] [cursor=pointer]: Points
+                      - generic [ref=f9e288]:
+                        - radio "Stacked lines" [ref=f9e289] [cursor=pointer]
+                        - generic [ref=f9e290] [cursor=pointer]: Stacked lines
+                      - generic [ref=f9e291]:
+                        - radio "Stacked bars" [ref=f9e292] [cursor=pointer]
+                        - generic [ref=f9e293] [cursor=pointer]: Stacked bars
+                  - list [ref=f9e311]:
+                    - listitem [ref=f9e312]:
+                      - button "keycloak — keycloak-1kc-lab-1" [ref=f9e316] [cursor=pointer]
+                    - listitem [ref=f9e317]:
+                      - button "keycloak — keycloak-1kc-lab-1" [ref=f9e321] [cursor=pointer]
+                    - listitem [ref=f9e322]:
+                      - button "keycloak — keycloak-0kc-lab-2" [ref=f9e326] [cursor=pointer]
+                    - listitem [ref=f9e327]:
+                      - button "keycloak — keycloak-0kc-lab-2" [ref=f9e331] [cursor=pointer]
+                    - listitem [ref=f9e332]:
+                      - button "keycloak — keycloak-0kc-lab-2" [ref=f9e336] [cursor=pointer]
+                    - listitem [ref=f9e337]:
+                      - button "node-exporter — kc-lab-1" [ref=f9e341] [cursor=pointer]
+                    - listitem [ref=f9e342]:
+                      - button "node-exporter — kc-lab-2" [ref=f9e346] [cursor=pointer]
+  - generic [ref=f9e351]:
+    - alert
+    - alert
+    - complementary
+    - complementary
@@ -15,6 +15,11 @@ Keycloak을 중심으로 네 가지 브라우저 인증 통합 패턴을 같은
 - AP3: Backend-for-Frontend (BFF)
 - AP4: Edge forward-auth

+세션 저장소·refresh token 경쟁·장애 복구는 네 패턴을 가로지르는 별도 축으로
+`develop-keycloak-session-store` 브랜치에서 진행합니다. 계획과 진행 상황은
+[`docs/session-store-lab-roadmap.md`](docs/session-store-lab-roadmap.md)에
+있습니다.
+
 현재 `develop`의 공통 baseline은 Keycloak, PostgreSQL, Spring Boot API,
 nginx를 Docker Compose로 실행하는 토대입니다. 패턴별 구현은 이 baseline
 위에서 별도 브랜치로 진행합니다.
@@ -1,6 +1,8 @@
 package com.example.keycloakpattern;

+import java.util.Collections;
 import java.util.LinkedHashMap;
+import java.util.List;
 import java.util.Map;

 import org.springframework.security.core.annotation.AuthenticationPrincipal;
@@ -9,6 +11,8 @@ import org.springframework.web.bind.annotation.GetMapping;
 import org.springframework.web.bind.annotation.RequestMapping;
 import org.springframework.web.bind.annotation.RestController;

+import jakarta.servlet.http.HttpServletRequest;
+
@RestController
@RequestMapping("/api")
 public class ApiController {
@@ -18,6 +22,38 @@ public class ApiController {
        return Map.of("status", "ok", "service", "keycloak-pattern-api");
    }

+    /**
+     * Reflects what actually reached the application after the proxy chain.
+     *
+     * <p>The reverse proxy contract is defined in {@code docs/reverse-proxy-headers.md}
+     * for a single nginx hop. The lab runs {@code nginx -> Traefik -> pod}, so this
+     * endpoint exists to measure the two-hop result instead of assuming it.
+     *
+     * <p>{@code scheme}, {@code secure} and {@code requestUrl} are the values Keycloak
+     * uses to build the {@code iss} claim and redirect URLs. If forwarded headers are
+     * lost or rewritten, the mismatch shows up here first.
+     */
+    @GetMapping("/echo")
+    public Map<String, Object> echo(HttpServletRequest request) {
+        Map<String, List<String>> headers = new LinkedHashMap<>();
+        for (String name : Collections.list(request.getHeaderNames())) {
+            headers.put(name.toLowerCase(), Collections.list(request.getHeaders(name)));
+        }
+
+        Map<String, Object> response = new LinkedHashMap<>();
+        response.put("headers", headers);
+        response.put("remoteAddr", request.getRemoteAddr());
+        // Pod IP. Identifies which replica answered, which is what makes the
+        // host nginx upstream distribution and the sticky-session switch observable.
+        response.put("localAddr", request.getLocalAddr());
+        response.put("scheme", request.getScheme());
+        response.put("secure", request.isSecure());
+        response.put("serverName", request.getServerName());
+        response.put("serverPort", request.getServerPort());
+        response.put("requestUrl", request.getRequestURL().toString());
+        return response;
+    }
+
    @GetMapping("/me")
    public Map<String, Object> currentUser(@AuthenticationPrincipal Jwt jwt) {
        Map<String, Object> response = new LinkedHashMap<>();
@@ -17,7 +17,8 @@ public class SecurityConfig {
            .sessionManagement(session ->
                session.sessionCreationPolicy(SessionCreationPolicy.STATELESS))
            .authorizeHttpRequests(authorize -> authorize
-                .requestMatchers("/actuator/health", "/actuator/health/**", "/api/public")
+                .requestMatchers("/actuator/health", "/actuator/health/**", "/api/public",
+                    "/api/echo")
                .permitAll()
                .anyRequest()
                .authenticated())
@@ -1,9 +1,19 @@
 server:
  port: ${SERVER_PORT:8081}
+  # Spring ignores X-Forwarded-* unless this is set, so scheme/secure/requestUrl
+  # report the raw connection by default. Keycloak has the same opt-in as
+  # KC_PROXY_HEADERS. Flipping this to "native" is what the two-hop measurement
+  # compares against.
+  forward-headers-strategy: ${SERVER_FORWARD_HEADERS_STRATEGY:none}

 spring:
  application:
    name: keycloak-pattern-api
+  jackson:
+    serialization:
+      # /api/echo is read by humans and captured as evidence screenshots, so the
+      # response is indented rather than relying on a browser's JSON viewer.
+      indent-output: true
  security:
    oauth2:
      resourceserver:
@@ -25,6 +25,18 @@ class ApiSecurityTest {
            .andExpect(jsonPath("$.status").value("ok"));
    }

+    @Test
+    void echoEndpointReflectsForwardedHeadersWithoutAuthentication() throws Exception {
+        mockMvc.perform(get("/api/echo")
+                .header("X-Forwarded-Proto", "https")
+                .header("X-Forwarded-Host", "app1.example.test"))
+            .andExpect(status().isOk())
+            .andExpect(jsonPath("$.headers['x-forwarded-proto'][0]").value("https"))
+            .andExpect(jsonPath("$.headers['x-forwarded-host'][0]").value("app1.example.test"))
+            .andExpect(jsonPath("$.requestUrl").exists())
+            .andExpect(jsonPath("$.remoteAddr").exists());
+    }
+
    @Test
    void protectedEndpointRejectsAnonymousRequests() throws Exception {
        mockMvc.perform(get("/api/me"))
@@ -0,0 +1,136 @@
+# Session store lab
+
+세션 저장소·refresh token 경쟁·장애 복구를 검증하는 2노드 k3s 실험대.
+네 인증 패턴(AP1~AP4)을 가로지르는 공통층이므로 별도 축으로 관리한다.
+
+이 문서는 **절차**만 담는다.
+
+| 문서 | 내용 |
+|---|---|
+| [`docs/session-store-lab-roadmap.md`](../../docs/session-store-lab-roadmap.md) | 이 축의 계획과 진행 상황 |
+| [`docs/session-lab-concepts.md`](../../docs/session-lab-concepts.md) | 등장 개념 전체 |
+| [`docs/session-lab-operations.md`](../../docs/session-lab-operations.md) | 관측 도구 · 자주 쓰는 명령 · 훈련 |
+| [`docs/two-hop-proxy-header-contract.md`](../../docs/two-hop-proxy-header-contract.md) | 첫 실험 결과 |
+
+## 토폴로지
+
+```
+  브라우저 / SSH  (tailnet)
+        │  https://{auth,app1,app2}.hyeonworks.com → 100.83.212.4
+        ▼
+  lab host  ── nginx :443  TLS 종료 · X-Forwarded-* 주입
+        │      nginx :80   301 → https
+        │
+        │  virbr0  192.168.122.0/24 (libvirt NAT)
+        ├──▶ kc-lab-1  .11  k3s server  Traefik :80
+        └──▶ kc-lab-2  .12  k3s agent   Traefik :80
+                            └──▶ Pod
+```
+
+`nginx → Traefik` **2홉**이 운영 구조와 같다는 점이 이 배치의 핵심이다.
+L7 프록시가 두 겹인 이유는 역할이 다르기 때문이다 — nginx는 바깥세상과의
+접점(TLS·인증서·헤더)을, Traefik은 클러스터 내부의 동적 라우팅을 맡는다.
+
+## 구성 요소
+
+| 경로 | 역할 |
+|---|---|
+| `cloud-init/kc-lab.yaml.example` | 게스트 부트스트랩 템플릿 |
+| `host/nginx-keycloak-lab.conf` | lab host의 `sites-available/keycloak-lab` |
+| `k8s/echo.yaml` | 2홉 헤더 계약 측정용 워크로드 |
+| `scripts/rebuild-seed.sh` | cloud-init 시드 ISO 재생성 + 풀 업로드 |
+| `scripts/build-and-import.sh` | 이미지 빌드 → 각 노드 containerd 반입 |
+| `scripts/measure-proxy-headers.sh` | 헤더 계약 실측 |
+| `scripts/verify-lab.sh` | 인프라 상태 점검 |
+
+## 상태 점검
+
+```bash
+./deploy/lab/scripts/verify-lab.sh      # lab host 에서
+```
+
+**`404`가 성공 신호다.** TLS가 종료되고 Traefik까지 도달했으나 매칭되는
+Ingress 규칙이 없다는 뜻이다. `502`나 연결 거부면 체인이 끊긴 것이다.
+
+## 첫 실험 — 2홉 헤더 계약
+
+[`docs/reverse-proxy-headers.md`](../../docs/reverse-proxy-headers.md)의 계약은
+nginx **1홉**을 가정하고 쓰였다. 실제 배치는 2홉이므로, nginx가 세팅한
+`X-Forwarded-*`를 Traefik이 그대로 넘기는지 덮어쓰는지 **측정해서 확인한다.**
+
+이 결론이 뒤의 모든 실험에 깔린다. Keycloak의 `iss` 클레임, redirect URL,
+쿠키 도메인 검증이 전부 이 헤더에 의존하기 때문이다.
+
+```bash
+# 워크스테이션: 이미지 빌드 후 두 노드에 반입
+./deploy/lab/scripts/build-and-import.sh
+
+# lab host: 배포
+kubectl apply -f deploy/lab/k8s/echo.yaml
+kubectl -n header-lab rollout status deployment/echo
+
+# 어디서든: 실측
+./deploy/lab/scripts/measure-proxy-headers.sh
+```
+
+관측 대상은 넷이다.
+
+1. `X-Forwarded-For` — Traefik이 **덧붙이는가 덮어쓰는가**
+2. `X-Forwarded-Proto` / `-Host` / `-Port` — 그대로 전달되는가
+3. **위조 내성** — 클라이언트가 직접 넣은 `X-Forwarded-*`가 앱까지 도달하는가
+4. `scheme` / `secure` / `requestUrl` — Keycloak이 URL을 만들 때 쓰는 값
+
+3번이 신뢰 경계의 핵심이다. 이 헤더들은 누구나 위조할 수 있는 평범한 HTTP
+헤더이므로, 신뢰 경계에 선 프록시가 **반드시 덮어써야** 한다.
+
+## 이미지 배포 경로
+
+k3s는 containerd를 쓰고 이 실험대에는 레지스트리가 없다.
+
+```
+워크스테이션  docker build → docker save
+      │ ssh (lab host 경유)
+      ▼
+게스트        sudo k3s ctr images import
+매니페스트    imagePullPolicy: Never
+```
+
+**두 노드 모두에 반입해야 한다.** 스케줄러가 어느 노드에 배치할지 모른다.
+Keycloak·PostgreSQL·Redis는 공식 이미지를 그대로 당겨오므로 이 경로가
+필요한 것은 자체 빌드 이미지뿐이다.
+
+**lab host에 Docker를 설치하지 않는다.** k3s의 containerd와 이미지 저장소가
+갈려서 `docker build`한 이미지를 k3s가 보지 못하게 된다.
+
+## 게스트 재생성
+
+파괴적 실험 후 초기화하는 경로다.
+
+```bash
+virsh destroy kc-lab-1
+virsh undefine kc-lab-1                       # --remove-all-storage 는 시드 ISO 까지 지운다
+virsh vol-delete --pool default kc-lab-1.qcow2
+
+./deploy/lab/scripts/rebuild-seed.sh 1        # user-data 를 고쳤을 때만
+
+virt-install --name kc-lab-1 --memory 3584 --vcpus 2 \
+  --disk size=20,backing_store=/var/lib/libvirt/images/base.qcow2 \
+  --disk vol=default/seed-kc-lab-1.iso,device=disk,bus=virtio,readonly=on \
+  --network network=default,mac=52:54:00:aa:bb:11 \
+  --import --os-variant debian12 --noautoconsole
+```
+
+시드는 **virtio 디스크**로 붙인다. `virt-install --cloud-init`은 시드를 SATA
+CD-ROM으로 붙이는데, Debian `genericcloud` 이미지는 크기를 줄이려고 물리
+하드웨어 드라이버를 제외해서 **AHCI 장치를 보지 못한다.** 그러면 cloud-init이
+데이터소스를 찾지 못하고 아무 오류도 남기지 않은 채 종료한다. 증상은
+hostname이 `localhost`로 남고 SSH가 `Permission denied (publickey)`로 거부되는
+것뿐이다.
+
+게스트에 들어갈 수 없을 때는 화면을 직접 뜬다.
+
+```bash
+virsh screenshot kc-lab-1 /tmp/kc1.ppm    # 확장자와 무관하게 PNG 로 저장된다
+```
+
+`localhost login:`이면 cloud-init 미실행, `kc-lab-1 login:`이면 실행된 것이다.
@@ -0,0 +1,37 @@
+#cloud-config
+# Template for both lab guests. scripts/rebuild-seed.sh substitutes __NODE__
+# and bakes this into a CIDATA seed image.
+#
+# Copy to kc-lab.yaml and fill the two placeholders. The real file is ignored by
+# git because plain_text_passwd is a credential, however disposable.
+#
+# Indentation is spaces only. YAML forbids tabs, and cloud-init fails silently
+# on a parse error: the guest boots as "localhost" with no user and no way in.
+hostname: kc-lab-__NODE__
+fqdn: kc-lab-__NODE__
+manage_etc_hosts: true
+
+users:
+  - name: donghyeon
+    groups: [sudo]
+    shell: /bin/bash
+    # NOPASSWD is required: the k3s installer and the fault-injection scripts
+    # run non-interactively and would block on a password prompt.
+    sudo: ['ALL=(ALL) NOPASSWD:ALL']
+    # Console-only escape hatch. Without it, a cloud-init failure leaves a guest
+    # that cannot be logged into at all, so its own failure log is unreadable.
+    # ssh_pwauth stays false, so this never widens SSH exposure.
+    lock_passwd: false
+    plain_text_passwd: CHANGE_ME
+    ssh_authorized_keys:
+      # Lab host key: needed because automation runs from the lab host, where
+      # agent forwarding is not available.
+      - CHANGE_ME_LAB_HOST_PUBLIC_KEY
+      # Workstation key: lets ProxyJump reach the guest directly.
+      - CHANGE_ME_WORKSTATION_PUBLIC_KEY
+
+ssh_pwauth: false
+package_update: true
+packages:
+  - curl
+  - nftables
@@ -0,0 +1,56 @@
+# Lab entry point. Deployed on the lab host as
+#   /etc/nginx/sites-available/keycloak-lab
+# and symlinked from sites-enabled/.
+#
+# Arch does not ship the Debian sites-available convention, so nginx.conf needs
+#   include /etc/nginx/sites-enabled/*;
+# inside its http { } block before this file has any effect.
+#
+# This is the outer of two L7 hops. It terminates TLS and hands plain HTTP to
+# the Traefik instance running on each k3s node.
+
+upstream k3s_traefik {
+    # Sticky-session switch. Keycloak recommends affinity on AUTH_SESSION_ID;
+    # ip_hash is the cheap stand-in for a single-browser lab. Leaving it off is
+    # the interesting case: Infinispan still routes correctly, only slower.
+    # ip_hash;
+    server 192.168.122.11:80;
+    server 192.168.122.12:80;
+}
+
+server {
+    listen 80 default_server;
+    server_name _;
+    return 301 https://$host$request_uri;
+}
+
+server {
+    listen 443 ssl default_server;
+    http2 on;
+    server_name _;
+
+    # fullchain.pem, never cert.pem: omitting the intermediates passes on
+    # desktop browsers and fails on mobile and curl.
+    ssl_certificate     /etc/letsencrypt/live/auth.hyeonworks.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/auth.hyeonworks.com/privkey.pem;
+    ssl_protocols       TLSv1.2 TLSv1.3;
+
+    location / {
+        proxy_pass http://k3s_traefik;
+        proxy_http_version 1.1;
+
+        proxy_set_header Host              $host;
+        proxy_set_header X-Forwarded-Host  $host;
+        proxy_set_header X-Forwarded-Proto https;
+        proxy_set_header X-Forwarded-Port  443;
+
+        # $remote_addr, not $proxy_add_x_forwarded_for. This is the trust
+        # boundary: a client-supplied X-Forwarded-For must be discarded, not
+        # extended, or nothing downstream can rely on the value.
+        proxy_set_header X-Forwarded-For   $remote_addr;
+        proxy_set_header X-Real-IP         $remote_addr;
+
+        proxy_read_timeout 3600s;
+        proxy_send_timeout 3600s;
+    }
+}
@@ -0,0 +1,46 @@
+# Experiment A-1 — cut the JGroups transport (TCP 7800) while leaving discovery alone.
+#
+# The point is to separate two things that are easy to conflate:
+#
+#   discovery   how the nodes FIND each other   -> PostgreSQL JGROUPS_PING table
+#   transport   how they actually TALK          -> TCP 7800
+#
+# Blocking only the transport produces a state that cannot happen on a single
+# node: both members stay registered in the database, so each believes the other
+# exists, yet no message gets through.
+#
+#   kubectl apply -f deploy/lab/k8s/a1-block-jgroups-transport.yaml
+#   kubectl -n keycloak-lab delete networkpolicy a1-block-jgroups-transport
+#
+# NetworkPolicy is an ALLOWLIST, not a firewall with deny rules. There is no way
+# to write "deny 7800". The moment a pod is selected by a policy carrying
+# policyTypes: [Ingress], every inbound port is denied unless a rule permits it.
+# So 7800 is blocked by *omission*: 8080 and 9000 are listed, 7800 is not.
+#
+# That makes the two allow rules load-bearing — get them wrong and the experiment
+# measures a dead Keycloak instead of a partitioned cluster:
+#
+#   8080  the HTTP endpoint. Traefik, the other pod's REST calls, and the probe
+#         traffic all arrive here.
+#   9000  the management port: /health/started, /health/ready, /health/live and
+#         /metrics. Losing it means the kubelet fails the readiness probe and
+#         kills the pod — the cluster would break for the wrong reason.
+#
+# Both rules deliberately omit `from:`, which allows those ports from any source.
+# Narrowing the source is not the subject here; the 2-hop experiment already
+# established how to do that by label when it matters.
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: a1-block-jgroups-transport
+  namespace: keycloak-lab
+spec:
+  podSelector:
+    matchLabels:
+      app: keycloak
+  policyTypes: [Ingress]
+  ingress:
+    - ports:
+        - { port: 8080, protocol: TCP }   # HTTP — must stay open
+        - { port: 9000, protocol: TCP }   # health + metrics — must stay open
+    # 7800 is absent on purpose. That is the whole experiment.
@@ -0,0 +1,62 @@
+# Restrict who may reach the echo pods.
+#
+# Traefik is configured to trust X-Forwarded-* from the whole pod CIDR, and the
+# app's Tomcat valve trusts every private range by default. Both are IP-range
+# decisions, so any pod in the cluster can forge those headers by talking to the
+# Service directly and bypassing Traefik entirely. Measured, not hypothetical:
+#
+#   kubectl -n header-lab run t --rm -i --restart=Never --image=curlimages/curl -- \
+#     curl -s http://echo:8081/api/echo -H 'X-Forwarded-Host: evil.example.com'
+#   → serverName evil.example.com, remoteAddr 1.2.3.4
+#
+# A NetworkPolicy closes that path. It selects by label rather than IP, so it
+# survives pod restarts and rescheduling — unlike the trustedIPs list, which
+# could not name Traefik because its IP changes.
+#
+# "Trusting forwarded headers" and "guaranteeing a proxy sits in front" are a
+# pair. Doing only the first leaves this hole.
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: echo-allow-traefik-only
+  namespace: header-lab
+spec:
+  podSelector:
+    matchLabels:
+      app: echo
+  policyTypes:
+    - Ingress
+  ingress:
+    # The proxy itself. namespaceSelector and podSelector in one list item are
+    # ANDed, so this is "traefik pods in kube-system" and nothing else.
+    - from:
+        - namespaceSelector:
+            matchLabels:
+              kubernetes.io/metadata.name: kube-system
+          podSelector:
+            matchLabels:
+              app.kubernetes.io/name: traefik
+      ports:
+        - protocol: TCP
+          port: 8081
+
+    # kubelet readiness/liveness probes originate from the node, not from a pod,
+    # so they need their own rule. Without it the probes fail and the pods are
+    # restarted in a loop.
+    #
+    # The probe's source address is the node's flannel bridge (cni0), which
+    # holds the first address of that node's /24:
+    #   kc-lab-1  10.42.0.1     kc-lab-2  10.42.1.1
+    # Listing them as /32 keeps this rule from re-admitting arbitrary pods,
+    # which a broader 10.42.0.0/16 block would do and would undo the policy.
+    #
+    # Adding a node means adding its gateway here. Verify with:
+    #   kubectl get nodes -o jsonpath='{range .items[*]}{.spec.podCIDR}{"\n"}{end}'
+    - from:
+        - ipBlock:
+            cidr: 10.42.0.1/32
+        - ipBlock:
+            cidr: 10.42.1.1/32
+      ports:
+        - protocol: TCP
+          port: 8081
@@ -0,0 +1,113 @@
+# Header echo workload for the two-hop proxy contract measurement.
+#
+#   browser -> host nginx (TLS termination) -> Traefik -> this pod
+#
+# The image is built from backend/ and imported straight into each node's
+# containerd, so imagePullPolicy must stay Never. See scripts/build-and-import.sh.
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: header-lab
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: echo
+  namespace: header-lab
+spec:
+  replicas: 2
+  selector:
+    matchLabels:
+      app: echo
+  template:
+    metadata:
+      labels:
+        app: echo
+    spec:
+      # One replica per node so the sticky-session switch on the host nginx
+      # upstream has something observable to route between.
+      topologySpreadConstraints:
+        - maxSkew: 1
+          topologyKey: kubernetes.io/hostname
+          whenUnsatisfiable: ScheduleAnyway
+          labelSelector:
+            matchLabels:
+              app: echo
+      containers:
+        - name: echo
+          image: keycloak-pattern-api:lab
+          imagePullPolicy: Never
+          ports:
+            - containerPort: 8081
+              name: http
+          env:
+            - name: SERVER_PORT
+              value: "8081"
+            # "none" makes the app report the raw connection, so scheme/secure/
+            # requestUrl show what arrives without any forwarded-header handling.
+            # Set to "native" and redeploy to see the same request interpreted
+            # with X-Forwarded-* honoured. Keycloak's KC_PROXY_HEADERS is the
+            # same opt-in, which is why measuring both sides matters here.
+            - name: SERVER_FORWARD_HEADERS_STRATEGY
+              value: "native"
+            # The JVM sizes its heap from the container limit, not the host.
+            - name: JAVA_TOOL_OPTIONS
+              value: "-XX:MaxRAMPercentage=70"
+            # /api/echo is permitAll, so the JWT decoder is never exercised.
+            # These stay pointed at the future Keycloak service name.
+            - name: SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_ISSUER_URI
+              value: "https://auth.hyeonworks.com/realms/keycloak-patterns"
+            - name: SPRING_SECURITY_OAUTH2_RESOURCESERVER_JWT_JWK_SET_URI
+              value: "https://auth.hyeonworks.com/realms/keycloak-patterns/protocol/openid-connect/certs"
+          readinessProbe:
+            httpGet:
+              path: /actuator/health/readiness
+              port: http
+            initialDelaySeconds: 15
+            periodSeconds: 5
+          livenessProbe:
+            httpGet:
+              path: /actuator/health/liveness
+              port: http
+            initialDelaySeconds: 45
+            periodSeconds: 15
+          resources:
+            requests:
+              memory: 320Mi
+              cpu: 100m
+            limits:
+              memory: 512Mi
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: echo
+  namespace: header-lab
+spec:
+  selector:
+    app: echo
+  ports:
+    - port: 8081
+      targetPort: http
+      name: http
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: echo
+  namespace: header-lab
+spec:
+  # k3s ships Traefik as the default ingress controller. Keeping it is what
+  # makes this lab a faithful two-hop replica.
+  ingressClassName: traefik
+  rules:
+    - host: app1.hyeonworks.com
+      http:
+        paths:
+          - path: /api
+            pathType: Prefix
+            backend:
+              service:
+                name: echo
+                port:
+                  number: 8081
@@ -0,0 +1,277 @@
+# Keycloak multi-node cluster with PostgreSQL.
+#
+# Goal of this manifest: two Keycloak pods on two different nodes must discover
+# each other and form one Infinispan cluster. Keycloak 26 discovers peers through
+# the database (jdbc-ping) rather than multicast, writing to a JGROUPS_PING table,
+# but the cluster traffic itself runs over TCP 7800 between the pods. Those are
+# two separate mechanisms, which is why "registered in the DB but not clustered"
+# is a real failure mode — and one that a single node cannot reproduce.
+#
+#   kubectl apply -f deploy/lab/k8s/keycloak-cluster.yaml
+#   kubectl -n keycloak-lab rollout status statefulset/keycloak --timeout=600s
+#
+# Secrets are plain here. Proper secret handling is roadmap item 11; keeping it
+# visible for now is deliberate so the gap is obvious rather than forgotten.
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: keycloak-lab
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: keycloak-lab-secrets
+  namespace: keycloak-lab
+type: Opaque
+stringData:
+  POSTGRES_PASSWORD: lab-postgres-change-me
+  KC_BOOTSTRAP_ADMIN_PASSWORD: lab-admin-change-me
+---
+# PostgreSQL. local-path binds the volume to whichever node the pod lands on, so
+# the database is effectively pinned to one node. That is not a flaw here: it is
+# what makes "the database node dies" a meaningful experiment later.
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: postgres-data
+  namespace: keycloak-lab
+spec:
+  accessModes: [ReadWriteOnce]
+  storageClassName: local-path
+  resources:
+    requests:
+      storage: 5Gi
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: postgres
+  namespace: keycloak-lab
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate          # RWO volume cannot be mounted by two pods at once
+  selector:
+    matchLabels:
+      app: postgres
+  template:
+    metadata:
+      labels:
+        app: postgres
+    spec:
+      containers:
+        - name: postgres
+          image: postgres:16-alpine
+          ports:
+            - containerPort: 5432
+              name: postgres
+          env:
+            - name: POSTGRES_DB
+              value: keycloak
+            - name: POSTGRES_USER
+              value: keycloak
+            - name: POSTGRES_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: keycloak-lab-secrets
+                  key: POSTGRES_PASSWORD
+            # The image refuses to initialise into a non-empty mount, and
+            # local-path volumes are clean, but this keeps the data one level
+            # down so a lost+found or similar never blocks initdb.
+            - name: PGDATA
+              value: /var/lib/postgresql/data/pgdata
+          volumeMounts:
+            - name: data
+              mountPath: /var/lib/postgresql/data
+          readinessProbe:
+            exec:
+              command: ["sh", "-c", "pg_isready -U keycloak -d keycloak"]
+            initialDelaySeconds: 10
+            periodSeconds: 5
+          resources:
+            requests:
+              memory: 192Mi
+              cpu: 50m
+            limits:
+              memory: 512Mi
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: postgres-data
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: postgres
+  namespace: keycloak-lab
+spec:
+  selector:
+    app: postgres
+  ports:
+    - port: 5432
+      targetPort: postgres
+---
+# Keycloak. A StatefulSet rather than a Deployment so each pod keeps a stable
+# name (keycloak-0, keycloak-1); cluster membership is far easier to read in
+# logs and in the JGROUPS_PING table when the identities do not churn.
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: keycloak
+  namespace: keycloak-lab
+spec:
+  serviceName: keycloak-headless
+  replicas: 2
+  podManagementPolicy: Parallel   # both pods start together, so they race to
+                                  # register — which is the interesting case
+  selector:
+    matchLabels:
+      app: keycloak
+  template:
+    metadata:
+      labels:
+        app: keycloak
+    spec:
+      # One pod per node. Two pods on one node would share a kernel and make the
+      # 7800 blocking experiment meaningless.
+      topologySpreadConstraints:
+        - maxSkew: 1
+          topologyKey: kubernetes.io/hostname
+          whenUnsatisfiable: ScheduleAnyway
+          labelSelector:
+            matchLabels:
+              app: keycloak
+      containers:
+        - name: keycloak
+          image: quay.io/keycloak/keycloak:26.7.0
+          # "start", not "start-dev". Dev mode forces cache=local and there is
+          # no cluster to form at all.
+          args: ["start"]
+          ports:
+            - containerPort: 8080
+              name: http
+            - containerPort: 9000
+              name: management
+            - containerPort: 7800
+              name: jgroups
+          env:
+            - name: KC_DB
+              value: postgres
+            - name: KC_DB_URL
+              value: jdbc:postgresql://postgres:5432/keycloak
+            - name: KC_DB_USERNAME
+              value: keycloak
+            - name: KC_DB_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: keycloak-lab-secrets
+                  key: POSTGRES_PASSWORD
+
+            # Settings confirmed by the two-hop header measurement.
+            # KC_HOSTNAME carries the full external URL, which pins scheme and
+            # host for issuer and redirect URLs regardless of headers.
+            # KC_PROXY_HEADERS is the separate opt-in that lets the forwarded
+            # client address through — the same kind of switch as Spring's
+            # forward-headers-strategy. See docs/two-hop-proxy-header-contract.md.
+            - name: KC_HOSTNAME
+              value: https://auth.hyeonworks.com
+            - name: KC_HOSTNAME_STRICT
+              value: "true"
+            - name: KC_PROXY_HEADERS
+              value: xforwarded
+            - name: KC_HTTP_ENABLED
+              value: "true"
+
+            - name: KC_HEALTH_ENABLED
+              value: "true"
+            - name: KC_METRICS_ENABLED
+              value: "true"
+
+            # Without an explicit cap the JVM sizes its heap from the container
+            # limit and this lab has roughly 3.8GB of guest headroom in total.
+            - name: JAVA_OPTS_KC_HEAP
+              value: "-Xms256m -Xmx512m"
+
+            - name: KC_BOOTSTRAP_ADMIN_USERNAME
+              value: admin
+            - name: KC_BOOTSTRAP_ADMIN_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: keycloak-lab-secrets
+                  key: KC_BOOTSTRAP_ADMIN_PASSWORD
+
+          # Keycloak serves health and metrics on the management port (9000),
+          # not on 8080, since version 25.
+          startupProbe:
+            httpGet:
+              path: /health/started
+              port: management
+            periodSeconds: 10
+            failureThreshold: 60      # first boot runs an implicit build
+          readinessProbe:
+            httpGet:
+              path: /health/ready
+              port: management
+            periodSeconds: 10
+          livenessProbe:
+            httpGet:
+              path: /health/live
+              port: management
+            periodSeconds: 30
+          resources:
+            requests:
+              memory: 640Mi
+              cpu: 100m
+            limits:
+              memory: 900Mi
+---
+# Headless service. Not required for jdbc-ping discovery, which goes through the
+# database, but it gives each pod a stable DNS name for direct inspection.
+apiVersion: v1
+kind: Service
+metadata:
+  name: keycloak-headless
+  namespace: keycloak-lab
+spec:
+  clusterIP: None
+  selector:
+    app: keycloak
+  ports:
+    - port: 8080
+      targetPort: http
+      name: http
+    - port: 9000
+      targetPort: management
+      name: management
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: keycloak
+  namespace: keycloak-lab
+spec:
+  selector:
+    app: keycloak
+  ports:
+    - port: 8080
+      targetPort: http
+      name: http
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: keycloak
+  namespace: keycloak-lab
+spec:
+  ingressClassName: traefik
+  rules:
+    - host: auth.hyeonworks.com
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: keycloak
+                port:
+                  number: 8080
@@ -0,0 +1,373 @@
+# Prometheus + node-exporter + Grafana.
+#
+# Purpose: during a fault-injection experiment, know *which signal moved first*.
+# Without a metrics store the only record is whatever scrolled past in a terminal,
+# and "the cluster recovered in about a minute" is not a measurement.
+#
+#   kubectl apply -f deploy/lab/k8s/observability.yaml
+#   kubectl -n observability rollout status deployment/prometheus --timeout=300s
+#
+# Placement decision — Prometheus and Grafana are pinned to the control-plane
+# node (kc-lab-1). An observability stack must not share a failure domain with
+# the thing it observes. With only two nodes that cannot be fully avoided, so the
+# rule here is: the node that gets killed in experiments is the *agent*
+# (kc-lab-2, holding keycloak-0 and postgres), and everything needed to watch
+# that happen lives on the server node.
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: observability
+---
+# Prometheus discovers scrape targets by querying the Kubernetes API, so it
+# needs read access to nodes, services, endpoints and pods. Without this the
+# kubernetes_sd_configs below silently return no targets.
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: prometheus
+  namespace: observability
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: prometheus
+rules:
+  - apiGroups: [""]
+    # nodes/proxy is required in addition to nodes/metrics: the kubelet job
+    # reaches each node through the API server's proxy subresource
+    # (/api/v1/nodes/<name>/proxy/metrics). Without it every kubelet target
+    # fails with 403 Forbidden while the other jobs stay green — a partial
+    # failure that is easy to miss unless the target list is checked.
+    resources: [nodes, nodes/metrics, nodes/proxy, services, endpoints, pods]
+    verbs: [get, list, watch]
+  - nonResourceURLs: ["/metrics"]
+    verbs: [get]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: prometheus
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: prometheus
+subjects:
+  - kind: ServiceAccount
+    name: prometheus
+    namespace: observability
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: prometheus-config
+  namespace: observability
+data:
+  prometheus.yml: |
+    global:
+      # 15s is short for production but right here: a node loss should show up
+      # within a couple of samples, not a minute later.
+      scrape_interval: 15s
+      evaluation_interval: 15s
+
+    scrape_configs:
+      # Prometheus scraping itself. Useful as a control: if this target is down,
+      # the problem is Prometheus, not the thing being measured.
+      - job_name: prometheus
+        static_configs:
+          - targets: ['localhost:9090']
+
+      # Keycloak. Metrics live on the management port 9000, not 8080 — the same
+      # split that the health probes use. KC_METRICS_ENABLED=true is already set
+      # on the StatefulSet.
+      #
+      # Discovery is by endpoints rather than a static list because pod IPs
+      # change on every restart; that was observed directly when the lab was
+      # power-cycled and every pod came back with a new address.
+      - job_name: keycloak
+        kubernetes_sd_configs:
+          - role: endpoints
+            namespaces:
+              names: [keycloak-lab]
+        relabel_configs:
+          - source_labels: [__meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
+            action: keep
+            regex: keycloak-headless;management
+          - source_labels: [__meta_kubernetes_pod_name]
+            target_label: pod
+          - source_labels: [__meta_kubernetes_pod_node_name]
+            target_label: node
+
+      # node-exporter, one per node via DaemonSet. This is what answers
+      # "did the machine die or did the process die".
+      - job_name: node-exporter
+        kubernetes_sd_configs:
+          - role: endpoints
+            namespaces:
+              names: [observability]
+        relabel_configs:
+          - source_labels: [__meta_kubernetes_service_name]
+            action: keep
+            regex: node-exporter
+          - source_labels: [__meta_kubernetes_pod_node_name]
+            target_label: node
+
+      # The kubelet's own metrics, reached through the API server proxy so no
+      # extra port needs opening.
+      - job_name: kubelet
+        scheme: https
+        tls_config:
+          ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
+          insecure_skip_verify: true
+        bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
+        kubernetes_sd_configs:
+          - role: node
+        relabel_configs:
+          - action: labelmap
+            regex: __meta_kubernetes_node_label_(.+)
+          - target_label: __address__
+            replacement: kubernetes.default.svc:443
+          - source_labels: [__meta_kubernetes_node_name]
+            regex: (.+)
+            target_label: __metrics_path__
+            replacement: /api/v1/nodes/${1}/proxy/metrics
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: prometheus-data
+  namespace: observability
+spec:
+  accessModes: [ReadWriteOnce]
+  storageClassName: local-path
+  resources:
+    requests:
+      storage: 5Gi
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: prometheus
+  namespace: observability
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate            # RWO volume; two pods cannot mount it at once
+  selector:
+    matchLabels:
+      app: prometheus
+  template:
+    metadata:
+      labels:
+        app: prometheus
+    spec:
+      serviceAccountName: prometheus
+      # See the placement note at the top of this file.
+      nodeSelector:
+        node-role.kubernetes.io/control-plane: "true"
+      securityContext:
+        fsGroup: 65534        # the image runs as nobody and must own the volume
+      containers:
+        - name: prometheus
+          image: prom/prometheus:v3.1.0
+          args:
+            - --config.file=/etc/prometheus/prometheus.yml
+            - --storage.tsdb.path=/prometheus
+            # 7 days is far more than an experiment needs and keeps the volume
+            # small enough that it never becomes the reason a node fills up.
+            - --storage.tsdb.retention.time=7d
+            - --web.enable-lifecycle
+          ports:
+            - containerPort: 9090
+              name: http
+          volumeMounts:
+            - name: config
+              mountPath: /etc/prometheus
+            - name: data
+              mountPath: /prometheus
+          readinessProbe:
+            httpGet: { path: /-/ready, port: http }
+            initialDelaySeconds: 10
+          livenessProbe:
+            httpGet: { path: /-/healthy, port: http }
+            initialDelaySeconds: 30
+          resources:
+            requests: { memory: 256Mi, cpu: 50m }
+            limits:   { memory: 640Mi }
+      volumes:
+        - name: config
+          configMap:
+            name: prometheus-config
+        - name: data
+          persistentVolumeClaim:
+            claimName: prometheus-data
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: prometheus
+  namespace: observability
+spec:
+  selector:
+    app: prometheus
+  ports:
+    - port: 9090
+      targetPort: http
+---
+# node-exporter. A DaemonSet so every node reports, including one that is about
+# to be killed — the last samples before it goes silent are the interesting part.
+apiVersion: apps/v1
+kind: DaemonSet
+metadata:
+  name: node-exporter
+  namespace: observability
+spec:
+  selector:
+    matchLabels:
+      app: node-exporter
+  template:
+    metadata:
+      labels:
+        app: node-exporter
+    spec:
+      # Host namespaces: the point is to measure the machine, not the container.
+      hostNetwork: true
+      hostPID: true
+      tolerations:
+        - operator: Exists      # must also run on tainted nodes
+      containers:
+        - name: node-exporter
+          image: prom/node-exporter:v1.8.2
+          args:
+            - --path.procfs=/host/proc
+            - --path.sysfs=/host/sys
+            - --path.rootfs=/host/root
+            - --collector.filesystem.mount-points-exclude=^/(dev|proc|sys|var/lib/docker/.+|var/lib/kubelet/.+)($|/)
+          ports:
+            - containerPort: 9100
+              name: metrics
+              hostPort: 9100
+          volumeMounts:
+            - { name: proc,   mountPath: /host/proc, readOnly: true }
+            - { name: sys,    mountPath: /host/sys,  readOnly: true }
+            - { name: rootfs, mountPath: /host/root, readOnly: true, mountPropagation: HostToContainer }
+          resources:
+            requests: { memory: 32Mi, cpu: 20m }
+            limits:   { memory: 96Mi }
+      volumes:
+        - { name: proc,   hostPath: { path: /proc } }
+        - { name: sys,    hostPath: { path: /sys } }
+        - { name: rootfs, hostPath: { path: / } }
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: node-exporter
+  namespace: observability
+spec:
+  clusterIP: None             # headless: Prometheus wants each pod, not a VIP
+  selector:
+    app: node-exporter
+  ports:
+    - port: 9100
+      targetPort: metrics
+      name: metrics
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: grafana
+  namespace: observability
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: grafana
+  template:
+    metadata:
+      labels:
+        app: grafana
+    spec:
+      nodeSelector:
+        node-role.kubernetes.io/control-plane: "true"
+      containers:
+        - name: grafana
+          image: grafana/grafana:11.4.0
+          ports:
+            - containerPort: 3000
+              name: http
+          env:
+            - name: GF_SECURITY_ADMIN_USER
+              value: admin
+            - name: GF_SECURITY_ADMIN_PASSWORD
+              value: lab-grafana-change-me
+            # Grafana builds absolute URLs for redirects and asset paths. Behind
+            # the nginx -> Traefik chain it must be told the external address,
+            # for exactly the reason Keycloak needs KC_HOSTNAME. Without it,
+            # login redirects come back as http://<pod-ip>:3000.
+            - name: GF_SERVER_ROOT_URL
+              value: https://app2.hyeonworks.com
+          volumeMounts:
+            - name: datasources
+              mountPath: /etc/grafana/provisioning/datasources
+          readinessProbe:
+            httpGet: { path: /api/health, port: http }
+            initialDelaySeconds: 15
+          resources:
+            requests: { memory: 128Mi, cpu: 50m }
+            limits:   { memory: 320Mi }
+      volumes:
+        - name: datasources
+          configMap:
+            name: grafana-datasources
+---
+# Provisioning the datasource as a file means Grafana comes up already wired to
+# Prometheus. Clicking through the UI would leave the configuration only in
+# Grafana's own database, which is emptyDir here and disappears on restart.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: grafana-datasources
+  namespace: observability
+data:
+  prometheus.yaml: |
+    apiVersion: 1
+    datasources:
+      - name: Prometheus
+        type: prometheus
+        access: proxy
+        url: http://prometheus.observability.svc:9090
+        isDefault: true
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: grafana
+  namespace: observability
+spec:
+  selector:
+    app: grafana
+  ports:
+    - port: 3000
+      targetPort: http
+---
+# Grafana is published on app2.hyeonworks.com because that name is already in
+# the wildcard-free certificate (auth / app1 / app2) and is otherwise unused.
+# It moves when app2 is needed for the SSO experiment.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: grafana
+  namespace: observability
+spec:
+  ingressClassName: traefik
+  rules:
+    - host: app2.hyeonworks.com
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: grafana
+                port:
+                  number: 3000
@@ -0,0 +1,43 @@
+# Make Traefik trust the X-Forwarded-* headers that the host nginx sets.
+#
+# Without this, Traefik rewrites every forwarded header from its own connection,
+# which is plain HTTP on port 80. The application then sees scheme=http even
+# though the browser connected over TLS. See docs/two-hop-proxy-header-contract.md.
+#
+# k3s installs Traefik through its bundled HelmChart, so values are overridden
+# with a HelmChartConfig rather than by editing the deployment. k3s reconciles
+# the chart and recreates the Traefik pod.
+#
+#   kubectl apply -f deploy/lab/k8s/traefik-forwarded-headers.yaml
+#   kubectl -n kube-system rollout status deploy/traefik --timeout=180s
+apiVersion: helm.cattle.io/v1
+kind: HelmChartConfig
+metadata:
+  name: traefik
+  namespace: kube-system
+spec:
+  valuesContent: |-
+    ports:
+      web:
+        forwardedHeaders:
+          # Requests arriving from these sources keep their existing
+          # X-Forwarded-* values instead of having them rewritten.
+          #
+          # 10.42.0.0/16 is the pod CIDR. It is required because the traefik
+          # Service uses externalTrafficPolicy: Cluster, so svclb SNATs the
+          # traffic and Traefik sees a pod-network address rather than the
+          # host nginx address.
+          #
+          # The node/host range is deliberately absent. Because svclb SNATs,
+          # the host nginx address never reaches Traefik — measured, not assumed.
+          # Trusting a range that cannot appear only widens the surface.
+          #
+          # Trusting the whole pod CIDR still means any pod in the cluster could
+          # forge these headers, which is why echo-network-policy.yaml restricts
+          # who may reach the application at all.
+          trustedIPs:
+            - 10.42.0.0/16
+      websecure:
+        forwardedHeaders:
+          trustedIPs:
+            - 10.42.0.0/16
@@ -0,0 +1,42 @@
+#!/usr/bin/env bash
+# Build the API image on this workstation and import it into each lab node's
+# containerd.
+#
+# k3s does not run Docker and the lab has no registry, so images are shipped as
+# a stream: docker save -> ssh through the lab host -> k3s ctr images import.
+# Every node needs its own copy because the scheduler may place the pod anywhere.
+#
+#   ./deploy/lab/scripts/build-and-import.sh
+#   IMAGE=keycloak-pattern-api:lab NODES="kc-lab-1" ./deploy/lab/scripts/build-and-import.sh
+set -euo pipefail
+
+IMAGE="${IMAGE:-keycloak-pattern-api:lab}"
+NODES="${NODES:-kc-lab-1 kc-lab-2}"
+LAB_HOST="${LAB_HOST:-test-server}"
+CONTEXT="${CONTEXT:-backend}"
+
+repo_root="$(git rev-parse --show-toplevel)"
+cd "$repo_root"
+
+echo "==> building ${IMAGE} from ${CONTEXT}/"
+docker build -t "$IMAGE" "$CONTEXT"
+
+for node in $NODES; do
+  echo "==> importing into ${node}"
+  # Nested ssh: the workstation cannot reach the guests directly because they
+  # sit behind the lab host's libvirt NAT. The lab host's ~/.ssh/config holds
+  # the kc-lab-* aliases.
+  docker save "$IMAGE" \
+    | ssh "$LAB_HOST" "ssh ${node} 'sudo k3s ctr images import -'"
+done
+
+echo "==> verifying"
+for node in $NODES; do
+  printf '  %-10s ' "$node"
+  ssh "$LAB_HOST" "ssh ${node} 'sudo k3s ctr images ls -q'" \
+    | grep -c "$IMAGE" \
+    | xargs -I{} echo "{} match(es)"
+done
+
+echo
+echo "next: kubectl rollout restart -n header-lab deployment/echo"
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+# Experiment 0c — where does a session entry actually live?
+#
+# Experiment 0b showed keycloak-1's session cache never moved when keycloak-0
+# handled a login. That leaves two explanations:
+#
+#   (a) a DISTRIBUTED cache with owners=1 — entries are spread across nodes by
+#       consistent hashing, and this one happened to land on keycloak-0;
+#   (b) a LOCAL cache — each node only ever caches what it handled itself.
+#
+# They are distinguished by driving logins at the OTHER node. Under (a) the
+# entries would keep landing on both nodes regardless of who was asked. Under
+# (b) the count rises only on the node that received the request.
+set -uo pipefail
+
+NS="${NS:-keycloak-lab}"
+N="${N:-5}"
+K0_IP=$(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.status.podIP}')
+K1_IP=$(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.status.podIP}')
+ADMIN_PW=$(kubectl -n "$NS" get secret keycloak-lab-secrets \
+             -o jsonpath='{.data.KC_BOOTSTRAP_ADMIN_PASSWORD}' | base64 -d)
+
+echo "수집 시각: $(date '+%Y-%m-%d %H:%M:%S %Z')"
+echo "  keycloak-0 = $K0_IP  ($(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.spec.nodeName}'))"
+echo "  keycloak-1 = $K1_IP  ($(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.spec.nodeName}'))"
+echo
+
+kubectl -n "$NS" run kc-own --rm -i --restart=Never \
+  --image=curlimages/curl:8.11.1 --quiet --command -- sh -c "
+O=/tmp/o; : > \$O
+ent() {
+  curl -s --retry 3 --max-time 20 http://\$1:9000/metrics \
+    | grep -E '^vendor_statistics_approximate_entries_unique.cache=.sessions' \
+    | awk '{print \$NF}'
+}
+login() { i=0; while [ \$i -lt $N ]; do
+  curl -s -o /dev/null -X POST http://\$1:8080/realms/master/protocol/openid-connect/token \
+    -d grant_type=password -d client_id=admin-cli \
+    -d username=admin -d 'password=$ADMIN_PW'
+  i=\$((i+1)); done; sleep 5; }
+{
+  printf '%-32s %12s %12s\n' '단계' 'k0 entries' 'k1 entries'
+  printf '%-32s %12s %12s\n' '시작' \"\$(ent $K0_IP)\" \"\$(ent $K1_IP)\"
+  login $K1_IP
+  printf '%-32s %12s %12s\n' 'keycloak-1 에 로그인 ${N}회' \"\$(ent $K0_IP)\" \"\$(ent $K1_IP)\"
+  login $K0_IP
+  printf '%-32s %12s %12s\n' 'keycloak-0 에 로그인 ${N}회' \"\$(ent $K0_IP)\" \"\$(ent $K1_IP)\"
+} >> \$O
+cat \$O
+" 2>&1 | grep -v '^pod .* deleted$'
+
+echo
+echo "=== 대조: PostgreSQL 에는 몇 건인가 ==="
+kubectl -n "$NS" exec deploy/postgres -- psql -U keycloak -d keycloak -tAc \
+  "select count(*) from offline_user_session where offline_flag='0'" 2>/dev/null | sed 's/^/  online 세션 /'
@@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+# Experiment 0b — does the Infinispan cache itself replicate, or do both nodes
+# merely agree because they read the same database?
+#
+# Experiment 0 proved the two nodes give the same answers. That alone does NOT
+# prove Infinispan replicated anything: with persistent-user-sessions (the
+# Keycloak 26 default) the session is written to PostgreSQL, so two nodes reading
+# one database would agree even with the cache disabled entirely.
+#
+# This script separates the two by measuring the cache counters on BOTH nodes
+# around a single login. If the write on keycloak-0 shows up as cache activity
+# on keycloak-1, the replication is real and not a database artifact.
+set -uo pipefail
+
+NS="${NS:-keycloak-lab}"
+K0_IP=$(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.status.podIP}')
+K1_IP=$(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.status.podIP}')
+ADMIN_PW=$(kubectl -n "$NS" get secret keycloak-lab-secrets \
+             -o jsonpath='{.data.KC_BOOTSTRAP_ADMIN_PASSWORD}' | base64 -d)
+
+echo "수집 시각: $(date '+%Y-%m-%d %H:%M:%S %Z')"
+echo
+
+# 파드 출력을 스트리밍으로 받으면 조각이 유실된다. 실제로 첫 시도에서
+# keycloak-1 의 스냅샷과 그 다음 마커가 통째로 사라져 델타가 0 으로 보였다.
+# 파드 안에서 파일로 모았다가 마지막에 한 번만 내보낸다.
+kubectl -n "$NS" run kc-delta --rm -i --restart=Never \
+  --image=curlimages/curl:8.11.1 --quiet --command -- sh -c "
+set -u
+K0='http://$K0_IP'; K1='http://$K1_IP'
+O=/tmp/o.txt; : > \$O
+snap() {
+  curl -s --retry 3 --retry-connrefused --max-time 20 \$1:9000/metrics \
+    | grep -E '^vendor_(statistics_(stores|hits|misses|approximate_entries_unique)|rpc_manager_replication_count)\{cache=\"(sessions|clientSessions)\"' \
+    | sed 's/,cache_manager=\"keycloak\"//; s/,node=\"[^\"]*\"//' >> \$O
+}
+echo '###BEFORE_K0' >> \$O; snap \$K0
+echo '###BEFORE_K1' >> \$O; snap \$K1
+echo '###LOGIN' >> \$O
+curl -s -o /dev/null -w 'http_code=%{http_code}\n' -X POST \
+  \"\$K0:8080/realms/master/protocol/openid-connect/token\" \
+  -d grant_type=password -d client_id=admin-cli \
+  -d username=admin -d 'password=$ADMIN_PW' >> \$O
+sleep 5
+echo '###AFTER_K0' >> \$O; snap \$K0
+echo '###AFTER_K1' >> \$O; snap \$K1
+echo '###END' >> \$O
+cat \$O
+" 2>&1 | grep -v '^pod .* deleted$' > /tmp/cache-delta.txt
+
+python3 - /tmp/cache-delta.txt <<'PY'
+import re, sys
+raw = open(sys.argv[1]).read()
+blocks, cur = {}, None
+for line in raw.splitlines():
+    if line.startswith('###'):
+        cur = line[3:]; blocks[cur] = {}
+    elif cur and '{' in line:
+        m = re.match(r'(\S+?)\{cache="(\w+)"\}\s+(\S+)', line)
+        if m:
+            blocks[cur][(m.group(1), m.group(2))] = float(m.group(3))
+
+print('=== 로그인은 keycloak-0 에만 보냈다 ===')
+code = [l for l in raw.splitlines() if l.startswith('http_code=')]
+print('  로그인 응답: ' + (code[0] if code else '없음'))
+for n in ('BEFORE_K0','BEFORE_K1','AFTER_K0','AFTER_K1'):
+    if not blocks.get(n):
+        print(f'  !! {n} 스냅샷이 비었다 — 델타를 신뢰할 수 없다')
+print()
+hdr = f"  {'계수기':<42} {'캐시':<15} {'전':>8} {'후':>8} {'증가':>7}"
+for node in ('K0', 'K1'):
+    who = 'keycloak-0 (로그인을 받은 노드)' if node == 'K0' else 'keycloak-1 (아무 요청도 받지 않은 노드)'
+    print(f'=== {who} ===')
+    print(hdr)
+    b, a = blocks.get(f'BEFORE_{node}', {}), blocks.get(f'AFTER_{node}', {})
+    for k in sorted(set(b) | set(a)):
+        before, after = b.get(k[0:2], 0.0), a.get(k[0:2], 0.0)
+        d = after - before
+        mark = '  ←' if d else ''
+        name = k[0].replace('vendor_statistics_', '').replace('vendor_rpc_manager_', 'rpc.')
+        print(f"  {name:<42} {k[1]:<15} {before:>8.0f} {after:>8.0f} {d:>+7.0f}{mark}")
+    print()
+PY
@@ -0,0 +1,108 @@
+#!/usr/bin/env bash
+# Experiment 0d — capture the actual SQL that the OTHER node runs.
+#
+# Experiments 0b/0c showed that session entries never appear in keycloak-1's
+# memory, yet keycloak-1 can use a session keycloak-0 created. The conclusion
+# "keycloak-1 reads it from PostgreSQL" was an inference, not an observation.
+#
+# This script turns on statement logging in PostgreSQL for a few seconds, sends
+# ONE refresh request to keycloak-1 for a session born on keycloak-0, and greps
+# the database log for that session id. If the inference is right, the SQL is
+# there, issued from keycloak-1's pod IP.
+#
+# It also checks whether serving that request makes keycloak-1 cache the session
+# — which sharpens "each node caches what it handled" from "what it logged in"
+# to "what it touched".
+set -uo pipefail
+
+NS="${NS:-keycloak-lab}"
+PSQL="kubectl -n $NS exec deploy/postgres -- psql -U keycloak -d keycloak -tAc"
+
+K0_IP=$(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.status.podIP}')
+K1_IP=$(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.status.podIP}')
+ADMIN_PW=$(kubectl -n "$NS" get secret keycloak-lab-secrets \
+             -o jsonpath='{.data.KC_BOOTSTRAP_ADMIN_PASSWORD}' | base64 -d)
+
+echo "수집 시각: $(date '+%Y-%m-%d %H:%M:%S %Z')"
+echo "  keycloak-0 = $K0_IP   (세션을 만드는 노드)"
+echo "  keycloak-1 = $K1_IP   (읽기만 하는 노드)"
+echo
+
+# %h 를 넣어야 어느 파드가 보낸 질의인지 로그에서 구분된다.
+echo "=== PostgreSQL 문장 로깅을 켠다 ==="
+$PSQL "alter system set log_statement='all'"      >/dev/null 2>&1
+$PSQL "alter system set log_line_prefix='%m [%p] %h '" >/dev/null 2>&1
+$PSQL "select pg_reload_conf()"                   >/dev/null 2>&1
+echo "  log_statement    = $($PSQL 'show log_statement' 2>/dev/null)"
+echo "  log_line_prefix  = $($PSQL 'show log_line_prefix' 2>/dev/null)"
+echo
+
+# 로그 커서를 잡아둔다. 이 줄 수 이후만 본다.
+LOG_BEFORE=$(kubectl -n "$NS" logs deploy/postgres --tail=-1 2>/dev/null | wc -l)
+
+RESULT=$(kubectl -n "$NS" run kc-readpath --rm -i --restart=Never \
+  --image=curlimages/curl:8.11.1 --quiet --command -- sh -c "
+O=/tmp/o; : > \$O
+TOKEN_EP='/realms/master/protocol/openid-connect/token'
+jget() { sed -n \"s/.*\\\"\$1\\\":\\\"\\([^\\\"]*\\)\\\".*/\\1/p\"; }
+ent() {
+  curl -s --retry 3 --max-time 20 http://\$1:9000/metrics \
+    | grep -E '^vendor_statistics_approximate_entries_unique.cache=.sessions' | awk '{print \$NF}'
+}
+# keycloak-0 에서 로그인한다
+L=\$(curl -s -X POST \"http://$K0_IP:8080\$TOKEN_EP\" -d grant_type=password \
+     -d client_id=admin-cli -d username=admin -d 'password=$ADMIN_PW')
+SID=\$(echo \"\$L\" | jget access_token | cut -d. -f2 | sed 's/\$/==/' | base64 -d 2>/dev/null | jget sid)
+RT=\$(echo \"\$L\" | jget refresh_token)
+echo \"SID=\$SID\" >> \$O
+echo \"K1_ENTRIES_BEFORE=\$(ent $K1_IP)\" >> \$O
+sleep 2
+# 반대편 노드에 refresh 를 딱 한 번 보낸다
+# 인용을 한 겹 더 쌓으면 curl 이 URL 을 통째로 못 읽는다. 실제로 000 이 나왔다.
+CODE=\$(curl -s -o /dev/null -w '%{http_code}' -X POST \
+  \"http://$K1_IP:8080\$TOKEN_EP\" \
+  -d grant_type=refresh_token -d client_id=admin-cli -d \"refresh_token=\$RT\")
+echo \"REFRESH_ON_K1=\$CODE\" >> \$O
+sleep 3
+echo \"K1_ENTRIES_AFTER=\$(ent $K1_IP)\" >> \$O
+cat \$O
+" 2>&1 | grep -v '^pod .* deleted$')
+
+echo "=== 요청 ==="
+echo "$RESULT" | sed 's/^/  /'
+SID=$(echo "$RESULT" | sed -n 's/^SID=//p')
+
+echo
+echo "=== PostgreSQL 문장 로깅을 끈다 ==="
+$PSQL "alter system reset log_statement"     >/dev/null 2>&1
+$PSQL "alter system reset log_line_prefix"   >/dev/null 2>&1
+$PSQL "select pg_reload_conf()"              >/dev/null 2>&1
+echo "  log_statement = $($PSQL 'show log_statement' 2>/dev/null)"
+
+echo
+echo "=== keycloak-1 이 실제로 보낸 SQL 문장 ==="
+echo "  (파라미터가 \$1 로 묶여 있어, sid 는 바로 아래 DETAIL 줄에 있다)"
+echo
+kubectl -n "$NS" logs deploy/postgres --tail=-1 2>/dev/null \
+  | tail -n +$((LOG_BEFORE + 1)) \
+  | grep -F "$K1_IP" | grep -E "LOG:  execute" \
+  | sed 's/.*execute [^:]*: //' | sed 's/^/  /' | head -12
+echo
+echo "=== 그 sid 를 언급한 SQL — 누가 보냈는가 ==="
+echo "  찾는 sid: $SID"
+echo
+kubectl -n "$NS" logs deploy/postgres --tail=-1 2>/dev/null \
+  | tail -n +$((LOG_BEFORE + 1)) \
+  | grep -F "$SID" \
+  | sed -e "s/$K0_IP/[keycloak-0]/g" -e "s/$K1_IP/[keycloak-1]/g" \
+  | cut -c1-220 \
+  | head -20
+
+echo
+echo "=== 요약: 파드별 질의 건수 ==="
+kubectl -n "$NS" logs deploy/postgres --tail=-1 2>/dev/null \
+  | tail -n +$((LOG_BEFORE + 1)) \
+  | grep -F "$SID" \
+  | grep -oE "^[0-9-]+ [0-9:.]+ [A-Z]+ \[[0-9]+\] [0-9.]+" \
+  | awk '{print $NF}' | sort | uniq -c \
+  | sed -e "s/$K0_IP/[keycloak-0]/" -e "s/$K1_IP/[keycloak-1]/" -e 's/^/  /'
@@ -0,0 +1,207 @@
+#!/usr/bin/env bash
+# Experiment 0 — is a session created on one Keycloak node usable on the other?
+#
+# Forming a cluster is not the same as sharing session state. The Infinispan log
+# says "cluster view (2)", but that only proves the members found each other.
+#
+# Design notes, learned the hard way:
+#
+#  * Every probe has a CONTROL. A result from the far node means nothing unless
+#    the same call against the issuing node is also measured. The first version
+#    of this script reported "403 on keycloak-1" as if it were a replication
+#    failure; the issuing node returned 403 too, and the cause was a missing
+#    openid scope. Measure both, always.
+#
+#  * Sessions are tracked by SID, not by count. Both the test login and the
+#    admin API calls create sessions for the same user, so counts are noisy.
+#    A specific session id either appears in a node's answer or it does not.
+#
+#  * The probe is the REFRESH TOKEN grant, not userinfo. userinfo only validates
+#    a signature and can succeed on a node that knows nothing about the session.
+#    Refreshing requires the node to find the session, check it is alive, and
+#    write back a new refresh time — it actually touches the session store.
+#
+# Talks to pod IPs directly: going through nginx/Traefik would hide which node
+# handled each request, which is the entire question.
+#
+#   ./deploy/lab/scripts/experiment-session-replication.sh
+set -uo pipefail
+
+NS="${NS:-keycloak-lab}"
+OUT="${OUT:-/tmp/session-replication}"
+mkdir -p "$OUT"
+
+PSQL="kubectl -n $NS exec deploy/postgres -- psql -U keycloak -d keycloak -tAc"
+
+echo "수집 시각: $(date '+%Y-%m-%d %H:%M:%S %Z')"
+echo
+
+K0_IP=$(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.status.podIP}')
+K1_IP=$(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.status.podIP}')
+K0_NODE=$(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.spec.nodeName}')
+K1_NODE=$(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.spec.nodeName}')
+ADMIN_PW=$(kubectl -n "$NS" get secret keycloak-lab-secrets \
+             -o jsonpath='{.data.KC_BOOTSTRAP_ADMIN_PASSWORD}' | base64 -d)
+
+echo "=== 대상 ==="
+printf '  keycloak-0  %-14s %s\n' "$K0_IP" "$K0_NODE"
+printf '  keycloak-1  %-14s %s\n' "$K1_IP" "$K1_NODE"
+echo
+
+echo "=== [0] 실험 전 DB 세션 ==="
+$PSQL "select offline_flag, count(*) from offline_user_session group by offline_flag" 2>/dev/null \
+  | sed 's/^/  offline_flag=/' || echo "  (없음)"
+echo
+
+# 파드 하나 안에서 전 단계를 실행한다. 단계마다 파드를 새로 띄우면 토큰을
+# 단계 사이로 넘길 수 없다.
+kubectl -n "$NS" run kc-probe --rm -i --restart=Never \
+  --image=curlimages/curl:8.11.1 --quiet --command -- sh -c "
+set -u
+K0='http://$K0_IP:8080'; K1='http://$K1_IP:8080'
+TOKEN_EP='/realms/master/protocol/openid-connect/token'
+jget() { sed -n \"s/.*\\\"\$1\\\":\\\"\\([^\\\"]*\\)\\\".*/\\1/p\"; }
+
+# ── [1] keycloak-0 에서 로그인. 이 노드가 세션의 출생지다 ──────────────────
+LOGIN=\$(curl -s -X POST \"\$K0\$TOKEN_EP\" \
+  -d grant_type=password -d client_id=admin-cli \
+  -d username=admin -d 'password=$ADMIN_PW')
+echo '###STEP1_LOGIN'; echo \"\$LOGIN\"
+
+AT=\$(echo \"\$LOGIN\" | jget access_token)
+RT=\$(echo \"\$LOGIN\" | jget refresh_token)
+
+# ── [2] 관리 API 조회용 토큰. 세션 오염을 피하려고 따로 하나만 더 만든다 ──
+ADMTOK=\$(curl -s -X POST \"\$K0\$TOKEN_EP\" \
+  -d grant_type=password -d client_id=admin-cli \
+  -d username=admin -d 'password=$ADMIN_PW' | jget access_token)
+CID=\$(curl -s -H \"Authorization: Bearer \$ADMTOK\" \
+  \"\$K0/admin/realms/master/clients?clientId=admin-cli\" | jget id | head -1)
+
+# ── [3] 두 노드에 같은 질문을 한다: admin-cli 의 세션 목록 ────────────────
+echo '###STEP3_SESSIONS_K0'
+curl -s -H \"Authorization: Bearer \$ADMTOK\" \
+  \"\$K0/admin/realms/master/clients/\$CID/user-sessions?max=100\"
+echo
+echo '###STEP3_SESSIONS_K1'
+curl -s -H \"Authorization: Bearer \$ADMTOK\" \
+  \"\$K1/admin/realms/master/clients/\$CID/user-sessions?max=100\"
+echo
+
+# ── [4] 대조군: keycloak-0 이 발급한 refresh token 을 keycloak-0 에 쓴다 ──
+#      먼저 반대편에 써야 하므로 여기서는 쓰지 않고, 순서를 [5] 뒤로 미룬다.
+#      refresh token 은 회전(rotation)되므로 한 번 쓰면 옛 것이 무효가 된다.
+#      따라서 '반대편 먼저'가 유일하게 의미 있는 순서다.
+
+# ── [5] 시험군: keycloak-0 이 발급한 refresh token 을 keycloak-1 에 쓴다 ──
+echo '###STEP5_REFRESH_ON_K1'
+curl -s -w '\nhttp_code=%{http_code}\n' -X POST \"\$K1\$TOKEN_EP\" \
+  -d grant_type=refresh_token -d client_id=admin-cli -d \"refresh_token=\$RT\"
+
+RT2=\$(curl -s -X POST \"\$K1\$TOKEN_EP\" \
+  -d grant_type=refresh_token -d client_id=admin-cli -d \"refresh_token=\$RT\" \
+  | jget refresh_token)
+
+# ── [6] 무효화가 반대 방향으로도 전파되는가 ───────────────────────────────
+#      keycloak-1 에서 로그아웃시키고, keycloak-0 에서 갱신을 시도한다.
+echo '###STEP6_LOGOUT_VIA_K1'
+curl -s -o /dev/null -w 'http_code=%{http_code}\n' -X POST \"\$K1/realms/master/protocol/openid-connect/logout\" \
+  -d client_id=admin-cli -d \"refresh_token=\$RT2\"
+
+echo '###STEP7_REFRESH_ON_K0_AFTER_LOGOUT'
+curl -s -w '\nhttp_code=%{http_code}\n' -X POST \"\$K0\$TOKEN_EP\" \
+  -d grant_type=refresh_token -d client_id=admin-cli -d \"refresh_token=\$RT2\"
+echo '###END'
+" > "$OUT/raw.txt" 2>&1
+
+sed -i '/^pod .* deleted$/d' "$OUT/raw.txt"
+
+python3 - "$OUT/raw.txt" <<'PY' | tee "$OUT/report.txt"
+import base64, json, sys
+
+raw = open(sys.argv[1]).read()
+blocks, cur = {}, None
+for line in raw.splitlines():
+    if line.startswith('###'):
+        cur = line[3:]; blocks[cur] = []
+    elif cur is not None:
+        blocks[cur].append(line)
+get = lambda k: '\n'.join(blocks.get(k, [])).strip()
+
+def j(s):
+    try:    return json.JSONDecoder().raw_decode(s.strip())[0]
+    except Exception: return None
+
+def claims(tok):
+    p = tok.split('.')[1]; p += '=' * (-len(p) % 4)
+    return json.loads(base64.urlsafe_b64decode(p))
+
+login = j(get('STEP1_LOGIN'))
+if not login or 'access_token' not in login:
+    print('로그인 실패:', get('STEP1_LOGIN')[:300]); sys.exit(1)
+
+ac = claims(login['access_token'])
+rc = claims(login['refresh_token'])
+SID = ac['sid']
+print('=== [1] keycloak-0 에서 로그인 ===')
+print(f"  sid          {SID}")
+print(f"  sub          {ac.get('sub')}")
+print(f"  iss          {ac.get('iss')}")
+print(f"  access 수명  {ac['exp']-ac['iat']}초")
+print(f"  refresh 수명 {rc['exp']-rc['iat']}초   typ={rc.get('typ')}")
+print(f"  refresh jti  {rc.get('jti')}")
+
+print()
+print('=== [3] 같은 sid 가 두 노드 모두에서 보이는가 ===')
+for step, who in (('STEP3_SESSIONS_K0', 'keycloak-0 (발급 노드)'),
+                  ('STEP3_SESSIONS_K1', 'keycloak-1 (반대편)')):
+    d = j(get(step))
+    if d is None:
+        print(f'  {who:24} 파싱 실패: {get(step)[:120]}'); continue
+    ids = [s.get('id') for s in d]
+    mark = '보임 ✔' if SID in ids else '없음 ✘'
+    print(f'  {who:24} 세션 {len(ids)}개 중 대상 sid → {mark}')
+    for s in d:
+        if s.get('id') == SID:
+            print(f"      ipAddress={s.get('ipAddress')}  start={s.get('start')}  lastAccess={s.get('lastAccess')}")
+
+def show(step, title, expect):
+    print(); print(f'=== {title} ===')
+    body = get(step)
+    code = [l for l in body.splitlines() if l.startswith('http_code=')]
+    code = code[0].split('=')[1] if code else '?'
+    d = j(body)
+    ok = '기대대로' if code == expect else f'기대({expect})와 다름'
+    print(f'  HTTP {code}   ← {ok}')
+    if d and 'access_token' in d:
+        c = claims(d['access_token'])
+        same = '동일 ✔' if c.get('sid') == SID else f"다름 ✘ ({c.get('sid')})"
+        print(f'  새 토큰의 sid → {same}')
+    elif d:
+        print(f"  error             {d.get('error')}")
+        print(f"  error_description {d.get('error_description')}")
+
+show('STEP5_REFRESH_ON_K1',
+     '[5] keycloak-0 이 발급한 refresh token 을 keycloak-1 에 사용', '200')
+
+print(); print('=== [6] keycloak-1 을 통해 로그아웃 ===')
+print('  ' + get('STEP6_LOGOUT_VIA_K1').strip())
+
+show('STEP7_REFRESH_ON_K0_AFTER_LOGOUT',
+     '[7] 로그아웃 후 keycloak-0 에서 갱신 시도 (무효화 전파)', '400')
+
+open('/tmp/session-replication/sid.txt','w').write(SID)
+PY
+
+SID=$(cat /tmp/session-replication/sid.txt 2>/dev/null)
+echo
+echo "=== [8] PostgreSQL 에서 그 sid 를 직접 확인 ==="
+echo "  대상 sid: $SID"
+$PSQL "select user_session_id, offline_flag, created_on, last_session_refresh
+       from offline_user_session where user_session_id='$SID'" 2>/dev/null \
+  | sed 's/^/  /' | grep -q . \
+  && $PSQL "select user_session_id||' | flag='||offline_flag||' | created='||created_on||' | refresh='||last_session_refresh
+            from offline_user_session where user_session_id='$SID'" 2>/dev/null | sed 's/^/  /' \
+  || echo "  행 없음 — 로그아웃으로 삭제되었다"
+echo
+echo "  전체 세션 수: $($PSQL 'select count(*) from offline_user_session' 2>/dev/null)"
@@ -0,0 +1,42 @@
+#!/usr/bin/env bash
+# Measure what the nginx -> Traefik chain actually delivers to the application.
+#
+# docs/reverse-proxy-headers.md documents a single-hop nginx contract. The lab
+# runs two hops, so the forwarded headers are measured rather than assumed.
+# Run from anywhere that can resolve the lab hostnames.
+#
+#   ./deploy/lab/scripts/measure-proxy-headers.sh
+set -euo pipefail
+
+HOST="${HOST:-app1.hyeonworks.com}"
+URL="https://${HOST}/api/echo"
+
+jqf() {
+  if command -v jq >/dev/null 2>&1; then jq "$@"; else python3 -m json.tool; fi
+}
+
+echo "=== 1. baseline: what the app sees for a normal request ==="
+curl -s "$URL" | jqf '{
+  scheme, secure, serverName, serverPort, requestUrl, remoteAddr,
+  forwarded: .headers | with_entries(select(.key | startswith("x-forwarded") or . == "x-real-ip" or . == "forwarded"))
+}' 2>/dev/null || curl -s "$URL"
+
+echo
+echo "=== 2. spoof test: client sends its own X-Forwarded-* ==="
+echo "    a trusted boundary must overwrite these, not append to them"
+curl -s "$URL" \
+  -H 'X-Forwarded-For: 1.2.3.4' \
+  -H 'X-Forwarded-Proto: http' \
+  -H 'X-Forwarded-Host: evil.example.com' \
+  -H 'X-Real-IP: 1.2.3.4' \
+  | jqf '.headers | with_entries(select(.key | startswith("x-forwarded") or . == "x-real-ip"))' 2>/dev/null
+
+echo
+echo "=== 3. which pod answered (host nginx upstream distribution) ==="
+for _ in 1 2 3 4; do
+  curl -s "$URL" | jqf -r '.headers["x-forwarded-server"] // "n/a"' 2>/dev/null
+done
+
+echo
+echo "=== 4. plain HTTP is redirected, not proxied ==="
+curl -s -o /dev/null -w '  http  -> %{http_code} %{redirect_url}\n' "http://${HOST}/api/echo"
@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+# Rebuild a guest's cloud-init seed image and publish it into the libvirt pool.
+# Run on the lab host.
+#
+#   ./rebuild-seed.sh 1
+#
+# The same content lives in three places: the source YAML, the ISO, and the
+# uploaded pool volume. Editing the YAML alone changes nothing, which is why
+# this is a script and not a set of remembered commands.
+#
+# A rebuilt seed only takes effect on a freshly created VM. cloud-init runs its
+# per-instance modules once per instance-id, so an existing guest ignores it.
+set -euo pipefail
+
+N="${1:?usage: rebuild-seed.sh <1|2>}"
+CLOUD_DIR="${CLOUD_DIR:-$HOME/workspace/cloud}"
+POOL="${POOL:-default}"
+export LIBVIRT_DEFAULT_URI="${LIBVIRT_DEFAULT_URI:-qemu:///system}"
+
+cd "$CLOUD_DIR"
+src="kc-lab-${N}.yaml"
+iso="seed-kc-lab-${N}.iso"
+meta="meta-kc-lab-${N}"
+
+[ -f "$src" ] || { echo "missing $CLOUD_DIR/$src" >&2; exit 1; }
+
+# A fresh instance-id makes cloud-init treat the guest as new and re-run the
+# per-instance modules.
+printf 'instance-id: kc-lab-%s-%s\nlocal-hostname: kc-lab-%s\n' \
+  "$N" "$(date +%s)" "$N" > "$meta"
+
+# NoCloud looks for a volume labelled cidata holding files named exactly
+# user-data and meta-data. -graft-points renames them inside the image so no
+# staging directory is needed.
+xorrisofs -quiet -output "$iso" -volid CIDATA -joliet -rock -graft-points \
+  "/user-data=${src}" "/meta-data=${meta}"
+
+size="$(stat -c%s "$iso")"
+virsh vol-delete --pool "$POOL" "$iso" >/dev/null 2>&1 || true
+virsh vol-create-as "$POOL" "$iso" "$size" --format raw >/dev/null
+virsh vol-upload --pool "$POOL" "$iso" "$iso"
+
+echo "$iso published to pool '$POOL' ($size bytes)"
+echo "attach it as a virtio disk, not a SATA cdrom:"
+echo "  --disk vol=${POOL}/${iso},device=disk,bus=virtio,readonly=on"
+echo "Debian genericcloud images carry no AHCI driver, so a SATA cdrom is invisible"
+echo "to the guest and cloud-init fails with no error anywhere."
@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+# Confirm the lab infrastructure is intact. Run on the lab host.
+#
+# A 404 from the HTTPS entry point is the success signal: TLS terminated and the
+# request reached Traefik, which simply had no matching ingress rule. A 502 or a
+# refused connection means the chain is broken somewhere.
+set -uo pipefail
+
+export LIBVIRT_DEFAULT_URI="${LIBVIRT_DEFAULT_URI:-qemu:///system}"
+HOSTS="${HOSTS:-auth.hyeonworks.com app1.hyeonworks.com app2.hyeonworks.com}"
+NODE_IPS="${NODE_IPS:-192.168.122.11 192.168.122.12}"
+fail=0
+
+check() { # description, expected, actual
+  if [ "$2" = "$3" ]; then printf '  ok    %-34s %s\n' "$1" "$3"
+  else printf '  FAIL  %-34s got %s, want %s\n' "$1" "$3" "$2"; fail=1; fi
+}
+
+echo "== guests =="
+for name in kc-lab-1 kc-lab-2; do
+  check "$name" running "$(virsh domstate "$name" 2>/dev/null || echo absent)"
+done
+
+echo "== k3s =="
+ready="$(kubectl get nodes --no-headers 2>/dev/null | grep -c ' Ready ')"
+check "nodes Ready" 2 "$ready"
+lb="$(kubectl -n kube-system get svc traefik \
+      -o jsonpath='{.status.loadBalancer.ingress[*].ip}' 2>/dev/null | wc -w)"
+check "traefik node IPs" 2 "$lb"
+
+echo "== host nginx =="
+check "service" active "$(systemctl is-active nginx)"
+check "cert renew timer" active "$(systemctl is-active certbot-renew.timer)"
+for ip in $NODE_IPS; do
+  check "traefik $ip" 404 "$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 "http://${ip}/")"
+done
+
+echo "== public entry point =="
+for h in $HOSTS; do
+  check "https://$h" 404 "$(curl -s -o /dev/null -w '%{http_code}' --max-time 8 "https://${h}/")"
+  check "tls verify $h" 0 "$(curl -s -o /dev/null -w '%{ssl_verify_result}' --max-time 8 "https://${h}/")"
+done
+check "http redirect" 301 "$(curl -s -o /dev/null -w '%{http_code}' --max-time 8 "http://${HOSTS%% *}/")"
+
+echo
+[ "$fail" -eq 0 ] && echo "lab is healthy" || echo "lab has failures"
+exit "$fail"
@@ -0,0 +1,6 @@
+# Keycloak receives HTTP only from the trusted reverse proxy.
+KC_HTTP_ENABLED=true
+KC_PROXY_HEADERS=xforwarded
+KC_HOSTNAME=https://auth.example.test
+KC_HOSTNAME_STRICT=true
+
@@ -0,0 +1,14 @@
+server {
+    listen 8080;
+    server_name auth.example.test;
+
+    location / {
+        proxy_pass http://keycloak:8080;
+        proxy_http_version 1.1;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-Host $host;
+        proxy_set_header X-Forwarded-Port 443;
+        proxy_set_header X-Forwarded-Proto https;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    }
+}
@@ -0,0 +1,10 @@
+auth.example.test {
+	tls /etc/tls/tls.crt /etc/tls/tls.key
+
+	reverse_proxy keycloak:8080 {
+		header_up Host {host}
+		header_up X-Forwarded-Host {host}
+		header_up X-Forwarded-Port 443
+		header_up X-Forwarded-Proto https
+	}
+}
@@ -0,0 +1,21 @@
+events {}
+
+http {
+    server {
+        listen 443 ssl;
+        server_name auth.example.test;
+
+        ssl_certificate /etc/tls/tls.crt;
+        ssl_certificate_key /etc/tls/tls.key;
+        ssl_protocols TLSv1.2 TLSv1.3;
+
+        location / {
+            proxy_pass http://keycloak:8080;
+            proxy_set_header Host $host;
+            proxy_set_header X-Forwarded-Host $host;
+            proxy_set_header X-Forwarded-Port 443;
+            proxy_set_header X-Forwarded-Proto https;
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        }
+    }
+}
@@ -0,0 +1,7 @@
+tunnel: 00000000-0000-0000-0000-000000000000
+credentials-file: /etc/cloudflared/00000000-0000-0000-0000-000000000000.json
+
+ingress:
+  - hostname: auth.example.test
+    service: http://reverse-proxy:8080
+  - service: http_status:404
@@ -0,0 +1,18 @@
+# Federated account key: `sub`, not email
+
+외부 IdP의 email은 표시·연락 속성이지 계정 식별자나 자동 연결 증명이 아니다.
+Keycloak의 federated identity는 provider alias와 provider user ID(`sub`)를
+로컬 사용자에 연결한다.
+
+정책:
+
+- 신규 identity의 email이 기존 로컬 계정과 충돌하면 기존 계정의 인증을 다시
+  요구하는 기본 First Broker Login flow를 사용한다.
+- `Automatically Set Existing User`를 production flow에 넣지 않는다.
+- upstream email 변경은 같은 `sub`의 계정 귀속을 바꾸지 않는다.
+- 마지막 로그인 수단을 unlink하는 UI에서는 먼저 다른 인증 수단을 등록하도록
+  안내한다.
+
+`verify-account-linking-sub-vs-email.sh`는 mock IdP 사용자의 email을 실제로
+변경하고 다시 로그인한다. 로컬 사용자 ID가 유지되고 federated `userId`가
+upstream `sub`와 같은지 확인한 후 원래 email을 복구한다.
@@ -0,0 +1,13 @@
+=== [기준선 1] 클러스터 뷰 — 양쪽 파드의 마지막 ISPN000094 ===
+  keycloak-0: [keycloak-1-48749(v=16.0.12)|5] (2) [keycloak-1-48749(v=16.0.12), keycloak-0-30843(v=16.0.12)]
+  keycloak-1: [keycloak-1-48749(v=16.0.12)|5] (2) [keycloak-1-48749(v=16.0.12), keycloak-0-30843(v=16.0.12)]
+
+=== [기준선 2] JGROUPS_PING — 디스커버리 등록 ===
+       name       |       ip        | coord |               coordinated_by                
+------------------+-----------------+-------+---------------------------------------------
+ keycloak-0-30843 | 10.42.1.43:7800 | f     | uuid://00000000-0000-0000-0000-000000000007
+ keycloak-1-48749 | 10.42.0.35:7800 | t     | uuid://00000000-0000-0000-0000-000000000007
+(2 rows)
+
+=== [기준선 3] 기존 NetworkPolicy ===
+No resources found in keycloak-lab namespace.
@@ -0,0 +1,17 @@
+=== [대조군] 차단 전 — keycloak-0 로그인 → keycloak-1 에서 refresh ===
+  sid                    tAWs2gCPr6SOcD4jDR9-_CzB
+  keycloak-1 에서 refresh: 200
+
+=== [기준선 4] JGroups 지표 — 양쪽 노드 ===
+--- K0 (10.42.1.43) ---
+vendor_jgroups_stats_bytes_sent_total 31476.0
+vendor_jgroups_merge3_get_num_merge_events 0.0
+vendor_jgroups_merge3_get_views 0.0
+vendor_jgroups_fd_sock2_get_num_suspected_members 0.0
+vendor_jgroups_nakack2_get_xmit_table_missing_messages 0.0
+--- K1 (10.42.0.35) ---
+vendor_jgroups_merge3_get_views 0.0
+vendor_jgroups_stats_bytes_sent_total 126765.0
+vendor_jgroups_nakack2_get_xmit_table_missing_messages 0.0
+vendor_jgroups_fd_sock2_get_num_suspected_members 0.0
+vendor_jgroups_merge3_get_num_merge_events 0.0
@@ -0,0 +1,8 @@
+=== 차단 적용 ===
+networkpolicy.networking.k8s.io/a1-block-jgroups-transport created
+a1-block-jgroups-transport   map[app:keycloak]
+
+적용 시각: 11:38:08
+=== FD_SOCK2 가 상대를 의심하기까지 기다린다 (15초 간격, 최대 3분) ===
+  +15초  suspected(k0 k1) = 
+  → 변화 감지
@@ -0,0 +1,26 @@
+차단 경과: 11:38:51 (적용 11:38:08)
+
+=== [차단 후 1] JGroups 지표 ===
+--- keycloak-0 ---
+vendor_jgroups_stats_bytes_sent_total 32857.0
+vendor_jgroups_merge3_get_num_merge_events 0.0
+vendor_jgroups_merge3_get_views 0.0
+vendor_jgroups_fd_sock2_get_num_suspected_members 0.0
+vendor_jgroups_nakack2_get_xmit_table_missing_messages 0.0
+--- keycloak-1 ---
+vendor_jgroups_merge3_get_views 0.0
+vendor_jgroups_stats_bytes_sent_total 129103.0
+vendor_jgroups_nakack2_get_xmit_table_missing_messages 0.0
+vendor_jgroups_fd_sock2_get_num_suspected_members 0.0
+vendor_jgroups_merge3_get_num_merge_events 0.0
+
+=== [차단 후 2] 클러스터 뷰 — 갈라졌는가 ===
+  keycloak-0: 
+  keycloak-1: 
+=== [차단 후 3] JGROUPS_PING — 디스커버리는 살아 있는가 ===
+       name       |       ip        | coord 
+------------------+-----------------+-------
+ keycloak-0-30843 | 10.42.1.43:7800 | f
+ keycloak-1-48749 | 10.42.0.35:7800 | t
+(2 rows)
+
@@ -0,0 +1,16 @@
+=== [문제 확정] NetworkPolicy 적용 후에도 기존 연결이 conntrack 에 살아 있다 ===
+--- kc-lab-1 ---
+  tcp      6 86398 ESTABLISHED src=10.42.0.35 dst=10.42.1.43 sport=40023 dport=7800 src=10.42.1.43 dst=10.42.0.35 sport=7800 dport=40023 [ASSURED] mark=0 use=1
+  tcp      6 79982 ESTABLISHED src=10.42.0.35 dst=10.42.1.43 sport=50477 dport=57800 src=10.42.1.43 dst=10.42.0.35 sport=57800 dport=50477 [ASSURED] mark=0 use=1
+--- kc-lab-2 ---
+  tcp      6 86398 ESTABLISHED src=10.42.0.35 dst=10.42.1.43 sport=40023 dport=7800 src=10.42.1.43 dst=10.42.0.35 sport=7800 dport=40023 [ASSURED] mark=0 use=1
+  tcp      6 33 SYN_SENT src=10.42.1.58 dst=10.42.0.35 sport=34824 dport=7800 [UNREPLIED] src=10.42.0.35 dst=10.42.1.58 sport=7800 dport=34824 mark=0 use=1
+  tcp      6 79982 ESTABLISHED src=10.42.0.35 dst=10.42.1.43 sport=50477 dport=57800 src=10.42.1.43 dst=10.42.0.35 sport=57800 dport=50477 [ASSURED] mark=0 use=1
+
+=== [조치] 7800 흐름의 conntrack 항목을 지운다 → 다음 패킷이 정책을 다시 탄다 ===
+  kc-lab-1: tcp      6 86398 ESTABLISHED src=10.42.0.35 dst=10.42.1.43 sport=40023 dport=7800 src=10.42.1.43 dst=10.42.0.35 sport=7800 dport=40023 [ASSURED] mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 0 flow entries have been deleted.
+  kc-lab-2: tcp      6 33 SYN_SENT src=10.42.1.58 dst=10.42.0.35 sport=34824 dport=7800 [UNREPLIED] src=10.42.0.35 dst=10.42.1.58 sport=7800 dport=34824 mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 0 flow entries have been deleted.
+
+=== 삭제 후 7800 conntrack ===
+  kc-lab-1: 2 건
+  kc-lab-2: 2 건
@@ -0,0 +1,13 @@
+관찰 시작: 11:42:03
+  +20초  suspected(k0 k1) = []
+  +40초  suspected(k0 k1) = []
+  +60초  suspected(k0 k1) = [0.0 0.0 0.0 0.0 ]
+  +80초  suspected(k0 k1) = []
+  +100초  suspected(k0 k1) = []
+  +120초  suspected(k0 k1) = []
+  +140초  suspected(k0 k1) = [0.0 ]
+  +160초  suspected(k0 k1) = [0.0 0.0 0.0 0.0 ]
+
+=== 클러스터 뷰 변화 (최근 8분) ===
+--- keycloak-0 ---
+--- keycloak-1 ---
@@ -0,0 +1,16 @@
+=== vendor_cluster_size — 지난 25분 (차단 11:38:08, conntrack 삭제 11:41) ===
+  keycloak-0:
+    11:20=2  11:21=2  11:22=2  11:23=2  11:24=2  11:25=2  11:26=2  11:27=2  11:28=2  11:29=2  11:30=2  11:31=2  11:32=2  11:33=2  11:34=2  11:35=2  11:36=2  11:37=2  11:38=2  11:39=2  11:40=2  11:41=2  11:42=2  11:43=2  11:44=2  11:45=2
+  keycloak-1:
+    11:20=2  11:21=2  11:22=2  11:23=2  11:24=2  11:25=2  11:26=2  11:27=2  11:28=2  11:29=2  11:30=2  11:31=2  11:32=2  11:33=2  11:34=2  11:35=2  11:36=2  11:37=2  11:38=2  11:39=2  11:40=2  11:41=2  11:42=2  11:43=2  11:44=2  11:45=2
+
+=== 현재 값 ===
+  keycloak-1   = 2 멤버
+  keycloak-0   = 2 멤버
+
+=== 7800 소켓 상태 (파드 내부) ===
+  keycloak-0  2
+  keycloak-1  2
+=== conntrack ===
+  kc-lab-1  1 건
+  kc-lab-2  1 건
@@ -0,0 +1,9 @@
+=== 정책이 걸린 상태에서 keycloak-0 을 재시작한다 → 재연결이 막힌다 ===
+재시작 시각: 11:46:07
+pod "keycloak-0" deleted from keycloak-lab namespace
+keycloak-0   false   10.42.1.67   2026-09-04T02:44:23Z
+
+=== cluster_size 추이 ===
+  keycloak-0: 11:45:27=1  11:45:57=1  11:46:27=1  11:46:57=1  11:47:27=1
+  keycloak-0: 11:40:57=2  11:41:27=2  11:41:57=2  11:42:27=2  11:42:57=2  11:43:27=2  11:43:57=2
+  keycloak-1: 11:40:57=2  11:41:27=2  11:41:57=2  11:42:27=2  11:42:57=2  11:43:27=2  11:43:57=2  11:44:27=1  11:44:57=1  11:45:27=1  11:45:57=1  11:46:27=1  11:46:57=1  11:47:27=1
@@ -0,0 +1,16 @@
+=== keycloak-0 헬스 상태 ===
+  keycloak-0 = 10.42.1.67   keycloak-1 = 10.42.0.35
+  PodReadyToStartContainers=True 
+  Initialized=True 
+  Ready=False ContainersNotReady
+  ContainersReady=False ContainersNotReady
+  PodScheduled=True 
+
+=== ★ 본 시험 — 분단 상태에서 교차 노드 세션이 되는가 ===
+  [1] keycloak-0 로그인          sid=nShl5TaBrZnKStDqaspjgmJB
+  [2] keycloak-1 에서 refresh     HTTP 200
+  [3] keycloak-1 에서 로그아웃    HTTP 204
+  [4] keycloak-0 에서 재갱신 시도 HTTP 200
+      (400 이면 무효화가 전파된 것)
+
+=== DB 세션 수 ===
@@ -0,0 +1,33 @@
+=== 그 sid 가 DB 에 남아 있는가 ===
+ user_session_id | offline_flag | last_session_refresh 
+-----------------+--------------+----------------------
+(0 rows)
+
+=== 전체 온라인 세션 수 ===
+  1
+
+=== 노드별 세션 캐시 엔트리 (Prometheus) ===
+  keycloak-1   kc-lab-1   = 0
+  keycloak-0   kc-lab-2   = 1
+
+=== keycloak-0 이 Ready 가 아닌 이유 — 헬스 응답 ===
+{
+    "status": "DOWN",
+    "checks": [
+        {
+            "name": "Graceful Shutdown",
+            "status": "UP"
+        },
+        {
+            "name": "Keycloak cluster health check",
+            "status": "DOWN",
+            "data": {
+                "Failing since": "2026-09-04 02:45:14,251"
+            }
+        },
+        {
+            "name": "Keycloak database connections async health check",
+            "status": "UP"
+        },
+        {
+            "name": "Keycloak Initialized",
@@ -0,0 +1,25 @@
+=== 양쪽 노드의 readiness — 둘 다 DOWN 이면 전면 장애다 ===
+Traceback (most recent call last):
+  File "<string>", line 3, in <module>
+    d=json.load(sys.stdin)
+  File "/usr/lib/python3.14/json/__init__.py", line 298, in load
+    return loads(fp.read(),
+        cls=cls, object_hook=object_hook,
+        parse_float=parse_float, parse_int=parse_int,
+        parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
+  File "/usr/lib/python3.14/json/__init__.py", line 352, in loads
+    return _default_decoder.decode(s)
+           ~~~~~~~~~~~~~~~~~~~~~~~^^^
+  File "/usr/lib/python3.14/json/decoder.py", line 348, in decode
+    raise JSONDecodeError("Extra data", s, end)
+json.decoder.JSONDecodeError: Extra data: line 21 column 2 (char 446)
+
+=== 파드 Ready 상태 ===
+keycloak-0                  false   0
+keycloak-1                  true    0
+
+=== ★ Service 엔드포인트 — 트래픽을 받는 파드가 남아 있는가 ===
+  ready 주소: [10.42.0.35]  notReady : [10.42.1.67]
+=== ★ 외부 진입점으로 실제 로그인이 되는가 (nginx→Traefik→Service) ===
+  https://auth.hyeonworks.com/realms/master  HTTP 200
+  토큰 발급                                   HTTP 200
@@ -0,0 +1,24 @@
+=== 차단 해제 ===
+해제 시각: 11:49:58
+networkpolicy.networking.k8s.io "a1-block-jgroups-transport" deleted from keycloak-lab namespace
+
+=== 자동으로 다시 붙는가 (30초 간격, 최대 4분) ===
+  +30초  keycloak-0=1 keycloak-1=1   | Ready 파드 2 개
+  +60초  keycloak-0=1 keycloak-1=1   | Ready 파드 2 개
+  +90초  keycloak-0=2 keycloak-1=2   | Ready 파드 3 개
+  → 클러스터 재형성
+
+=== 복구 로그 ===
+  keycloak-0: [keycloak-0-26403(v=16.0.12)|0] (1) [keycloak-0-26403(v=16.0.12)]
+  keycloak-1: [keycloak-1-48749(v=16.0.12)|6] (1) [keycloak-1-48749(v=16.0.12)]
+
+=== MERGE3 가 합쳤는가 ===
+  merge_events keycloak-1   = 1
+  merge_events keycloak-0   = 1
+=== JGROUPS_PING — 코디네이터가 하나로 돌아왔는가 ===
+       name       |       ip        | coord 
+------------------+-----------------+-------
+ keycloak-0-26403 | 10.42.1.67:7800 | t
+ keycloak-1-48749 | 10.42.0.35:7800 | f
+(2 rows)
+
@@ -0,0 +1,26 @@
+# A-1 — JGroups 트랜스포트(7800) 차단 증거
+
+2026-09-04 11:38–11:52 KST · Keycloak 26.7.0 / Infinispan 16.0.12
+해설: [`docs/experiment-a1-jgroups-transport-block.md`](../../experiment-a1-jgroups-transport-block.md)
+
+| 파일 | 무엇을 보여주는가 |
+|---|---|
+| `01-baseline-cluster.txt` | 차단 전 — 양쪽이 뷰 ID 5·멤버 2로 일치, `JGROUPS_PING` 코디네이터 1명 |
+| `02-control-before-block.txt` | **대조군** — 차단 전 교차 노드 refresh `200`, JGroups 지표 전부 0 |
+| `03-block-applied.txt` | NetworkPolicy 적용. **빈 측정값을 "변화 감지"로 오판한 기록** |
+| `04-after-block-state.txt` | 차단 43초 후 — 지표 무변화, `JGROUPS_PING` 그대로 |
+| `05-conntrack-problem.txt` | **핵심 문제** — `ESTABLISHED [ASSURED]` 로 기존 연결이 살아 있음. FD_SOCK2 의 **57800** 포트도 함께 드러남 |
+| `06-partition-observed.txt` | 임시 curl 파드 폴링의 실패 — 빈 값·개수 불일치 |
+| `07-cluster-size.txt` | **`vendor_cluster_size` 가 25분 내내 2** — 분단이 일어나지 않았다는 결정적 증거 |
+| `08-restart-forced-partition.txt` | 재연결 강제 후 `2 → 1` |
+| `09-cross-node-under-partition.txt` | **본 시험** — 교차 refresh `200`(예측 적중), **로그아웃 후 재갱신 `200`(예측 빗나감)** |
+| `10-logout-not-propagated.txt` | 기제 확정 — **DB 행 0건인데 keycloak-0 캐시에 1건**, 헬스체크 `cluster health: DOWN` |
+| `11-service-impact.txt` | **분단 노드가 Service 에서 빠짐.** `ready=[10.42.0.35] notReady=[10.42.1.67]`, 외부 로그인 `200` |
+| `12-recovery.txt` | 90초 만에 자동 재형성, `merge3_get_num_merge_events = 1`, 코디네이터 재선출 |
+| `a1-cluster-size-partition-recovery.png` | Grafana — `vendor_cluster_size` 가 `2 → 1 → 2` 로 움직이는 전 구간 |
+
+## 핵심 세 줄
+
+1. **NetworkPolicy 만으로는 이미 붙어 있는 클러스터를 못 끊는다.** conntrack 의 ESTABLISHED 가 먼저 통과시킨다.
+2. **세션 공유는 분단을 견딘다(200).** 통념이 틀렸고 A-0 모델이 맞다.
+3. **로그아웃 무효화는 7800 을 탄다.** DB 행이 지워져도 반대편은 낡은 캐시로 200 을 준다 — A-0 의 인과 해석을 정정한다.
@@ -0,0 +1,14 @@
+=== A-2 기준선 — 클러스터가 정상으로 돌아왔는가 ===
+keycloak-0                  true   10.42.1.67   kc-lab-2
+keycloak-1                  true   10.42.0.35   kc-lab-1
+postgres-7b474b88c8-sn9ff   true   10.42.1.24   kc-lab-2
+
+  cluster_size keycloak-1   = 2
+  cluster_size keycloak-0   = 2
+
+=== 노드별 세션 캐시 (실험 설계에 필요) ===
+  keycloak-1   kc-lab-1   = 0 건
+  keycloak-0   kc-lab-2   = 0 건
+
+=== DB 온라인 세션 ===
+  2
@@ -0,0 +1,10 @@
+pod/a2-probe condition met
+  keycloak-0=10.42.1.67  keycloak-1=10.42.0.35
+
+=== [준비] 양쪽 노드에 세션을 하나씩 만든다 ===
+  keycloak-0 에서 로그인  sid=EAXV5HcG2J1BZ3vnwONf64AQ  토큰길이=613
+  keycloak-1 에서 로그인  sid=McyTj5lj3n_JqApCXeuAHExc  토큰길이=613
+
+=== [확인] 세션이 각자 노드에만 캐시되었는가 ===
+  keycloak-1   = 0 건
+  keycloak-0   = 1 건
@@ -0,0 +1,16 @@
+=== [1] 토큰을 새로 발급 (access 수명 60초) ===
+  발급 완료 sid=RKXQGAgkuLtouFMVPTFmp_0_
+
+=== [2] PostgreSQL 정지 ===
+  정지 시각: 11:56:04
+deployment.apps/postgres scaled
+pod/postgres-7b474b88c8-sn9ff condition met
+  삭제 완료: 11:56:04
+
+=== [3] 네 경로를 즉시 시험 ===
+  ④ 이미 발급된 access token 으로 관리 API   HTTP 000000{"error":"HTTP 401 Unauthorized"}401
+  ① 캐시를 가진 노드(keycloak-0)에서 refresh  HTTP 500
+  ② 캐시가 없는 노드(keycloak-1)에서 refresh  HTTP 500
+  ③ 새 로그인                                 HTTP 500
+  --- 오류 본문 (새 로그인) ---
+{"error":"unknown_error","error_description":"For more on this error consult the server log."}
@@ -0,0 +1,29 @@
+=== 파드 Ready 상태 — DB 가 없으면 어떻게 되는가 ===
+keycloak-0   false   0
+keycloak-1   false   0
+
+=== Service 엔드포인트 ===
+Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
+Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
+  notReady: [10.42.0.35 10.42.1.67]
+=== health/ready 상세 ===
+  전체: DOWN
+    Graceful Shutdown                                    UP
+    Keycloak cluster health check                        UP
+    Keycloak database connections async health check     DOWN
+    Keycloak Initialized                                 UP
+
+=== ④ 다시 — 서명 검증만 필요한 경로는 살아 있는가 ===
+  JWKS 엔드포인트(realm 공개키)  HTTP 200
+  realm 메타데이터(.well-known)  HTTP 200
+  관리 API(세션 조회 필요)       HTTP 500
+
+=== 외부 진입점 ===
+  https://auth.hyeonworks.com/realms/master   HTTP 503
+
+=== Keycloak 로그 — 실제 오류 ===
+	at io.agroal.pool.ConnectionPool$CreateConnectionTask.call(ConnectionPool.java:664)
+	at io.agroal.pool.ConnectionPool$CreateConnectionTask.call(ConnectionPool.java:645)
+Caused by: java.net.ConnectException: Connection refused
+	at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:219)
+	at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:365)
@@ -0,0 +1,21 @@
+=== ★ up 지표는 무엇을 말하는가 (프로세스는 살아 있다) ===
+  up{pod=keycloak-1} = 1   ← 1 인데 서비스는 503 이다
+  up{pod=keycloak-0} = 1   ← 1 인데 서비스는 503 이다
+
+=== 복구 — PostgreSQL 재기동 ===
+  재기동 시각: 11:57:09
+deployment.apps/postgres scaled
+Waiting for deployment "postgres" rollout to finish: 0 out of 1 new replicas have been updated...
+Waiting for deployment "postgres" rollout to finish: 0 of 1 updated replicas are available...
+deployment "postgres" successfully rolled out
+
+=== Keycloak 이 스스로 회복하는가 (재시작 없이) ===
+  +15초  keycloak-0                  true keycloak-1                  true  | 외부 HTTP 200
+  → 서비스 복귀
+
+=== 재시작 횟수 — 파드가 죽었다 살아난 것인가, 그대로 회복한 것인가 ===
+keycloak-0                  0
+keycloak-1                  0
+
+=== 정지 전 세션이 살아남았는가 ===
+  online 세션 5
@@ -0,0 +1,19 @@
+# A-2 — PostgreSQL 정지 증거
+
+2026-09-04 11:56–11:58 KST · Keycloak 26.7.0
+해설: [`docs/experiment-a2-database-loss.md`](../../experiment-a2-database-loss.md)
+
+| 파일 | 무엇을 보여주는가 |
+|---|---|
+| `01-baseline.txt` | 정지 전 — 양쪽 Ready, `cluster_size=2` |
+| `02-setup-sessions.txt` | 양쪽 노드에 세션 하나씩. 캐시는 각자 노드에만 |
+| `03-four-paths.txt` | **네 경로 전부 `500`.** 캐시를 가진 노드도 실패 — refresh 는 쓰기다 |
+| `04-health-and-service.txt` | **전면 장애 증거** — Ready 파드 0개, `ready 주소=[]`, 외부 **503**, `database connections: DOWN`. JWKS·.well-known 은 `200` |
+| `05-recovery.txt` | **`up=1` 인 채로 503.** DB 복귀 15초 후 재시작 0회로 자동 회복, 세션 5건 생존 |
+| `a2-up-stayed-1-during-outage.png` | Grafana — `up{job="keycloak"}` 이 전면 장애 내내 **1에 평평** |
+
+## 핵심 세 줄
+
+1. **DB 는 단일 장애점이다.** Keycloak 을 몇 대로 늘려도 같이 죽는다 — Ready 파드 0개, 외부 503.
+2. **캐시는 읽기를 대신할 뿐 쓰기를 못 한다.** refresh 는 `UPDATE LAST_SESSION_REFRESH` 를 하므로 캐시가 있어도 실패한다.
+3. **`up` 은 이 장애를 못 잡는다.** 알림은 readiness 와 외부 응답 코드에 걸어야 한다.
@@ -0,0 +1,19 @@
+=== [준비] 손실 측정 설계 확인 ===
+  LAST_SESSION_REFRESH 는 integer(초) — 200ms 손실은 보이지 않는다
+ created_on           | integer                 |           | not null | 
+ last_session_refresh | integer                 |           | not null | 0
+    "idx_user_session_expiration_created" btree (realm_id, offline_flag, remember_me, created_on, user_session_id, user_id)
+    "idx_user_session_expiration_last_refresh" btree (realm_id, offline_flag, remember_me, last_session_refresh, user_session_id, user_id)
+  → 대신 행 존재 여부로 잰다. 로그인 하나 = 행 하나 = 이진 판정
+
+  전역 synchronous_commit: on
+
+=== [1] 빠른 연속 로그인을 백그라운드로 시작 ===
+  루프 시작
+  6초 경과 — 지금까지 성공한 로그인: 0
+
+=== [2] PostgreSQL 강제 종료 (SIGKILL) ===
+  종료 시각: 12:00:26.511
+pod "postgres-7b474b88c8-xc2vt" force deleted from keycloak-lab namespace
+  삭제 반환: 12:00:26.586
+  클라이언트가 200 을 받은 로그인 수: 0
@@ -0,0 +1,14 @@
+deployment "postgres" successfully rolled out
+
+=== crash recovery 가 실행되었는가 (강제 종료의 흔적) ===
+2026-09-04 02:58:41.036 UTC [1] LOG:  database system is ready to accept connections
+
+=== [설계 확인] 로그인 트랜잭션도 synchronous_commit 을 끄는가 ===
+  --- 로그인 트랜잭션 (INSERT 가 있는 것) ---
+2:BEGIN
+5:COMMIT
+6:BEGIN
+9:insert into OFFLINE_USER_SESSION (BROKER_SESSION_ID,CREATED_ON,DATA,LAST_SESSION_REFRESH,REALM_ID,REMEMBER_ME,USER_ID,VERSION,OFFLINE_FLAG,USER_SESSION_ID) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)
+10:insert into OFFLINE_CLIENT_SESSION (DATA,REALM_ID,TIMESTAMP,VERSION,CLIENT_ID,CLIENT_STORAGE_PROVIDER,EXTERNAL_CLIENT_ID,OFFLINE_FLAG,USER_SESSION_ID) values ($1,$2,$3,$4,$5,$6,$7,$8,$9)
+11:SET LOCAL synchronous_commit TO OFF
+12:COMMIT
@@ -0,0 +1,17 @@
+=== [1] 로그인 루프 시작 (호스트에서 백그라운드로 exec — 세션이 살아 있어야 한다) ===
+  8초 동안 클라이언트가 200 을 받은 로그인: 106 건
+
+=== [2] SIGKILL ===
+  종료: 12:01:32.981
+  반환: 12:01:33.236
+  최종 성공 로그인 수: 110 건
+    마지막 sid: FimM-krSybBACP2qIvshLWwU
+    마지막 sid: EwFfFwOIfqiv8N5GQ5OjtsVq
+    마지막 sid: CJX-PxFkS7rUc_9FQgB7iw1f
+    마지막 sid: 1EFK7SgUA4M7tq_SkC_BD2er
+    마지막 sid: _LiqTczuyxlpOs3T3xs25SLv
+
+=== [3] PostgreSQL 재기동 후 crash recovery 확인 ===
+Waiting for deployment "postgres" rollout to finish: 0 of 1 updated replicas are available...
+deployment "postgres" successfully rolled out
+  2026-09-04 02:59:48.427 UTC [1] LOG:  database system is ready to accept connections
@@ -0,0 +1,27 @@
+=== [4] 클라이언트가 받은 sid 가 DB 에 있는가 ===
+  클라이언트가 200 을 받은 sid: 291 건
+  DB 온라인 세션 총계: 375
+
+  --- 마지막 15건을 하나씩 조회 ---
+    TCCOYnVlN30Y2fGyJEsVJ_Lq   있음
+    vBcllIkKWovh-FXN9tSzmxAe   있음
+    eMpN_ywUdRTks9uBE9aTumPK   있음
+    aPC_T0yrlMjrlskcLAp0AvY6   있음
+    UBPxmduB-ahGHBg636sY3AEz   있음
+    HLnBloNX9R3qQJSkpOnkNqL4   있음
+    _KPJS30IAqVhkTJGHcCxKvrM   있음
+    rq3caZ9MkyMYlFSSzRjQLygD   있음
+    ivvQm70hjl55DPpF7vpYmz_E   있음
+    81mx-rmi-tAeogHx3-su3z2q   있음
+    WnvNDH93uzcz1XNFbaMSXk1A   있음
+    DXPAIhjO5sGpCUlcD8IEoS8L   있음
+    aZMvl4IwPdK-rC_7bG005Z5C   있음
+    eIuBCprfWA5x0glcgSKYrrX0   있음
+    ozES5kEeu2IFf_cfcC_jFlbF   있음
+
+  마지막 15건 중 유실: 0 건
+
+=== [5] 전체 대조 — 몇 건이나 사라졌는가 ===
+  클라이언트 성공: 291 건
+  DB 에 존재:      291 건
+  ★ 유실:          0 건
@@ -0,0 +1,12 @@
+=== [정리] 세션 테이블 비우고 루프 잔여 확인 ===
+DELETE 375
+  남은 세션: 0
+
+=== [재주입] postmaster(PID 1)에 SIGKILL — 진짜 크래시 ===
+  8초 후 성공 로그인: 110 건
+  SIGKILL: 12:03:21.441
+  최종 성공 로그인: 139 건
+
+=== [검증] 이번엔 crash recovery 가 돌았는가 ===
+deployment "postgres" successfully rolled out
+  2026-09-04 02:59:48.427 UTC [1] LOG:  database system is ready to accept connections
@@ -0,0 +1,17 @@
+=== 로그인 루프 시작 ===
+  8초 후: 112 건
+
+=== 백엔드 프로세스에 SIGKILL → postmaster 가 재초기화한다 ===
+  시각: 12:04:22.063
+  최종 성공 로그인: 153 건
+
+=== [검증] crash recovery 가 돌았는가 ===
+  2026-09-04 02:59:48.427 UTC [1] LOG:  database system is ready to accept connections
+  2026-09-04 03:02:35.807 UTC [1] LOG:  server process (PID 40) was terminated by signal 9: Killed
+  2026-09-04 03:02:35.807 UTC [1] LOG:  terminating any other active server processes
+  2026-09-04 03:02:35.814 UTC [1] LOG:  all server processes terminated; reinitializing
+  2026-09-04 03:02:35.896 UTC [2585] LOG:  database system was not properly shut down; automatic recovery in progress
+  2026-09-04 03:02:35.899 UTC [2585] LOG:  redo starts at 0/23CAB68
+  2026-09-04 03:02:35.904 UTC [2585] LOG:  redo done at 0/2529E40 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
+  2026-09-04 03:02:35.923 UTC [2586] LOG:  checkpoint complete: wrote 113 buffers (0.7%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.004 s, sync=0.004 s, total=0.015 s; sync files=27, longest=0.003 s, average=0.001 s; distance=1405 kB, estimate=1405 kB; lsn=0/252A048, redo lsn=0/252A048
+  2026-09-04 03:02:35.926 UTC [1] LOG:  database system is ready to accept connections
@@ -0,0 +1,19 @@
+=== 크래시 전후 대조 ===
+  클라이언트가 200 과 토큰을 받은 로그인 : 153 건
+  그중 DB 에 실제로 존재                 : 149 건
+  ★ 유실                                 : 4 건
+  DB 전체 온라인 세션                    : 150 건
+
+=== 유실된 sid 목록 ===
+    ★ CQUfg9HLH29xvhiu6pVlfWOo  ← 토큰은 발급됐는데 세션이 없다
+    ★ 5gLP4fqmpZBbjhH_d-0TPMMr  ← 토큰은 발급됐는데 세션이 없다
+    ★ hkcOv1QskUFmYveMLB6Hljra  ← 토큰은 발급됐는데 세션이 없다
+    ★ p5XybeQIYmAs818gO4Vl_5ea  ← 토큰은 발급됐는데 세션이 없다
+
+=== 그 토큰이 지금 실제로 쓰이는가 (마지막 sid 로 확인) ===
+  마지막 sid: 8do0Bw6tkVLDVxgxotE7GosH
+     user_session_id      | created_on | last_session_refresh 
+--------------------------+------------+----------------------
+ 8do0Bw6tkVLDVxgxotE7GosH | 1788490958 |           1788490958
+(1 row)
+
@@ -0,0 +1,20 @@
+# A-3 — DB 강제 종료와 데이터 손실 증거
+
+2026-09-04 12:00–12:05 KST · Keycloak 26.7.0 / PostgreSQL 16
+해설: [`docs/experiment-a3-database-crash.md`](../../experiment-a3-database-crash.md)
+
+| 파일 | 무엇을 보여주는가 |
+|---|---|
+| `01-crash-injection.txt` | 첫 시도 실패 — 파드 안 백그라운드 루프가 `exec` 종료와 함께 죽어 0건 수집 |
+| `02-design-check.txt` | **핵심 설계 확인** — 로그인 트랜잭션도 `SET LOCAL synchronous_commit TO OFF` 로 커밋한다 |
+| `03-loss-measurement.txt` | `--grace-period=0 --force` 주입 |
+| `04-comparison.txt` | **유실 0건** — 그러나 crash recovery 가 안 돌았다. 죽인 적이 없는 것 |
+| `05-true-crash.txt` | `kill -9 1` 시도 — **컨테이너 안에서 PID 1 은 SIGKILL 을 무시한다** |
+| `06-backend-kill-crash.txt` | **성공한 주입** — 백엔드에 SIGKILL → `not properly shut down` / `redo starts` / `redo done` |
+| `07-loss-result.txt` | **결과: 153건 중 4건 유실.** 토큰은 발급됐는데 세션 행이 없는 sid 목록 |
+
+## 핵심 세 줄
+
+1. **로그인도 비동기 커밋이다.** refresh 시각뿐 아니라 **로그인 자체**가 사라질 수 있다.
+2. **153건 중 4건(약 2.6%) 유실** — 초당 19건 기준 마지막 0.2초 분량, `wal_writer_delay` 기본값과 일치.
+3. **주입을 세 번 시도해 세 번째에 성공했다.** 앞의 둘은 "손실 0"으로 보였지만 실제로는 크래시가 아니었다.
@@ -0,0 +1,20 @@
+=== A-4 기준선 ===
+kc-lab-1   Ready   true
+kc-lab-2   Ready   <none>
+
+a2-probe                    true   kc-lab-2
+keycloak-0                  true   kc-lab-2
+keycloak-1                  true   kc-lab-1
+postgres-7b474b88c8-2gf27   true   kc-lab-2
+
+=== PVC 가 어느 노드에 묶여 있는가 (재배치 가능성) ===
+  persistentvolumeclaim/postgres-data → kc-lab-2
+
+=== 서비스 정상 확인 ===
+  https://auth.hyeonworks.com/realms/master  HTTP 200
+
+=== VM 상태 ===
+--------------------------
+ 1    kc-lab-1   running
+ 2    kc-lab-2   running
+
@@ -0,0 +1,17 @@
+=== 워커 노드(kc-lab-2) 전원 차단 — virsh destroy 는 종료 신호가 없다 ===
+차단 시각: 12:07:43
+Domain 'kc-lab-2' destroyed
+
+
+  +15초  node=Ready | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 000
+  +30초  node=Ready | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 000
+  +45초  node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+  +60초  node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+  +75초  node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+  +90초  node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+  +105초  node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+  +120초  node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+  +135초  node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+  +150초  node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+  +165초  node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+  +180초  node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
@@ -0,0 +1,28 @@
+=== 파드 상태의 진실 — Running 인데 노드가 없다 ===
+a2-probe                    Running   true    kc-lab-2   <none>
+keycloak-0                  Running   true    kc-lab-2   <none>
+keycloak-1                  Running   false   kc-lab-1   <none>
+postgres-7b474b88c8-2gf27   Running   true    kc-lab-2   <none>
+
+=== 재배치가 시도되었는가 ===
+10m         Warning   Unhealthy           pod/keycloak-0                   Readiness probe failed: Get "http://10.42.1.67:9000/health/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
+3m15s       Warning   NodeNotReady        pod/postgres-7b474b88c8-2gf27    Node is not ready
+3m15s       Warning   NodeNotReady        pod/keycloak-0                   Node is not ready
+3m15s       Warning   NodeNotReady        pod/a2-probe                     Node is not ready
+2m27s       Warning   Unhealthy           pod/keycloak-1                   Readiness probe failed: Get "http://10.42.0.35:9000/health/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
+2s          Warning   Unhealthy           pod/keycloak-1                   Readiness probe failed: HTTP probe failed with statuscode: 503
+
+=== 노드 taint — 쿠버네티스가 붙인 것 ===
+  node.kubernetes.io/unreachable=:NoSchedule
+  node.kubernetes.io/unreachable=:NoExecute
+
+=== Prometheus 가 본 것 (kc-lab-1 에 있어 살아남았다) ===
+  up{job=keycloak       pod=keycloak-1                } = 1
+  up{job=keycloak       pod=keycloak-0                } = 0
+  up{job=kubelet        pod=-                         } = 1
+  up{job=kubelet        pod=-                         } = 0
+  up{job=node-exporter  pod=kc-lab-1                  } = 1
+  up{job=node-exporter  pod=kc-lab-2                  } = 0
+  up{job=prometheus     pod=-                         } = 1
+
+=== 진입점이 처음 40초간 000 이었던 이유 — nginx upstream ===
@@ -0,0 +1,15 @@
+=== nginx 설정 위치 찾기 ===
+
+=== NoExecute taint 의 tolerationSeconds — 언제 축출되는가 ===
+  node.kubernetes.io/not-ready NoExecute tolerationSeconds=300
+  node.kubernetes.io/unreachable NoExecute tolerationSeconds=300
+
+=== 5분 축출 시점까지 관찰 ===
+  +210초  a2-probe:Running keycloak-0:Running keycloak-1:Running postgres-7b474b88c8-2gf27:Running 
+  +240초  a2-probe:Running keycloak-0:Running keycloak-1:Running postgres-7b474b88c8-2gf27:Running 
+  +270초  a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending 
+  +300초  a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending 
+  +330초  a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending 
+  +360초  a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending 
+  +390초  a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending 
+  +420초  a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending 
@@ -0,0 +1,18 @@
+=== 새 postgres 가 Pending 인 이유 ===
+Events:
+  Type     Reason            Age    From               Message
+  ----     ------            ----   ----               -------
+  Warning  FailedScheduling  4m45s  default-scheduler  0/2 nodes are available: 1 node(s) didn't match PersistentVolume's node affinity, 1 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.
+
+=== keycloak-0 대체 파드가 안 생기는 이유 (StatefulSet) ===
+keycloak   2     <none>   1
+keycloak-0                  1/1   Terminating   0     30m
+keycloak-1                  0/1   Running       0     143m
+
+=== 복구 — 노드 재기동 ===
+재기동 시각: 12:16:31
+Domain 'kc-lab-2' started
+
+  +30초  node=Ready | Running 파드 3 개 | 외부 HTTP 503
+  +60초  node=Ready | Running 파드 3 개 | 외부 HTTP 200
+  → 서비스 복귀
@@ -0,0 +1,19 @@
+=== 복구 확인 ===
+keycloak-0                  1/1   Running   0     68s
+keycloak-1                  1/1   Running   0     144m
+postgres-7b474b88c8-9cmsv   1/1   Running   0     4m20s
+
+=== kc-lab-1(k3s server)에 무엇이 있는가 — 이게 곧 영향 범위다 ===
+  keycloak-lab     keycloak-1
+  kube-system      coredns-54996dc9b4-8k8fj
+  kube-system      helm-install-traefik-crd-q29b5
+  kube-system      local-path-provisioner-77b9867795-g27z8
+  kube-system      metrics-server-6dc596dfb8-7xxq4
+  kube-system      svclb-traefik-5eb6a9a1-qwwk5
+  kube-system      traefik-5d6fcf895-wpfhr
+  observability    grafana-845b5678cf-b6gvc
+  observability    node-exporter-9qk9w
+  observability    prometheus-6774f94f7c-pzr2t
+
+=== Traefik replica 수 (진입점의 단일 장애점인가) ===
+traefik   1     1
@@ -0,0 +1,19 @@
+=== 컨트롤 플레인 노드(kc-lab-1) 전원 차단 ===
+차단 시각: 12:18:08
+Domain 'kc-lab-1' destroyed
+
+
+  +20초  외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp 
+  +40초  외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp 
+  +60초  외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp 
+  +80초  외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp 
+  +100초  외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp 
+  +120초  외부 auth=000 grafana=502 | kubectl: Unable to connect to the server: dial tcp 
+  +140초  외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp 
+  +160초  외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp 
+
+=== 살아 있는 노드에서 직접 확인 — 워크로드는 도는가 ===
+  CONTAINER           IMAGE               CREATED             STATE               NAME                ATTEMPT             POD ID              POD                 NAMESPACE
+  e5f777900b762       60e153026e8f5       4 minutes ago       Running             keycloak            0                   640d4dafaefb3       keycloak-0          keycloak-lab
+  
+  6
@@ -0,0 +1,13 @@
+=== 컨트롤 플레인 노드 복구 ===
+재기동: 12:23:39
+Domain 'kc-lab-1' started
+
+  +30초  외부=502 | kc-lab-1=Ready kc-lab-2=Ready 
+  +60초  외부=200 | kc-lab-1=Ready kc-lab-2=Ready 
+  → 서비스 복귀 (총 60초)
+
+=== 최종 상태 ===
+keycloak-0                  1/1   Running   0                   7m57s
+keycloak-1                  1/1   Running   1 (<invalid> ago)   151m
+postgres-7b474b88c8-9cmsv   1/1   Running   0                   11m
+  (prometheus port-forward 재연결 필요)
@@ -0,0 +1,23 @@
+# A-4 — 노드 전원 차단 증거
+
+2026-09-04 12:07–12:24 KST
+해설: [`docs/experiment-a4-node-loss.md`](../../experiment-a4-node-loss.md)
+
+| 파일 | 무엇을 보여주는가 |
+|---|---|
+| `01-baseline.txt` | 차단 전 — 양쪽 Ready, **PVC 가 kc-lab-2 에 못박혀 있음**(재배치 불가의 원인), 외부 200 |
+| `02-worker-node-killed.txt` | `virsh destroy kc-lab-2` — **40초간 노드가 Ready 로 남아 있고** 외부는 이미 `000`. 이후 `503` |
+| `03-state-during-loss.txt` | **죽은 파드가 `ready=true`, 산 파드가 `ready=false`.** `up` 은 정확히 0. `unreachable` taint |
+| `04-eviction-timing.txt` | `tolerationSeconds=300` — **5분 뒤** 축출, 새 postgres 는 `Pending` |
+| `05-recovery.txt` | `FailedScheduling: didn't match PersistentVolume's node affinity`, StatefulSet `DESIRED=2 CURRENT=1`. 노드 복귀 후 **60초** |
+| `06-control-plane-inventory.txt` | kc-lab-1 에 있는 것 목록 — **Traefik `replicas=1`** |
+| `07-control-plane-loss.txt` | `virsh destroy kc-lab-1` — 외부 `000`, `kubectl` 불통. **그런데 `crictl ps` 로 보면 keycloak-0 은 Running** |
+| `08-control-plane-recovery.txt` | 60초 만에 복귀 |
+| `a4-up-dropped-per-node.png` | Grafana — `up` 이 노드별로 0 으로 떨어지는 구간. 12:18–12:23 은 **0 이 아니라 데이터 없음**(관측자가 같이 죽음) |
+
+## 핵심 네 줄
+
+1. **쿠버네티스는 40초 동안 노드가 살아 있다고 말한다.** 사용자는 이미 장애를 겪는 중이다.
+2. **죽은 파드의 상태는 화석이다.** `ready=true` 인 파드가 꺼진 기계 위에 있다.
+3. **StatefulSet 은 대체 파드를 만들지 않고, PVC 는 재배치를 막는다.** 사람이 개입해야 한다.
+4. **컨트롤 플레인 상실 ≠ 워크로드 상실.** 컨테이너는 계속 돌고, 들어갈 문만 사라진다.
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
DongHyeonkaandClaude Opus 5	d0666c5ba0	docs: A-4 — a dead pod reports healthier than a live one Kubernetes keeps calling the node Ready for forty seconds while users already see failures, and the pod on the powered-off machine stays ready=true because its kubelet can no longer contradict itself. Eviction waits another five minutes, then the StatefulSet refuses to recreate its pod and the replacement Deployment pod cannot schedule because the local-path volume is pinned to the dead node. Killing the server node instead shows the opposite shape: containerd keeps the workload running while the API server, Traefik and the observability stack disappear, so the outage is the missing path rather than the missing application. Traefik at one replica is the ingress single point of failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 12:25:30 +09:00
DongHyeonkaandClaude Opus 5	99b689e715	docs: A-3 — four logins returned tokens for sessions the crash erased Keycloak commits the login INSERT with synchronous_commit off, so a crash loses whole sessions and not just refresh timestamps. Measured 4 of 153 lost, matching the default wal_writer_delay window. Two injections failed silently first: --grace-period=0 --force lets the container runtime send SIGTERM so PostgreSQL flushes and shuts down cleanly, and SIGKILL to PID 1 from inside its own namespace is ignored by the kernel. Killing a backend makes the postmaster reinitialize, which is a real crash recovery. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 12:05:08 +09:00
DongHyeonkaandClaude Opus 5	4177fb6a48	docs: A-2 — losing the database takes every node down while up stays 1 Both pods go NotReady, the Service endpoint list empties and the front door returns 503, so adding Keycloak replicas buys nothing against database loss. The node holding the session in cache fails too, because a refresh writes LAST_SESSION_REFRESH. Recovery was automatic in about fifteen seconds with no restart, which is what readiness rather than liveness buys. The observability finding matters as much: up stayed at 1 through a total outage, so alerting on it would have caught nothing. kube-state-metrics is missing and pod readiness is therefore not recorded as a metric. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 11:57:55 +09:00
DongHyeonkaandClaude Opus 5	2a98ef1090	docs: A-1 — sessions survive a JGroups partition but logout invalidation does not Cutting TCP 7800 leaves cross-node refresh working (200), confirming sessions travel through PostgreSQL rather than the cluster transport. Logout is the opposite: the database row is deleted but the other node answers from its stale local cache, so the A-0 conclusion that invalidation rides the database is corrected here. Two things the plan did not anticipate: a NetworkPolicy cannot sever an established connection because conntrack accepts it before policy evaluation, and Keycloak reports the partition through its readiness probe so the split node removes itself from the Service. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 11:52:49 +09:00
DongHyeonkaandClaude Opus 5	9c3cde457e	docs: close nine gaps found by re-reading every open question in full Cross-checked each question's 남은 미지수, 다음 검증 and 제약 against the plan item by item. Adds B-0 (autoconfiguration actually chosen), B-6 (encryption key rotation) and B-7 (oauth2-proxy cookie secret rotation) as new experiments, plus lock-holder death, rotation-disabled comparison, partial-logout recovery, store latency and the Q4 design checklist. Restores the Redis persistence comparison and records the correct index URL. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 11:16:59 +09:00
DongHyeonkaandClaude Opus 5	f1e8c35805	docs: plan all remaining experiments with architecture, injection points and predictions Twenty experiments across four layers, each with a topology diagram marking where the fault goes in, the metrics to watch, a falsifiable prediction written before the run, and a pass/fail rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 10:50:23 +09:00
DongHyeonkaandClaude Opus 5	1de6108157	docs: add the prerequisite knowledge this lab assumes Builds up from HTTP statelessness to why session storage location determines the operational response, so the measurements have context to land in. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 10:24:21 +09:00
DongHyeonkaandClaude Opus 5	22d873eb4f	docs: capture the SQL the other node actually runs, and correct the replication claim PostgreSQL statement logging shows keycloak-1 reading and updating the session created on keycloak-0. The same transaction reveals optimistic locking via VERSION, SKIP LOCKED, and synchronous_commit turned off. Fixes the earlier concept note that credited Infinispan with cross-node propagation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 10:14:45 +09:00
DongHyeonkaandClaude Opus 5	e5ebaeb623	docs: prove sessions are shared by PostgreSQL, not Infinispan replication Experiment 0 with three probes: cross-node refresh/logout, cache counter deltas around a single login, and cache entry ownership. Each node caches only what it handled; cache totals sum exactly to the database count. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 09:58:10 +09:00
DongHyeonkaandClaude Opus 5	006da7d490	docs: record observability setup and add concept layers 10-13 Adds Kubernetes resources (StatefulSet, PVC, Secret, RBAC, placement), Keycloak clustering internals (Infinispan, JGroups), Prometheus concepts and virtualization operations. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 09:21:45 +09:00
DongHyeonkaandClaude Opus 5	d5cc2b55a9	fix: grant nodes/proxy so kubelet metrics can be scraped Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 09:14:51 +09:00
DongHyeonkaandClaude Opus 5	0bb0e0ac49	feat: add Prometheus, node-exporter and Grafana for fault-injection observability Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 09:13:41 +09:00
DongHyeonkaandClaude Opus 5	33878e8880	docs: record multi-node cluster setup, rationale and formation evidence Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 17:26:45 +09:00
DongHyeonkaandClaude Opus 5	6dce35ec83	feat: deploy Keycloak multi-node cluster with PostgreSQL Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 17:17:25 +09:00
DongHyeonkaandClaude Opus 5	001efd624a	docs: correct the memory analysis to distinguish host and guest headroom Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 17:02:11 +09:00
DongHyeonkaandClaude Opus 5	d6f8b9f8b3	docs: restructure roadmap into A/B/operations layers and add five operational items Adds backup rehearsal, version upgrade, observability, secret management and certificate renewal. Corrects the experiment order so the refresh-token contention test runs after the shared store exists. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 16:56:13 +09:00
DongHyeonkaandClaude Opus 5	df4d3b4345	docs: map published open questions to lab coverage and fix experiment order Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 16:44:12 +09:00
DongHyeonkaandClaude Opus 5	61ba5db259	docs: add lab operations guide with tooling, commands and drills Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 16:39:45 +09:00
DongHyeonkaandClaude Opus 5	bc784fcd6e	docs: surface the spoofing bypass finding in the roadmap Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 16:24:52 +09:00
DongHyeonkaandClaude Opus 5	ddcb1c08e6	docs: record the session store lab roadmap and progress Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 16:20:38 +09:00
DongHyeonkaandClaude Opus 5	e1ba9c5626	docs: record proxy-bypass closure with before and after evidence Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 16:17:24 +09:00
DongHyeonkaandClaude Opus 5	3af52bb66a	feat: narrow Traefik trusted range and restrict echo ingress to Traefik Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 16:15:28 +09:00
DongHyeonkaandClaude Opus 5	98874b0c6c	docs: retake staged evidence screenshots with indented output Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 15:52:47 +09:00
DongHyeonkaandClaude Opus 5	b708c8d503	feat: indent echo responses for readable evidence Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 15:48:57 +09:00
DongHyeonkaandClaude Opus 5	7737787937	docs: record the staged fix and post-fix evidence Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 15:33:40 +09:00
DongHyeonkaandClaude Opus 5	1c1b86e849	feat: let the app interpret forwarded headers Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 15:30:31 +09:00
DongHyeonkaandClaude Opus 5	2294c52095	feat: make Traefik trust forwarded headers from the host nginx Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 15:29:18 +09:00
DongHyeonkaandClaude Opus 5	69d4502757	docs: collect evidence for the two-hop header contract failure Control experiment isolates two independent causes: the nginx 443 block still emits X-Forwarded-Proto http, and Traefik rewrites forwarded headers regardless. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 15:04:38 +09:00
DongHyeonkaandClaude Opus 5	ae1f391598	docs: expand two-hop header contract with switch locations and per-pattern impact Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 14:57:08 +09:00
DongHyeonkaandClaude Opus 5	844d6f1d33	docs: measure and diagnose the two-hop proxy header contract Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 14:35:27 +09:00
DongHyeonkaandClaude Opus 5	a831792c5c	docs: map deploy/ assets and record why unapplied configs are kept Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 14:18:08 +09:00
DongHyeonkaandClaude Opus 5	bcfdeb93ee	docs: explain every setting used in the echo manifest Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 14:11:18 +09:00
DongHyeonkaandClaude Opus 5	deae8966b8	docs: record why Docker must not be installed on the lab host Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 13:53:36 +09:00
DongHyeonkaandClaude Opus 5	6c90468c5a	feat: report pod address from echo endpoint Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-03 13:44:29 +09:00
DongHyeonka	51bb055d61	feat: 2홉 구성 진행	2026-09-03 11:46:56 +09:00
DongHyeonka	c22b217fbd	chore: 메타데이터 gitignore	2026-09-02 14:44:06 +09:00
donghyeon-ka	c07593c471	merge: four-pattern tradeoff matrix	2026-07-25 16:31:44 +09:00
donghyeon-ka	d01a60964a	docs: compare four authentication patterns	2026-07-25 16:31:44 +09:00
donghyeon-ka	5d7544256a	merge: public domain tunnel profile	2026-07-25 16:31:13 +09:00
donghyeon-ka	eacc0e86c9	feat: add validated named tunnel profile	2026-07-25 16:31:13 +09:00
donghyeon-ka	ac912cb354	merge: HTTPS termination profiles	2026-07-25 16:30:25 +09:00
donghyeon-ka	e4cee2a06d	feat: add validated HTTPS termination profiles	2026-07-25 16:30:25 +09:00
donghyeon-ka	006b405ad5	merge: reverse proxy header contract	2026-07-25 16:29:32 +09:00
donghyeon-ka	8539d1bf5b	feat: define trusted reverse proxy header contract	2026-07-25 16:29:32 +09:00
donghyeon-ka	3473875d9a	merge: Google redirect URI policy	2026-07-25 16:28:53 +09:00
donghyeon-ka	f077e5038e	docs: define exact Google redirect URI policy	2026-07-25 16:28:53 +09:00
donghyeon-ka	e4eb7e54f7	merge: federated subject identity contract	2026-07-25 16:28:00 +09:00
donghyeon-ka	bdde0feb86	test: verify broker identity uses subject not email	2026-07-25 16:28:00 +09:00
donghyeon-ka	3da8e609ae	merge: broker claim-to-role mapping	2026-07-25 16:26:59 +09:00
donghyeon-ka	ed064473dc	feat: map broker claims to realm roles	2026-07-25 16:26:59 +09:00
donghyeon-ka	a84f7d50a1	merge: Google claim attribute mapping	2026-07-25 16:24:24 +09:00
donghyeon-ka	98b07b4fdf	feat: map upstream Google identity claims	2026-07-25 16:24:24 +09:00
donghyeon-ka	98566a713d	merge: hardened First Broker Login flow	2026-07-25 16:17:54 +09:00
donghyeon-ka	aeb783e592	test: reproduce and block unsafe broker auto-link	2026-07-25 16:17:54 +09:00
donghyeon-ka	3bbbaf5230	merge: Google broker configuration profiles	2026-07-25 15:30:47 +09:00
				`@@ -0,0 +1 @@`
				`[ 149ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0`
				`@@ -0,0 +1 @@`
				`[ 103ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0`
				`@@ -0,0 +1 @@`
				`[ 95ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0`
				`@@ -0,0 +1 @@`
				`[ 132ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0`
				`@@ -0,0 +1 @@`
				`[ 239ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0`
				`@@ -0,0 +1 @@`
				`[ 105ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0`
				`@@ -0,0 +1 @@`
				`[ 108ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0`