diff --git a/.playwright-mcp/console-2026-09-04T04-16-32-330Z.log b/.playwright-mcp/console-2026-09-04T04-16-32-330Z.log index 765c675..e0577d3 100644 --- a/.playwright-mcp/console-2026-09-04T04-16-32-330Z.log +++ b/.playwright-mcp/console-2026-09-04T04-16-32-330Z.log @@ -10,3 +10,11 @@ [ 52154ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 [ 67608ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 [ 86965ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 108668ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 129158ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 137544ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 152699ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 159898ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 173491ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 187925ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 197824ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 diff --git a/.playwright-mcp/console-2026-09-04T04-19-58-373Z.log b/.playwright-mcp/console-2026-09-04T04-19-58-373Z.log new file mode 100644 index 0000000..483a316 --- /dev/null +++ b/.playwright-mcp/console-2026-09-04T04-19-58-373Z.log @@ -0,0 +1,11 @@ +[ 635ms] [WARNING] is deprecated. Please include @ https://app2.hyeonworks.com/explore?schemaVersion=1&orgId=1&panes=%7B%22a%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_statistics_approximate_entries_unique%7Bcache%3D%5C%22sessions%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%2C%7B%22refId%22%3A%22B%22%2C%22expr%22%3A%22vendor_cluster_size%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22cluster_size%20%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-15m%22%2C%22to%22%3A%22now%22%7D%7D%7D:0 +[ 686ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1451ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 3030ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 4308ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 5296ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 7691ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 18337ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 20184ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 32473ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 50697ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 diff --git a/.playwright-mcp/page-2026-09-04T04-19-59-064Z.yml b/.playwright-mcp/page-2026-09-04T04-19-59-064Z.yml new file mode 100644 index 0000000..5588751 --- /dev/null +++ b/.playwright-mcp/page-2026-09-04T04-19-59-064Z.yml @@ -0,0 +1,56 @@ +- generic [ref=f18e4]: + - link "Skip to main content" [ref=f18e5] [cursor=pointer]: + - /url: "#pageContent" + - banner [ref=f18e7]: + - generic [ref=f18e8]: + - link [ref=f18e10] [cursor=pointer]: + - /url: / + - img "Grafana" [ref=f18e11] + - generic [ref=f18e14]: + - button "Search or jump to..." [ref=f18e18] [cursor=pointer] + - generic [ref=f18e19]: ctrl+k + - generic [ref=f18e23]: + - button "New" [ref=f18e24] [cursor=pointer] + - button "Help" [ref=f18e30] [cursor=pointer] + - button "News" [ref=f18e33] [cursor=pointer] + - button "Profile" [ref=f18e36] [cursor=pointer]: + - img "User avatar" [ref=f18e37] + - generic [ref=f18e38]: + - button "Open menu" [ref=f18e40] [cursor=pointer] + - navigation "Breadcrumbs" [ref=f18e43]: + - list [ref=f18e44]: + - listitem [ref=f18e45]: + - link "Home" [ref=f18e46] [cursor=pointer]: + - /url: / + - listitem [ref=f18e50]: + - link "Explore" [ref=f18e51] [cursor=pointer]: + - /url: /explore + - listitem [ref=f18e55]: + - generic "Prometheus" [ref=f18e56] + - generic [ref=f18e57]: + - button "Show more items" [ref=f18e60] [cursor=pointer] + - button "Toggle top search bar" [ref=f18e64] [cursor=pointer] + - main [ref=f18e70]: + - generic [ref=f18e72]: + - heading "Explore" [level=1] [ref=f18e73] + - generic [ref=f18e78]: + - navigation "Explore toolbar" [ref=f18e80]: + - navigation "Search links" [ref=f18e82]: + - generic [ref=f18e83]: + - button "Content outline" [expanded] [ref=f18e85] [cursor=pointer]: + - generic [ref=f18e88]: Outline + - generic [ref=f18e93] [cursor=pointer]: + - img "Prometheus logo" [ref=f18e95] + - textbox "Select a data source" [ref=f18e96]: + - /placeholder: "" + - button "Show more items" [ref=f18e102] [cursor=pointer] + - generic [ref=f18e106]: + - button "Collapse outline" [expanded] [ref=f18e112] [cursor=pointer]: + - img "arrow-from-right" [ref=f18e113] + - generic [ref=f18e120]: + - generic [ref=f18e123]: + - button "Add query" [ref=f18e124] [cursor=pointer] + - button "Query history" [ref=f18e128] [cursor=pointer] + - button "Query inspector" [ref=f18e132] [cursor=pointer] + - generic: + - main \ No newline at end of file diff --git a/docs/evidence/a8-rolling-restart/01-restart-availability.txt b/docs/evidence/a8-rolling-restart/01-restart-availability.txt new file mode 100644 index 0000000..4584f20 --- /dev/null +++ b/docs/evidence/a8-rolling-restart/01-restart-availability.txt @@ -0,0 +1,17 @@ +=== [1] 재시작 전 로그인 — 토큰을 파드 안에 보관 === + sid = XLcgQWRiJrTkuNZcJsNeT_2j + DB 세션 수: 151 + +=== [2] 롤링 재시작 중 가용성 — 5초 간격으로 외부 진입점 확인 === +statefulset.apps/keycloak restarted +200 Waiting for partitioned roll out to finish: 0 out of 2 new pods have been updated... +Waiting for 1 pods to be ready... +Waiting for 1 pods to be ready... +Waiting for 1 pods to be ready... +200 200 200 200 Waiting for partitioned roll out to finish: 1 out of 2 new pods have been updated... +Waiting for 1 pods to be ready... +Waiting for 1 pods to be ready... +Waiting for 1 pods to be ready... +200 200 200 200 partitioned roll out complete: 2 new pods have been updated... + + (위 숫자열이 재시작 중 외부 응답 코드의 시계열) diff --git a/docs/evidence/a8-rolling-restart/02-session-survival.txt b/docs/evidence/a8-rolling-restart/02-session-survival.txt new file mode 100644 index 0000000..367869d --- /dev/null +++ b/docs/evidence/a8-rolling-restart/02-session-survival.txt @@ -0,0 +1,19 @@ +=== [3] 재시작 전 발급한 refresh token 이 아직 통하는가 === + 대상 sid: XLcgQWRiJrTkuNZcJsNeT_2j + keycloak-0 에서 refresh HTTP 200 + +=== [4] DB 에 그 세션이 남아 있는가 === + user_session_id | created_on | last_session_refresh +--------------------------+------------+---------------------- + XLcgQWRiJrTkuNZcJsNeT_2j | 1788495513 | 1788495577 +(1 row) + + 전체 온라인 세션: 151 (재시작 전 151) + +=== [5] 캐시는 어떻게 되었는가 === + keycloak-0 sessions 캐시 0.0 건 / cluster_size 2.0 + keycloak-1 sessions 캐시 1.0 건 / cluster_size 2.0 + +=== [6] 파드 나이 — 정말 재시작되었나 === +keycloak-0 1/1 Running 0 44s +keycloak-1 1/1 Running 0 66s diff --git a/docs/evidence/a8-rolling-restart/README.md b/docs/evidence/a8-rolling-restart/README.md new file mode 100644 index 0000000..c55475d --- /dev/null +++ b/docs/evidence/a8-rolling-restart/README.md @@ -0,0 +1,16 @@ +# A-8 — 롤링 재시작 증거 + +2026-09-04 13:38–13:41 KST +해설: [`docs/experiment-a8-rolling-restart.md`](../../experiment-a8-rolling-restart.md) + +| 파일 | 무엇을 보여주는가 | +|---|---| +| `01-restart-availability.txt` | 재시작 전 로그인(sid 기록), 롤링 재시작 중 외부 진입점 **9회 모두 `200`** | +| `02-session-survival.txt` | 재시작 전 토큰으로 refresh **`200`**, DB 행 생존(`last_session_refresh` 갱신 확인), **세션 수 151 → 151**, 캐시 0으로 초기화, 파드 나이 44초/66초 | +| `a8-cache-reset-cluster-reformed.png` | Grafana — 세션 캐시가 0 으로 떨어지고 `cluster_size` 가 다시 2 가 되는 구간 | + +## 핵심 세 줄 + +1. **무중단이었다.** 한 번에 하나씩 내리고 readiness 가 전환을 맞춰준다 — replica ≥ 2 가 전제. +2. **세션은 살아남고 캐시만 사라진다.** DB 151건 그대로, 재시작 전 토큰이 그대로 통한다. +3. **이것이 `persistent-user-sessions` 를 켜는 진짜 이유다.** A-7(volatile)에서 정반대가 나와야 한다. diff --git a/docs/evidence/a8-rolling-restart/a8-cache-reset-cluster-reformed.png b/docs/evidence/a8-rolling-restart/a8-cache-reset-cluster-reformed.png new file mode 100644 index 0000000..4297b51 Binary files /dev/null and b/docs/evidence/a8-rolling-restart/a8-cache-reset-cluster-reformed.png differ diff --git a/docs/experiment-a8-rolling-restart.md b/docs/experiment-a8-rolling-restart.md new file mode 100644 index 0000000..7ca66bc --- /dev/null +++ b/docs/experiment-a8-rolling-restart.md @@ -0,0 +1,175 @@ +# A-8 — 배포할 때마다 로그아웃되는가 + +브랜치 `feature/keycloak-a8-rolling-restart` · +증거 [`docs/evidence/a8-rolling-restart/`](evidence/a8-rolling-restart/) · +2026-09-04 13:38–13:41 KST + +**운영에서 가장 자주 겪는 일이다.** 장애가 아니라 정상 작업인데도 +사용자가 로그아웃되면 그건 사고다. + +--- + +## 0. 결론부터 + +| 확인 | 결과 | +|---|---| +| 재시작 중 서비스 중단 | **없음.** 전 구간 `200` | +| 재시작 전 발급한 refresh token | **여전히 통한다** (`200`) | +| DB 세션 수 | **151 → 151** 그대로 | +| 세션 캐시 | **0 으로 초기화** | +| 클러스터 | 자동 재형성 (`cluster_size = 2`) | + +**세션은 살아남고 캐시만 사라진다.** 이것이 `persistent-user-sessions` 를 +켜는 진짜 이유다. + +--- + +## 1. 방법 + +```bash +# 1. 재시작 전 로그인 — 토큰을 상주 파드 안에 보관한다 +kubectl -n keycloak-lab run a8-probe --image=curlimages/curl:8.11.1 \ + --restart=Never --command -- sleep 3600 +kubectl -n keycloak-lab exec a8-probe -- sh -c '<로그인 후 /tmp/rt, /tmp/sid 에 저장>' + +# 2. 재시작하면서 5초 간격으로 외부 진입점을 찍는다 +kubectl -n keycloak-lab rollout restart statefulset/keycloak +( for i in $(seq 1 48); do + curl -s -o /dev/null -w '%{http_code} ' --max-time 4 https://auth.hyeonworks.com/realms/master + sleep 5 + done ) & +kubectl -n keycloak-lab rollout status statefulset/keycloak --timeout=420s +``` + +**탐침 파드가 StatefulSet 밖에 있어야** 재시작을 넘어 토큰을 들고 있을 수 있다. + +--- + +## 2. 가용성 — 무중단이었다 + +``` +statefulset.apps/keycloak restarted +200 Waiting for partitioned roll out to finish: 0 out of 2 new pods have been updated... + Waiting for 1 pods to be ready... +200 200 200 200 Waiting for partitioned roll out to finish: 1 out of 2 new pods have been updated... + Waiting for 1 pods to be ready... +200 200 200 200 partitioned roll out complete: 2 new pods have been updated... +``` + +**9번 찍어서 9번 다 `200`.** 한 번도 끊기지 않았다. + +### 왜 무중단이 되는가 + +``` + StatefulSet 롤링 재시작 + │ + ├─ keycloak-1 종료 → Service 엔드포인트에서 빠짐 + │ └─ 이 동안 keycloak-0 이 전부 받는다 + ├─ keycloak-1 기동 → readiness UP → 엔드포인트 복귀 + │ + └─ keycloak-0 종료 → ... (반복) +``` + +**한 번에 하나씩** 내리므로 항상 최소 하나는 Ready 다. +readiness 프로브가 이 전환을 정확히 맞춰준다 — A-2 에서 본 그 메커니즘이 +여기서는 **정상 작업을 안전하게** 만든다. + +> 다만 이 실험대는 **파드가 2개**다. replica 1 이면 반드시 끊긴다. +> 무중단은 공짜가 아니라 **replica ≥ 2 와 readiness 의 조합**이다. + +--- + +## 3. 세션 생존 + +``` +=== 재시작 전 발급한 refresh token 이 아직 통하는가 === + 대상 sid: XLcgQWRiJrTkuNZcJsNeT_2j + keycloak-0 에서 refresh HTTP 200 + +=== DB 에 그 세션이 남아 있는가 === + user_session_id | created_on | last_session_refresh +--------------------------+------------+---------------------- + XLcgQWRiJrTkuNZcJsNeT_2j | 1788495513 | 1788495577 + + 전체 온라인 세션: 151 (재시작 전 151) +``` + +**`last_session_refresh` 가 `created_on` 보다 64초 뒤**다. 재시작 후의 refresh +가 **실제로 DB 에 기록**되었다는 뜻이다 — 응답 코드만 200 인 게 아니라 +쓰기까지 정상이다. + +**파드가 통째로 바뀌었는데(44초/66초 나이) 세션은 그대로다.** + +--- + +## 4. 캐시는 사라진다 + +``` + keycloak-0 sessions 캐시 0.0 건 / cluster_size 2.0 + keycloak-1 sessions 캐시 1.0 건 / cluster_size 2.0 +``` + +![캐시 초기화와 클러스터 재형성](evidence/a8-rolling-restart/a8-cache-reset-cluster-reformed.png) + +**캐시는 프로세스 메모리이므로 재시작에 사라진다.** `keycloak-1` 의 1건은 +방금 refresh 를 처리하며 새로 담은 것이다. + +``` + 재시작 전: 캐시 N건 + DB 151건 + 재시작 후: 캐시 0건 + DB 151건 ← 진실은 DB 에 있다 +``` + +**A-0 의 모델이 그대로 확인된다.** 캐시가 통째로 날아가도 정확성은 유지되고 +**첫 접근만 느려진다** (룩어사이드 캐시의 성질). + +--- + +## 5. 이것이 `persistent-user-sessions` 를 켜는 진짜 이유다 + +| | persistent (KC 26 기본) | volatile (KC 24 이전 방식) | +|---|---|---| +| 롤링 재시작 후 | **세션 유지** | **전원 로그아웃** | +| 배포 빈도 | 자유롭다 | 배포가 곧 사고다 | +| 대가 | DB 쓰기 (A-6 에서 본 지연) | 없음 | + +**A-7 에서 volatile 로 바꿔 같은 실험을 반복하면 여기가 정반대가 될 것이다.** +그 비교가 이 실험의 짝이다. + +--- + +## 6. 재현 절차 (명령어) + +```bash +# 상주 탐침 (StatefulSet 밖에 있어야 한다) +kubectl -n keycloak-lab run a8-probe --image=curlimages/curl:8.11.1 \ + --restart=Never --command -- sleep 3600 +kubectl -n keycloak-lab wait --for=condition=Ready pod/a8-probe --timeout=120s + +# 로그인하고 토큰 보관 +kubectl -n keycloak-lab exec a8-probe -- sh -c \ + 'curl -s -X POST http://:8080/realms/master/protocol/openid-connect/token \ + -d grant_type=password -d client_id=admin-cli -d username=admin -d password= > /tmp/tok' + +# 재시작 + 가용성 감시 +kubectl -n keycloak-lab rollout restart statefulset/keycloak +kubectl -n keycloak-lab rollout status statefulset/keycloak --timeout=420s + +# 세션 생존 확인 +kubectl -n keycloak-lab exec a8-probe -- sh -c \ + 'curl -s -o /dev/null -w "%{http_code}\n" -X POST http://:8080/realms/master/protocol/openid-connect/token \ + -d grant_type=refresh_token -d client_id=admin-cli -d refresh_token=$(cat /tmp/rt)' + +# DB 대조 +kubectl -n keycloak-lab exec deploy/postgres -- psql -U keycloak -d keycloak \ + -c "select user_session_id, created_on, last_session_refresh from offline_user_session where user_session_id=''" +``` + +--- + +## 7. 다음 실험에 남기는 것 + +| 실험 | 이 실험이 준 것 | +|---|---| +| **A-7** volatile 비교 | **이 실험을 그대로 반복하면 정반대 결과가 나와야 한다** | +| **D-2** 버전 업그레이드 | 롤링 재시작이 안전하다는 것이 업그레이드의 전제 | +| 구성 | 무중단은 **replica ≥ 2 + readiness** 의 조합이다 |