docs: record multi-node cluster setup, rationale and formation evidence

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
DongHyeonka
2026-09-03 17:26:45 +09:00
co-authored by Claude Opus 5
parent 6dce35ec83
commit 33878e8880
3 changed files with 495 additions and 0 deletions
@@ -0,0 +1,56 @@
수집 시각: 2026-09-03 17:24:54 KST
대상: Keycloak 26.7.0 × 2 + PostgreSQL 16, k3s 2노드
=== [1] 파드 배치 ===
keycloak-0 1/1 10.42.1.18 kc-lab-2
keycloak-1 1/1 10.42.0.16 kc-lab-1
postgres-7b474b88c8-bw7b8 1/1 10.42.1.19 kc-lab-2
=== [2] 클러스터 뷰 로그 (Infinispan) ===
-- keycloak-0 --
2026-09-03 08:18:23,359 INFO [org.infinispan.CLUSTER] (executor-thread-1) ISPN000094: Received new cluster view for channel ISPN: [keycloak-1-26938(v=16.0.12)|1] (2) [keycloak-1-26938(v=16.0.12), keycloak-0-49501(v=16.0.12)]
2026-09-03 08:18:23,433 INFO [org.infinispan.CLUSTER] (executor-thread-1) ISPN000079: Channel `ISPN` local address is `keycloak-0-49501`, physical addresses are `[10.42.1.18:7800]`
-- keycloak-1 --
2026-09-03 08:18:23,269 INFO [org.infinispan.CLUSTER] (jgroups-5,keycloak-1-26938(v=16.0.12)) ISPN000094: Received new cluster view for channel ISPN: [keycloak-1-26938(v=16.0.12)|1] (2) [keycloak-1-26938(v=16.0.12), keycloak-0-49501(v=16.0.12)]
2026-09-03 08:18:23,282 INFO [org.infinispan.CLUSTER] (jgroups-5,keycloak-1-26938(v=16.0.12)) ISPN100000: Node keycloak-0-49501 joined the cluster
2026-09-03 08:18:23,286 INFO [org.infinispan.CLUSTER] (jgroups-5,keycloak-1-26938(v=16.0.12)) ISPN100000: Node keycloak-0-49501 joined the cluster
=== [3] JGROUPS_PING 테이블 구조 ===
Table "public.jgroups_ping"
Column | Type | Collation | Nullable | Default
----------------+------------------------+-----------+----------+---------
address | character varying(200) | | not null |
name | character varying(200) | | |
cluster_name | character varying(200) | | not null |
ip | character varying(200) | | not null |
coord | boolean | | |
last_update | bigint | | |
coordinated_by | character varying(200) | | |
Indexes:
"constraint_jgroups_ping" PRIMARY KEY, btree (address)
=== [4] JGROUPS_PING 등록 내역 ===
name | cluster_name | ip | coord
------------------+--------------+-----------------+-------
keycloak-0-49501 | ISPN | 10.42.1.18:7800 | f
keycloak-1-26938 | ISPN | 10.42.0.16:7800 | t
(2 rows)
=== [5] 외부 접근 — OIDC discovery ===
issuer https://auth.hyeonworks.com/realms/master
authorization_endpoint https://auth.hyeonworks.com/realms/master/protocol/openid-connect/auth
token_endpoint https://auth.hyeonworks.com/realms/master/protocol/openid-connect/token
end_session_endpoint https://auth.hyeonworks.com/realms/master/protocol/openid-connect/logout
jwks_uri https://auth.hyeonworks.com/realms/master/protocol/openid-connect/certs
★ 전부 https. 첫 실험에서 확정한 KC_HOSTNAME + KC_PROXY_HEADERS 조합이 작동한다.
=== [6] 자원 사용 ===
keycloak-0 8m 594Mi
keycloak-1 9m 593Mi
postgres-7b474b88c8-bw7b8 3m 67Mi
--- 노드 ---
kc-lab-1 2248Mi (65%)
kc-lab-2 1447Mi (58%)
@@ -0,0 +1,62 @@
# 증거 — Keycloak 멀티노드 클러스터 형성
`docs/keycloak-multinode-cluster.md`의 근거 자료.
**정상적으로 클러스터가 형성된 상태**에서 수집했으며, 이후 고장을 주입한
뒤 이것과 대조한다.
수집 시각: 2026-09-03 17:24 KST
| 파일 | 내용 |
|---|---|
| `01-cluster-formed.txt` | 파드 배치·클러스터 뷰 로그·JGROUPS_PING·OIDC discovery·자원 |
## 이 상태에서 확인된 것
**클러스터 뷰가 멤버 2를 보고한다**
```
ISPN000094: Received new cluster view for channel ISPN:
[keycloak-1-26938|1] (2) [keycloak-1-26938, keycloak-0-49501]
ISPN100000: Node keycloak-0-49501 joined the cluster
ISPN000079: physical addresses are [10.42.1.18:7800]
```
**디스커버리와 통신 경로가 한 테이블에 다 보인다**
```
name | cluster_name | ip | coord
------------------+--------------+-----------------+-------
keycloak-0-49501 | ISPN | 10.42.1.18:7800 | f
keycloak-1-26938 | ISPN | 10.42.0.16:7800 | t
```
`name`/`cluster_name`은 **DB 디스커버리**의 결과이고, `ip``:7800`
**실제 통신 경로**다. 7800을 막으면 이 표는 그대로 채워지면서 클러스터 뷰만
깨질 것으로 예상한다 — 다음 실험의 가설이다.
`coord = t``keycloak-1`이 코디네이터다.
**배치** — 서로 다른 노드에 하나씩. PostgreSQL은 `kc-lab-2`에 있으므로
**그 노드를 죽이면 Keycloak 하나와 DB가 동시에 사라진다.**
```
keycloak-0 10.42.1.18 kc-lab-2
keycloak-1 10.42.0.16 kc-lab-1
postgres 10.42.1.19 kc-lab-2
```
**issuer가 https로 발급된다** — 첫 실험(2홉 헤더 계약)의 결론이 적용된 결과다.
## 재수집
```bash
kubectl -n keycloak-lab get pods -o wide
kubectl -n keycloak-lab logs keycloak-0 | grep -E 'ISPN000094|ISPN000079|ISPN100000'
PG=$(kubectl -n keycloak-lab get pod -l app=postgres -o name | head -1)
kubectl -n keycloak-lab exec "$PG" -- \
psql -U keycloak -d keycloak -c "SELECT name, cluster_name, ip, coord FROM jgroups_ping ORDER BY name;"
curl -s https://auth.hyeonworks.com/realms/master/.well-known/openid-configuration | python3 -m json.tool
kubectl -n keycloak-lab top pods
```