donghyeon.kang/keycloak-pattern
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Compare commits

merge into: donghyeon.kang/keycloak-pattern:feature/keycloak-b6-key-rotation
Branches Tags
donghyeon.kang/keycloak-pattern:main donghyeon.kang/keycloak-pattern:feature/keycloak-d4a-deploy-hook donghyeon.kang/keycloak-pattern:feature/keycloak-followup-untested-items donghyeon.kang/keycloak-pattern:feature/keycloak-d4-certificate-renewal donghyeon.kang/keycloak-pattern:feature/keycloak-d3-secret-management donghyeon.kang/keycloak-pattern:feature/keycloak-d2-version-upgrade donghyeon.kang/keycloak-pattern:feature/keycloak-d1-backup-restore donghyeon.kang/keycloak-pattern:feature/keycloak-c2-backchannel-logout donghyeon.kang/keycloak-pattern:feature/keycloak-c1-multi-app-sso donghyeon.kang/keycloak-pattern:feature/keycloak-b7-cookie-secret-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-b6-key-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-b5-redis-loss-persistence donghyeon.kang/keycloak-pattern:feature/keycloak-b4-edge-authorization-scope donghyeon.kang/keycloak-pattern:feature/keycloak-b3-refresh-token-contention donghyeon.kang/keycloak-pattern:feature/keycloak-b2-multi-instance-session donghyeon.kang/keycloak-pattern:feature/keycloak-b1-redis-session-store donghyeon.kang/keycloak-pattern:feature/keycloak-b0-bff-redis-deploy donghyeon.kang/keycloak-pattern:feature/keycloak-a7-volatile-comparison donghyeon.kang/keycloak-pattern:feature/keycloak-a8-rolling-restart donghyeon.kang/keycloak-pattern:feature/keycloak-a6-latency-injection donghyeon.kang/keycloak-pattern:feature/keycloak-a5-asymmetric-partition donghyeon.kang/keycloak-pattern:feature/keycloak-a4-node-loss donghyeon.kang/keycloak-pattern:feature/keycloak-a3-database-crash donghyeon.kang/keycloak-pattern:feature/keycloak-a2-database-loss donghyeon.kang/keycloak-pattern:feature/keycloak-a1-jgroups-transport-block donghyeon.kang/keycloak-pattern:feature/keycloak-multinode-cluster-jdbc-ping donghyeon.kang/keycloak-pattern:develop-keycloak-session-store donghyeon.kang/keycloak-pattern:develop donghyeon.kang/keycloak-pattern:develop-keycloak-pattern1 donghyeon.kang/keycloak-pattern:develop-keycloak-pattern4 donghyeon.kang/keycloak-pattern:feature/keycloak-traefik-forwardauth-alternative donghyeon.kang/keycloak-pattern:feature/keycloak-edge-forwardauth-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-edge-forwardauth-no-google donghyeon.kang/keycloak-pattern:develop-keycloak-pattern3 donghyeon.kang/keycloak-pattern:feature/keycloak-bff-vs-spa-direct donghyeon.kang/keycloak-pattern:feature/keycloak-single-ec2-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-single-ec2-no-google donghyeon.kang/keycloak-pattern:feature/keycloak-three-leg-trust-chain donghyeon.kang/keycloak-pattern:feature/keycloak-internal-spa-direct-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-internal-spa-direct-no-google donghyeon.kang/keycloak-pattern:feature/keycloak-federation-spa-zero-change donghyeon.kang/keycloak-pattern:feature/keycloak-account-linking-spa-ux donghyeon.kang/keycloak-pattern:feature/keycloak-refresh-token-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-pkce-flow-stages donghyeon.kang/keycloak-pattern:feature/keycloak-spring-rs-role-mapping donghyeon.kang/keycloak-pattern:develop-keycloak-pattern2 donghyeon.kang/keycloak-pattern:feature/keycloak-four-pattern-tradeoff-matrix donghyeon.kang/keycloak-pattern:feature/keycloak-public-domain-tunneling donghyeon.kang/keycloak-pattern:feature/keycloak-https-termination-caddy-nginx donghyeon.kang/keycloak-pattern:feature/keycloak-reverse-proxy-headers donghyeon.kang/keycloak-pattern:feature/keycloak-google-redirect-uri-policy donghyeon.kang/keycloak-pattern:feature/keycloak-account-linking-sub-vs-email donghyeon.kang/keycloak-pattern:feature/keycloak-idp-mappers-claim-to-role donghyeon.kang/keycloak-pattern:feature/keycloak-google-claim-attribute-mapping donghyeon.kang/keycloak-pattern:feature/keycloak-first-broker-login-flow donghyeon.kang/keycloak-pattern:feature/keycloak-idp-brokering-google-client donghyeon.kang/keycloak-pattern:feature/keycloak-patterns donghyeon.kang/keycloak-pattern:feature/keycloak-header-spoofing-defense donghyeon.kang/keycloak-pattern:feature/keycloak-nginx-auth-request-integration donghyeon.kang/keycloak-pattern:feature/keycloak-oauth2-proxy-oidc-flow donghyeon.kang/keycloak-pattern:feature/keycloak-bff-csrf-samesite-defense donghyeon.kang/keycloak-pattern:feature/keycloak-bff-oauth2login-session donghyeon.kang/keycloak-pattern:feature/keycloak-token-mediating-access-handoff donghyeon.kang/keycloak-pattern:feature/keycloak-token-mediating-confidential-client donghyeon.kang/keycloak-pattern:feature/keycloak-refresh-rotation-and-logout donghyeon.kang/keycloak-pattern:feature/keycloak-spa-token-storage-tradeoff donghyeon.kang/keycloak-pattern:feature/keycloak-iss-claim-hostname-mismatch donghyeon.kang/keycloak-pattern:feature/keycloak-spring-rs-audience-validator donghyeon.kang/keycloak-pattern:feature/keycloak-vanilla-js-spa-pkce donghyeon.kang/keycloak-pattern:feature/keycloak-realm-client-export donghyeon.kang/keycloak-pattern:feature/keycloak-docker-compose-stack
..
pull from: donghyeon.kang/keycloak-pattern:feature/keycloak-d4a-deploy-hook
Branches Tags
donghyeon.kang/keycloak-pattern:feature/keycloak-d4a-deploy-hook donghyeon.kang/keycloak-pattern:feature/keycloak-followup-untested-items donghyeon.kang/keycloak-pattern:feature/keycloak-d4-certificate-renewal donghyeon.kang/keycloak-pattern:feature/keycloak-d3-secret-management donghyeon.kang/keycloak-pattern:feature/keycloak-d2-version-upgrade donghyeon.kang/keycloak-pattern:feature/keycloak-d1-backup-restore donghyeon.kang/keycloak-pattern:feature/keycloak-c2-backchannel-logout donghyeon.kang/keycloak-pattern:feature/keycloak-c1-multi-app-sso donghyeon.kang/keycloak-pattern:feature/keycloak-b7-cookie-secret-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-b6-key-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-b5-redis-loss-persistence donghyeon.kang/keycloak-pattern:feature/keycloak-b4-edge-authorization-scope donghyeon.kang/keycloak-pattern:feature/keycloak-b3-refresh-token-contention donghyeon.kang/keycloak-pattern:feature/keycloak-b2-multi-instance-session donghyeon.kang/keycloak-pattern:feature/keycloak-b1-redis-session-store donghyeon.kang/keycloak-pattern:feature/keycloak-b0-bff-redis-deploy donghyeon.kang/keycloak-pattern:feature/keycloak-a7-volatile-comparison donghyeon.kang/keycloak-pattern:feature/keycloak-a8-rolling-restart donghyeon.kang/keycloak-pattern:feature/keycloak-a6-latency-injection donghyeon.kang/keycloak-pattern:feature/keycloak-a5-asymmetric-partition donghyeon.kang/keycloak-pattern:feature/keycloak-a4-node-loss donghyeon.kang/keycloak-pattern:feature/keycloak-a3-database-crash donghyeon.kang/keycloak-pattern:feature/keycloak-a2-database-loss donghyeon.kang/keycloak-pattern:feature/keycloak-a1-jgroups-transport-block donghyeon.kang/keycloak-pattern:feature/keycloak-multinode-cluster-jdbc-ping donghyeon.kang/keycloak-pattern:develop-keycloak-session-store donghyeon.kang/keycloak-pattern:develop donghyeon.kang/keycloak-pattern:develop-keycloak-pattern1 donghyeon.kang/keycloak-pattern:develop-keycloak-pattern4 donghyeon.kang/keycloak-pattern:feature/keycloak-traefik-forwardauth-alternative donghyeon.kang/keycloak-pattern:feature/keycloak-edge-forwardauth-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-edge-forwardauth-no-google donghyeon.kang/keycloak-pattern:develop-keycloak-pattern3 donghyeon.kang/keycloak-pattern:feature/keycloak-bff-vs-spa-direct donghyeon.kang/keycloak-pattern:feature/keycloak-single-ec2-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-single-ec2-no-google donghyeon.kang/keycloak-pattern:feature/keycloak-three-leg-trust-chain donghyeon.kang/keycloak-pattern:feature/keycloak-internal-spa-direct-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-internal-spa-direct-no-google donghyeon.kang/keycloak-pattern:feature/keycloak-federation-spa-zero-change donghyeon.kang/keycloak-pattern:feature/keycloak-account-linking-spa-ux donghyeon.kang/keycloak-pattern:feature/keycloak-refresh-token-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-pkce-flow-stages donghyeon.kang/keycloak-pattern:feature/keycloak-spring-rs-role-mapping donghyeon.kang/keycloak-pattern:develop-keycloak-pattern2 donghyeon.kang/keycloak-pattern:feature/keycloak-four-pattern-tradeoff-matrix donghyeon.kang/keycloak-pattern:feature/keycloak-public-domain-tunneling donghyeon.kang/keycloak-pattern:feature/keycloak-https-termination-caddy-nginx donghyeon.kang/keycloak-pattern:feature/keycloak-reverse-proxy-headers donghyeon.kang/keycloak-pattern:feature/keycloak-google-redirect-uri-policy donghyeon.kang/keycloak-pattern:feature/keycloak-account-linking-sub-vs-email donghyeon.kang/keycloak-pattern:feature/keycloak-idp-mappers-claim-to-role donghyeon.kang/keycloak-pattern:feature/keycloak-google-claim-attribute-mapping donghyeon.kang/keycloak-pattern:feature/keycloak-first-broker-login-flow donghyeon.kang/keycloak-pattern:feature/keycloak-idp-brokering-google-client donghyeon.kang/keycloak-pattern:feature/keycloak-patterns donghyeon.kang/keycloak-pattern:feature/keycloak-header-spoofing-defense donghyeon.kang/keycloak-pattern:feature/keycloak-nginx-auth-request-integration donghyeon.kang/keycloak-pattern:feature/keycloak-oauth2-proxy-oidc-flow donghyeon.kang/keycloak-pattern:feature/keycloak-bff-csrf-samesite-defense donghyeon.kang/keycloak-pattern:feature/keycloak-bff-oauth2login-session donghyeon.kang/keycloak-pattern:feature/keycloak-token-mediating-access-handoff donghyeon.kang/keycloak-pattern:feature/keycloak-token-mediating-confidential-client donghyeon.kang/keycloak-pattern:feature/keycloak-refresh-rotation-and-logout donghyeon.kang/keycloak-pattern:feature/keycloak-spa-token-storage-tradeoff donghyeon.kang/keycloak-pattern:feature/keycloak-iss-claim-hostname-mismatch donghyeon.kang/keycloak-pattern:feature/keycloak-spring-rs-audience-validator donghyeon.kang/keycloak-pattern:feature/keycloak-vanilla-js-spa-pkce donghyeon.kang/keycloak-pattern:feature/keycloak-realm-client-export donghyeon.kang/keycloak-pattern:feature/keycloak-docker-compose-stack donghyeon.kang/keycloak-pattern:main
28 Commits
feature/keycloak-b6-key-rotation .. feature/keycloak-d4a-deploy-hook
Author SHA1 Message Date
DongHyeonka 9465582b5d chore: 실행 환경 구성 문서 추가 및 수정 2026-09-10 15:55:36 +09:00
DongHyeonkaandClaude Opus 5 6f6ab86345 docs(guides): reproduction guides for all 26 experiments 
Written by subagents running under the writing-practitioner-guides skill,
one guide per experiment, 22,566 lines. Each walks a reader from baseline
capture through injection, injection verification, observation and recovery.

Section 3 carries the weight in most of them. Injection failed silently nine
times in this lab, and a failed injection looks exactly like no effect — so
the guides verify the target is actually in the intended state before
reading any result. A-4 makes virsh list the only proof because the node
reads Ready for 40 seconds after the machine is off; A-5 makes the packet
counter the sole go/no-go because a rule on the wrong node produces an empty
result that reads like a finding; A-6 quotes the run where 적용완료 was
printed between four Cannot find device "eth0" lines.

The traps the guides are built around are ones that invert a conclusion
rather than merely annoy:

  A-0   emptying the session table without a restart leaves cache entries
        that get counted as replication arriving
  A-2   dropping -o /dev/null fuses body and status into one string
  A-3   presence of "ready to accept connections" instead of its timestamp
  B-2   row count alone reads an UPDATE as nothing having happened
  B-4   tr ',' '\n' splits ["admin","editor"] so only admin is seen
  B-7   no login screen means the cookie died and SSO re-authenticated
  C-1   counting sessions without joining realm counts your own kcadm one
  D-1   kubectl exec without -i restores nothing and still exits 0
  D-4a  "ran with error output" is what success looks like

Every quoted block is copied from docs/evidence/ and marked 실측; reshaped
commands are marked 미검증 rather than passed off as measured. Where a source
document carries a ★ correction the guides follow the corrected claim — A-7's
REVOKED_TOKEN hypothesis, C-1's session count, B-2's schema attribution.

Two hazards are stated rather than smoothed over: B-6 deletes a key that
cannot be recreated, and D-1/D-4 need host sudo, which asks for a password,
so those steps say a person must type them.

Audit over all 26: 672 interpretation pairs, 486 evidence citations, 117
undo sections, and zero occurrences of the patterns the skill forbids —
no python data processing, no deprecated kubectl get endpoints, no
placeholders, no bare kcadm.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-07 18:29:00 +09:00
DongHyeonkaandClaude Opus 5 8062cc9a19 docs(guides): bring the seven setup guides in line with the practitioner skill 
The guides were written before the skill existed and a scored audit found
three gaps. Fixed by a subagent running under the skill, with measured
values, versions, IPs and quoted output declared off limits.

The big one: every step showed a command and its output, and almost none
said which line to look at or what it meant. 64 interpretation pairs added
across the seven files, weighted where the reading is hardest — 20 in the
Keycloak stage, where a Secret existing and a pod having received it are
different facts.

Only the extracting form of curl appeared. Where the reader meets a response
for the first time the guides now open with curl -I or curl -v and name the
lines worth reading; -w '%{http_code}' survives only where the code is a
value being compared — two upstream nodes against each other, or the 900-run
control loop.

Listing Secret keys went from a three-stage pipe to kubectl describe secret,
which prints the key names and their byte counts in one native command
without exposing a value.

And a tool assumption: jq and yamllint are installed on neither the lab host
nor the guests. The guides now say so where JSON is read by eye, rather than
sending the reader to install something mid-diagnosis. cloud-init schema is
on the guests and is now the guest-side check.

Also removes a stray Playwright screenshot committed at the repository root
in 919547a; the evidence copy under docs/evidence/b7a-orphan-session/ is the
one the document references.

Four things the audit left standing are recorded in the agent's report rather
than papered over — notably that 04's reload measurements are stated without
a reproduction procedure, and that 05 and 06 reference each other as
prerequisites.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-07 17:33:31 +09:00
DongHyeonkaandClaude Opus 5 88b7bd4bf0 docs(guides): hands-on guides for the seven setup stages 
The experiment documents record what was found. These record what to type to
reproduce it, in folders per stage.

Two kinds of command are kept apart. 하기/확인 is what somebody actually types
at a terminal — short, one at a time. 근거를 재려면 is the long measuring form
this lab used to put evidence in a document, marked as not needed day to day.
The same split applies to curl: -I to look once, -w '%{http_code}' only when
comparing across repetitions.

No placeholders. Where a value is needed the command that produces it is
given, and secrets are checked by length rather than printed:

  TOKEN=$(ssh kc-lab-1 'sudo cat /var/lib/rancher/k3s/server/node-token')
  echo "${#TOKEN} 자"

Stage 05 verifies resources in layers, because a Secret existing and a pod
having received it are different facts: keys, then length, then the value
inside the container, then which env var came from which Secret. Same for
workloads — Deployment to ReplicaSet to Pod, with the seven ReplicaSets this
cluster actually carries as the worked example.

Two commands were wrong and re-running them caught it. kubectl get endpoints
prints a deprecation warning on v1.33+, so the guide uses describe svc and
EndpointSlice. And the Keycloak image has no curl, so reading metrics from
inside the container fails with exit 127 — the guide asks Prometheus instead,
or runs a throwaway curl pod.

Read-only checks were executed against the running lab and their output is
quoted verbatim. Creating commands could not be re-run without destroying the
lab, so they are the ones used at build time; the README says which is which.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-07 16:48:10 +09:00
DongHyeonkaandClaude Opus 5 cdac9b8178 docs: give the twelve experiments that had no architecture diagram one 
An audit against the standard the series set — concepts, procedure,
commands, architecture diagram, evidence table, terminal output — found the
three new experiments met it while twelve of the original ones had no
diagram at all: A-0, A-1, A-3, A-4, A-5, A-6, A-8, B-0, B-2, B-7, C-2, D-2.

Each now has one drawn from what that experiment actually found, not filler:
A-0 shows sharing going through PostgreSQL rather than between the caches;
A-3 the gap between the 200 and the WAL flush, with both failed injections;
A-5 the three silent injection failures; A-6 the two places latency is
multiplied; B-0 the repository keyed by principal with no session id; B-2
the primary key that causes the overwrite; D-2 why the rolling update
stopped the accident halfway.

Also corrected the index's stale claim of 11 experiments without a
screenshot — it is 14, and the reason is recorded: those experiments were
measured from terminals, the database and logs, and the observability stack
does not scrape Redis, the BFF or PostgreSQL, so there is no console to
photograph.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 21:44:36 +09:00
DongHyeonkaandClaude Opus 5 a2f749292a docs: add architecture diagrams for the three follow-up experiments 
Each of A-7a, B-7a and D-4a had the concepts, procedure, commands and
terminal evidence but no diagram, which the rest of the series carries.

  a7a-cache-temperature.svg  the three cache states and the SQL each issues
  b7a-orphan-lifecycle.svg   ticket structure, accumulation across two
                             rotations, and why TTL is the only signal
  d4a-hook-timeline.svg      the 38m25s gap against the 1-2s one, on one axis

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 21:37:00 +09:00
DongHyeonkaandClaude Opus 5 e9baac21d5 docs(d4a): the deploy hook works — issuance to serving drops from 38m25s to 1-2s 
D-4 named the fix and never ran it. Running it:

  /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh
    #!/bin/sh
    nginx -t && nginx -s reload

certbot reports "Hook 'deploy-hook' ran", the master stays at PID 585 and
the worker is replaced 28829 -> 37252, and the served certificate changes.

One trap worth naming: certbot prefixes the hook output with "ran with error
output" whenever the hook writes anything to stderr, and nginx's routine
types_hash warning goes to stderr. Everything inside is success — "test is
successful", "signal process started". A monitor that greps for "error"
would read a working hook as a failure. The worker PID is what to check.

Timing needed a clock correction. test-server has NTP off and runs 106
seconds fast; the dev machine matches Google and the Let's Encrypt ACME
endpoint exactly. Corrected, the hook's nginx -t lands at 12:27:50 UTC
against the new certificate's SCT at 12:27:49.054 — one second. The
correction validates itself: uncorrected, the hook would have run 104
seconds before the certificate existed.

That same skew is why D-4's gap was reported 106 seconds short; corrected to
2305s in the previous commit.

Remaining and left alone: whether certbot-renew.timer performs a real
renewal, which cannot be tested for about 89 days.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 21:35:00 +09:00
DongHyeonkaandClaude Opus 5 3d7778bd3c docs(d4): correct the gap to 38m25s — the two timestamps came from different clocks 
The 2199 seconds reported for D-4 subtracted a test-server timestamp
(archive/cert2.pem mtime) from a dev-machine timestamp (the serial change
observed by the poll), without noting they are different clocks.

Checked against external references: the dev machine matches Google and the
Let's Encrypt ACME endpoint to the second, while test-server is 105 seconds
fast and reports NTPSynchronized=no. Three round-trip measurements put the
offset at +106.1s every time.

Corrected:

  new certificate written to disk  08:20:27 UTC   (mtime 17:22:13 KST - 106s)
  actually served                  08:58:52 UTC   (dev observation, no correction)
  gap                              2305s = 38m25s

The correction validates itself in D-4a, where the new certificate's SCT —
signed by CT logs on their own accurate clock at 12:27:49.054 GMT — lands
one second before the skew-corrected hook time. Without the correction the
hook would appear to have run 104 seconds before the certificate existed.

Updated across the experiment doc, the index, the follow-up doc, the verdict
evidence file and the SVG.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 21:33:01 +09:00
DongHyeonkaandClaude Opus 5 919547a025 docs(b7a): the orphan sessions can be deleted — oauth2-proxy just cannot do it 
B-7 stopped at "could not delete the server-side session". The reason it
gave was right: the ticket carries the session id, the ticket is encrypted
with the cookie secret, so after a rotation the proxy cannot work out which
Redis key to remove. But that is a limitation of the proxy, not of Redis.

Measured across two rotations:

- The orphan does expire. TTL falls one second per second and is not
  refreshed by requests (the startup log says refresh:disabled, and
  --cookie-refresh is unset), so it dies exactly one hour after creation.
- An operator can delete it. `redis-cli del` returned 1, dbsize went 2 to 1,
  and the live session answered /oauth2/userinfo with 200 immediately after.
- But nothing in Redis says which key is the orphan. Same name prefix, same
  type, the same 3510 bytes, and the values are encrypted.
- TTL is the only signal, and because it is never refreshed it is an exact
  function of creation time. Anything created before the rotation is an
  orphan. Derived creation time 11:30:26 against the AuthSuccess log line at
  11:30:27 — one second out. The rule was then run and removed the orphan
  while leaving the live session.
- They accumulate: the session that survived the first rotation became the
  orphan of the second.

The caveat is recorded too: turning on --cookie-refresh breaks the
derivation, and at that point flushing and forcing everyone to
re-authenticate is the more honest option.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 20:37:11 +09:00
DongHyeonkaandClaude Opus 5 b9f4ef7bc2 docs(a7a): the volatile refresh failure is a client-scope lookup, not REVOKED_TOKEN 
A-7 left the cause as a hypothesis and said to confirm it with the statement
logging A-3 used. Doing that shows the hypothesis was wrong.

Login issues no SQL at all in volatile mode. Refresh issues exactly one
statement, and it is not the one A-7 guessed:

  select cscme1_0.SCOPE_ID from CLIENT_SCOPE_CLIENT cscme1_0
   where cscme1_0.CLIENT_ID=$1 and cscme1_0.DEFAULT_SCOPE=$2
   parameters: $1 = '131a9912-...', $2 = 'f'

REVOKED_TOKEN never appears. DEFAULT_SCOPE='f' means this is the optional
client scope list, which refresh needs because it recomputes which scopes go
into the new access token.

The larger finding is that A-7's table was conditional and did not say so.
The single statement is issued once and then cached, so the same
configuration produces three different outcomes depending only on cache
warmth — all three reproduced:

  fully cold      login 400  (select ce1_0.ID from CLIENT ...)
  CLIENT warm     login 200, refresh 500  <- what A-7 measured
  fully warm      login 200, refresh 200

So "volatile means you can log in without a database" is also conditional:
on a cold start the client lookup itself fails. In each case the Keycloak
log names the failing SQL directly, so this is determined rather than
inferred.

A-7 keeps its original wording with a correction banner above it, so what
was mis-guessed and why stays visible.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 20:27:23 +09:00
DongHyeonkaandClaude Opus 5 9dbee18a42 docs(b4): item 3 is no longer unmeasured — link it to the follow-up result 
B-4 left role propagation open because oauth2-proxy was not deployed yet.
B-7 deployed it and the follow-up measured it: the value does not change
with request count, only when a new session is created.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 18:10:43 +09:00
DongHyeonkaandClaude Opus 5 905392947d docs: close the three untested items — all three are now measured 
D-2 forward upgrade and B-4 role propagation were filled earlier. D-4 forced
renewal was the third, and it needed a person at the keyboard because the
host asks for a sudo password.

It turned out to hold the largest finding of the three: the renewal
succeeded and went unserved for 36 minutes 39 seconds, because nothing in
the setup reloads nginx — no ExecStartPost, three empty hook directories,
no nginx plugin. And the half of the plan's question that was still an
expectation is now a measurement: the reload is graceful, 8856 connections
with zero failures and an in-flight request surviving intact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 18:10:04 +09:00
DongHyeonkaandClaude Opus 5 faf55dc1be docs(d4): the renewal succeeded and was not served for 36 minutes 39 seconds 
The forced renewal did work. What did not work was anything making nginx
notice.

  disk     cert2.pem written 2026-09-04 17:22:13 KST
  network  old serial through 08:58:47 UTC, new serial from 08:58:52 UTC
  gap      2199 seconds, and the old certificate was observed 428 times in it

The gap closed only because a person ran `nginx -s reload`. Nothing else
would have: certbot-renew.service is just `ExecStart=/usr/bin/certbot -q
renew` with no ExecStartPost, all three of renewal-hooks/{deploy,post,pre}
are empty, and certbot 5.7.0 here has no nginx plugin (dns-cloudflare,
manual, null, standalone, webroot). Three paths, all empty. Any one of them
would have been enough.

nginx holds the certificate in memory from startup, and certbot swaps the
live/ symlink rather than the path — so the config still looks right while
the served certificate is stale. Master 585 and worker 586 sharing a start
time 22.4 hours old is what proved no reload had happened.

This is invisible for 88 days. The timer ran twice today and exited
0/SUCCESS both times, because a renewal is not due yet. The day it becomes
due, the symptom is an expired certificate and the log still says SUCCESS.

The other half of the plan's question now has a measurement rather than an
expectation. The reload is genuinely graceful: 8856 polled connections, zero
non-200, p95 205.7ms before against 204.3ms after. And an in-flight request
— 845KB pulled at 20k/s, so still transferring 12 seconds in when the reload
landed — completed with all 845361 bytes over a single connection.

The earlier note that nginx reload "is graceful by design but must not be
written that way without checking" was right to hold back. The neighbouring
belief, that renewing gets the new certificate served, was the one that was
false, and there was no way to tell in advance which it would be.

Adds diagrams/d4-renewal-gap.svg and evidence 09-13.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 18:04:44 +09:00
DongHyeonkaandClaude Opus 5 0d84921a56 docs(d4): the renewal unit has no reload, and nginx has not reloaded in 22 hours 
The forced renewal did not change the certificate being served — 161 serial
samples over 13 minutes, all identical, notBefore still Sep 3.

Two things were readable without sudo and both matter.

nginx has never been reloaded: master 585 and worker 586 share a start time
and both are 80529 seconds old. A reload keeps the master and replaces the
workers, so the original fork still being there means no reload has happened
since Sep 3 19:00. nginx reads the certificate at startup and holds it in
memory; without a reload it serves the old one no matter what is on disk.

And the unit that is supposed to renew has nothing that would reload it:

  [Service]
  Type=oneshot
  ExecStart=/usr/bin/certbot -q renew
  PrivateTmp=true

No ExecStartPost, no --deploy-hook. The timer runs twice a day and exited
0/SUCCESS both times today, which is precisely why this stays invisible —
the renewal is not due for 88 days, so nothing has failed yet.

What remains undetermined is whether the user's --force-renewal failed, or
succeeded and left new files that nginx never picked up. Telling those apart
needs /etc/letsencrypt, which is root-only. If it is the latter, that is the
most valuable thing this lab has produced: "renewal succeeded" and "the new
certificate is being served" are different events.

CT logs looked like a way around the permission problem and were not: the
served certificate carries two valid SCTs, yet crt.sh returns zero rows for
auth.hyeonworks.com. Same shape as A-2's `up` metric — the observing tool
sees a subset of the truth.

Also recorded: 76 in-flight failures at 08:15:04 are a local artifact, not a
server event. The concurrent 0.2s poll was clean across the same second,
num_connects was 0, and time_total was 50 microseconds. Not reproducible in
100 retries. The monitor now records curl's exit code so a recurrence is
diagnosable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 17:28:01 +09:00
DongHyeonkaandClaude Opus 5 716e62524a docs(d4): measure the controls before the injection that needs a password 
The forced renewal needs sudo on the host, and the host asks for a password.
That blocks the injection, not the experiment — the part worth doing first
was the control anyway.

Control 1 — new connections, 0.2s x 900 over 180s: 900/900 = 200, zero
failures. The noise floor is 0, so a single non-200 during the renewal can
be attributed to the renewal. Without this the observation would have been
uninterpretable, which is the mistake A-6 made calling a -41% control
"no effect" and A-8 made claiming zero-downtime from 9 samples.

Control 2 — the poll cannot answer the question the plan actually asked.
TLS handshakes were 900/900, meaning every request is a fresh connection,
so it measures "are new connections accepted", not "what happens to a
request already in flight". A separate device: the 845KB admin console
bundle pulled at --limit-rate 20k, holding one request open for 42 seconds.
Baseline 200 / 845361 bytes / 1 connection.

Monitors are running under setsid, and stop three minutes after the
certificate serial changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 17:12:31 +09:00
DongHyeonkaandClaude Opus 5 74c9b3cea7 docs: replace prose placeholders in reproduction steps with executable commands 
The audit found ~80 placeholders, and the damaging ones were where the
measuring apparatus itself was prose rather than a command:

  a6  "( curl ... ) & 를 20개 띄우고 wait"  — the 22.2s headline came from this
  a3  "<로그인 반복, sid 를 /tmp/sids 에>"  — the whole RPO measurement
  a3  "<sid 목록>"                          — the control it is compared against
  a5  "<수신 파드IP>"                       — the injection
  a8  writes /tmp/tok, reads /tmp/rt        — self-inconsistent, sent an empty token
  b3  $KC / $RT / $NEW never assigned
  c2  bare kcadm.sh with no kubectl exec
  a1  conntrack tuples written by hand, though the direction flips per restart

Each is now a shell-expandable form: pod IPs from jsonpath, the admin password
from the secret, ids from kcadm --format csv, conntrack tuples derived from
"conntrack -L" with awk rather than transcribed.

Then the rewritten commands were executed against the live cluster, and one
of them failed — the 20-way load generator, written as "kubectl run --rm -i",
lost its output stream twice in a row. That is a trap this series already hit
once, and the rewrite reintroduced it. A-6 now uses a resident probe pod that
collects into a file and is cat-ed once; verified 20/20 lines.

Evidence: docs/evidence/followup/05-command-reproducibility.txt

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 17:01:54 +09:00
DongHyeonkaandClaude Opus 5 f3f3a8da46 docs: re-measure B-4 role propagation and record it as evidence 
The twelve-request result and the post-reauth value were printed in the document but never written to a file; the audit flagged it as a conclusion standing without evidence. Measured again with timestamps into 03-b4-role-propagation.txt, which also surfaced a 107-second clock skew between the browser host and the lab host that the file now records so the two timelines can be compared.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 16:54:05 +09:00
DongHyeonkaandClaude Opus 5 e0d27d47ce docs: correct the places where documents contradicted their own evidence 
An independent audit found ten documents printing values their evidence files do not contain. C-1 printed a session count of 0 where the evidence says 4, C-2 printed a success readback for a command that exited 1, and A-1 credited the conntrack flush with a split that the timestamps attribute to a pod restart four seconds earlier.

Also measured wal_writer_delay, which A-3 had asserted as matching without ever querying it, relabelled the A-6 control that moved 41 percent, noted A-8's nine-sample resolution, corrected D-1's RTO to the 41 seconds its own timeline shows, and added a correction banner to D-2. Every experiment document now links its evidence files with their real collection times, and the duplicate screenshots are documented as duplicates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 16:35:49 +09:00
DongHyeonkaandClaude Opus 5 78b270559c docs: add SVG diagrams, explicit concept sections and the diagram convention 
Twelve SVG architecture diagrams cover the experiments whose documents had little or no structure drawing, embedded under a 구조 heading with a shared convention file. Seven documents carried their concepts under narrative headings and now have an explicit 개념 section so they can be found.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 16:20:05 +09:00
DongHyeonkaandClaude Opus 5 98a74e90a5 docs: fill the untested items and record why the B layer has no graphs 
The forward upgrade to 26.7.3 was zero downtime across 87 samples, and since databasechangelog stayed at 210 the rollback to 26.7.0 also succeeded, which narrows D-2's conclusion: rolling back fails when the schema moved, not because of the version number. The row count is the check.

Role changes never reach the upstream through request repetition; the session is a snapshot taken at login and only a new session picks up the new claim. Auditing the docs also surfaced that Prometheus scrapes only keycloak, kubelet, node-exporter and itself, so the B-layer experiments have no metrics to screenshot rather than missing screenshots.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 16:18:33 +09:00
DongHyeonkaandClaude Opus 5 b5528fae87 docs: index all 23 experiments with what each measured 
One table per experiment with its branch and result, plus the nine injections that silently did nothing and the five predictions that turned out wrong.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 15:11:32 +09:00
DongHyeonkaandClaude Opus 5 4864d837f1 docs: D-4 — the certificate is fine and the renewal itself went untested 
Three SAN entries and no wildcard is the constraint that cost something real in B-7, where oauth2-proxy had to borrow Grafana's app2 hostname because a fourth name was not available. The served chain is four deep and verifies, so fullchain.pem is configured rather than the cert.pem mistake that only breaks clients without a cached intermediate.

The forced renewal and the reload behaviour could not be measured because sudo on the host asks for a password, the same silent failure first noticed in B-7. nginx reload is graceful by design, but this lab has repeatedly shown that by design is not the same as measured, so it is recorded as untested rather than assumed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 15:09:36 +09:00
DongHyeonkaandClaude Opus 5 027c24ee27 docs: D-3 — only RBAC actually hides anything 
Every secret in the lab prints in four commands, while kubectl describe shows just a byte count and creates the impression that something is hidden. k3s reports encryption at rest disabled and the plaintext password is present in state.db, so one node disk carries the whole cluster's secrets, and inside the pod they are ordinary environment variables visible to exec, /proc and crash dumps.

The default service account cannot read secrets, which makes RBAC the one control doing real work here and the thing worth tightening.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 15:07:35 +09:00
DongHyeonkaandClaude Opus 5 df140ab218 docs: D-2 — rolling back the image does not roll back the schema 
Downgrading from 26.7.0 to 26.0 fails with liquibase ValidationFailedException on a changeset checksum, which is stricter than an unknown migration: the old version knows the changeset but its definition differs. The pod goes CrashLoopBackOff and never starts.

The StatefulSet stopped the rollout at the first pod, so the other kept serving and the front door stayed at 200, which replica 1 would not have done. The failed start never touched the schema, so restoring the image was enough; had the migration already applied, the D-1 database restore would have been the only way back.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 15:05:36 +09:00
DongHyeonkaandClaude Opus 5 df5af95cb3 docs: D-1 — an empty database still answered 200 
Dropping the schema left Keycloak serving realm metadata and JWKS from its Infinispan cache, so the front door stayed at 200 while only the paths that read the database failed. That is a different shape from A-2, where the connection itself broke and readiness pulled the pods out of the Service; here the connection is fine and the tables are simply gone, which the health check does not notice.

Restoring the pg_dump took one second with zero errors and no pod restart, and the row counts matched the backup exactly, sessions included. The real RPO is the backup interval plus the synchronous_commit loss measured in A-3, and this dump sits in the host's /tmp, which is the same failure domain as the thing it protects.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 14:59:56 +09:00
DongHyeonkaandClaude Opus 5 6c310c93b7 docs: C-2 — nothing propagates because nobody implemented the receiving end 
Neither client had a backchannel logout URL and the BFF has no oidcLogout configuration, so the three candidate paths all answer 302, which is the authentication redirect rather than a handler. Setting the URL on the identity provider alone changed nothing: with a live session, logging the user out emptied the Keycloak side and left the Redis session untouched.

Reachability is not the blocker here, since a Keycloak pod fetches the app's public URL with a 200, but that is a property of this tailnet split-DNS lab and is the assumption most likely to fail in production, where it fails silently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 14:56:58 +09:00
DongHyeonkaandClaude Opus 5 e856e7af4d docs: C-1 — killing the SSO session logs nobody out 
One user session carries a client session per application, so visiting the second app skips the login screen. Deleting the identity provider session leaves both application sessions untouched and both apps keep serving, because the identity provider, the application session and the access token each have their own lifetime.

That inverts the B-2 finding: there the app session was cleared and the surviving SSO session let the user straight back in. Either way, clearing one side leaves the other. It also means an identity provider outage is a single point of failure for logging in, not for already-authenticated users, and the failure arrives late and all at once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 14:49:36 +09:00
DongHyeonkaandClaude Opus 5 bcb563a04e docs: B-7 — the cookie secret has no overlap window and rotation orphans sessions 
oauth2-proxy carries the authorization request in a signed cookie, so the callback can land on a different replica and still succeed, which is the opposite of the BFF failure in B-0. Sharing is therefore just sharing one Secret.

Rotating it is all-or-nothing: --cookie-secret is singular, so there is no second key to read old tickets with, and the log shows both the validation failure and Error removing session, leaving the Redis session orphaned because the key cannot be derived from a ticket that will not decode.

Getting there required two diagnoses: the callback 502 came from the full session riding in Set-Cookie past nginx's buffer, and every earlier attempt to read nginx config returned nothing because sudo on the host asks for a password while the guests do not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 14:44:00 +09:00
219 changed files with 35186 additions and 76 deletions
Expand all files Collapse all files
.playwright-mcp/console-2026-09-04T05-35-58-200Z.log
+2
View File
@@ -0,0 +1,2 @@
[ 275ms] [ERROR] Failed to load resource: the server responded with a status of 502 () @ https://app2.hyeonworks.com/oauth2/callback?state=G2-BDWkehNWO7hGwhYCxXBVRKZ6AomLIrSLBtIXr0Gw%3A%2Fapi%2Fecho&session_state=vsW8xDlLJN3DUl-0B-X1WL7Q&iss=https%3A%2F%2Fauth.hyeonworks.com%2Frealms%2Fkeycloak-patterns&code=4155f58e-6a58-e47b-93bd-7e2b625c2b91.vsW8xDlLJN3DUl-0B-X1WL7Q.80431dbc-af81-4673-9790-ad06d1570b2e:0
[ 408ms] [ERROR] Failed to load resource: the server responded with a status of 502 () @ https://app2.hyeonworks.com/oauth2/callback?state=Da-7OcMB4f7vyHgr-6CqrTtJpmj1R_SfRfcE3CUJNzE%3A%2Ffavicon.ico&session_state=vsW8xDlLJN3DUl-0B-X1WL7Q&iss=https%3A%2F%2Fauth.hyeonworks.com%2Frealms%2Fkeycloak-patterns&code=96d66247-91b0-0cc1-89dc-e527c2b69bf2.vsW8xDlLJN3DUl-0B-X1WL7Q.80431dbc-af81-4673-9790-ad06d1570b2e:0
.playwright-mcp/console-2026-09-04T05-36-21-060Z.log
+2
View File
@@ -0,0 +1,2 @@
[ 266ms] [ERROR] Failed to load resource: the server responded with a status of 502 () @ https://app2.hyeonworks.com/oauth2/callback?state=TZnQvjIWCEySrf4PMg2WLVFoOfkyOJWB58f2LGjVfpo%3A%2Fapi%2Fecho&session_state=vsW8xDlLJN3DUl-0B-X1WL7Q&iss=https%3A%2F%2Fauth.hyeonworks.com%2Frealms%2Fkeycloak-patterns&code=8ae913a1-2647-0ed9-625e-3d80e1565024.vsW8xDlLJN3DUl-0B-X1WL7Q.80431dbc-af81-4673-9790-ad06d1570b2e:0
[ 416ms] [ERROR] Failed to load resource: the server responded with a status of 502 () @ https://app2.hyeonworks.com/oauth2/callback?state=uAYwZp59ncz95XjkJXAlIgsT7oksBQBViiQS07t2eow%3A%2Ffavicon.ico&session_state=vsW8xDlLJN3DUl-0B-X1WL7Q&iss=https%3A%2F%2Fauth.hyeonworks.com%2Frealms%2Fkeycloak-patterns&code=7b7fc816-0b27-93a0-83b8-656488813253.vsW8xDlLJN3DUl-0B-X1WL7Q.80431dbc-af81-4673-9790-ad06d1570b2e:0
.playwright-mcp/console-2026-09-04T05-38-20-492Z.log
+2
View File
@@ -0,0 +1,2 @@
[ 287ms] [ERROR] Failed to load resource: the server responded with a status of 502 () @ https://app2.hyeonworks.com/oauth2/callback?state=0EOFj1PoLyPil0dgukpi7zKW4JKnGZTP9Wj6EhTR-lw%3A%2Fapi%2Fecho&session_state=vsW8xDlLJN3DUl-0B-X1WL7Q&iss=https%3A%2F%2Fauth.hyeonworks.com%2Frealms%2Fkeycloak-patterns&code=d13cc206-133f-00a9-9908-599988c4d7cf.vsW8xDlLJN3DUl-0B-X1WL7Q.80431dbc-af81-4673-9790-ad06d1570b2e:0
[ 457ms] [ERROR] Failed to load resource: the server responded with a status of 502 () @ https://app2.hyeonworks.com/oauth2/callback?state=kbYC5O4_ELsoQFw85vyYfgWEqlI2yWImB4rmelbmSTM%3A%2Ffavicon.ico&session_state=vsW8xDlLJN3DUl-0B-X1WL7Q&iss=https%3A%2F%2Fauth.hyeonworks.com%2Frealms%2Fkeycloak-patterns&code=91f9fde9-f376-b6f3-4622-a1ba674cc4fd.vsW8xDlLJN3DUl-0B-X1WL7Q.80431dbc-af81-4673-9790-ad06d1570b2e:0
.playwright-mcp/console-2026-09-04T05-41-10-043Z.log
+1
View File
@@ -0,0 +1 @@
[ 307ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T05-42-18-456Z.log
+1
View File
@@ -0,0 +1 @@
[ 261ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T05-46-37-048Z.log
+1
View File
@@ -0,0 +1 @@
[ 882ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T05-48-09-177Z.log
+1
View File
@@ -0,0 +1 @@
[ 178ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T06-25-14-829Z.log
+2
View File
@@ -0,0 +1,2 @@
[ 19340ms] [ERROR] Failed to load resource: the server responded with a status of 403 () @ https://app2.hyeonworks.com/oauth2/callback?state=rKipZCUv8W5a-xgYheJbjBsInoD5Il1AaF1RlM_RB2s%3A%2Fapi%2Fecho&session_state=Mw52KcQijFB9Bq4rN-C4SF5Y&iss=https%3A%2F%2Fauth.hyeonworks.com%2Frealms%2Fkeycloak-patterns&code=f9a4835a-2af3-b886-bd04-10b5347ee8d2.Mw52KcQijFB9Bq4rN-C4SF5Y.80431dbc-af81-4673-9790-ad06d1570b2e:0
[ 20374ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T06-25-42-707Z.log
+1
View File
@@ -0,0 +1 @@
[ 210ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T06-26-41-840Z.log
+1
View File
@@ -0,0 +1 @@
[ 423ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T07-14-56-538Z.log
+8
View File
@@ -0,0 +1,8 @@
[ 1127ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2377ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&panes=%7B%22h4a%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_cluster_size%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22cluster_size+%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%2C%7B%22refId%22%3A%22B%22%2C%22expr%22%3A%22up%7Bjob%3D%5C%22keycloak%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22up+%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-30m%22%2C%22to%22%3A%22now%22%7D%7D%7D&orgId=1:0
[ 2472ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 3501ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 5119ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 8511ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 14956ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 28065ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T07-15-33-300Z.log
+10
View File
@@ -0,0 +1,10 @@
[ 1362ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1874ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&orgId=1&panes=%7B%22a%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_cluster_size%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22cluster_size%20%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%2C%7B%22refId%22%3A%22B%22%2C%22expr%22%3A%22up%7Bjob%3D%5C%22keycloak%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22up%20%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%221788502680000%22%2C%22to%22%3A%221788503520000%22%7D%7D%7D:0
[ 2907ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 3998ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 6253ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 9426ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 12495ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 24486ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 31338ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 46196ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T07-16-34-745Z.log
+198
View File
@@ -0,0 +1,198 @@
[ 1319ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&orgId=1&panes=%7B%22a%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_cluster_size%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22cluster_size%20%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%2C%7B%22refId%22%3A%22B%22%2C%22expr%22%3A%22vendor_statistics_approximate_entries_unique%7Bcache%3D%5C%22sessions%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22sessions%20%5Cuce90%5Cuc2dc%20%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%221788497040000%22%2C%22to%22%3A%221788499080000%22%7D%7D%7D:0
[ 5941ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 11107ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 14234ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 17005ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 23049ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 30565ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 44135ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 54992ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 63653ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 70658ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 90768ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 105475ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 114995ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 125443ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 137321ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 147252ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 167227ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 179100ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 187598ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 207362ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 213255ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 230544ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 248833ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 257549ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 262324ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 274443ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 294124ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 310689ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 312049ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 327025ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 339933ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 342391ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 362305ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 370693ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 378685ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 397930ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 414007ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 419130ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 420355ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 428557ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 442088ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 445041ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 446677ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 460087ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 462505ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 463673ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 482762ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 495123ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 513442ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 516414ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 536581ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 543441ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 563624ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 580510ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 588408ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 594209ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 604166ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 605295ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 621987ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 639705ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 653320ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 660911ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 678105ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 693662ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 703089ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 704712ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 719727ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 725514ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 730429ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 735135ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 736677ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 751626ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 767197ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 782052ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 797775ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 803343ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 822168ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 828683ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 845938ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 865903ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 883116ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 894375ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 898471ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 914886ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 933184ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 953353ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 963902ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 982846ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 998975ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1000721ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1003229ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1009879ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1016334ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1023805ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1043684ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1049467ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1059107ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1075856ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1076956ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1094159ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1102825ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1116077ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1134809ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1136553ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1141191ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1158221ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1173507ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1190834ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1198506ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1217761ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1231342ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1233528ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1243017ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1253381ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1256056ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1258513ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1271317ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1284530ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1301830ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1307562ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1310330ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1317970ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1328668ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1336338ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1340155ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1346989ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1348836ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1352113ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1367871ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1371158ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1390720ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1405259ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1420920ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1433518ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1450513ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1453685ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1460654ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1470589ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1489330ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1503666ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1521587ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1537970ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1540426ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1548105ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1549441ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1567671ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1585493ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1597054ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1602669ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1607866ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1623290ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1631460ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1640086ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1645979ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1650742ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1652863ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1663908ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1666587ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1668427ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1676825ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1687882ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1691062ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1698532ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1709597ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1725669ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1739294ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1744412ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1753319ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1761792ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1765298ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1784381ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1786396ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1798982ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1807180ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1823569ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1837292ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1838720ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1848387ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1856847ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1866742ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1879249ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1884700ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1900774ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1912256ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1931088ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1937844ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1948191ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1951360ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1957512ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1958836ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1979011ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1993551ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2011369ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2024782ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2040763ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2044650ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2061337ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 302 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T07-51-02-373Z.log
+1
View File
@@ -0,0 +1 @@
[ 12202ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T07-52-53-917Z.log
+1
View File
@@ -0,0 +1 @@
[ 451ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T11-28-55-952Z.log
+1
View File
@@ -0,0 +1 @@
[ 14441ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ chrome-error://chromewebdata/:0
.playwright-mcp/console-2026-09-04T11-30-27-075Z.log
+1
View File
@@ -0,0 +1 @@
[ 296ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ chrome-error://chromewebdata/:0
.playwright-mcp/console-2026-09-04T11-32-26-121Z.log
+1
View File
@@ -0,0 +1 @@
[ 158ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T11-32-49-840Z.log
+1
View File
@@ -0,0 +1 @@
[ 67ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T11-33-54-917Z.log
+2
View File
@@ -0,0 +1,2 @@
[ 40ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/oauth2/userinfo:0
[ 162ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ https://app2.hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-04T11-34-00-240Z.log
+1
View File
@@ -0,0 +1 @@
[ 117ms] [ERROR] Failed to load resource: the server responded with a status of 401 () @ chrome-error://chromewebdata/:0
.playwright-mcp/page-2026-09-04T05-35-58-525Z.yml
+4
View File
@@ -0,0 +1,4 @@
- generic [active] [ref=f40e1]:
- heading "502 Bad Gateway" [level=1] [ref=f40e3]
- separator [ref=f40e4]
- generic [ref=f40e5]: nginx/1.30.4
.playwright-mcp/page-2026-09-04T05-36-21-375Z.yml
+4
View File
@@ -0,0 +1,4 @@
- generic [active] [ref=f41e1]:
- heading "502 Bad Gateway" [level=1] [ref=f41e3]
- separator [ref=f41e4]
- generic [ref=f41e5]: nginx/1.30.4
.playwright-mcp/page-2026-09-04T05-38-20-832Z.yml
+4
View File
@@ -0,0 +1,4 @@
- generic [active] [ref=f42e1]:
- heading "502 Bad Gateway" [level=1] [ref=f42e3]
- separator [ref=f42e4]
- generic [ref=f42e5]: nginx/1.30.4
.playwright-mcp/page-2026-09-04T05-41-10-348Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f43e1]: "{ \"headers\" : { \"host\" : [ \"app2.hyeonworks.com\" ], \"user-agent\" : [ \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36\" ], \"accept\" : [ \"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7\" ], \"accept-encoding\" : [ \"gzip, deflate, br, zstd\" ], \"accept-language\" : [ \"en-US,en;q=0.9\" ], \"cookie\" : [ \"grafana_session=60c3e7ae41ffc00665f4a2c377def399; grafana_session_expiry=1788497124; _oauth2_proxy=djIuWDI5aGRYUm9NbDl3Y205NGVTMWlNall4TVRGbVltUXhabVJoWWpOaFpUSXhPREpsTWpnM01EQXhZakF5WVEuTk81VE82RHRod2NWZWstaHpPZVg1Zw==|1788500470|iPSRUlwHDB0XgC6sUdU4dq1EHq9WQDPYrDoezajKVUA=\" ], \"priority\" : [ \"u=0, i\" ], \"sec-ch-ua\" : [ \"\\\"Chromium\\\";v=\\\"152\\\", \\\"Not?A_Brand\\\";v=\\\"24\\\", \\\"Google Chrome\\\";v=\\\"152\\\"\" ], \"sec-ch-ua-mobile\" : [ \"?0\" ], \"sec-ch-ua-platform\" : [ \"\\\"Linux\\\"\" ], \"sec-fetch-dest\" : [ \"document\" ], \"sec-fetch-mode\" : [ \"navigate\" ], \"sec-fetch-site\" : [ \"none\" ], \"sec-fetch-user\" : [ \"?1\" ], \"upgrade-insecure-requests\" : [ \"1\" ], \"x-forwarded-email\" : [ \"labuser@example.com\" ], \"x-forwarded-host\" : [ \"app2.hyeonworks.com\" ], \"x-forwarded-port\" : [ \"443\" ], \"x-forwarded-preferred-username\" : [ \"labuser\" ], \"x-forwarded-proto\" : [ \"https\" ], \"x-forwarded-server\" : [ \"traefik-5d6fcf895-wpfhr\" ], \"x-forwarded-user\" : [ \"27df5ea9-8703-4ec5-badd-d972c583e1ff\" ], \"x-real-ip\" : [ \"100.123.124.30\" ] }, \"remoteAddr\" : \"100.123.124.30\", \"localAddr\" : \"10.42.0.53\", \"scheme\" : \"https\", \"secure\" : true, \"serverName\" : \"app2.hyeonworks.com\", \"serverPort\" : 443, \"requestUrl\" : \"https://app2.hyeonworks.com/api/echo\" }"
.playwright-mcp/page-2026-09-04T05-42-18-721Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f44e1]: "{ \"headers\" : { \"host\" : [ \"app2.hyeonworks.com\" ], \"user-agent\" : [ \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36\" ], \"accept\" : [ \"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7\" ], \"accept-encoding\" : [ \"gzip, deflate, br, zstd\" ], \"accept-language\" : [ \"en-US,en;q=0.9\" ], \"cookie\" : [ \"grafana_session=60c3e7ae41ffc00665f4a2c377def399; grafana_session_expiry=1788497124; _oauth2_proxy=djIuWDI5aGRYUm9NbDl3Y205NGVTMDVOemhrWm1GbFptSmtZV1JqWTJJNU5tTTNZbVV4TmpJMVpHSmhOVFl4TmcucmoxSnJPYjJKOW1ZV191aXVWa2FCZw==|1788500538|RoiStOeQcIDldxB3cckyO-OAiMgBjBfw5gOSvUsgTFU=\" ], \"priority\" : [ \"u=0, i\" ], \"sec-ch-ua\" : [ \"\\\"Chromium\\\";v=\\\"152\\\", \\\"Not?A_Brand\\\";v=\\\"24\\\", \\\"Google Chrome\\\";v=\\\"152\\\"\" ], \"sec-ch-ua-mobile\" : [ \"?0\" ], \"sec-ch-ua-platform\" : [ \"\\\"Linux\\\"\" ], \"sec-fetch-dest\" : [ \"document\" ], \"sec-fetch-mode\" : [ \"navigate\" ], \"sec-fetch-site\" : [ \"none\" ], \"sec-fetch-user\" : [ \"?1\" ], \"upgrade-insecure-requests\" : [ \"1\" ], \"x-forwarded-email\" : [ \"labuser@example.com\" ], \"x-forwarded-host\" : [ \"app2.hyeonworks.com\" ], \"x-forwarded-port\" : [ \"443\" ], \"x-forwarded-preferred-username\" : [ \"labuser\" ], \"x-forwarded-proto\" : [ \"https\" ], \"x-forwarded-server\" : [ \"traefik-5d6fcf895-wpfhr\" ], \"x-forwarded-user\" : [ \"27df5ea9-8703-4ec5-badd-d972c583e1ff\" ], \"x-real-ip\" : [ \"100.123.124.30\" ] }, \"remoteAddr\" : \"100.123.124.30\", \"localAddr\" : \"10.42.1.132\", \"scheme\" : \"https\", \"secure\" : true, \"serverName\" : \"app2.hyeonworks.com\", \"serverPort\" : 443, \"requestUrl\" : \"https://app2.hyeonworks.com/api/echo\" }"
.playwright-mcp/page-2026-09-04T05-46-05-665Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f45e3]:
- banner [ref=f45e4]:
- generic [ref=f45e5]: keycloak-patterns
- main [ref=f45e6]:
- heading "Sign in to your account" [level=1] [ref=f45e8]
- generic [ref=f45e12]:
- generic [ref=f45e13]:
- generic [ref=f45e14]: Username or email
- textbox "Username or email" [active] [ref=f45e17]
- generic [ref=f45e18]:
- generic [ref=f45e19]: Password
- generic [ref=f45e21]:
- textbox "Password" [ref=f45e24]
- button "Show password" [ref=f45e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f45e27]:
- button "Sign In" [ref=f45e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T05-46-18-702Z.yml
+7
View File
@@ -0,0 +1,7 @@
- main [ref=f46e2]:
- heading "AP3 · Backend-for-Frontend" [level=1] [ref=f46e3]
- paragraph [ref=f46e4]: 브라우저에는 OAuth token이 전혀 전달되지 않습니다. HttpOnly session cookie로 BFF만 호출하고, BFF가 서버 보관 access token을 Resource Server 요청에 붙입니다.
- button "Keycloak 로그인" [ref=f46e5] [cursor=pointer]
- button "token 경계 확인" [ref=f46e6] [cursor=pointer]
- button "BFF 경유 API 호출" [ref=f46e7] [cursor=pointer]
- button "CSRF token으로 상태 변경" [ref=f46e8] [cursor=pointer]
.playwright-mcp/page-2026-09-04T05-46-37-934Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f47e1]: "{ \"headers\" : { \"host\" : [ \"app2.hyeonworks.com\" ], \"user-agent\" : [ \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36\" ], \"accept\" : [ \"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7\" ], \"accept-encoding\" : [ \"gzip, deflate, br, zstd\" ], \"accept-language\" : [ \"en-US,en;q=0.9\" ], \"cookie\" : [ \"grafana_session=60c3e7ae41ffc00665f4a2c377def399; grafana_session_expiry=1788497124; _oauth2_proxy=djIuWDI5aGRYUm9NbDl3Y205NGVTMDJZakF5T0dFM01HWTJPV000WmpCa1lUazVOalpsWWpNMk9UY3laR1ptTWcuWmpUamNTbGxVSHV1RmJjQ2ZRd2lsUQ==|1788500797|17Hbo3RDtzOldnLZx2xOt3e34lHo_v0yqRNdqdKUx-g=\" ], \"priority\" : [ \"u=0, i\" ], \"sec-ch-ua\" : [ \"\\\"Chromium\\\";v=\\\"152\\\", \\\"Not?A_Brand\\\";v=\\\"24\\\", \\\"Google Chrome\\\";v=\\\"152\\\"\" ], \"sec-ch-ua-mobile\" : [ \"?0\" ], \"sec-ch-ua-platform\" : [ \"\\\"Linux\\\"\" ], \"sec-fetch-dest\" : [ \"document\" ], \"sec-fetch-mode\" : [ \"navigate\" ], \"sec-fetch-site\" : [ \"none\" ], \"sec-fetch-user\" : [ \"?1\" ], \"upgrade-insecure-requests\" : [ \"1\" ], \"x-forwarded-email\" : [ \"labuser@example.com\" ], \"x-forwarded-host\" : [ \"app2.hyeonworks.com\" ], \"x-forwarded-port\" : [ \"443\" ], \"x-forwarded-preferred-username\" : [ \"labuser\" ], \"x-forwarded-proto\" : [ \"https\" ], \"x-forwarded-server\" : [ \"traefik-5d6fcf895-wpfhr\" ], \"x-forwarded-user\" : [ \"27df5ea9-8703-4ec5-badd-d972c583e1ff\" ], \"x-real-ip\" : [ \"100.123.124.30\" ] }, \"remoteAddr\" : \"100.123.124.30\", \"localAddr\" : \"10.42.0.53\", \"scheme\" : \"https\", \"secure\" : true, \"serverName\" : \"app2.hyeonworks.com\", \"serverPort\" : 443, \"requestUrl\" : \"https://app2.hyeonworks.com/api/echo\" }"
.playwright-mcp/page-2026-09-04T05-48-04-197Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f48e1]: "{\"pattern\":\"AP3-backend-for-frontend\",\"principal\":\"labuser\",\"accessTokenStoredOnServer\":true,\"refreshTokenStoredOnServer\":true,\"browserTokenCount\":0,\"csrfProtectionEnabled\":true}"
.playwright-mcp/page-2026-09-04T05-48-09-366Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f49e1]: "{ \"headers\" : { \"host\" : [ \"app2.hyeonworks.com\" ], \"user-agent\" : [ \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36\" ], \"accept\" : [ \"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7\" ], \"accept-encoding\" : [ \"gzip, deflate, br, zstd\" ], \"accept-language\" : [ \"en-US,en;q=0.9\" ], \"cookie\" : [ \"grafana_session=60c3e7ae41ffc00665f4a2c377def399; grafana_session_expiry=1788497124; _oauth2_proxy=djIuWDI5aGRYUm9NbDl3Y205NGVTMDJZakF5T0dFM01HWTJPV000WmpCa1lUazVOalpsWWpNMk9UY3laR1ptTWcuWmpUamNTbGxVSHV1RmJjQ2ZRd2lsUQ==|1788500797|17Hbo3RDtzOldnLZx2xOt3e34lHo_v0yqRNdqdKUx-g=\" ], \"priority\" : [ \"u=0, i\" ], \"sec-ch-ua\" : [ \"\\\"Chromium\\\";v=\\\"152\\\", \\\"Not?A_Brand\\\";v=\\\"24\\\", \\\"Google Chrome\\\";v=\\\"152\\\"\" ], \"sec-ch-ua-mobile\" : [ \"?0\" ], \"sec-ch-ua-platform\" : [ \"\\\"Linux\\\"\" ], \"sec-fetch-dest\" : [ \"document\" ], \"sec-fetch-mode\" : [ \"navigate\" ], \"sec-fetch-site\" : [ \"none\" ], \"sec-fetch-user\" : [ \"?1\" ], \"upgrade-insecure-requests\" : [ \"1\" ], \"x-forwarded-email\" : [ \"labuser@example.com\" ], \"x-forwarded-host\" : [ \"app2.hyeonworks.com\" ], \"x-forwarded-port\" : [ \"443\" ], \"x-forwarded-preferred-username\" : [ \"labuser\" ], \"x-forwarded-proto\" : [ \"https\" ], \"x-forwarded-server\" : [ \"traefik-5d6fcf895-wpfhr\" ], \"x-forwarded-user\" : [ \"27df5ea9-8703-4ec5-badd-d972c583e1ff\" ], \"x-real-ip\" : [ \"100.123.124.30\" ] }, \"remoteAddr\" : \"100.123.124.30\", \"localAddr\" : \"10.42.0.53\", \"scheme\" : \"https\", \"secure\" : true, \"serverName\" : \"app2.hyeonworks.com\", \"serverPort\" : 443, \"requestUrl\" : \"https://app2.hyeonworks.com/api/echo\" }"
.playwright-mcp/page-2026-09-04T05-51-26-882Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f50e1]: "{\"pattern\":\"AP3-backend-for-frontend\",\"principal\":\"labuser\",\"accessTokenStoredOnServer\":true,\"refreshTokenStoredOnServer\":true,\"browserTokenCount\":0,\"csrfProtectionEnabled\":true}"
.playwright-mcp/page-2026-09-04T05-52-09-485Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f51e3]:
- banner [ref=f51e4]:
- generic [ref=f51e5]: keycloak-patterns
- main [ref=f51e6]:
- heading "Sign in to your account" [level=1] [ref=f51e8]
- generic [ref=f51e12]:
- generic [ref=f51e13]:
- generic [ref=f51e14]: Username or email
- textbox "Username or email" [active] [ref=f51e17]
- generic [ref=f51e18]:
- generic [ref=f51e19]: Password
- generic [ref=f51e21]:
- textbox "Password" [ref=f51e24]
- button "Show password" [ref=f51e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f51e27]:
- button "Sign In" [ref=f51e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T05-52-19-135Z.yml
+7
View File
@@ -0,0 +1,7 @@
- main [ref=f52e2]:
- heading "AP3 · Backend-for-Frontend" [level=1] [ref=f52e3]
- paragraph [ref=f52e4]: 브라우저에는 OAuth token이 전혀 전달되지 않습니다. HttpOnly session cookie로 BFF만 호출하고, BFF가 서버 보관 access token을 Resource Server 요청에 붙입니다.
- button "Keycloak 로그인" [ref=f52e5] [cursor=pointer]
- button "token 경계 확인" [ref=f52e6] [cursor=pointer]
- button "BFF 경유 API 호출" [ref=f52e7] [cursor=pointer]
- button "CSRF token으로 상태 변경" [ref=f52e8] [cursor=pointer]
.playwright-mcp/page-2026-09-04T05-54-31-795Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f53e3]:
- banner [ref=f53e4]:
- generic [ref=f53e5]: keycloak-patterns
- main [ref=f53e6]:
- heading "Sign in to your account" [level=1] [ref=f53e8]
- generic [ref=f53e12]:
- generic [ref=f53e13]:
- generic [ref=f53e14]: Username or email
- textbox "Username or email" [ref=f53e17]
- generic [ref=f53e18]:
- generic [ref=f53e19]: Password
- generic [ref=f53e21]:
- textbox "Password" [ref=f53e24]
- button "Show password" [ref=f53e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f53e27]:
- button "Sign In" [ref=f53e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T05-54-45-185Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f53e3]:
- banner [ref=f53e4]:
- generic [ref=f53e5]: keycloak-patterns
- main [ref=f53e6]:
- heading "Sign in to your account" [level=1] [ref=f53e8]
- generic [ref=f53e12]:
- generic [ref=f53e13]:
- generic [ref=f53e14]: Username or email
- textbox "Username or email" [ref=f53e17]: labuser
- generic [ref=f53e18]:
- generic [ref=f53e19]: Password
- generic [ref=f53e21]:
- textbox "Password" [active] [ref=f53e24]: labpass
- button "Show password" [ref=f53e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f53e27]:
- button "Sign In" [ref=f53e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T05-54-50-691Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f53e3]:
- banner [ref=f53e4]:
- generic [ref=f53e5]: keycloak-patterns
- main [ref=f53e6]:
- heading "Sign in to your account" [level=1] [ref=f53e8]
- generic [ref=f53e12]:
- generic [ref=f53e13]:
- generic [ref=f53e14]: Username or email
- textbox "Username or email" [ref=f53e17]: labuser
- generic [ref=f53e18]:
- generic [ref=f53e19]: Password
- generic [ref=f53e21]:
- textbox "Password" [active] [ref=f53e24]: labpass
- button "Show password" [ref=f53e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f53e27]:
- button "Sign In" [ref=f53e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T05-55-00-942Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f53e3]:
- banner [ref=f53e4]:
- generic [ref=f53e5]: keycloak-patterns
- main [ref=f53e6]:
- heading "Sign in to your account" [level=1] [ref=f53e8]
- generic [ref=f53e12]:
- generic [ref=f53e13]:
- generic [ref=f53e14]: Username or email
- textbox "Username or email" [ref=f53e17]: labuser
- generic [ref=f53e18]:
- generic [ref=f53e19]: Password
- generic [ref=f53e21]:
- textbox "Password" [active] [ref=f53e24]: labpass
- button "Show password" [ref=f53e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f53e27]:
- button "Sign In" [ref=f53e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T05-55-08-213Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f54e3]:
- banner [ref=f54e4]:
- generic [ref=f54e5]: keycloak-patterns
- main [ref=f54e6]:
- heading "Sign in to your account" [level=1] [ref=f54e8]
- generic [ref=f54e12]:
- generic [ref=f54e13]:
- generic [ref=f54e14]: Username or email
- textbox "Username or email" [ref=f54e17]
- generic [ref=f54e18]:
- generic [ref=f54e19]: Password
- generic [ref=f54e21]:
- textbox "Password" [ref=f54e24]
- button "Show password" [ref=f54e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f54e27]:
- button "Sign In" [ref=f54e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T05-55-18-791Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f54e3]:
- banner [ref=f54e4]:
- generic [ref=f54e5]: keycloak-patterns
- main [ref=f54e6]:
- heading "Sign in to your account" [level=1] [ref=f54e8]
- generic [ref=f54e12]:
- generic [ref=f54e13]:
- generic [ref=f54e14]: Username or email
- textbox "Username or email" [ref=f54e17]: labuser
- generic [ref=f54e18]:
- generic [ref=f54e19]: Password
- generic [ref=f54e21]:
- textbox "Password" [active] [ref=f54e24]: labpass
- button "Show password" [ref=f54e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f54e27]:
- button "Sign In" [ref=f54e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T06-25-15-925Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f55e3]:
- banner [ref=f55e4]:
- generic [ref=f55e5]: keycloak-patterns
- main [ref=f55e6]:
- heading "Sign in to your account" [level=1] [ref=f55e8]
- generic [ref=f55e12]:
- generic [ref=f55e13]:
- generic [ref=f55e14]: Username or email
- textbox "Username or email" [ref=f55e17]
- generic [ref=f55e18]:
- generic [ref=f55e19]: Password
- generic [ref=f55e21]:
- textbox "Password" [ref=f55e24]
- button "Show password" [ref=f55e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f55e27]:
- button "Sign In" [ref=f55e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T06-25-26-485Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f55e3]:
- banner [ref=f55e4]:
- generic [ref=f55e5]: keycloak-patterns
- main [ref=f55e6]:
- heading "Sign in to your account" [level=1] [ref=f55e8]
- generic [ref=f55e12]:
- generic [ref=f55e13]:
- generic [ref=f55e14]: Username or email
- textbox "Username or email" [ref=f55e17]: labuser
- generic [ref=f55e18]:
- generic [ref=f55e19]: Password
- generic [ref=f55e21]:
- textbox "Password" [ref=f55e24]: labpass
- button "Show password" [ref=f55e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f55e27]:
- button "Sign In" [ref=f55e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T06-25-35-208Z.yml
+17
View File
@@ -0,0 +1,17 @@
- generic [ref=f56e1]:
- generic [ref=f56e3]:
- generic [ref=f56e4]: "403"
- heading "Forbidden" [level=1] [ref=f56e6]
- generic [ref=f56e8]:
- paragraph [ref=f56e9]: More Info
- generic [ref=f56e10] [cursor=pointer]:
- separator [ref=f56e12]
- generic [ref=f56e13]:
- button "Go back" [ref=f56e16] [cursor=pointer]
- button "Sign in" [ref=f56e19] [cursor=pointer]
- contentinfo [ref=f56e20]:
- paragraph [ref=f56e22]:
- text: Secured with
- link "OAuth2 Proxy" [ref=f56e23] [cursor=pointer]:
- /url: https://github.com/oauth2-proxy/oauth2-proxy#oauth2_proxy
- text: version v7.7.1
.playwright-mcp/page-2026-09-04T06-25-42-934Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [ref=f57e1]: "{ \"headers\" : { \"host\" : [ \"app2.hyeonworks.com\" ], \"user-agent\" : [ \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36\" ], \"accept\" : [ \"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7\" ], \"accept-encoding\" : [ \"gzip, deflate, br, zstd\" ], \"accept-language\" : [ \"en-US,en;q=0.9\" ], \"cookie\" : [ \"grafana_session=60c3e7ae41ffc00665f4a2c377def399; grafana_session_expiry=1788497124; _oauth2_proxy=djIuWDI5aGRYUm9NbDl3Y205NGVTMDNZMkZrTUdZM01tRmlZekkwWldFell6a3lOREJpTW1KaE5UZGpOVEJoWWcuZVVmckp5VHRqY1VsXzdiV1hiX3hwdw==|1788503135|yDSo7VdgRSlvoVfj9raHPClKTlQiWDg7FauLfoUayw4=\" ], \"priority\" : [ \"u=0, i\" ], \"sec-ch-ua\" : [ \"\\\"Chromium\\\";v=\\\"152\\\", \\\"Not?A_Brand\\\";v=\\\"24\\\", \\\"Google Chrome\\\";v=\\\"152\\\"\" ], \"sec-ch-ua-mobile\" : [ \"?0\" ], \"sec-ch-ua-platform\" : [ \"\\\"Linux\\\"\" ], \"sec-fetch-dest\" : [ \"document\" ], \"sec-fetch-mode\" : [ \"navigate\" ], \"sec-fetch-site\" : [ \"none\" ], \"sec-fetch-user\" : [ \"?1\" ], \"upgrade-insecure-requests\" : [ \"1\" ], \"x-forwarded-email\" : [ \"labuser@example.com\" ], \"x-forwarded-host\" : [ \"app2.hyeonworks.com\" ], \"x-forwarded-port\" : [ \"443\" ], \"x-forwarded-preferred-username\" : [ \"labuser\" ], \"x-forwarded-proto\" : [ \"https\" ], \"x-forwarded-server\" : [ \"traefik-5d6fcf895-wpfhr\" ], \"x-forwarded-user\" : [ \"27df5ea9-8703-4ec5-badd-d972c583e1ff\" ], \"x-real-ip\" : [ \"100.123.124.30\" ] }, \"remoteAddr\" : \"100.123.124.30\", \"localAddr\" : \"10.42.0.53\", \"scheme\" : \"https\", \"secure\" : true, \"serverName\" : \"app2.hyeonworks.com\", \"serverPort\" : 443, \"requestUrl\" : \"https://app2.hyeonworks.com/api/echo\" }"
.playwright-mcp/page-2026-09-04T06-26-42-277Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [ref=f58e1]: "{ \"headers\" : { \"host\" : [ \"app2.hyeonworks.com\" ], \"user-agent\" : [ \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36\" ], \"accept\" : [ \"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7\" ], \"accept-encoding\" : [ \"gzip, deflate, br, zstd\" ], \"accept-language\" : [ \"en-US,en;q=0.9\" ], \"cookie\" : [ \"grafana_session=60c3e7ae41ffc00665f4a2c377def399; grafana_session_expiry=1788497124; _oauth2_proxy=djIuWDI5aGRYUm9NbDl3Y205NGVTMHpabUV5TVdKaVpEa3lOemRrTlRFMU9USTBaV00wWWpGaE16bGhNak0zT1EuN2tTa3dnWUdISDkwMGFSSTVOSUFFUQ==|1788503202|snWKU5IRfRLoD9-bXEodGjEgfHeAw8PQaoHnecpFH90=\" ], \"priority\" : [ \"u=0, i\" ], \"sec-ch-ua\" : [ \"\\\"Chromium\\\";v=\\\"152\\\", \\\"Not?A_Brand\\\";v=\\\"24\\\", \\\"Google Chrome\\\";v=\\\"152\\\"\" ], \"sec-ch-ua-mobile\" : [ \"?0\" ], \"sec-ch-ua-platform\" : [ \"\\\"Linux\\\"\" ], \"sec-fetch-dest\" : [ \"document\" ], \"sec-fetch-mode\" : [ \"navigate\" ], \"sec-fetch-site\" : [ \"none\" ], \"sec-fetch-user\" : [ \"?1\" ], \"upgrade-insecure-requests\" : [ \"1\" ], \"x-forwarded-email\" : [ \"changed-labuser@example.com\" ], \"x-forwarded-host\" : [ \"app2.hyeonworks.com\" ], \"x-forwarded-port\" : [ \"443\" ], \"x-forwarded-preferred-username\" : [ \"labuser\" ], \"x-forwarded-proto\" : [ \"https\" ], \"x-forwarded-server\" : [ \"traefik-5d6fcf895-wpfhr\" ], \"x-forwarded-user\" : [ \"27df5ea9-8703-4ec5-badd-d972c583e1ff\" ], \"x-real-ip\" : [ \"100.123.124.30\" ] }, \"remoteAddr\" : \"100.123.124.30\", \"localAddr\" : \"10.42.1.132\", \"scheme\" : \"https\", \"secure\" : true, \"serverName\" : \"app2.hyeonworks.com\", \"serverPort\" : 443, \"requestUrl\" : \"https://app2.hyeonworks.com/api/echo\" }"
.playwright-mcp/page-2026-09-04T07-14-59-023Z.yml
+175
View File
@@ -0,0 +1,175 @@
- generic [ref=f59e1]:
- generic [ref=f59e4]:
- link "Skip to main content" [ref=f59e5] [cursor=pointer]:
- /url: "#pageContent"
- banner [ref=f59e7]:
- generic [ref=f59e8]:
- link [ref=f59e10] [cursor=pointer]:
- /url: /
- img "Grafana" [ref=f59e11]
- generic [ref=f59e14]:
- button "Search or jump to..." [ref=f59e18] [cursor=pointer]
- generic [ref=f59e19]: ctrl+k
- generic [ref=f59e23]:
- button "New" [ref=f59e24] [cursor=pointer]
- button "Help" [ref=f59e30] [cursor=pointer]
- button "News" [ref=f59e33] [cursor=pointer]
- button "Profile" [ref=f59e36] [cursor=pointer]:
- img "User avatar" [ref=f59e37]
- generic [ref=f59e38]:
- button "Open menu" [ref=f59e40] [cursor=pointer]
- navigation "Breadcrumbs" [ref=f59e43]:
- list [ref=f59e44]:
- listitem [ref=f59e45]:
- link "Home" [ref=f59e46] [cursor=pointer]:
- /url: /
- listitem [ref=f59e50]:
- link "Explore" [ref=f59e51] [cursor=pointer]:
- /url: /explore
- listitem [ref=f59e55]:
- generic "Prometheus" [ref=f59e56]
- generic [ref=f59e57]:
- generic [ref=f59e60]:
- button "Copy shortened URL" [ref=f59e61] [cursor=pointer]
- button "Open copy link options" [ref=f59e64] [cursor=pointer]
- button "Toggle top search bar" [ref=f59e68] [cursor=pointer]
- main [ref=f59e74]:
- generic [ref=f59e76]:
- heading "Explore" [level=1] [ref=f59e77]
- generic [ref=f59e82]:
- navigation "Explore toolbar" [ref=f59e84]:
- navigation "Search links" [ref=f59e86]:
- generic [ref=f59e87]:
- button "Content outline" [expanded] [ref=f59e89] [cursor=pointer]:
- generic [ref=f59e92]: Outline
- generic [ref=f59e97] [cursor=pointer]:
- img "Prometheus logo" [ref=f59e99]
- textbox "Select a data source" [ref=f59e100]:
- /placeholder: Prometheus
- generic [ref=f59e104]:
- button "Split the pane" [ref=f59e106] [cursor=pointer]:
- generic [ref=f59e109]: Split
- button "Add" [ref=f59e111] [cursor=pointer]
- generic [ref=f59e116]:
- 'button "Time range selected: Last 30 minutes" [ref=f59e117] [cursor=pointer]'
- button "Zoom out time range" [ref=f59e122] [cursor=pointer]
- generic [ref=f59e126]:
- button "Cancel" [ref=f59e127] [cursor=pointer]
- button "Auto refresh turned off. Choose refresh time interval" [ref=f59e129] [cursor=pointer]
- generic [ref=f59e133]:
- generic [ref=f59e137]:
- button "Collapse outline" [expanded] [ref=f59e139] [cursor=pointer]:
- img "arrow-from-right" [ref=f59e140]
- generic [ref=f59e142]:
- button "Content outline item collapse button" [ref=f59e143] [cursor=pointer]:
- img "angle-right" [ref=f59e144]
- button "Queries" [ref=f59e146] [cursor=pointer]:
- img "arrow" [ref=f59e147]
- generic [ref=f59e154]:
- generic [ref=f59e156]:
- generic [ref=f59e157]:
- generic "Query editor row" [ref=f59e159]:
- generic [ref=f59e160]:
- generic [ref=f59e162]:
- generic [ref=f59e163]:
- button "Collapse query row" [expanded] [ref=f59e164] [cursor=pointer]
- generic [ref=f59e167]:
- button "Query editor row title A" [ref=f59e168] [cursor=pointer]:
- generic [ref=f59e169]: A
- emphasis [ref=f59e170]: (Prometheus)
- generic [ref=f59e171]:
- button "Show data source help" [ref=f59e173] [cursor=pointer]
- button "Duplicate query" [ref=f59e177] [cursor=pointer]
- button "Hide response" [ref=f59e181] [cursor=pointer]
- button "Remove query" [ref=f59e185] [cursor=pointer]
- button "Drag and drop to reorder" [ref=f59e188]:
- img "Drag and drop to reorder" [ref=f59e189]
- generic [ref=f59e192]:
- generic [ref=f59e193]:
- button "Kick start your query" [ref=f59e194] [cursor=pointer]
- generic [ref=f59e197]:
- generic [ref=f59e198] [cursor=pointer]: Explain
- generic [ref=f59e199]:
- checkbox "Explain Toggle switch" [ref=f59e200]
- generic "Toggle switch" [ref=f59e201] [cursor=pointer]
- radiogroup [ref=f59e206]:
- generic [ref=f59e207]:
- radio "Builder" [ref=f59e208] [cursor=pointer]
- generic [ref=f59e209] [cursor=pointer]: Builder
- generic [ref=f59e210]:
- radio "Code" [checked] [ref=f59e211] [cursor=pointer]
- generic [ref=f59e212] [cursor=pointer]: Code
- generic [ref=f59e214]:
- generic [ref=f59e216]:
- button "Loading metrics..." [disabled] [ref=f59e217] [cursor=pointer]
- code [ref=f59e224]:
- generic [ref=f59e225]:
- generic [ref=f59e230]: vendor_cluster_size
- textbox "Editor content;Press Alt+F1 for Accessibility Options." [ref=f59e235]: vendor_cluster_size
- 'button "Options Legend: cluster_size {{pod}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f59e241] [cursor=pointer]':
- generic [ref=f59e245]:
- heading "Options" [level=6] [ref=f59e246]
- generic [ref=f59e247]:
- generic [ref=f59e248]: "Legend: cluster_size {{pod}}"
- generic [ref=f59e249]: "Format: Time series"
- generic [ref=f59e250]: "Step: auto"
- generic [ref=f59e251]: "Type: Range"
- generic [ref=f59e252]: "Exemplars: false"
- generic "Query editor row" [ref=f59e254]:
- generic [ref=f59e255]:
- generic [ref=f59e257]:
- generic [ref=f59e258]:
- button "Collapse query row" [expanded] [ref=f59e259] [cursor=pointer]
- generic [ref=f59e262]:
- button "Query editor row title B" [ref=f59e263] [cursor=pointer]:
- generic [ref=f59e264]: B
- emphasis [ref=f59e265]: (Prometheus)
- generic [ref=f59e266]:
- button "Show data source help" [ref=f59e268] [cursor=pointer]
- button "Duplicate query" [ref=f59e272] [cursor=pointer]
- button "Hide response" [ref=f59e276] [cursor=pointer]
- button "Remove query" [ref=f59e280] [cursor=pointer]
- button "Drag and drop to reorder" [ref=f59e283]:
- img "Drag and drop to reorder" [ref=f59e284]
- generic [ref=f59e287]:
- generic [ref=f59e288]:
- button "Kick start your query" [ref=f59e289] [cursor=pointer]
- generic [ref=f59e292]:
- generic [ref=f59e293] [cursor=pointer]: Explain
- generic [ref=f59e294]:
- checkbox "Explain Toggle switch" [ref=f59e295]
- generic "Toggle switch" [ref=f59e296] [cursor=pointer]
- radiogroup [ref=f59e301]:
- generic [ref=f59e302]:
- radio "Builder" [ref=f59e303] [cursor=pointer]
- generic [ref=f59e304] [cursor=pointer]: Builder
- generic [ref=f59e305]:
- radio "Code" [checked] [ref=f59e306] [cursor=pointer]
- generic [ref=f59e307] [cursor=pointer]: Code
- generic [ref=f59e309]:
- generic [ref=f59e311]:
- button "Loading metrics..." [disabled] [ref=f59e312] [cursor=pointer]
- code [ref=f59e319]:
- generic [ref=f59e320]:
- generic [ref=f59e325]: "up{job=\"keycloak\"}"
- textbox "Editor content;Press Alt+F1 for Accessibility Options." [ref=f59e330]: "up{job=\"keycloak\"}"
- 'button "Options Legend: up {{pod}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f59e336] [cursor=pointer]':
- generic [ref=f59e340]:
- heading "Options" [level=6] [ref=f59e341]
- generic [ref=f59e342]:
- generic [ref=f59e343]: "Legend: up {{pod}}"
- generic [ref=f59e344]: "Format: Time series"
- generic [ref=f59e345]: "Step: auto"
- generic [ref=f59e346]: "Type: Range"
- generic [ref=f59e347]: "Exemplars: false"
- generic [ref=f59e348]:
- button "Add query" [ref=f59e349] [cursor=pointer]
- button "Query history" [ref=f59e353] [cursor=pointer]
- button "Query inspector" [ref=f59e357] [cursor=pointer]
- generic:
- main
- generic [ref=f59e364]:
- alert
- alert
- complementary
- complementary
.playwright-mcp/page-2026-09-04T07-15-35-378Z.yml
+56
View File
@@ -0,0 +1,56 @@
- generic [ref=f62e4]:
- link "Skip to main content" [ref=f62e5] [cursor=pointer]:
- /url: "#pageContent"
- banner [ref=f62e7]:
- generic [ref=f62e8]:
- link [ref=f62e10] [cursor=pointer]:
- /url: /
- img "Grafana" [ref=f62e11]
- generic [ref=f62e14]:
- button "Search or jump to..." [ref=f62e18] [cursor=pointer]
- generic [ref=f62e19]: ctrl+k
- generic [ref=f62e23]:
- button "New" [ref=f62e24] [cursor=pointer]
- button "Help" [ref=f62e30] [cursor=pointer]
- button "News" [ref=f62e33] [cursor=pointer]
- button "Profile" [ref=f62e36] [cursor=pointer]:
- img "User avatar" [ref=f62e37]
- generic [ref=f62e38]:
- button "Open menu" [ref=f62e40] [cursor=pointer]
- navigation "Breadcrumbs" [ref=f62e43]:
- list [ref=f62e44]:
- listitem [ref=f62e45]:
- link "Home" [ref=f62e46] [cursor=pointer]:
- /url: /
- listitem [ref=f62e50]:
- link "Explore" [ref=f62e51] [cursor=pointer]:
- /url: /explore
- listitem [ref=f62e55]:
- generic "Prometheus" [ref=f62e56]
- generic [ref=f62e57]:
- button "Show more items" [ref=f62e60] [cursor=pointer]
- button "Toggle top search bar" [ref=f62e64] [cursor=pointer]
- main [ref=f62e70]:
- generic [ref=f62e72]:
- heading "Explore" [level=1] [ref=f62e73]
- generic [ref=f62e78]:
- navigation "Explore toolbar" [ref=f62e80]:
- navigation "Search links" [ref=f62e82]:
- generic [ref=f62e83]:
- button "Content outline" [expanded] [ref=f62e85] [cursor=pointer]:
- generic [ref=f62e88]: Outline
- generic [ref=f62e93] [cursor=pointer]:
- img "Prometheus logo" [ref=f62e95]
- textbox "Select a data source" [ref=f62e96]:
- /placeholder: Prometheus
- button "Show more items" [ref=f62e102] [cursor=pointer]
- generic [ref=f62e106]:
- button "Collapse outline" [expanded] [ref=f62e112] [cursor=pointer]:
- img "arrow-from-right" [ref=f62e113]
- generic [ref=f62e119]:
- generic [ref=f62e122]:
- button "Add query" [ref=f62e123] [cursor=pointer]
- button "Query history" [ref=f62e127] [cursor=pointer]
- button "Query inspector" [ref=f62e131] [cursor=pointer]
- generic:
- main
.playwright-mcp/page-2026-09-04T07-16-36-528Z.yml
+29
View File
@@ -0,0 +1,29 @@
- generic [ref=f65e4]:
- link "Skip to main content" [ref=f65e5] [cursor=pointer]:
- /url: "#pageContent"
- banner [ref=f65e7]:
- generic [ref=f65e8]:
- link [ref=f65e10] [cursor=pointer]:
- /url: /
- img "Grafana" [ref=f65e11]
- generic [ref=f65e14]:
- button "Search or jump to..." [ref=f65e18] [cursor=pointer]
- generic [ref=f65e19]: ctrl+k
- generic [ref=f65e23]:
- button "New" [ref=f65e24] [cursor=pointer]
- button "Help" [ref=f65e30] [cursor=pointer]
- button "News" [ref=f65e33] [cursor=pointer]
- button "Profile" [ref=f65e36] [cursor=pointer]:
- img "User avatar" [ref=f65e37]
- generic [ref=f65e38]:
- button "Open menu" [ref=f65e40] [cursor=pointer]
- navigation "Breadcrumbs" [ref=f65e43]:
- list [ref=f65e44]:
- listitem [ref=f65e45]:
- link "Home" [ref=f65e46] [cursor=pointer]:
- /url: /
- listitem [ref=f65e50]:
- generic "Explore" [ref=f65e51]
- button "Toggle top search bar" [ref=f65e53] [cursor=pointer]
- main [ref=f65e59]:
- heading "Explore" [level=1] [ref=f65e62]
.playwright-mcp/page-2026-09-04T07-51-03-233Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=f68e3]:
- banner [ref=f68e4]:
- generic [ref=f68e5]: keycloak-patterns
- main [ref=f68e6]:
- heading "Sign in to your account" [level=1] [ref=f68e8]
- generic [ref=f68e12]:
- generic [ref=f68e13]:
- generic [ref=f68e14]: Username or email
- textbox "Username or email" [ref=f68e17]
- generic [ref=f68e18]:
- generic [ref=f68e19]: Password
- generic [ref=f68e21]:
- textbox "Password" [ref=f68e24]
- button "Show password" [ref=f68e26] [cursor=pointer]:
- generic [aria-hidden] [ref=f68e27]:
- button "Sign In" [ref=f68e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T07-51-15-548Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f69e1]: "{ \"headers\" : { \"host\" : [ \"app2.hyeonworks.com\" ], \"user-agent\" : [ \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36\" ], \"accept\" : [ \"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7\" ], \"accept-encoding\" : [ \"gzip, deflate, br, zstd\" ], \"accept-language\" : [ \"en-US,en;q=0.9\" ], \"cache-control\" : [ \"max-age=0\" ], \"cookie\" : [ \"grafana_session=bfb5f2c2406b3f163ea01063ac1b8b83; grafana_session_expiry=1788508427; _oauth2_proxy=djIuWDI5aGRYUm9NbDl3Y205NGVTMHhNamhoWXprNFpHSmxORGRoT0RObU16UmlNemMxTTJJd016ZGlZakpqTWcuQVVUYldxYkFlejYxNXozUUZBcGh1dw==|1788508274|biz4o9E4vZXX07LnF32LU7tIEMrvfwxwCIBsZkxpwbY=\" ], \"priority\" : [ \"u=0, i\" ], \"sec-ch-ua\" : [ \"\\\"Chromium\\\";v=\\\"152\\\", \\\"Not?A_Brand\\\";v=\\\"24\\\", \\\"Google Chrome\\\";v=\\\"152\\\"\" ], \"sec-ch-ua-mobile\" : [ \"?0\" ], \"sec-ch-ua-platform\" : [ \"\\\"Linux\\\"\" ], \"sec-fetch-dest\" : [ \"document\" ], \"sec-fetch-mode\" : [ \"navigate\" ], \"sec-fetch-site\" : [ \"same-site\" ], \"sec-fetch-user\" : [ \"?1\" ], \"upgrade-insecure-requests\" : [ \"1\" ], \"x-forwarded-email\" : [ \"labuser@example.com\" ], \"x-forwarded-host\" : [ \"app2.hyeonworks.com\" ], \"x-forwarded-port\" : [ \"443\" ], \"x-forwarded-preferred-username\" : [ \"labuser\" ], \"x-forwarded-proto\" : [ \"https\" ], \"x-forwarded-server\" : [ \"traefik-5d6fcf895-wpfhr\" ], \"x-forwarded-user\" : [ \"27df5ea9-8703-4ec5-badd-d972c583e1ff\" ], \"x-real-ip\" : [ \"100.123.124.30\" ] }, \"remoteAddr\" : \"100.123.124.30\", \"localAddr\" : \"10.42.1.132\", \"scheme\" : \"https\", \"secure\" : true, \"serverName\" : \"app2.hyeonworks.com\", \"serverPort\" : 443, \"requestUrl\" : \"https://app2.hyeonworks.com/api/echo\" }"
.playwright-mcp/page-2026-09-04T07-52-54-362Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [ref=f70e1]: "{ \"headers\" : { \"host\" : [ \"app2.hyeonworks.com\" ], \"user-agent\" : [ \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36\" ], \"accept\" : [ \"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7\" ], \"accept-encoding\" : [ \"gzip, deflate, br, zstd\" ], \"accept-language\" : [ \"en-US,en;q=0.9\" ], \"cookie\" : [ \"grafana_session=bfb5f2c2406b3f163ea01063ac1b8b83; grafana_session_expiry=1788508427; _oauth2_proxy=djIuWDI5aGRYUm9NbDl3Y205NGVTMWhOakZqTlRjM1ptSTVORFF5TVdWbU1qQmpaak5pWXpka1pXRTFPR1F6WWcuOXF3TmRyQzJIOFlrSDh6T1pTMl91QQ==|1788508374|Ehcd4hjNk0noEy2MSPI-C5jg84JZmilWcKhuj8OH3uw=\" ], \"priority\" : [ \"u=0, i\" ], \"sec-ch-ua\" : [ \"\\\"Chromium\\\";v=\\\"152\\\", \\\"Not?A_Brand\\\";v=\\\"24\\\", \\\"Google Chrome\\\";v=\\\"152\\\"\" ], \"sec-ch-ua-mobile\" : [ \"?0\" ], \"sec-ch-ua-platform\" : [ \"\\\"Linux\\\"\" ], \"sec-fetch-dest\" : [ \"document\" ], \"sec-fetch-mode\" : [ \"navigate\" ], \"sec-fetch-site\" : [ \"none\" ], \"sec-fetch-user\" : [ \"?1\" ], \"upgrade-insecure-requests\" : [ \"1\" ], \"x-forwarded-email\" : [ \"changed-labuser@example.com\" ], \"x-forwarded-host\" : [ \"app2.hyeonworks.com\" ], \"x-forwarded-port\" : [ \"443\" ], \"x-forwarded-preferred-username\" : [ \"labuser\" ], \"x-forwarded-proto\" : [ \"https\" ], \"x-forwarded-server\" : [ \"traefik-5d6fcf895-wpfhr\" ], \"x-forwarded-user\" : [ \"27df5ea9-8703-4ec5-badd-d972c583e1ff\" ], \"x-real-ip\" : [ \"100.123.124.30\" ] }, \"remoteAddr\" : \"100.123.124.30\", \"localAddr\" : \"10.42.0.53\", \"scheme\" : \"https\", \"secure\" : true, \"serverName\" : \"app2.hyeonworks.com\", \"serverPort\" : 443, \"requestUrl\" : \"https://app2.hyeonworks.com/api/echo\" }"
.playwright-mcp/page-2026-09-04T11-28-57-010Z.yml
+16
View File
@@ -0,0 +1,16 @@
- generic [ref=e3]:
- banner [ref=e4]:
- generic [ref=e5]: keycloak-patterns
- main [ref=e6]:
- heading "Sign in to your account" [level=1] [ref=e8]
- generic [ref=e12]:
- generic [ref=e13]:
- generic [ref=e14]: Username or email
- textbox "Username or email" [active] [ref=e17]
- generic [ref=e18]:
- generic [ref=e19]: Password
- generic [ref=e21]:
- textbox "Password" [ref=e24]
- button "Show password" [ref=e26] [cursor=pointer]:
- generic [aria-hidden] [ref=e27]:
- button "Sign In" [ref=e30] [cursor=pointer]
.playwright-mcp/page-2026-09-04T11-29-10-434Z.yml
+6
View File
@@ -0,0 +1,6 @@
- generic [ref=f1e3]:
- generic [ref=f1e6]:
- heading "This page isn’t working" [level=1] [ref=f1e7]
- paragraph [ref=f1e8]: If the problem continues, contact the site owner.
- generic [ref=f1e9]: HTTP ERROR 401
- button "Reload" [ref=f1e12] [cursor=pointer]
.playwright-mcp/page-2026-09-04T11-32-26-282Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f3e1]: "{\"user\":\"27df5ea9-8703-4ec5-badd-d972c583e1ff\",\"email\":\"labuser@example.com\",\"preferredUsername\":\"labuser\"}"
.playwright-mcp/page-2026-09-04T11-32-49-912Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f4e1]: "{\"user\":\"27df5ea9-8703-4ec5-badd-d972c583e1ff\",\"email\":\"labuser@example.com\",\"preferredUsername\":\"labuser\"}"
.playwright-mcp/page-2026-09-04T11-33-54-983Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f5e1]: Unauthorized
deploy/lab/README.md
+18 -6
View File
@@ -18,15 +18,23 @@
브라우저 / SSH (tailnet) 브라우저 / SSH (tailnet)
│ https://{auth,app1,app2}.hyeonworks.com → 100.83.212.4 │ https://{auth,app1,app2}.hyeonworks.com → 100.83.212.4
▼ ▼
lab host ── nginx :443 TLS 종료 · X-Forwarded-* 주입 lab host ── nftables DNAT :80,:443 → 192.168.122.10
│ nginx :80 301 → https │ (물리 호스트가 실험대를 위해 하는 일의 전부)
│ │
│ virbr0 192.168.122.0/24 (libvirt NAT) │ virbr0 192.168.122.0/24 (libvirt NAT)
├──▶ kc-lab-1 .11 k3s server Traefik :80 ├──▶ kc-lab-edge .10 nginx :443 TLS 종료 · X-Forwarded-* 주입
└──▶ kc-lab-2 .12 k3s agent Traefik :80 │ │ nginx :80 301 → https
└──▶ Pod │ │ certbot · 갱신 타이머 · deploy 훅
│ ├──▶ kc-lab-1 .11 Traefik :80 ──▶ Pod
│ └──▶ kc-lab-2 .12 Traefik :80 ──▶ Pod
├──▶ kc-lab-1 .11 k3s server
└──▶ kc-lab-2 .12 k3s agent
``` ```
**L7 홉은 두 겹 그대로다**(엣지 nginx → Traefik). 앞에 늘어난 것은 커널이
하는 L4 전달 한 번뿐이고, 그 대가로 **인증서·nginx 설정·certbot 이 전부
일회용 게스트 안**으로 들어갔다.
`nginx → Traefik` **2홉**이 운영 구조와 같다는 점이 이 배치의 핵심이다. `nginx → Traefik` **2홉**이 운영 구조와 같다는 점이 이 배치의 핵심이다.
L7 프록시가 두 겹인 이유는 역할이 다르기 때문이다 — nginx는 바깥세상과의 L7 프록시가 두 겹인 이유는 역할이 다르기 때문이다 — nginx는 바깥세상과의
접점(TLS·인증서·헤더)을, Traefik은 클러스터 내부의 동적 라우팅을 맡는다. 접점(TLS·인증서·헤더)을, Traefik은 클러스터 내부의 동적 라우팅을 맡는다.
@@ -36,7 +44,11 @@ L7 프록시가 두 겹인 이유는 역할이 다르기 때문이다 — nginx
| 경로 | 역할 | | 경로 | 역할 |
|---|---| |---|---|
| `cloud-init/kc-lab.yaml.example` | 게스트 부트스트랩 템플릿 | | `cloud-init/kc-lab.yaml.example` | 게스트 부트스트랩 템플릿 |
| `host/nginx-keycloak-lab.conf` | lab host의 `sites-available/keycloak-lab` | | `edge/nginx-keycloak-lab.conf` | `kc-lab-edge` 의 `sites-available/keycloak-lab` |
| `edge/reload-nginx.sh` | certbot deploy 훅. 없으면 갱신이 서빙에 반영되지 않는다 (D-4) |
| `edge/lab-edge-dnat.nft` | 물리 호스트의 유일한 트래픽 규칙 |
| `edge/lab-edge-dnat.service` | 위 규칙을 부팅 때 적용 |
| `scripts/migrate-to-edge.sh` | 엣지 계층을 호스트에서 게스트로 옮긴다 |
| `k8s/echo.yaml` | 2홉 헤더 계약 측정용 워크로드 | | `k8s/echo.yaml` | 2홉 헤더 계약 측정용 워크로드 |
| `scripts/rebuild-seed.sh` | cloud-init 시드 ISO 재생성 + 풀 업로드 | | `scripts/rebuild-seed.sh` | cloud-init 시드 ISO 재생성 + 풀 업로드 |
| `scripts/build-and-import.sh` | 이미지 빌드 → 각 노드 containerd 반입 | | `scripts/build-and-import.sh` | 이미지 빌드 → 각 노드 containerd 반입 |
deploy/lab/cloud-init/kc-lab.yaml.example
+12 -1
View File
@@ -17,7 +17,12 @@ users:
shell: /bin/bash shell: /bin/bash
# NOPASSWD is required: the k3s installer and the fault-injection scripts # NOPASSWD is required: the k3s installer and the fault-injection scripts
# run non-interactively and would block on a password prompt. # run non-interactively and would block on a password prompt.
sudo: ['ALL=(ALL) NOPASSWD:ALL'] #
# A string, not a list. The list form still boots, but `cloud-init schema -c`
# (22.4.2 on the guests) rejects it and prints the whole users.0 block with
# "is not valid under any of the given schemas" — naming no key. That makes
# the guide's own validation step look broken when it is not.
sudo: "ALL=(ALL) NOPASSWD:ALL"
# Console-only escape hatch. Without it, a cloud-init failure leaves a guest # Console-only escape hatch. Without it, a cloud-init failure leaves a guest
# that cannot be logged into at all, so its own failure log is unreadable. # that cannot be logged into at all, so its own failure log is unreadable.
# ssh_pwauth stays false, so this never widens SSH exposure. # ssh_pwauth stays false, so this never widens SSH exposure.
@@ -35,3 +40,9 @@ package_update: true
packages: packages:
- curl - curl
- nftables - nftables
# kc-lab-edge only. The k3s nodes do not need these, and the edge does not need
# anything else — nginx terminates TLS and certbot renews the certificate, both
# inside this disposable guest.
# - nginx
# - certbot
# - python3-certbot-dns-cloudflare
deploy/lab/edge/lab-edge-dnat.nft
+33
View File
@@ -0,0 +1,33 @@
#!/usr/sbin/nft -f
# Forward the tailnet entry point to the edge guest.
#
# This is the ONLY lab traffic rule the physical host carries. Everything else
# that used to live here — nginx config, certificates, certbot, the deploy hook
# — now lives on kc-lab-edge and is destroyed with it.
#
# DNAT only, never SNAT. The guests' default route is the host, so replies come
# back through here and conntrack reverses the translation on its own. Adding a
# masquerade would rewrite the source and the edge would see 192.168.122.1 for
# every client — which would silently invalidate the X-Forwarded-For contract
# that this lab measures.
#
# PREROUTING nat runs before the routing decision, so this wins over any local
# socket on :80/:443. That makes the cutover atomic and the rollback a single
# `nft delete table ip lab_edge`.
table ip lab_edge
delete table ip lab_edge
table ip lab_edge {
chain prerouting {
type nat hook prerouting priority dstnat; policy accept;
iifname "tailscale0" tcp dport { 80, 443 } dnat to 192.168.122.10
}
# libvirt's own forward rules accept RELATED,ESTABLISHED into the guest
# subnet but not a NEW inbound connection. This runs ahead of them.
chain forward {
type filter hook forward priority filter - 10; policy accept;
ip daddr 192.168.122.10 tcp dport { 80, 443 } ct state new accept
}
}
deploy/lab/edge/lab-edge-dnat.service
+13
View File
@@ -0,0 +1,13 @@
[Unit]
Description=Lab edge DNAT (tailnet :80/:443 -> kc-lab-edge)
After=network-online.target libvirtd.service
Wants=network-online.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/nft -f /etc/nftables.d/lab-edge-dnat.nft
ExecStop=/usr/sbin/nft delete table ip lab_edge
[Install]
WantedBy=multi-user.target
deploy/lab/host/nginx-keycloak-lab.conf → deploy/lab/edge/nginx-keycloak-lab.conf
+4 -2
View File
@@ -25,8 +25,10 @@ server {
} }
server { server {
listen 443 ssl default_server; # The http2 parameter of listen, not the separate `http2 on;` directive:
http2 on; # that directive needs nginx >= 1.25.1 and the edge guest is Debian 12
# (nginx 1.22). This form works on both and is what the lab actually runs.
listen 443 ssl http2 default_server;
server_name _; server_name _;
# fullchain.pem, never cert.pem: omitting the intermediates passes on # fullchain.pem, never cert.pem: omitting the intermediates passes on
deploy/lab/edge/reload-nginx.sh
Executable
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
# certbot deploy hook. Install as
# /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh (chmod +x)
#
# deploy/ runs only when a certificate was actually renewed (RENEWED_LINEAGE is
# set). post/ would run twice a day whether or not anything changed, reloading
# nginx for nothing.
#
# Without this, D-4 measured the failure exactly: the renewal succeeds, the
# timer reports SUCCESS, and the old certificate keeps being served for 38m25s
# — with no error anywhere.
nginx -t && nginx -s reload
deploy/lab/host/teardown-host.sh
Executable
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
# Remove the lab's host layer from test-server. Packages stay.
#
# sudo bash deploy/lab/host/teardown-host.sh
#
# The host's sudo asks for a password, so run this in a terminal — not over a
# non-interactive ssh, where sudo fails silently into an empty result.
#
# ★ Certificates are BACKED UP, NOT DELETED. Let's Encrypt allows 5 duplicate
# certificates per week for the same name set, and this lab's names resolve to
# a tailnet address (100.64.0.0/10, not routable from the public internet), so
# an HTTP-01 reissue cannot be validated from here. Deleting the files turns a
# free restore into a problem that has to be solved first. Restoring is:
#
# sudo tar xzf ~/letsencrypt-backup-<stamp>.tgz -C /etc
set -u
STAMP="$(date +%Y%m%d-%H%M%S)"
HOME_DIR="${SUDO_USER:+/home/$SUDO_USER}"
HOME_DIR="${HOME_DIR:-$HOME}"
echo "===== 1) 인증서 백업 (지우지 않는다) ====="
if [ -d /etc/letsencrypt ]; then
out="$HOME_DIR/letsencrypt-backup-$STAMP.tgz"
tar czf "$out" -C /etc letsencrypt
chown "${SUDO_USER:-root}" "$out"
echo "백업: $out ($(du -h "$out" | cut -f1))"
echo "현재 인증서:"
certbot certificates 2>/dev/null | grep -E "Certificate Name|Domains|Expiry Date" || true
echo "검증 방식 (재발급이 되는지의 답):"
grep -H authenticator /etc/letsencrypt/renewal/*.conf 2>/dev/null || echo " (renewal 설정 없음)"
else
echo "/etc/letsencrypt 없음 — 건너뜀"
fi
echo
echo "===== 2) nginx 실험대 설정 제거 ====="
if [ -f /etc/nginx/sites-available/keycloak-lab ]; then
cp /etc/nginx/sites-available/keycloak-lab "$HOME_DIR/keycloak-lab.nginx.$STAMP.bak"
echo "백업: $HOME_DIR/keycloak-lab.nginx.$STAMP.bak"
fi
rm -fv /etc/nginx/sites-enabled/keycloak-lab
rm -fv /etc/nginx/sites-available/keycloak-lab
systemctl disable --now nginx
echo
echo "===== 3) certbot 갱신 타이머 정지 ====="
# 인증서 파일은 남기지만, 갱신 시도는 멈춘다. 지금 DNS 로는 HTTP-01 검증이
# 실패하고, 실패가 로그에만 쌓이면서 「왜 안 되지」의 원인이 된다.
systemctl disable --now certbot-renew.timer 2>/dev/null || true
echo
echo "===== 4) 엣지 DNAT (있으면) ====="
systemctl disable --now lab-edge-dnat.service 2>/dev/null || true
rm -fv /etc/systemd/system/lab-edge-dnat.service /etc/nftables.d/lab-edge-dnat.nft
systemctl daemon-reload
nft delete table ip lab_edge 2>/dev/null || true
echo
echo "===== 5) 확인 ====="
echo "-- nginx: $(systemctl is-active nginx) / $(systemctl is-enabled nginx 2>&1)"
echo "-- certbot timer: $(systemctl is-active certbot-renew.timer 2>&1) / $(systemctl is-enabled certbot-renew.timer 2>&1)"
echo "-- 80/443 리스너:"; ss -tlnp | grep -E ':(80|443) ' || echo " (없음 — 정상)"
echo "-- sites-enabled:"; ls -A /etc/nginx/sites-enabled 2>/dev/null || echo " (비었음 — 정상)"
echo "-- letsencrypt:"; ls /etc/letsencrypt/live 2>/dev/null || echo " (없음)"
echo "-- libvirt 도메인:"; virsh list --all 2>/dev/null | tail -n +3 | grep -v '^$' || echo " (없음 — 정상)"
echo
echo "완료. 패키지(nginx · libvirt · qemu · certbot · kubectl)와 base.qcow2 는 남아 있다."
deploy/lab/k8s/b7-oauth2-proxy.yaml
+128
View File
@@ -0,0 +1,128 @@
# Experiment B-7 — oauth2-proxy, to measure how replicas share a cookie secret
# and what happens when it is rotated (Q1, unknown 7).
#
# This is a different shape of problem from the BFF. The BFF keeps state on the
# server, so the question was "which store". oauth2-proxy keeps no server state
# at all: the whole session rides in a cookie that is signed and encrypted with
# --cookie-secret. So there is nothing to share and nothing to lose on restart —
# instead, every replica must hold the *same* secret, and changing it invalidates
# every cookie at once.
#
# kubectl apply -f deploy/lab/k8s/b7-oauth2-proxy.yaml
#
# app2.hyeonworks.com is borrowed from Grafana for the duration of this
# experiment; the certificate only covers auth / app1 / app2, so a fourth name
# is not available. Grafana's Ingress is restored afterwards.
apiVersion: v1
kind: Secret
metadata:
name: oauth2-proxy-secrets
namespace: keycloak-lab
type: Opaque
stringData:
# oauth2-proxy requires exactly 16, 24 or 32 bytes. This is the value whose
# rotation the experiment is about.
COOKIE_SECRET_A: "lab-cookie-secret-aaaaaaaaaaaaaa"
COOKIE_SECRET_B: "lab-cookie-secret-bbbbbbbbbbbbbb"
CLIENT_SECRET: proxy-lab-secret
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: oauth2-proxy
namespace: keycloak-lab
spec:
# Two replicas is the point: Q1 asks how they share the secret.
replicas: 2
selector:
matchLabels: { app: oauth2-proxy }
template:
metadata:
labels: { app: oauth2-proxy }
spec:
# See B-1: Kubernetes injects <SVCNAME>_PORT as a tcp:// URL and it
# collides with ordinary configuration names.
enableServiceLinks: false
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels: { app: oauth2-proxy }
containers:
- name: oauth2-proxy
image: quay.io/oauth2-proxy/oauth2-proxy:v7.7.1
args:
- --provider=oidc
- --oidc-issuer-url=https://auth.hyeonworks.com/realms/keycloak-patterns
- --client-id=oauth2-proxy
- --redirect-url=https://app2.hyeonworks.com/oauth2/callback
- --email-domain=*
- --http-address=0.0.0.0:4180
# The upstream is the same echo app the B-4 header experiment used,
# so what the proxy forwards can be read straight off the response.
- --upstream=http://echo.header-lab.svc:8081
# ★ 이 옵션을 켜면 세션(=쿠키)에 access token 이 들어간다.
# 그러면 Set-Cookie 가 커져 프록시 앞단에서 502 가 났다.
# B-4 에서 본 헤더 크기 절벽이 이번에는 응답 쪽에서 나타난 것이다.
# - --pass-authorization-header=true
- --set-xauthrequest=true
- --reverse-proxy=true
- --cookie-secure=true
# One hour, matching the value Q1 records for the current setup.
- --cookie-expire=1h
- --skip-provider-button=true
# ★ 쿠키에 세션 전체를 담으면 Set-Cookie 가 커지고, 그 응답이
# 앞단 nginx 의 proxy_buffer 를 넘겨 502 가 났다(측정됨).
# Redis 로 옮기면 쿠키에는 티켓만 남는다 — 그리고 그 순간
# "replica 가 secret 을 공유해야 한다"는 문제의 성격도 바뀐다.
- --session-store-type=redis
- --redis-connection-url=redis://redis.keycloak-lab.svc:6379
env:
- name: OAUTH2_PROXY_CLIENT_SECRET
valueFrom:
secretKeyRef: { name: oauth2-proxy-secrets, key: CLIENT_SECRET }
# Which of the two secrets is in use is switched here. Both replicas
# read the same key, which is exactly the sharing Q1 asks about.
- name: OAUTH2_PROXY_COOKIE_SECRET
valueFrom:
secretKeyRef: { name: oauth2-proxy-secrets, key: COOKIE_SECRET_A }
ports:
- containerPort: 4180
name: http
readinessProbe:
httpGet: { path: /ping, port: http }
initialDelaySeconds: 5
resources:
requests: { memory: 32Mi, cpu: 20m }
limits: { memory: 128Mi }
---
apiVersion: v1
kind: Service
metadata:
name: oauth2-proxy
namespace: keycloak-lab
spec:
selector: { app: oauth2-proxy }
ports:
- port: 4180
targetPort: http
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: oauth2-proxy
namespace: keycloak-lab
spec:
ingressClassName: traefik
rules:
- host: app2.hyeonworks.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: oauth2-proxy
port:
number: 4180
docs/diagrams/_style.md
+12
View File
@@ -0,0 +1,12 @@
# 다이어그램 규약
| 표현 | 뜻 |
|---|---|
| 실선 상자 | 살아 있는 구성 요소 |
| 붉은 점선 상자 | 이 실험에서 죽이거나 막은 것 |
| ✂ 붉은 X | 주입 지점 |
| 실선 화살표 | 정상 경로 |
| 붉은 점선 화살표 | 실험에서 깨진 경로 |
| 회색 글씨 | 측정값 |
SVG 는 GitHub 에서 그대로 렌더링되며 외부 폰트를 쓰지 않는다.
docs/diagrams/a0-session-replication.svg
+48
View File
@@ -0,0 +1,48 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 400" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">A-0 — 세션을 공유하는 것은 Infinispan 이 아니라 PostgreSQL 이다</text>
<text x="20" y="44" class="s">클러스터가 형성됐다는 것과 세션이 복제된다는 것은 다른 얘기였다</text>
<rect x="30" y="76" width="330" height="120" class="box"/>
<text x="46" y="98" class="tb">keycloak-0 (kc-lab-2)</text>
<rect x="46" y="112" width="298" height="34" class="box"/>
<text x="58" y="134" class="t">Infinispan sessions 캐시</text>
<text x="46" y="166" class="s">자기가 처리한 로그인만 들어 있다</text>
<text x="46" y="184" class="s">entries_unique 는 노드마다 다르다</text>
<rect x="500" y="76" width="330" height="120" class="box"/>
<text x="516" y="98" class="tb">keycloak-1 (kc-lab-1)</text>
<rect x="516" y="112" width="298" height="34" class="box"/>
<text x="528" y="134" class="t">Infinispan sessions 캐시</text>
<text x="516" y="166" class="s">여기에도 자기 것만 있다</text>
<text x="516" y="184" class="s">상대 세션은 들어오지 않는다</text>
<path d="M362 130 L498 130" class="r" stroke-dasharray="5 3" marker-end="url(#ar)"/>
<path d="M498 146 L362 146" class="r" stroke-dasharray="5 3" marker-end="url(#ar)"/>
<text x="430" y="122" class="tb" text-anchor="middle" fill="#cf222e">복제 없음</text>
<text x="430" y="170" class="s" text-anchor="middle" fill="#cf222e">세션 엔트리는 노드 사이를 건너가지 않는다</text>
<rect x="300" y="250" width="260" height="76" class="ok"/>
<text x="316" y="272" class="tb">PostgreSQL</text>
<text x="316" y="292" class="s">OFFLINE_USER_SESSION (offline_flag='0')</text>
<text x="316" y="310" class="s">두 노드가 같은 행을 본다</text>
<path d="M150 198 L330 246" class="g" marker-end="url(#ag)"/>
<path d="M710 198 L530 246" class="g" marker-end="url(#ag)"/>
<text x="180" y="232" class="s">SELECT / INSERT</text>
<text x="600" y="232" class="s">SELECT / INSERT</text>
<rect x="30" y="346" width="800" height="40" class="box"/>
<text x="46" y="370" class="s">근거 — 노드 A 로 로그인하고 노드 B 로 refresh 했을 때, 반대편 노드가 날린 SQL 을 문장 로깅으로 직접 잡았다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.2 KiB

docs/diagrams/a1-transport-block.svg
+47
View File
@@ -0,0 +1,47 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 430" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">A-1 — 7800 을 막아도 세션 공유는 안 깨진다. 깨지는 것은 로그아웃 전파다</text>
<text x="20" y="44" class="s">예측 하나가 빗나갔고, 예상 못 한 것이 둘 나왔다</text>
<rect x="30" y="72" width="300" height="104" class="box"/>
<text x="46" y="94" class="tb">keycloak-0</text>
<text x="46" y="116" class="s">로그인 처리 · 세션을 DB 에 쓴다</text>
<rect x="530" y="72" width="300" height="104" class="box"/>
<text x="546" y="94" class="tb">keycloak-1</text>
<text x="546" y="116" class="s">DB 를 읽어 같은 세션을 안다</text>
<rect x="352" y="86" width="156" height="34" class="dead"/>
<text x="430" y="107" class="t" text-anchor="middle" fill="#cf222e">TCP 7800 차단</text>
<text x="430" y="140" class="s" text-anchor="middle">NetworkPolicy 는 허용목록이다</text>
<text x="430" y="156" class="s" text-anchor="middle">8080·9000 만 열고 7800 은 누락시킨다</text>
<rect x="300" y="206" width="260" height="64" class="ok"/>
<text x="316" y="228" class="tb">PostgreSQL</text>
<text x="316" y="248" class="s">세션은 여기 있다 → 교차 노드 refresh 200</text>
<path d="M150 178 L320 202" class="g" marker-end="url(#ag)"/>
<path d="M710 178 L540 202" class="g" marker-end="url(#ag)"/>
<rect x="30" y="290" width="390" height="76" class="dead"/>
<text x="46" y="312" class="tb">깨진 것 — 로그아웃 전파</text>
<text x="46" y="332" class="s">무효화 통지가 7800 을 탄다</text>
<text x="46" y="350" class="s">400 이어야 할 refresh 가 200 을 반환했다</text>
<rect x="440" y="290" width="390" height="76" class="warn"/>
<text x="456" y="312" class="tb">★ 주입이 먹지 않았다 — conntrack</text>
<text x="456" y="332" class="s">ESTABLISHED 연결은 규칙 평가를 건너뛴다</text>
<text x="456" y="350" class="s">cluster_size 가 25분간 2 로 남았다 → 삭제 후에야 갈렸다</text>
<text x="30" y="392" class="s">그리고 예상 못 한 둘째 — 분단된 노드가 readiness 실패로 스스로 로드밸런서에서 빠진다</text>
<text x="30" y="410" class="s">덕분에 외부에서는 장애가 보이지 않는다. 관측 지점을 밖에만 두면 이 실험은 「아무 일도 없음」으로 보인다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.3 KiB

docs/diagrams/a2-database-loss.svg
+26
View File
@@ -0,0 +1,26 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 700 300" width="700" height="300" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
.bad{stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;fill:none;marker-end:url(#b)}
</style>
<defs>
<marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker>
<marker id="b" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#cf222e"/></marker>
</defs>
<text class="h" x="16" y="24">A-2 · PostgreSQL 정지 — 살아남는 노드가 없다</text>
<rect class="box" x="30" y="46" width="180" height="52"/><text class="t" x="120" y="68" text-anchor="middle">keycloak-1</text><text class="s" x="120" y="86" text-anchor="middle">캐시: 세션 N개</text>
<rect class="box" x="480" y="46" width="180" height="52"/><text class="t" x="570" y="68" text-anchor="middle">keycloak-0</text><text class="s" x="570" y="86" text-anchor="middle">캐시: 세션 M개</text>
<path class="ln" d="M210,72 L480,72"/><text class="s" x="345" y="66" text-anchor="middle">7800 · 살아 있다</text>
<path class="bad" d="M120,100 L300,150"/><path class="bad" d="M570,100 L400,150"/>
<rect class="dead" x="270" y="156" width="160" height="52"/>
<text class="r" x="350" y="178" text-anchor="middle">postgres ✗</text><text class="s" x="350" y="196" text-anchor="middle">replicas=0</text>
<rect class="dead" x="30" y="228" width="290" height="52"/>
<text class="r" x="175" y="250" text-anchor="middle">양쪽 모두 NotReady</text><text class="s" x="175" y="268" text-anchor="middle">ready 주소 = [] · 외부 503</text>
<rect class="box" x="370" y="228" width="290" height="52"/>
<text class="t" x="515" y="250" text-anchor="middle">up{job="keycloak"} = 1</text><text class="s" x="515" y="268" text-anchor="middle">프로세스는 살아 있다 — up 은 못 잡는다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.4 KiB

docs/diagrams/a3-database-crash.svg
+55
View File
@@ -0,0 +1,55 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 400" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">A-3 — 200 을 받은 로그인 153건 중 4건이 DB 에 없다</text>
<text x="20" y="44" class="s">버그가 아니라 synchronous_commit OFF 의 대가를 실측한 것이다</text>
<rect x="30" y="74" width="180" height="70" class="box"/>
<text x="46" y="96" class="tb">클라이언트</text>
<text x="46" y="116" class="s">로그인 요청</text>
<text x="46" y="134" class="s">200 + 토큰 수신</text>
<rect x="250" y="74" width="200" height="70" class="box"/>
<text x="266" y="96" class="tb">Keycloak</text>
<text x="266" y="116" class="s">트랜잭션마다</text>
<text x="266" y="134" class="s">SET LOCAL synchronous_commit OFF</text>
<rect x="490" y="74" width="180" height="70" class="warn"/>
<text x="506" y="96" class="tb">PostgreSQL</text>
<text x="506" y="116" class="s">COMMIT 을 즉시 반환</text>
<text x="506" y="134" class="s">WAL 은 아직 메모리에</text>
<rect x="710" y="74" width="120" height="70" class="box"/>
<text x="726" y="96" class="tb">디스크</text>
<text x="726" y="116" class="s">wal_writer_delay</text>
<text x="726" y="134" class="s">200ms 뒤 기록</text>
<path d="M212 108 L246 108" class="k" marker-end="url(#a)"/>
<path d="M452 108 L486 108" class="k" marker-end="url(#a)"/>
<path d="M672 108 L706 108" class="r" stroke-dasharray="4 3" marker-end="url(#ar)"/>
<text x="676" y="164" class="s" fill="#cf222e">이 구간이 비어 있다</text>
<rect x="620" y="180" width="210" height="40" class="dead"/>
<text x="636" y="205" class="t" fill="#cf222e">✂ 여기서 강제 종료</text>
<rect x="30" y="250" width="380" height="96" class="dead"/>
<text x="46" y="272" class="tb">결과 — RPO 는 0 이 아니다</text>
<text x="46" y="296" class="t">클라이언트가 200 받은 로그인 153 건</text>
<text x="46" y="316" class="t">그중 DB 에 실제로 존재 149 건</text>
<text x="46" y="336" class="t" fill="#cf222e">★ 유실 4 건</text>
<rect x="430" y="250" width="400" height="96" class="box"/>
<text x="446" y="272" class="tb">주입도 두 번 실패했다</text>
<text x="446" y="294" class="s">kubectl delete --force 는 크래시가 아니다 — 런타임이 SIGTERM 을 보내</text>
<text x="446" y="310" class="s">PostgreSQL 이 정상 플러시했고 유실이 0 이었다</text>
<text x="446" y="332" class="s">kill -9 1 도 무시된다 — PID 1 은 자기 네임스페이스의 SIGKILL 을 받지 않는다</text>
<text x="30" y="372" class="s">백엔드 프로세스를 죽여 postmaster 가 reinitialize 하게 만들자 비로소 "not properly shut down / redo starts" 가 찍혔다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.7 KiB

docs/diagrams/a4-node-loss.svg
+52
View File
@@ -0,0 +1,52 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 420" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">A-4 — 둘 다 전면 장애지만 이유가 다르다</text>
<text x="20" y="44" class="s">그리고 죽은 파드가 산 파드보다 건강해 보인다</text>
<rect x="30" y="70" width="390" height="150" class="box"/>
<text x="46" y="92" class="tb">4a — 워커(kc-lab-2) 상실</text>
<rect x="46" y="104" width="170" height="46" class="dead"/>
<text x="58" y="124" class="t">keycloak-0</text>
<text x="58" y="142" class="s">PostgreSQL 도 여기 있었다</text>
<rect x="236" y="104" width="170" height="46" class="warn"/>
<text x="248" y="124" class="t">keycloak-1 살아있음</text>
<text x="248" y="142" class="s">그러나 DB 가 없다</text>
<text x="46" y="172" class="t">외부 응답 <tspan fill="#cf222e" font-weight="600">503</tspan> · kubectl 정상</text>
<text x="46" y="194" class="s">DB 가 같이 죽어서 장애다. 남은 파드는 돌지만 아무것도 못 한다</text>
<text x="46" y="212" class="s">PVC 가 local-path 라 다른 노드로 재배치되지 않는다</text>
<rect x="440" y="70" width="390" height="150" class="box"/>
<text x="456" y="92" class="tb">4b — 컨트롤 플레인(kc-lab-1) 상실</text>
<rect x="456" y="104" width="170" height="46" class="ok"/>
<text x="468" y="124" class="t">keycloak-0</text>
<text x="468" y="142" class="s">★ 계속 돌고 있다</text>
<rect x="646" y="104" width="170" height="46" class="dead"/>
<text x="658" y="124" class="t">API 서버 · traefik</text>
<text x="658" y="142" class="s">들어갈 길이 없다</text>
<text x="456" y="172" class="t">외부 응답 <tspan fill="#cf222e" font-weight="600">000</tspan> · kubectl <tspan fill="#cf222e">불통</tspan></text>
<text x="456" y="194" class="s">워크로드는 멀쩡한데 도달할 수 없어 장애다</text>
<text x="456" y="212" class="s">진입점이 단일 노드에 있으면 워크로드 이중화는 의미가 없다</text>
<rect x="30" y="242" width="800" height="86" class="warn"/>
<text x="46" y="264" class="tb">★ 예상하지 못한 것 셋</text>
<text x="46" y="286" class="s">1. 죽은 파드가 산 파드보다 건강해 보인다 — kubelet 이 사라져 상태가 갱신되지 않으니 Running 으로 남는다</text>
<text x="46" y="304" class="s">2. StatefulSet 은 Terminating 파드의 대체를 만들지 않는다 — 이름이 같아야 하므로 지워지기를 기다린다</text>
<text x="46" y="322" class="s">3. node-monitor-grace-period 40초 + tolerationSeconds 300초 = 축출까지 5분 40초</text>
<rect x="30" y="348" width="800" height="46" class="ok"/>
<text x="46" y="370" class="tb">복구는 둘 다 virsh start 이후 60초</text>
<text x="46" y="388" class="s">장애 시간의 대부분은 복구가 아니라 "누가 죽은 것을 알아채는 데" 걸린 시간이다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.9 KiB

docs/diagrams/a5-asymmetric-partition.svg
+49
View File
@@ -0,0 +1,49 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 420" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">A-5 — 비대칭 차단은 클러스터를 가르지 못한다. 그리고 갈라도 서비스는 계속된다</text>
<text x="20" y="44" class="s">주입을 세 번 실패했고, 세 번 모두 「아무 일도 없었다」로 보였다</text>
<rect x="30" y="72" width="390" height="112" class="box"/>
<text x="46" y="94" class="tb">① 한 방향만 차단</text>
<rect x="46" y="106" width="130" height="34" class="box"/>
<text x="58" y="128" class="t">keycloak-0</text>
<rect x="290" y="106" width="116" height="34" class="box"/>
<text x="302" y="128" class="t">keycloak-1</text>
<path d="M178 116 L286 116" class="r" stroke-dasharray="4 3" marker-end="url(#ar)"/>
<path d="M286 132 L178 132" class="g" marker-end="url(#ag)"/>
<text x="46" y="164" class="s" fill="#1a7f37">열린 방향으로 재연결한다 → cluster_size 2 유지. 가르지 못한다</text>
<rect x="440" y="72" width="390" height="112" class="box"/>
<text x="456" y="94" class="tb">② 양방향 완전 차단</text>
<rect x="456" y="106" width="130" height="34" class="dead"/>
<text x="468" y="128" class="t">keycloak-0</text>
<rect x="700" y="106" width="116" height="34" class="ok"/>
<text x="712" y="128" class="t">keycloak-1</text>
<path d="M588 116 L696 116" class="r" stroke-dasharray="4 3" marker-end="url(#ar)"/>
<path d="M696 132 L588 132" class="r" stroke-dasharray="4 3" marker-end="url(#ar)"/>
<text x="456" y="164" class="s">양쪽 모두 멤버 1개. 그런데 <tspan fill="#cf222e" font-weight="600">한쪽만 DOWN</tspan> 이 된다 — 코디네이터 쪽이 살아남는다</text>
<rect x="30" y="204" width="800" height="48" class="ok"/>
<text x="46" y="226" class="tb">그래서 전면 장애 경로가 없다 — 외부 응답 200 유지</text>
<text x="46" y="244" class="s">분단된 쪽이 스스로 readiness 를 떨어뜨려 로드밸런서에서 빠지고, 남은 쪽이 계속 응답한다</text>
<rect x="30" y="272" width="800" height="130" class="warn"/>
<text x="46" y="294" class="tb">★ 세 번의 주입 실패 — 전부 「조용히」 실패했다</text>
<text x="46" y="318" class="s">1. iptables -I FORWARD 1 이 무시됐다 — kube-router 가 자기 체인을 FORWARD 맨 위에 다시 끼워 넣는다 (패킷 0)</text>
<text x="46" y="338" class="s">2. 방향이 뒤집혀 있었다 — JGroups 의 client/server 역할은 재시작마다 바뀐다. raw 규칙이 엉뚱한 노드에 걸렸다</text>
<text x="46" y="358" class="s">3. dev eth0 이 없다 — Debian 은 enp1s0 이고, 게다가 flannel VXLAN 이 이미 캡슐화해 파드 IP 가 안 보인다</text>
<text x="46" y="382" class="s" fill="#cf222e">해결: raw 테이블 PREROUTING(conntrack 보다 먼저) + flannel.1 인터페이스에서 필터</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.8 KiB

docs/diagrams/a6-latency-injection.svg
+51
View File
@@ -0,0 +1,51 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 400" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">A-6 — 200밀리초가 22초가 된다</text>
<text x="20" y="44" class="s">지연은 왕복 횟수만큼 곱해지고, 커넥션 풀에서 한 번 더 곱해진다</text>
<rect x="30" y="76" width="180" height="60" class="ok"/>
<text x="46" y="98" class="tb">대조군 keycloak-0</text>
<text x="46" y="120" class="s">DB 와 같은 노드 · 66ms</text>
<rect x="250" y="76" width="180" height="60" class="dead"/>
<text x="266" y="98" class="tb">시험군 keycloak-1</text>
<text x="266" y="120" class="s">tc netem delay 200ms</text>
<text x="230" y="112" class="tb" fill="#cf222e">✂</text>
<rect x="470" y="76" width="180" height="60" class="warn"/>
<text x="486" y="98" class="tb">단일 요청</text>
<text x="486" y="120" class="t">1,872 ms <tspan class="s">(28배)</tspan></text>
<rect x="690" y="76" width="140" height="60" class="dead"/>
<text x="706" y="98" class="tb">동시 20건</text>
<text x="706" y="120" class="t" fill="#cf222e">22.2 초</text>
<path d="M432 106 L466 106" class="k" marker-end="url(#a)"/>
<path d="M652 106 L686 106" class="r" marker-end="url(#ar)"/>
<text x="20" y="176" class="tb">왜 200ms 가 1,872ms 가 되는가 — 왕복 횟수</text>
<rect x="30" y="190" width="800" height="46" class="box"/>
<text x="46" y="210" class="s">로그인 한 번은 DB 왕복을 여러 번 한다: 클라이언트 조회 · 사용자 조회 · 세션 INSERT · 커밋 …</text>
<text x="46" y="228" class="s">왕복마다 200ms 가 더해진다. 애플리케이션은 아무것도 잘못하지 않았다</text>
<text x="20" y="266" class="tb">왜 1,872ms 가 22.2초가 되는가 — 커넥션 풀</text>
<rect x="30" y="280" width="390" height="76" class="dead"/>
<text x="46" y="302" class="s">요청이 커넥션을 오래 붙들고 있으면</text>
<text x="46" y="320" class="s">뒤의 요청은 풀에서 대기한다</text>
<text x="46" y="342" class="t" fill="#cf222e">agroal 커넥션 획득 대기 최대 20,000 ms</text>
<rect x="440" y="280" width="390" height="76" class="warn"/>
<text x="456" y="302" class="tb">그리고 파드가 죽는다</text>
<text x="456" y="322" class="s">readiness 프로브가 타임아웃으로 실패 →</text>
<text x="456" y="342" class="s">느린 노드가 로드밸런서에서 빠진다. 느림이 장애로 승격된다</text>
<text x="30" y="384" class="s">측정 장치 주의 — 동시 20건을 kubectl run --rm -i 로 돌리면 출력 스트림이 유실된다. 상주 탐침에 파일로 모아야 한다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.6 KiB

docs/diagrams/a7-volatile-inversion.svg
+36
View File
@@ -0,0 +1,36 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 720 330" width="720" height="330" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}.g{font-size:12px;fill:#1a7f37;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
</style>
<defs><marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker></defs>
<text class="h" x="16" y="24">A-7 · 같은 주입, 같은 관측, 정반대 결과</text>
<text class="h" x="180" y="52" text-anchor="middle">persistent (KC 26 기본)</text>
<rect class="box" x="30" y="62" width="300" height="54"/>
<text class="t" x="180" y="84" text-anchor="middle">keycloak ×2 — 로컬 캐시</text>
<path class="ln" d="M180,116 L180,140"/>
<rect class="ok" x="70" y="142" width="220" height="40"/>
<text class="g" x="180" y="167" text-anchor="middle">PostgreSQL — 진실의 원천</text>
<text class="h" x="540" y="52" text-anchor="middle">volatile (KC 24 이전 방식)</text>
<rect class="box" x="390" y="62" width="300" height="54"/>
<text class="t" x="540" y="84" text-anchor="middle">keycloak ×2 — 캐시가 곧 진실</text>
<path class="ln" d="M470,116 L470,140"/><path class="ln" d="M610,140 L610,116"/>
<rect class="ok" x="430" y="142" width="220" height="40"/>
<text class="g" x="540" y="167" text-anchor="middle">클러스터 복제 (7800)</text>
<rect class="box" x="30" y="202" width="660" height="112"/>
<text class="t" x="360" y="224" text-anchor="middle">뒤집힌 세 결과</text>
<text class="s" x="200" y="248" text-anchor="middle">A-1 7800 차단 후 교차 refresh</text>
<text class="g" x="430" y="248" text-anchor="middle">200</text><text class="r" x="560" y="248" text-anchor="middle">400 Session not active</text>
<text class="s" x="200" y="272" text-anchor="middle">A-8 롤링 재시작 후 refresh</text>
<text class="g" x="430" y="272" text-anchor="middle">200</text><text class="r" x="560" y="272" text-anchor="middle">400 Session not active</text>
<text class="s" x="200" y="296" text-anchor="middle">A-2 DB 정지 중 새 로그인</text>
<text class="r" x="430" y="296" text-anchor="middle">500</text><text class="g" x="560" y="296" text-anchor="middle">200</text>
<text class="s" x="430" y="230" text-anchor="middle">persistent</text><text class="s" x="560" y="230" text-anchor="middle">volatile</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.8 KiB

docs/diagrams/a7a-cache-temperature.svg
+70
View File
@@ -0,0 +1,70 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 470" font-family="-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">A-7a — volatile + DB 정지의 결과는 「무엇을 하느냐」가 아니라 「캐시가 더운가」로 갈린다</text>
<text x="20" y="44" class="s">문장 로깅(log_statement=all)으로 측정 · 세 상태 모두 재현</text>
<rect x="20" y="66" width="380" height="120" class="box"/>
<text x="34" y="88" class="tb">로그인이 쏘는 SQL</text>
<rect x="36" y="100" width="348" height="34" class="ok"/>
<text x="48" y="122" class="t">0개 — realm·사용자·클라이언트가 전부 캐시</text>
<text x="36" y="152" class="s">MARK_LOGIN_START / MARK_LOGIN_END 사이가 비어 있다</text>
<text x="36" y="170" class="s">volatile 이라 세션 쓰기도 없다</text>
<rect x="440" y="66" width="400" height="120" class="box"/>
<text x="454" y="88" class="tb">refresh 가 쏘는 SQL</text>
<rect x="456" y="100" width="368" height="34" class="warn"/>
<text x="466" y="115" class="t">select cscme1_0.SCOPE_ID from CLIENT_SCOPE_CLIENT</text>
<text x="466" y="129" class="s">where CLIENT_ID=$1 and DEFAULT_SCOPE=$2 ('f' = optional)</text>
<text x="454" y="152" class="s">★ REVOKED_TOKEN 은 한 번도 나오지 않는다 — A-7 의 가설은 틀렸다</text>
<text x="454" y="170" class="s">그리고 첫 refresh 만 쏜다. 이후 3회는 SQL 0건 = 캐시됨</text>
<text x="20" y="222" class="tb">그래서 같은 설정이 세 가지 답을 낸다 — PostgreSQL 정지 시</text>
<rect x="20" y="240" width="266" height="120" class="dead"/>
<text x="34" y="262" class="tb">① 완전 냉시동</text>
<text x="34" y="282" class="s">재시작 직후, 아무것도 안 함</text>
<text x="34" y="306" class="t">로그인 <tspan fill="#cf222e" font-weight="600">400</tspan> refresh 400</text>
<text x="34" y="328" class="s">select ce1_0.ID from CLIENT</text>
<text x="34" y="344" class="s">클라이언트 조회조차 캐시에 없다</text>
<rect x="298" y="240" width="266" height="120" class="warn"/>
<text x="312" y="262" class="tb">② CLIENT 만 더움</text>
<text x="312" y="282" class="s">로그인 1회 뒤 · ← A-7 이 본 상태</text>
<text x="312" y="306" class="t">로그인 200 refresh <tspan fill="#bf8700" font-weight="600">500</tspan></text>
<text x="312" y="328" class="s">CLIENT_SCOPE_CLIENT 조회 실패</text>
<text x="312" y="344" class="s">Keycloak 로그가 이 SQL 을 직접 지목</text>
<rect x="576" y="240" width="264" height="120" class="ok"/>
<text x="590" y="262" class="tb">③ 완전히 더움</text>
<text x="590" y="282" class="s">refresh 3회로 캐시를 채운 뒤</text>
<text x="590" y="306" class="t">로그인 200 refresh <tspan fill="#1a7f37" font-weight="600">200</tspan></text>
<text x="590" y="328" class="s">SQL 0건 — DB 를 아예 안 본다</text>
<text x="590" y="344" class="s">A-7 의 표와 정반대</text>
<path d="M286 300 L294 300" class="k" marker-end="url(#a)"/>
<path d="M564 300 L572 300" class="k" marker-end="url(#a)"/>
<text x="292" y="290" class="s">로그인 1회</text>
<text x="570" y="290" class="s">refresh 3회</text>
<rect x="20" y="384" width="820" height="66" class="box"/>
<text x="34" y="406" class="tb">남기는 것</text>
<text x="34" y="426" class="s">· A-7 의 표에는 조건이 빠져 있었다 — 「volatile 이면 DB 없이 로그인된다」도 냉시동에서는 거짓이다</text>
<text x="34" y="442" class="s">· 상태가 결과를 바꾸는데 그 상태가 안 보인다. A-1 의 conntrack 과 같은 계열의 함정이다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.5 KiB

docs/diagrams/a8-rolling-restart.svg
+47
View File
@@ -0,0 +1,47 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 380" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">A-8 — 세션은 살아남고 캐시만 사라진다</text>
<text x="20" y="44" class="s">이것이 persistent-user-sessions 를 켜는 진짜 이유다</text>
<rect x="30" y="74" width="800" height="30" class="box"/>
<text x="46" y="94" class="tb">kubectl rollout restart statefulset/keycloak — 한 번에 한 파드씩</text>
<rect x="30" y="124" width="250" height="110" class="dead"/>
<text x="46" y="146" class="tb">사라지는 것 — 메모리</text>
<text x="46" y="168" class="s">Infinispan sessions 캐시</text>
<text x="46" y="188" class="t">entries_unique → <tspan fill="#cf222e" font-weight="600">0</tspan></text>
<text x="46" y="212" class="s">파드와 함께 없어진다</text>
<rect x="305" y="124" width="250" height="110" class="ok"/>
<text x="321" y="146" class="tb">남는 것 — PostgreSQL</text>
<text x="321" y="168" class="s">OFFLINE_USER_SESSION</text>
<text x="321" y="188" class="t">151 → <tspan fill="#1a7f37" font-weight="600">151</tspan> 그대로</text>
<text x="321" y="212" class="s">재시작과 무관하다</text>
<rect x="580" y="124" width="250" height="110" class="ok"/>
<text x="596" y="146" class="tb">그래서 사용자는</text>
<text x="596" y="168" class="s">재시작 전 발급한 refresh token</text>
<text x="596" y="188" class="t">여전히 <tspan fill="#1a7f37" font-weight="600">200</tspan></text>
<text x="596" y="212" class="s">전 구간 중단 없음</text>
<path d="M282 178 L301 178" class="k" marker-end="url(#a)"/>
<path d="M557 178 L576 178" class="k" marker-end="url(#a)"/>
<rect x="30" y="254" width="800" height="52" class="box"/>
<text x="46" y="276" class="tb">클러스터는 자동 재형성된다</text>
<text x="46" y="296" class="s">JGROUPS_PING 테이블에 다시 등록되고 cluster_size 가 2 로 돌아온다 — 사람이 할 일이 없다</text>
<rect x="30" y="322" width="800" height="44" class="warn"/>
<text x="46" y="344" class="s">★ A-7 과 대조 — volatile 이면 같은 재시작에서 refresh 가 400 Session not active 가 된다.</text>
<text x="46" y="360" class="s">세션이 메모리에만 있었으므로 캐시가 사라지는 순간 세션도 사라진다. 이 실험의 결론은 버전 조건부다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.3 KiB

docs/diagrams/b0-autoconfiguration.svg
+45
View File
@@ -0,0 +1,45 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 400" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">B-0 — 아무것도 설정하지 않으면 Spring 이 무엇을 고르는가</text>
<text x="20" y="44" class="s">추측이 맞았지만, 추측으로 두면 안 되는 이유가 둘째 줄에 있다</text>
<rect x="30" y="72" width="800" height="118" class="box"/>
<text x="46" y="94" class="tb">/actuator/beans 가 실제로 답한 것</text>
<text x="46" y="118" class="t">authorizedClientService → InMemoryOAuth2AuthorizedClientService</text>
<text x="46" y="138" class="t">authorizedClientRepository → AuthenticatedPrincipalOAuth2AuthorizedClientRepository</text>
<text x="46" y="158" class="t">SessionRepository → 없음 (서블릿 컨테이너 in-memory)</text>
<text x="46" y="178" class="t" fill="#cf222e">Redis / Spring Session → 없음</text>
<rect x="30" y="212" width="390" height="150" class="warn"/>
<text x="46" y="234" class="tb">★ 이름이 곧 설명이다</text>
<text x="46" y="256" class="s">AuthenticatedPrincipal…Repository 는</text>
<text x="46" y="276" class="t">principal 이름으로 찾는다</text>
<rect x="46" y="288" width="358" height="30" class="dead"/>
<text x="58" y="308" class="s">조회 키에 session id 가 없다</text>
<text x="46" y="336" class="s">그래서 세션 저장소를 Redis 로 옮겨도</text>
<text x="46" y="354" class="s">토큰은 따라오지 않는다 — B-1 이 겪는 문제의 뿌리</text>
<rect x="440" y="212" width="390" height="150" class="box"/>
<text x="456" y="234" class="tb">두 개가 서로 다른 것을 저장한다</text>
<rect x="456" y="248" width="358" height="40" class="box"/>
<text x="468" y="266" class="t">Application Session</text>
<text x="468" y="282" class="s">누가 로그인했는지 · 세션 id 로 찾는다</text>
<rect x="456" y="298" width="358" height="40" class="box"/>
<text x="468" y="316" class="t">OAuth2AuthorizedClient</text>
<text x="468" y="332" class="s">access / refresh token · principal 이름으로 찾는다</text>
<text x="456" y="356" class="s">이 둘을 하나로 생각하면 B-1·B-2 의 결과를 해석할 수 없다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.2 KiB

docs/diagrams/b1-store-split.svg
+33
View File
@@ -0,0 +1,33 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 700 330" width="700" height="330" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
.bad{stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;fill:none;marker-end:url(#b)}
</style>
<defs>
<marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker>
<marker id="b" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#cf222e"/></marker>
</defs>
<text class="h" x="16" y="24">B-1 · Redis 는 세션만 옮기고 토큰은 두고 간다</text>
<rect class="box" x="30" y="46" width="140" height="46"/><text class="t" x="100" y="66" text-anchor="middle">bff-0</text><text class="s" x="100" y="82" text-anchor="middle">kc-lab-1</text>
<rect class="box" x="30" y="104" width="140" height="46"/><text class="t" x="100" y="124" text-anchor="middle">bff-1</text><text class="s" x="100" y="140" text-anchor="middle">kc-lab-2</text>
<rect class="ok" x="330" y="46" width="330" height="60"/>
<text class="t" x="495" y="68" text-anchor="middle">Redis — Application Session</text>
<text class="s" x="495" y="86" text-anchor="middle">sessionRepository → RedisSessionRepository ✔ 옮겨졌다</text>
<text class="s" x="495" y="100" text-anchor="middle">필드: SPRING_SECURITY_CONTEXT · TTL 1772초</text>
<rect class="dead" x="330" y="122" width="330" height="60"/>
<text class="r" x="495" y="144" text-anchor="middle">프로세스 메모리 — OAuth2AuthorizedClient</text>
<text class="s" x="495" y="162" text-anchor="middle">InMemoryOAuth2AuthorizedClientService ✗ 그대로</text>
<text class="s" x="495" y="176" text-anchor="middle">access token · refresh token 이 여기 있다</text>
<path class="ln" d="M170,69 L330,69"/><path class="ln" d="M170,127 L330,80"/>
<path class="bad" d="M170,140 L330,150"/>
<rect class="box" x="30" y="210" width="630" height="90"/>
<text class="t" x="345" y="234" text-anchor="middle">그 결과 사용자에게 보이는 것</text>
<text class="s" x="345" y="256" text-anchor="middle">principal: labuser ← 로그인은 되어 있다</text>
<text class="s" x="345" y="272" text-anchor="middle">accessTokenStoredOnServer: false ← 토큰이 없다</text>
<text class="r" x="345" y="292" text-anchor="middle">완전히 로그아웃되는 편이 차라리 낫다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.8 KiB

docs/diagrams/b2-store-split-consequences.svg
+55
View File
@@ -0,0 +1,55 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 440" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">B-2 — 저장소를 나눠 문제를 풀자 다른 두 문제가 남았다</text>
<text x="20" y="44" class="s">Q1 의 네 항목 중 둘이 통과, 둘이 실패</text>
<rect x="30" y="72" width="240" height="90" class="box"/>
<text x="46" y="94" class="tb">bff 인스턴스 2개</text>
<text x="46" y="116" class="s">어느 쪽으로 요청이 가도 된다</text>
<text x="46" y="136" class="s">enableServiceLinks: false</text>
<text x="46" y="154" class="s">(REDIS_PORT 충돌 회피)</text>
<rect x="330" y="72" width="230" height="90" class="ok"/>
<text x="346" y="94" class="tb">Redis</text>
<text x="346" y="116" class="s">Application Session</text>
<text x="346" y="136" class="s">세션 id 로 조회</text>
<text x="346" y="154" class="s">로그아웃 시 정리됨</text>
<rect x="600" y="72" width="230" height="90" class="warn"/>
<text x="616" y="94" class="tb">PostgreSQL</text>
<text x="616" y="116" class="s">OAuth2AuthorizedClient</text>
<text x="616" y="136" class="s">principal 이름으로 조회</text>
<text x="616" y="154" class="s">로그아웃해도 남는다</text>
<path d="M272 108 L326 108" class="g" marker-end="url(#ag)"/>
<path d="M562 108 L596 108" class="g" marker-end="url(#ag)"/>
<rect x="30" y="188" width="390" height="100" class="ok"/>
<text x="46" y="210" class="tb">통과</text>
<text x="46" y="234" class="t">① 다른 인스턴스로 요청해도 된다</text>
<text x="46" y="256" class="t">② 재시작 후에도 로그인 유지</text>
<text x="46" y="278" class="s">저장소를 밖으로 뺐으니 당연한 결과다</text>
<rect x="440" y="188" width="390" height="100" class="dead"/>
<text x="456" y="210" class="tb">실패</text>
<text x="456" y="234" class="t" fill="#cf222e">③ 같은 사용자의 다른 브라우저가 덮어쓴다</text>
<text x="456" y="256" class="t" fill="#cf222e">④ 로그아웃해도 한쪽만 정리된다</text>
<text x="456" y="278" class="s">둘 다 저장소 선택의 문제가 아니라 스키마의 문제다</text>
<rect x="30" y="308" width="800" height="70" class="warn"/>
<text x="46" y="330" class="tb">★ 뿌리는 DDL 한 줄이다</text>
<text x="46" y="352" class="t">PRIMARY KEY (client_registration_id, principal_name)</text>
<text x="46" y="370" class="s">세션 id 가 키에 없다 → 같은 사용자의 두 세션이 같은 행을 쓴다 → 나중 로그인이 앞의 토큰을 덮어쓴다</text>
<rect x="30" y="392" width="800" height="40" class="dead"/>
<text x="46" y="416" class="s">로그아웃 후: Redis 세션 0 키 (정리됨) · PostgreSQL 토큰 1 행 — 평문 refresh token 이 그대로 남는다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.7 KiB

docs/diagrams/b3-refresh-contention.svg
+29
View File
@@ -0,0 +1,29 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 700 300" width="700" height="300" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
.bad{stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;fill:none;marker-end:url(#b)}
</style>
<defs>
<marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker>
<marker id="b" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#cf222e"/></marker>
</defs>
<text class="h" x="16" y="24">B-3 · 동시 refresh — 경쟁이 아니라 세션 파괴</text>
<rect class="box" x="30" y="46" width="150" height="40"/><text class="t" x="105" y="71" text-anchor="middle">같은 refresh token ×5</text>
<path class="ln" d="M180,66 L280,66"/>
<rect class="box" x="280" y="42" width="180" height="48"/><text class="t" x="370" y="62" text-anchor="middle">Keycloak</text>
<text class="s" x="370" y="80" text-anchor="middle">rotation ON · maxReuse=0</text>
<path class="ln" d="M460,58 L560,58"/><rect class="box" x="560" y="42" width="110" height="24"/><text class="s" x="615" y="58" text-anchor="middle">1× HTTP 200</text>
<path class="bad" d="M460,78 L560,78"/><rect class="dead" x="560" y="66" width="110" height="24"/><text class="s" x="615" y="82" text-anchor="middle">4× HTTP 400</text>
<rect class="dead" x="120" y="120" width="460" height="70"/>
<text class="r" x="350" y="144" text-anchor="middle">재사용 탐지가 client session 을 제거한다</text>
<text class="s" x="350" y="164" text-anchor="middle">user_session 은 남고 client_session = 0 (정상 세션은 1)</text>
<text class="s" x="350" y="180" text-anchor="middle">그래서 오류가 "Session doesn't have required client"</text>
<rect class="dead" x="120" y="210" width="460" height="60"/>
<text class="r" x="350" y="234" text-anchor="middle">★ 이긴 요청의 새 토큰도 곧바로 400</text>
<text class="s" x="350" y="254" text-anchor="middle">재시도로 회복 불가 → Q2 의 판정은 lock</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.5 KiB

docs/diagrams/b4-header-forgery.svg
+37
View File
@@ -0,0 +1,37 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 700 290" width="700" height="290" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}.g{font-size:12px;fill:#1a7f37;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
.bad{stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;fill:none;marker-end:url(#b)}
</style>
<defs>
<marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker>
<marker id="b" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#cf222e"/></marker>
</defs>
<text class="h" x="16" y="24">B-4 · edge 가 설정하지 않은 헤더는 그대로 통과한다</text>
<rect class="box" x="30" y="46" width="130" height="60"/><text class="t" x="95" y="68" text-anchor="middle">공격자</text>
<text class="s" x="95" y="86" text-anchor="middle">X-Auth-Request-</text><text class="s" x="95" y="100" text-anchor="middle">Roles: admin</text>
<path class="ln" d="M160,76 L250,76"/>
<rect class="box" x="250" y="40" width="180" height="72"/><text class="t" x="340" y="62" text-anchor="middle">nginx</text>
<text class="s" x="340" y="82" text-anchor="middle">proxy_set_header 한 것만 덮어쓴다</text>
<text class="r" x="340" y="100" text-anchor="middle">X-Auth-Request-* 는 설정이 없다</text>
<path class="bad" d="M430,76 L520,76"/>
<rect class="dead" x="520" y="46" width="150" height="60"/>
<text class="r" x="595" y="68" text-anchor="middle">upstream</text>
<text class="s" x="595" y="86" text-anchor="middle">['viewer','admin']</text><text class="s" x="595" y="100" text-anchor="middle">둘 다 도착 · 검증 없음</text>
<rect class="box" x="30" y="134" width="310" height="66"/>
<text class="t" x="185" y="156" text-anchor="middle">구분자 문제</text>
<text class="s" x="185" y="176" text-anchor="middle">"admin,editor" 와 "role-with,comma" 가</text>
<text class="s" x="185" y="192" text-anchor="middle">도착 시점에 구별되지 않는다</text>
<rect class="box" x="360" y="134" width="310" height="66"/>
<text class="t" x="515" y="156" text-anchor="middle">크기는 절벽이다</text>
<text class="s" x="515" y="176" text-anchor="middle">4KB 통과 · 8KB → Tomcat 400</text>
<text class="s" x="515" y="192" text-anchor="middle">16KB → 연결 끊김 (nginx)</text>
<rect class="dead" x="30" y="222" width="640" height="50"/>
<text class="r" x="350" y="244" text-anchor="middle">헤더가 인가 근거가 되면 위조 가능성이 곧 권한 상승이다</text>
<text class="s" x="350" y="262" text-anchor="middle">Q4 의 5문항 중 2·4번 해당 → Q4 자신의 기준으로 BFF 구조</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.1 KiB

docs/diagrams/b5-redis-loss.svg
+35
View File
@@ -0,0 +1,35 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 700 300" width="700" height="300" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}.g{font-size:12px;fill:#1a7f37;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
.bad{stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;fill:none;marker-end:url(#b)}
</style>
<defs>
<marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker>
<marker id="b" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#cf222e"/></marker>
</defs>
<text class="h" x="16" y="24">B-5 · 파드가 Ready 인 채로 계속 실패한다</text>
<rect class="box" x="30" y="46" width="180" height="76"/>
<text class="t" x="120" y="68" text-anchor="middle">bff ×2</text>
<text class="g" x="120" y="90" text-anchor="middle">Ready = true</text>
<text class="s" x="120" y="110" text-anchor="middle">Service 에 그대로 남는다</text>
<path class="bad" d="M210,84 L330,84"/>
<rect class="dead" x="330" y="58" width="160" height="52"/>
<text class="r" x="410" y="80" text-anchor="middle">redis ✗</text><text class="s" x="410" y="98" text-anchor="middle">replicas=0</text>
<rect class="box" x="30" y="146" width="310" height="110"/>
<text class="t" x="185" y="168" text-anchor="middle">health group 이 갈랐다</text>
<text class="r" x="185" y="192" text-anchor="middle">/actuator/health → 503</text>
<text class="g" x="185" y="214" text-anchor="middle">/actuator/health/readiness → 200 UP</text>
<text class="s" x="185" y="238" text-anchor="middle">redis 지표가 readiness 그룹에 없다</text>
<rect class="box" x="360" y="146" width="310" height="110"/>
<text class="t" x="515" y="168" text-anchor="middle">A-2 와 정반대</text>
<text class="s" x="515" y="192" text-anchor="middle">A-2 Keycloak: DB 검사가 readiness 에</text>
<text class="s" x="515" y="208" text-anchor="middle">→ NotReady → 503 (명확)</text>
<text class="s" x="515" y="230" text-anchor="middle">B-5 BFF: 없음 → Ready 유지</text>
<text class="r" x="515" y="248" text-anchor="middle">→ HTTP 000 (멈춤)</text>
<text class="s" x="16" y="284">영속화: 볼륨 없이 AOF 만 켜면 appendonlydir 은 생기지만 파드 삭제로 전부 사라진다 — 볼륨이 먼저다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.8 KiB

docs/diagrams/b6-key-rotation.svg
+34
View File
@@ -0,0 +1,34 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 700 260" width="700" height="260" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}.g{font-size:12px;fill:#1a7f37;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
.bad{stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;fill:none;marker-end:url(#b)}
</style>
<defs>
<marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker>
<marker id="b" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#cf222e"/></marker>
</defs>
<text class="h" x="16" y="24">B-6 · 회전은 안전하고 옛 키를 버리는 순간이 위험하다</text>
<rect class="ok" x="30" y="46" width="190" height="76"/>
<text class="t" x="125" y="68" text-anchor="middle">t0 — 키 A 만</text>
<text class="s" x="125" y="88" text-anchor="middle">발급 A · 검증 A</text>
<text class="s" x="125" y="108" text-anchor="middle">JWKS RS256 1개</text>
<path class="ln" d="M220,84 L255,84"/>
<rect class="ok" x="255" y="46" width="190" height="76"/>
<text class="t" x="350" y="68" text-anchor="middle">t1 — B 추가 (priority 200)</text>
<text class="g" x="350" y="88" text-anchor="middle">발급 B · 검증 A+B</text>
<text class="s" x="350" y="108" text-anchor="middle">옛 토큰 200 · 새 토큰 200</text>
<path class="bad" d="M445,84 L480,84"/>
<rect class="dead" x="480" y="46" width="190" height="76"/>
<text class="t" x="575" y="68" text-anchor="middle">t2 — A 제거</text>
<text class="r" x="575" y="88" text-anchor="middle">옛 토큰 즉시 401</text>
<text class="s" x="575" y="108" text-anchor="middle">캐시가 유예를 주지 않는다</text>
<rect class="box" x="30" y="146" width="640" height="60"/>
<text class="t" x="350" y="168" text-anchor="middle">겹침 구간(t1~t2)의 최소 길이 = 옛 키로 서명된 것 중 가장 오래 사는 것의 수명</text>
<text class="s" x="350" y="190" text-anchor="middle">access token 60초 · refresh token 1800초 → 최소 30분</text>
<text class="s" x="16" y="234">모르는 kid 를 만나면 JWKS 를 다시 받으므로 제거가 즉시 반영된다. 유예는 옛 키를 남겨두는 기간으로 만든다.</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.7 KiB

docs/diagrams/b7-cookie-secret.svg
+52
View File
@@ -0,0 +1,52 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 420" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">B-7 — BFF 와 정반대의 성질: 공유할 상태가 없고, 대신 겹침 구간도 없다</text>
<text x="20" y="44" class="s">oauth2-proxy 는 세션을 쿠키에 담는다</text>
<rect x="30" y="72" width="380" height="120" class="box"/>
<text x="46" y="94" class="tb">replica 는 무엇을 공유하나</text>
<rect x="46" y="106" width="160" height="34" class="box"/>
<text x="58" y="128" class="t">oauth2-proxy A</text>
<rect x="250" y="106" width="146" height="34" class="box"/>
<text x="262" y="128" class="t">oauth2-proxy B</text>
<rect x="106" y="152" width="230" height="30" class="ok"/>
<text x="118" y="172" class="s">같은 k8s Secret 을 읽는다</text>
<path d="M126 142 L180 150" class="g"/>
<path d="M316 142 L262 150" class="g"/>
<text x="46" y="196" class="s"> </text>
<rect x="440" y="72" width="390" height="120" class="ok"/>
<text x="456" y="94" class="tb">그래서 얻는 것</text>
<text x="456" y="118" class="s">공유할 서버 상태가 없다 → 콜백이 다른 replica 로 가도 된다</text>
<text x="456" y="140" class="s">replica 를 늘려도 세션 저장소 고민이 없다</text>
<text x="456" y="164" class="s">BFF 가 Redis·PostgreSQL 로 풀던 문제가 아예 생기지 않는다</text>
<rect x="30" y="212" width="380" height="106" class="dead"/>
<text x="46" y="234" class="tb">★ 대신 겹침 구간을 만들 수 없다</text>
<text x="46" y="258" class="t">--cookie-secret 은 단수다</text>
<text x="46" y="280" class="s">「옛 secret 도 당분간 받아준다」가 불가능하다</text>
<text x="46" y="300" class="s">교체하는 순간 모든 쿠키가 한꺼번에 무효가 된다</text>
<rect x="440" y="212" width="390" height="106" class="warn"/>
<text x="456" y="234" class="tb">교체하면 벌어지는 일</text>
<text x="456" y="258" class="s">옛 쿠키 → session ticket cookie failed validation</text>
<text x="456" y="278" class="s">Keycloak SSO 가 살아 있으면 로그인 화면 없이 조용히 재인증</text>
<text x="456" y="300" class="s" fill="#cf222e">서버 쪽 세션은 고아로 남는다 — 티켓을 못 풀어 지우지도 못한다</text>
<rect x="30" y="338" width="800" height="66" class="ok"/>
<text x="46" y="360" class="tb">→ B-7a 가 이어받았다</text>
<text x="46" y="382" class="s">「지울 수 없다」는 oauth2-proxy 의 한계일 뿐이었다. 운영자는 지울 수 있고, TTL 역산으로 고아만 골라낼 수 있다</text>
<text x="46" y="398" class="s">그리고 고아는 생성 후 정확히 1시간에 사라진다 — TTL 이 요청으로 갱신되지 않기 때문이다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.7 KiB

docs/diagrams/b7a-orphan-lifecycle.svg
+74
View File
@@ -0,0 +1,74 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 500" font-family="-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">B-7a — 고아 세션은 지울 수 있다. oauth2-proxy 가 못 할 뿐이다</text>
<text x="20" y="44" class="s">회전 2회 측정 · Redis + oauth2-proxy 로그 + 브라우저</text>
<text x="20" y="76" class="tb">왜 프록시는 못 지우는가 — 티켓의 구조</text>
<rect x="20" y="88" width="400" height="86" class="box"/>
<text x="34" y="110" class="t">쿠키에 담기는 것 = 티켓</text>
<rect x="36" y="120" width="180" height="30" class="box"/>
<text x="48" y="140" class="s">세션 ID → Redis 키 이름</text>
<rect x="224" y="120" width="180" height="30" class="box"/>
<text x="236" y="140" class="s">암호화 키 → 값 복호화</text>
<text x="34" y="166" class="s">티켓 전체가 cookie secret 으로 암호화되어 있다</text>
<rect x="444" y="88" width="396" height="86" class="dead"/>
<text x="458" y="110" class="tb">secret 을 바꾸면</text>
<text x="458" y="130" class="s">티켓을 못 연다 → 세션 ID 조차 못 읽는다</text>
<text x="458" y="148" class="s">→ 어느 Redis 키를 지울지 모른다</text>
<text x="458" y="166" class="s">Error removing session: error decoding ticket to clear session</text>
<path d="M424 131 L440 131" class="r" marker-end="url(#ar)"/>
<text x="20" y="208" class="tb">회전할 때마다 누적한다</text>
<line x1="70" y1="256" x2="800" y2="256" class="k"/>
<line x1="130" y1="244" x2="130" y2="268" stroke="#cf222e" stroke-width="2"/>
<line x1="430" y1="244" x2="430" y2="268" stroke="#cf222e" stroke-width="2"/>
<text x="92" y="288" class="s">1차 회전 11:29:56</text>
<text x="392" y="288" class="s">2차 회전 11:33:27</text>
<rect x="70" y="224" width="60" height="16" class="ok"/>
<text x="76" y="236" class="s">세션 A</text>
<rect x="130" y="224" width="300" height="16" class="dead"/>
<text x="150" y="236" class="s" fill="#cf222e">A 는 고아 · TTL 만료까지 남는다</text>
<rect x="130" y="224" width="0" height="0"/>
<rect x="150" y="304" width="280" height="16" class="ok"/>
<text x="156" y="316" class="s">세션 B (1차 회전 후 생성)</text>
<rect x="430" y="304" width="370" height="16" class="dead"/>
<text x="450" y="316" class="s" fill="#cf222e">B 도 고아가 된다</text>
<rect x="450" y="224" width="350" height="16" class="ok"/>
<text x="456" y="236" class="s">세션 C (2차 회전 후 생성)</text>
<text x="70" y="338" class="s">회전 1회 = 그 시점 로그인 사용자 수만큼의 고아</text>
<rect x="20" y="356" width="400" height="126" class="dead"/>
<text x="34" y="378" class="tb">Redis 만 보고는 구분할 수 없다</text>
<text x="34" y="400" class="s">이름 _oauth2_proxy-&lt;불투명한 32자 hex&gt; (동일 형식)</text>
<text x="34" y="418" class="s">type string (동일)</text>
<text x="34" y="436" class="s">크기 3510 바이트 (바이트까지 동일)</text>
<text x="34" y="454" class="s">값 암호화됨 (읽을 수 없다)</text>
<text x="34" y="474" class="s" fill="#cf222e">→ 다른 것은 TTL 하나뿐이다</text>
<rect x="444" y="356" width="396" height="126" class="ok"/>
<text x="458" y="378" class="tb">그래서 TTL 이 정리 규칙이 된다</text>
<text x="458" y="398" class="s">--cookie-refresh 가 없어 TTL 이 갱신되지 않는다 (refresh:disabled)</text>
<text x="458" y="416" class="s">→ TTL 은 생성 시각의 정확한 함수다</text>
<rect x="460" y="424" width="364" height="26" class="box"/>
<text x="470" y="442" class="t">생성시각 = 지금 − (cookie-expire − TTL)</text>
<text x="458" y="466" class="s" fill="#1a7f37">이 값이 회전 시각보다 이르면 고아. 검증: 역산 11:30:26 vs 로그 11:30:27 — 1초 오차</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.8 KiB

docs/diagrams/c1-sso-structure.svg
+31
View File
@@ -0,0 +1,31 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 700 300" width="700" height="300" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}.g{font-size:12px;fill:#1a7f37;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
.bad{stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;fill:none;marker-end:url(#b)}
</style>
<defs>
<marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker>
<marker id="b" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#cf222e"/></marker>
</defs>
<text class="h" x="16" y="24">C-1 · SSO 의 구조와 IdP 로그아웃의 한계</text>
<rect class="box" x="220" y="42" width="260" height="52"/>
<text class="t" x="350" y="64" text-anchor="middle">Keycloak user session ×1</text>
<text class="s" x="350" y="82" text-anchor="middle">oqOjHekin4JU-BZjgQLjUByW</text>
<path class="ln" d="M300,96 L180,130"/><path class="ln" d="M400,96 L520,130"/>
<rect class="box" x="60" y="134" width="240" height="52"/>
<text class="t" x="180" y="156" text-anchor="middle">client session — bff-confidential</text>
<text class="s" x="180" y="174" text-anchor="middle">app1 · Redis 세션 + PostgreSQL 토큰</text>
<rect class="box" x="400" y="134" width="240" height="52"/>
<text class="t" x="520" y="156" text-anchor="middle">client session — oauth2-proxy</text>
<text class="s" x="520" y="174" text-anchor="middle">app2 · 쿠키 티켓 + Redis 세션</text>
<rect class="dead" x="220" y="206" width="260" height="40"/>
<text class="r" x="350" y="231" text-anchor="middle">IdP 세션 삭제 ✗</text>
<path class="bad" d="M300,246 L200,262"/><path class="bad" d="M400,246 L500,262"/>
<rect class="ok" x="60" y="256" width="240" height="34"/><text class="g" x="180" y="278" text-anchor="middle">app1 그대로 동작</text>
<rect class="ok" x="400" y="256" width="240" height="34"/><text class="g" x="520" y="278" text-anchor="middle">app2 그대로 동작</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.4 KiB

docs/diagrams/c2-backchannel-logout.svg
+44
View File
@@ -0,0 +1,44 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 400" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">C-2 — 로그아웃이 안 퍼지는 이유는 단순했다. 아무도 구현하지 않았다</text>
<text x="20" y="44" class="s">C-1 이 관측한 현상의 원인</text>
<text x="20" y="76" class="tb">백채널 로그아웃이 동작하려면 양쪽이 다 있어야 한다</text>
<rect x="30" y="94" width="380" height="110" class="dead"/>
<text x="46" y="116" class="tb">① IdP 쪽 — Keycloak 클라이언트 설정</text>
<text x="46" y="140" class="t">backchannelLogoutUrl</text>
<text x="46" y="162" class="t" fill="#cf222e">두 클라이언트 모두 비어 있다</text>
<text x="46" y="186" class="s">Keycloak 은 부를 주소를 모른다</text>
<rect x="440" y="94" width="390" height="110" class="dead"/>
<text x="456" y="116" class="tb">② 앱 쪽 — 수신 엔드포인트</text>
<text x="456" y="140" class="t">/logout/connect/back-channel/{registrationId}</text>
<text x="456" y="162" class="t" fill="#cf222e">BFF 소스에 oidcLogout 설정이 없다</text>
<text x="456" y="186" class="s">주소를 알려줘도 받을 곳이 없다</text>
<rect x="30" y="228" width="800" height="60" class="warn"/>
<text x="46" y="250" class="tb">★ 그래서 IdP 쪽만 설정하면 되는 줄 알면 틀린다</text>
<text x="46" y="272" class="s">backchannelLogoutUrl 을 넣어 봤지만 앱 세션은 그대로 남았다. 받는 쪽이 없기 때문이다</text>
<rect x="30" y="308" width="380" height="72" class="ok"/>
<text x="46" y="330" class="tb">네트워크 문제가 아니라는 확인</text>
<text x="46" y="352" class="s">Keycloak 파드에서 앱 URL 로 요청 → HTTP 200</text>
<text x="46" y="372" class="s">닿기는 한다. 닿아도 처리할 코드가 없을 뿐이다</text>
<rect x="440" y="308" width="390" height="72" class="box"/>
<text x="456" y="330" class="tb">남기는 것</text>
<text x="456" y="352" class="s">「설정이 빠졌다」와 「기능이 없다」는 다르게 고쳐야 한다</text>
<text x="456" y="372" class="s">여기는 둘 다였고, 확인 순서를 바꿨다면 한쪽만 고치고 끝냈을 것이다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.2 KiB

docs/diagrams/d1-backup-restore.svg
+30
View File
@@ -0,0 +1,30 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 700 270" width="700" height="270" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}.g{font-size:12px;fill:#1a7f37;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
.bad{stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;fill:none;marker-end:url(#b)}
</style>
<defs>
<marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker>
<marker id="b" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#cf222e"/></marker>
</defs>
<text class="h" x="16" y="24">D-1 · 빈 데이터베이스가 200 을 냈다</text>
<rect class="box" x="30" y="46" width="180" height="46"/><text class="t" x="120" y="66" text-anchor="middle">pg_dump</text><text class="s" x="120" y="82" text-anchor="middle">395KB · 101 테이블 · 세션 포함</text>
<path class="ln" d="M210,69 L270,69"/>
<rect class="dead" x="270" y="42" width="180" height="54"/>
<text class="r" x="360" y="64" text-anchor="middle">DROP SCHEMA CASCADE</text><text class="s" x="360" y="84" text-anchor="middle">남은 테이블 0</text>
<path class="ln" d="M450,69 L510,69"/>
<rect class="ok" x="510" y="42" width="160" height="54"/>
<text class="g" x="590" y="64" text-anchor="middle">복구 1초</text><text class="s" x="590" y="84" text-anchor="middle">오류 0건 · 재시작 0회</text>
<rect class="box" x="30" y="118" width="640" height="76"/>
<text class="t" x="350" y="140" text-anchor="middle">테이블이 0개일 때 무엇이 깨졌는가 — 전부가 아니다</text>
<text class="g" x="350" y="162" text-anchor="middle">/protocol/openid-connect/certs → 200 (realm 키가 캐시에 있다)</text>
<text class="r" x="350" y="182" text-anchor="middle">/.well-known → 500 토큰 발급 → 400</text>
<rect class="dead" x="30" y="210" width="640" height="46"/>
<text class="r" x="350" y="232" text-anchor="middle">헬스체크는 "DB 가 살아 있다"만 보고 "데이터가 있다"는 안 본다</text>
<text class="s" x="350" y="250" text-anchor="middle">RPO = 백업 주기 + A-3 의 synchronous_commit 손실 · 덤프는 같은 호스트 /tmp 에 있었다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.6 KiB

docs/diagrams/d2-version-upgrade.svg
+48
View File
@@ -0,0 +1,48 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 420" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">D-2 — 롤백은 안 된다. 다만 롤링 업데이트가 사고를 절반에서 멈춰줬다</text>
<text x="20" y="44" class="s">그리고 이 결론은 조건부였다 — 스키마가 안 바뀌면 롤백된다(후속에서 정밀화)</text>
<rect x="30" y="72" width="380" height="126" class="box"/>
<text x="46" y="94" class="tb">앞으로 (26.7.0 → 26.7.3)</text>
<text x="46" y="118" class="t" fill="#1a7f37">된다. 무중단</text>
<text x="46" y="140" class="s">87회 요청 전부 200</text>
<text x="46" y="160" class="s">databasechangelog 210 → 210 (스키마 변경 없음)</text>
<text x="46" y="180" class="s">세션 유지 · Infinispan 16.0.12 → 16.0.14</text>
<rect x="440" y="72" width="390" height="126" class="dead"/>
<text x="456" y="94" class="tb">뒤로 (스키마가 이미 움직였을 때)</text>
<text x="456" y="118" class="t" fill="#cf222e">안 된다</text>
<text x="456" y="140" class="s">liquibase ValidationFailedException:</text>
<text x="456" y="158" class="s">1 changesets check sum</text>
<text x="456" y="180" class="s">새 버전이 남긴 체크섬을 옛 버전이 거부한다</text>
<rect x="30" y="222" width="800" height="72" class="ok"/>
<text x="46" y="244" class="tb">★ 그런데 서비스는 살아 있었다 — StatefulSet 롤링 업데이트 덕분이다</text>
<text x="46" y="266" class="s">한 번에 한 파드씩 바꾼다. 첫 파드가 기동에 실패하면 거기서 멈추고 나머지는 건드리지 않는다</text>
<text x="46" y="286" class="s">그래서 남은 파드가 외부 200 을 계속 냈다. 「롤백 계획」이 없어도 사고가 전면화되지 않았다</text>
<rect x="30" y="314" width="380" height="90" class="warn"/>
<text x="46" y="336" class="tb">그러면 무엇으로 판단하나</text>
<text x="46" y="358" class="t">select count(*) from databasechangelog</text>
<text x="46" y="380" class="s">업그레이드 전후 이 수가 같으면 롤백 가능</text>
<text x="46" y="398" class="s">늘었으면 스키마가 움직였다 = 롤백 불가</text>
<rect x="440" y="314" width="390" height="90" class="box"/>
<text x="456" y="336" class="tb">전제 — 백업이 먼저다 (D-1)</text>
<text x="456" y="358" class="s">롤백이 막히는 상황에서 되돌릴 방법은 덤프 복원뿐이다</text>
<text x="456" y="380" class="s">D-1 이 그 절차를 재고, D-2 가 그 절차가 필요한 이유를 만든다</text>
<text x="456" y="398" class="s">순서를 바꾸면 되돌릴 수 없는 상태에서 백업을 배우게 된다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.6 KiB

docs/diagrams/d3-secret-exposure.svg
+31
View File
@@ -0,0 +1,31 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 700 280" width="700" height="280" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}.g{font-size:12px;fill:#1a7f37;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
.bad{stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;fill:none;marker-end:url(#b)}
</style>
<defs>
<marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker>
<marker id="b" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#cf222e"/></marker>
</defs>
<text class="h" x="16" y="24">D-3 · 네 경로 중 RBAC 만 실제로 감춘다</text>
<rect class="dead" x="30" y="46" width="310" height="60"/>
<text class="r" x="185" y="68" text-anchor="middle">kubectl get -o jsonpath | base64 -d</text>
<text class="s" x="185" y="88" text-anchor="middle">POSTGRES_PASSWORD = lab-postgres-change-me</text>
<rect class="dead" x="360" y="46" width="310" height="60"/>
<text class="r" x="515" y="68" text-anchor="middle">저장소 (at rest)</text>
<text class="s" x="515" y="88" text-anchor="middle">Encryption Disabled · state.db 에 평문</text>
<rect class="dead" x="30" y="120" width="310" height="60"/>
<text class="r" x="185" y="142" text-anchor="middle">파드 안</text>
<text class="s" x="185" y="162" text-anchor="middle">KEYCLOAK_CLIENT_SECRET=... 환경변수</text>
<rect class="ok" x="360" y="120" width="310" height="60"/>
<text class="g" x="515" y="142" text-anchor="middle">RBAC</text>
<text class="s" x="515" y="162" text-anchor="middle">default SA 는 get secrets 불가</text>
<rect class="box" x="30" y="200" width="640" height="60"/>
<text class="t" x="350" y="222" text-anchor="middle">describe 는 "14 bytes" 만 보여줘 감춰졌다는 착각을 준다</text>
<text class="s" x="350" y="244" text-anchor="middle">base64 는 감추기 위한 것이 아니라 YAML 에 임의 바이트를 담기 위한 인코딩이다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.4 KiB

docs/diagrams/d4-cert-chain.svg
+30
View File
@@ -0,0 +1,30 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 700 250" width="700" height="250" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.r{font-size:12px;fill:#cf222e;font-weight:600}.g{font-size:12px;fill:#1a7f37;font-weight:600}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
.bad{stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;fill:none;marker-end:url(#b)}
</style>
<defs>
<marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker>
<marker id="b" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#cf222e"/></marker>
</defs>
<text class="h" x="16" y="24">D-4 · 인증서 체인과 SAN 제약</text>
<rect class="ok" x="30" y="46" width="300" height="104"/>
<text class="t" x="180" y="68" text-anchor="middle">체인 4단계 · Verify return code: 0</text>
<text class="s" x="180" y="90" text-anchor="middle">0 CN=auth.hyeonworks.com</text>
<text class="s" x="180" y="106" text-anchor="middle">1 Let's Encrypt YE2</text>
<text class="s" x="180" y="122" text-anchor="middle">2 ISRG Root YE</text>
<text class="s" x="180" y="138" text-anchor="middle">3 ISRG Root X2</text>
<rect class="box" x="360" y="46" width="310" height="104"/>
<text class="t" x="515" y="68" text-anchor="middle">SAN 3개 · 와일드카드 아님</text>
<text class="s" x="515" y="90" text-anchor="middle">auth · app1 · app2</text>
<text class="r" x="515" y="114" text-anchor="middle">네 번째 이름이 없다</text>
<text class="s" x="515" y="134" text-anchor="middle">B-7 에서 Grafana 의 app2 를 빌려야 했다</text>
<rect class="box" x="30" y="168" width="640" height="60"/>
<text class="t" x="350" y="190" text-anchor="middle">단계가 1개면 cert.pem, 2개 이상이면 fullchain.pem 이다</text>
<text class="s" x="350" y="212" text-anchor="middle">브라우저는 중간 인증서를 캐시하므로 cert.pem 실수는 캐시 없는 클라이언트에서만 드러난다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.4 KiB

docs/diagrams/d4-renewal-gap.svg
+94
View File
@@ -0,0 +1,94 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 520" font-family="-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">D-4 — 「갱신 성공」과 「새 인증서 서빙」 사이의 38분 25초</text>
<text x="20" y="44" class="s">측정: 2026-09-04 · 일련번호 5초 간격 564표본 · 새 연결 0.2초 간격 8856건</text>
<!-- 좌: 디스크 -->
<rect x="20" y="70" width="250" height="196" class="box"/>
<text x="34" y="92" class="tb">디스크 — certbot</text>
<rect x="36" y="104" width="218" height="46" class="box"/>
<text x="48" y="122" class="t">cert1.pem</text>
<text x="48" y="138" class="s">09-03 10:47:40 · notAfter Dec 2</text>
<rect x="36" y="158" width="218" height="46" class="ok"/>
<text x="48" y="176" class="t">cert2.pem</text>
<text x="48" y="192" class="s">09-04 17:22:13 · notAfter Dec 3</text>
<text x="36" y="224" class="s">certbot renew --force-renewal</text>
<text x="36" y="240" class="s">→ 성공. live/ 링크가 2번을 가리킨다</text>
<text x="36" y="256" class="s">경로는 그대로, 내용만 바뀐다</text>
<!-- 중: 끊긴 고리 -->
<rect x="300" y="96" width="260" height="144" class="dead"/>
<text x="316" y="118" class="tb">★ 잇는 것이 없다</text>
<text x="316" y="140" class="t">1. ExecStartPost</text>
<text x="470" y="140" class="t" fill="#cf222e">없음</text>
<text x="316" y="160" class="t">2. renewal-hooks/deploy/</text>
<text x="470" y="160" class="t" fill="#cf222e">비었음</text>
<text x="316" y="180" class="t">3. certbot nginx 플러그인</text>
<text x="470" y="180" class="t" fill="#cf222e">없음</text>
<text x="316" y="206" class="s">post/ · pre/ 도 비어 있다 (total 8)</text>
<text x="316" y="222" class="s">셋 다 비면 아무도 reload 를 부르지 않는다</text>
<path d="M270 168 L296 168" class="r" marker-end="url(#ar)"/>
<path d="M300 230 L560 108" stroke="#cf222e" stroke-width="1.2" stroke-dasharray="4 3"/>
<path d="M300 108 L560 230" stroke="#cf222e" stroke-width="1.2" stroke-dasharray="4 3"/>
<!-- 우: nginx -->
<rect x="590" y="70" width="250" height="196" class="box"/>
<text x="604" y="92" class="tb">nginx — 메모리</text>
<rect x="606" y="104" width="218" height="60" class="dead"/>
<text x="618" y="124" class="t">기동 시 읽은 cert1</text>
<text x="618" y="140" class="s">master 585 / worker 586</text>
<text x="618" y="156" class="s">둘 다 09-03 19:00:39, 22.4시간째</text>
<text x="606" y="184" class="s">reload 없이는 디스크를 다시 안 본다.</text>
<text x="606" y="200" class="s">reload 하면 마스터는 남고</text>
<text x="606" y="216" class="s">워커만 새로 뜬다 → 워커 PID 로 확인</text>
<text x="606" y="240" class="s">ssl_certificate .../fullchain.pem</text>
<path d="M564 168 L586 168" class="k" marker-end="url(#a)"/>
<!-- 타임라인 -->
<text x="20" y="306" class="tb">밖에서 본 것 — 일련번호 (5초 간격)</text>
<line x1="60" y1="356" x2="800" y2="356" class="k"/>
<line x1="60" y1="350" x2="60" y2="362" class="k"/>
<line x1="300" y1="350" x2="300" y2="362" class="k"/>
<line x1="660" y1="344" x2="660" y2="368" stroke="#1a7f37" stroke-width="2"/>
<line x1="800" y1="350" x2="800" y2="362" class="k"/>
<text x="46" y="382" class="s">08:10:51</text>
<text x="278" y="382" class="s">08:22:13</text>
<text x="620" y="382" class="s">08:58:52</text>
<text x="770" y="382" class="s">09:02</text>
<text x="278" y="398" class="s">디스크 기록</text>
<text x="618" y="398" class="s">nginx -s reload</text>
<rect x="60" y="330" width="600" height="16" class="dead"/>
<text x="300" y="342" class="s" text-anchor="middle" fill="#cf222e">0520BB…B853 (옛 인증서) — 이 구간에서 428회 관측</text>
<rect x="660" y="330" width="140" height="16" class="ok"/>
<text x="730" y="342" class="s" text-anchor="middle" fill="#1a7f37">06C7CB…EA1D</text>
<path d="M300 416 L660 416" class="r" marker-end="url(#ar)"/>
<path d="M660 416 L300 416" class="r" marker-end="url(#ar)"/>
<text x="480" y="410" class="tb" text-anchor="middle" fill="#cf222e">38분 25초</text>
<text x="480" y="434" class="s" text-anchor="middle">사람이 reload 했기에 36분이다. 아무도 안 했다면 다음 재시작까지 — 무기한</text>
<!-- reload 판정 -->
<rect x="20" y="452" width="400" height="52" class="ok"/>
<text x="34" y="472" class="tb">reload 는 무중단이었다</text>
<text x="34" y="490" class="s">새 연결 8856건 전부 200 · p95 205.7 → 204.3ms (변화 없음)</text>
<rect x="440" y="452" width="400" height="52" class="ok"/>
<text x="454" y="472" class="tb">진행 중이던 요청도 살아남았다</text>
<text x="454" y="490" class="s">전송 12초째에 reload · 845361바이트 전량 · 연결수 1</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 5.6 KiB

docs/diagrams/d4a-hook-timeline.svg
+57
View File
@@ -0,0 +1,57 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 430" font-family="-apple-system,BlinkMacSystemFont,'Segoe UI',Helvetica,sans-serif">
<style>
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.dead{fill:#fff5f5;stroke:#cf222e;stroke-width:1.6;stroke-dasharray:5 3;rx:6}
.ok{fill:#f6fdf6;stroke:#1a7f37;stroke-width:1.6;rx:6}
.warn{fill:#fff8f0;stroke:#bf8700;stroke-width:1.5;rx:6}
.t{font-size:12px;fill:#1f2328}
.tb{font-size:12.5px;fill:#1f2328;font-weight:600}
.s{font-size:10.5px;fill:#656d76}
.ttl{font-size:14px;fill:#1f2328;font-weight:600}
.r{stroke:#cf222e;stroke-width:1.6;fill:none}
.g{stroke:#1a7f37;stroke-width:1.6;fill:none}
.k{stroke:#8c959f;stroke-width:1.3;fill:none}
</style>
<defs>
<marker id="a" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#8c959f"/></marker>
<marker id="ar" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#cf222e"/></marker>
<marker id="ag" markerWidth="8" markerHeight="8" refX="7" refY="3" orient="auto"><path d="M0,0 L7,3 L0,6 z" fill="#1a7f37"/></marker>
</defs>
<text x="20" y="26" class="ttl">D-4a — deploy 훅 하나로 38분 25초가 1~2초가 된다</text>
<text x="20" y="44" class="s">시각은 전부 실제 UTC (test-server 시계에서 106초 보정)</text>
<rect x="20" y="68" width="820" height="112" class="dead"/>
<text x="34" y="90" class="tb">D-4 — 훅이 없을 때</text>
<line x1="60" y1="130" x2="800" y2="130" class="k"/>
<line x1="90" y1="120" x2="90" y2="140" stroke="#cf222e" stroke-width="2"/>
<line x1="740" y1="120" x2="740" y2="140" stroke="#1a7f37" stroke-width="2"/>
<rect x="90" y="122" width="650" height="16" class="dead"/>
<text x="415" y="134" class="s" text-anchor="middle" fill="#cf222e">옛 인증서를 계속 서빙 — 428회 관측</text>
<text x="62" y="158" class="s">08:20:27 디스크 기록</text>
<text x="660" y="158" class="s">08:58:52 사람이 nginx -s reload</text>
<text x="415" y="112" class="tb" text-anchor="middle" fill="#cf222e">2305초 = 38분 25초</text>
<text x="34" y="174" class="s">아무도 reload 하지 않았다면 다음 nginx 재시작까지 = 사실상 무기한</text>
<rect x="20" y="196" width="820" height="112" class="ok"/>
<text x="34" y="218" class="tb">D-4a — deploy 훅을 넣었을 때</text>
<line x1="60" y1="258" x2="800" y2="258" class="k"/>
<line x1="90" y1="248" x2="90" y2="268" stroke="#1a7f37" stroke-width="2"/>
<line x1="104" y1="248" x2="104" y2="268" stroke="#1a7f37" stroke-width="2"/>
<rect x="90" y="250" width="14" height="16" class="ok"/>
<text x="62" y="286" class="s">12:27:49.05 발급 (SCT — CT 로그의 독립 시계)</text>
<text x="360" y="286" class="s">12:27:50 nginx -t · 새 워커 37252 · 12:27:51 reload</text>
<text x="150" y="244" class="tb" fill="#1a7f37">1~2초, 자동</text>
<text x="34" y="302" class="s">certbot 이 파일을 쓴 직후 같은 실행 안에서 deploy 훅을 부른다 — 사람이 개입할 자리가 없다</text>
<rect x="20" y="324" width="400" height="92" class="box"/>
<text x="34" y="346" class="tb">판정은 문구가 아니라 워커 PID 로</text>
<text x="34" y="366" class="s">master 585 → 585 (유지)</text>
<text x="34" y="384" class="s">worker 28829 → 37252 (교체 = reload 됨)</text>
<text x="34" y="404" class="s">nginx 는 인증서를 기동 시 읽어 메모리에 든다</text>
<rect x="444" y="324" width="396" height="92" class="warn"/>
<text x="458" y="346" class="tb">★ 함정 — 성공을 실패로 오독한다</text>
<text x="458" y="366" class="s">Hook 'deploy-hook' ran with error output:</text>
<text x="458" y="384" class="s"> [warn] could not build optimal types_hash …</text>
<text x="458" y="402" class="s">nginx 경고가 stderr 로 나갔을 뿐. 내용은 test is successful</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.8 KiB

docs/diagrams/lab-topology.svg
+57
View File
@@ -0,0 +1,57 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 760 470" width="760" height="470" font-family="system-ui,-apple-system,Segoe UI,Roboto,sans-serif">
<style>
.t{font-size:13px;fill:#1f2328}.s{font-size:11px;fill:#59636e}.h{font-size:14px;font-weight:600;fill:#1f2328}
.box{fill:#fff;stroke:#8c959f;stroke-width:1.4;rx:6}
.host{fill:#f6f8fa;stroke:#59636e;stroke-width:1.6;rx:8}
.node{fill:#fff;stroke:#0969da;stroke-width:1.6;rx:8}
.ln{stroke:#59636e;stroke-width:1.4;fill:none;marker-end:url(#a)}
</style>
<defs><marker id="a" markerWidth="9" markerHeight="9" refX="8" refY="4.5" orient="auto"><path d="M0,0 L9,4.5 L0,9 z" fill="#59636e"/></marker></defs>
<rect class="box" x="290" y="12" width="180" height="38"/>
<text class="t" x="380" y="30" text-anchor="middle">개발 노트북</text>
<text class="s" x="380" y="44" text-anchor="middle">브라우저 · kubectl · Playwright</text>
<path class="ln" d="M380,52 L380,80"/>
<text class="s" x="392" y="70">https · tailnet 100.x · split DNS</text>
<rect class="host" x="40" y="84" width="680" height="66"/>
<text class="h" x="56" y="106">test-server</text>
<text class="s" x="56" y="122">Arch Linux · 12GB · WiFi only · sudo 는 비밀번호 필요</text>
<rect class="box" x="430" y="94" width="270" height="46"/>
<text class="t" x="565" y="112" text-anchor="middle">nginx :443 — TLS 종료</text>
<text class="s" x="565" y="128" text-anchor="middle">auth / app1 / app2 (SAN 3개, 와일드카드 아님)</text>
<path class="ln" d="M240,152 L200,186"/><path class="ln" d="M520,152 L560,186"/>
<text class="s" x="330" y="172" text-anchor="middle">http · libvirt NAT (virbr0)</text>
<rect class="node" x="40" y="190" width="320" height="250"/>
<text class="h" x="56" y="212">kc-lab-1 · 5120MB</text>
<text class="s" x="56" y="228">k3s server · 10.42.0.0/24</text>
<rect class="box" x="56" y="238" width="130" height="26"/><text class="t" x="121" y="255" text-anchor="middle">traefik ×1</text>
<rect class="box" x="196" y="238" width="148" height="26"/><text class="t" x="270" y="255" text-anchor="middle">coredns</text>
<rect class="box" x="56" y="272" width="130" height="26"/><text class="t" x="121" y="289" text-anchor="middle">keycloak-1</text>
<rect class="box" x="196" y="272" width="148" height="26"/><text class="t" x="270" y="289" text-anchor="middle">bff (1/2)</text>
<rect class="box" x="56" y="306" width="288" height="26"/><text class="t" x="200" y="323" text-anchor="middle">oauth2-proxy (1/2)</text>
<rect class="box" x="56" y="340" width="288" height="46"/>
<text class="t" x="200" y="358" text-anchor="middle">prometheus (PVC) · grafana</text>
<text class="s" x="200" y="374" text-anchor="middle">관측 스택은 여기 고정 — 죽이지 않는다</text>
<rect class="node" x="400" y="190" width="320" height="250"/>
<text class="h" x="416" y="212">kc-lab-2 · 4096MB</text>
<text class="s" x="416" y="228">k3s agent · 10.42.1.0/24</text>
<rect class="box" x="416" y="238" width="288" height="26"/><text class="t" x="560" y="255" text-anchor="middle">keycloak-0</text>
<rect class="box" x="416" y="272" width="140" height="26"/><text class="t" x="486" y="289" text-anchor="middle">bff (2/2)</text>
<rect class="box" x="566" y="272" width="138" height="26"/><text class="t" x="635" y="289" text-anchor="middle">oauth2-proxy</text>
<rect class="box" x="416" y="306" width="140" height="46"/>
<text class="t" x="486" y="324" text-anchor="middle">postgres</text><text class="s" x="486" y="340" text-anchor="middle">PVC (노드 고정)</text>
<rect class="box" x="566" y="306" width="138" height="46"/>
<text class="t" x="635" y="324" text-anchor="middle">redis</text><text class="s" x="635" y="340" text-anchor="middle">PVC + AOF</text>
<text class="s" x="560" y="374" text-anchor="middle">장애 주입은 여기</text>
<path d="M360,290 L400,290" stroke="#0969da" stroke-width="1.6" fill="none" marker-end="url(#a)"/>
<path d="M400,300 L360,300" stroke="#0969da" stroke-width="1.6" fill="none" marker-end="url(#a)"/>
<text class="s" x="380" y="284" text-anchor="middle">7800</text>
<text class="s" x="380" y="318" text-anchor="middle">JGroups</text>
<text class="s" x="40" y="460">A-0 에서 확인: 세션은 이 7800 이 아니라 postgres 를 통해 공유된다</text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.3 KiB

docs/evidence/a3-database-crash/08-wal-settings.txt
+9
View File
@@ -0,0 +1,9 @@
=== A-3 이 가정만 하고 재지 않은 값 ===
name | setting | unit | source
------------------------+---------+------+---------
commit_delay | 0 | | default
synchronous_commit | on | | default
wal_writer_delay | 200 | ms | default
wal_writer_flush_after | 128 | 8kB | default
(4 rows)
docs/evidence/a5-asymmetric-partition/02-injection-verify.txt
+19
View File
@@ -0,0 +1,19 @@
# 주의 — 이 파일은 원 실험 시점에 0바이트로 저장됐다.
# 리다이렉션이 stdout 만 받았는데 출력이 stderr 로 갔거나 tee 앞 파이프가
# 비어 있었던 것으로 보인다. README 는 그 사이 파일 내용을 서술하고 있었는데,
# 그것은 화면에서 본 것을 적은 것이지 이 파일에서 온 것이 아니었다.
#
# 아래는 사후에 다시 수집한 것이며, 원 시점의 DROP 규칙(0 패킷)은 이미
# 제거되어 재현되지 않는다. 구조적 사실(kube-router 가 자기 체인을 FORWARD
# 최상단에 유지한다)만 확인할 수 있다.
# 원 실험의 결정적 증거는 04-correct-direction.txt 의 패킷 카운터 19/21 이다.
=== A-5 재수집 — filter 테이블 규칙이 CNI 체인에 밀리는 것 ===
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 1690 3386K KUBE-ROUTER-FORWARD 0 -- * * 0.0.0.0/0 0.0.0.0/0 /* kube-router netpol - TEMCG2JMHZYE7H7T */
2 7 612 KUBE-PROXY-FIREWALL 0 -- * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW /* kubernetes load balancer firewall */
3 40 13196 KUBE-FORWARD 0 -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
(원 실험 시점의 규칙은 이미 제거됐다. 아래는 kube-router 가 자기 체인을
FORWARD 최상단에 유지한다는 구조적 사실만 보여준다 — 그것이 실패 원인이었다.)
docs/evidence/a7a-volatile-cause/01-cause-determined.txt
+117
View File
@@ -0,0 +1,117 @@
A-7a — volatile 모드에서 refresh 가 500 인 진짜 이유
=======================================================
수집: 2026-09-04 11:18 ~ 11:24 UTC · kc-lab-1 에서 sudo kubectl
A-7 이 남긴 가설
----------------
> **측정은 확실하지만 원인은 확정하지 못했다.** 유력한 후보는
> `REVOKED_TOKEN` 테이블이다 — refresh token 회전에서 이미 쓴 토큰인지
> 확인하려면 그 테이블을 봐야 하고, 그 경로는 캐시되지 않는다.
★ 이 가설은 틀렸다.
방법 — PostgreSQL 문장 로깅 (A-3 기법)
---------------------------------------
ALTER SYSTEM SET log_statement = 'all'; SELECT pg_reload_conf();
표식을 넣어 로그인과 refresh 가 각각 어떤 SQL 을 쏘는지 구분했다.
select 'MARK_LOGIN_START' ... 로그인 ... select 'MARK_LOGIN_END'
select 'MARK_REFRESH_START' ... refresh ... select 'MARK_REFRESH_END'
volatile 전환 확인
args: ["start","--features-disabled=persistent-user-sessions"]
로그인 200 · offline_user_session 행수 = 0 ← volatile 맞다
[측정 1] 로그인은 SQL 을 0개 쏜다
----------------------------------
11:18:49.461 statement: select 'MARK_LOGIN_START'
11:18:49.743 statement: select 'MARK_LOGIN_END'
↑ 사이에 아무것도 없다
realm·사용자·클라이언트가 전부 Infinispan 캐시에 있어 DB 를 안 본다.
[측정 2] ★ refresh 는 CLIENT_SCOPE_CLIENT 를 본다
--------------------------------------------------
11:18:52.009 statement: select 'MARK_REFRESH_START'
11:18:52.137 statement: BEGIN
11:18:52.137 execute <unnamed>/C_107:
select cscme1_0.SCOPE_ID from CLIENT_SCOPE_CLIENT cscme1_0
where cscme1_0.CLIENT_ID=$1 and cscme1_0.DEFAULT_SCOPE=$2
parameters: $1 = '131a9912-b578-4b9c-b16a-97518704077e', $2 = 'f'
11:18:52.148 execute S_2: COMMIT
11:18:52.253 statement: select 'MARK_REFRESH_END'
REVOKED_TOKEN 은 **한 번도 나오지 않는다.**
`DEFAULT_SCOPE='f'` 이므로 **선택적(optional) 클라이언트 스코프** 조회다.
[측정 3] 그 조회는 한 번뿐이다 — 캐시에 들어간다
-------------------------------------------------
연속 refresh 3회, 전부 200. 표식 사이 SQL:
statement: select 'MARK_R1'
statement: select 'MARK_R2'
statement: select 'MARK_R3'
statement: select 'MARK_R_END'
↑ SQL 0건
첫 refresh 가 캐시를 채우고, 이후로는 DB 를 보지 않는다.
════ 그래서 A-7 의 표 자체가 조건부였다 ════
같은 설정에서 **캐시 온도만으로 결과가 셋으로 갈린다.** 전부 측정했다.
┌──────────────────────┬────────┬─────────┬──────────────────────────────┐
│ 캐시 상태 │ 로그인 │ refresh │ 실패한 SQL │
├──────────────────────┼────────┼─────────┼──────────────────────────────┤
│ 완전 냉시동 │ 400 │ 400 │ select ce1_0.ID from CLIENT │
│ (재시작 직후) │ │ │ where CLIENT_ID=? REALM_ID=?│
├──────────────────────┼────────┼─────────┼──────────────────────────────┤
│ CLIENT 만 더움 │ 200 │ ★ 500 │ select cscme1_0.SCOPE_ID │
│ ← A-7 이 본 상태 │ │ │ from CLIENT_SCOPE_CLIENT │
├──────────────────────┼────────┼─────────┼──────────────────────────────┤
│ 완전히 더움 │ 200 │ 200 │ 없음 (SQL 0건) │
└──────────────────────┴────────┴─────────┴──────────────────────────────┘
[재현 A] 완전 냉시동 — 로그인부터 400
keycloak 재시작 → postgres 정지 → 로그인
로그인 400 {"error":"unauthorized_client",
"error_description":"Unexpected error when authenticating client"}
ERROR [org.keycloak.services] KC-SERVICES0015: Unexpected error when
authenticating client: org.hibernate.exception.GenericJDBCException:
JDBC exception executing SQL [FATAL: terminating connection due to
administrator command]
[select ce1_0.ID from CLIENT ce1_0 where ce1_0.CLIENT_ID=? and ce1_0.REALM_ID=?]
→ A-7 은 "volatile 이면 DB 없이 로그인된다"고 적었지만,
**냉시동에서는 그것도 안 된다.**
[재현 B] ★ A-7 이 본 그 조건 — 로그인 200, refresh 500
keycloak 재시작 → (DB 살아있을 때) 로그인 1회 → postgres 정지 → refresh
로그인 200
refresh 500 {"error":"unknown_error"}
JDBC exception executing SQL [FATAL: terminating connection due to
administrator command]
[select cscme1_0.SCOPE_ID from CLIENT_SCOPE_CLIENT cscme1_0
where cscme1_0.CLIENT_ID=? and cscme1_0.DEFAULT_SCOPE=?]
★ 로그가 실패한 SQL 을 직접 지목한다. CLIENT_SCOPE_CLIENT 다.
[재현 C] 완전히 더움 — 둘 다 200
refresh 를 3회 미리 돌려 캐시를 채운 뒤 postgres 정지
로그인 200
refresh 200 ← A-7 의 표와 정반대다
════ 결론 ════
1. **원인은 REVOKED_TOKEN 이 아니라 선택적 클라이언트 스코프 조회다.**
`CLIENT_SCOPE_CLIENT` 를 `DEFAULT_SCOPE='f'` 로 조회하는 그 한 문장이다.
2. **A-7 의 "volatile → refresh 500" 은 캐시가 차가울 때만 참이다.**
더워지면 200 이 된다. A-7 은 그 조건을 몰랐고, 그래서 표에 적지 않았다.
3. **"volatile 이면 DB 없이 로그인된다" 도 조건부다.**
냉시동에서는 400 이다. 클라이언트 조회조차 캐시에 없기 때문이다.
4. **일반화** — volatile 모드에서 DB 정지 시의 동작은 "무엇을 하느냐"가
아니라 **"그 경로가 이미 캐시를 채웠느냐"** 로 결정된다. 그래서 같은
명령이 재시작 직후와 얼마 쓴 뒤에 다른 답을 낸다. 이런 종류의 결과는
**한 번 재보고 표로 적으면 안 된다** — A-7 이 그렇게 했다.
docs/evidence/a7a-volatile-cause/README.md
+15
View File
@@ -0,0 +1,15 @@
# A-7a — volatile refresh 500 의 원인 확정 증거
2026-09-04 11:18 – 11:24 UTC
해설: [`docs/experiment-a7a-volatile-cause.md`](../../experiment-a7a-volatile-cause.md)
| 파일 | 무엇을 보여주는가 |
|---|---|
| `01-cause-determined.txt` | **가설이 틀렸다.** `REVOKED_TOKEN` 이 아니라 `CLIENT_SCOPE_CLIENT` 조회다. 문장 로깅으로 잡고, 실패 로그가 그 SQL 을 직접 지목한다. 그리고 **같은 설정에서 캐시 온도만으로 400/500/200 셋이 나온다** |
## 핵심 네 줄
1. **원인은 선택적 클라이언트 스코프 조회다.** `select cscme1_0.SCOPE_ID from CLIENT_SCOPE_CLIENT where CLIENT_ID=? and DEFAULT_SCOPE=?` — A-7 이 지목한 `REVOKED_TOKEN` 은 로그에 한 번도 안 나온다.
2. **로그인은 SQL 을 0개 쏜다.** refresh 만 이 한 문장을 쏘고, 그것도 **첫 번째만** 쏜다. 이후로는 캐시된다.
3. **A-7 의 표가 조건부였다.** 완전 냉시동이면 로그인부터 400, 캐시가 완전히 더우면 refresh 도 200. A-7 이 본 「로그인 200 · refresh 500」은 그 사이의 한 상태다 — 셋 다 재현했다.
4. **교훈** — volatile + DB 정지의 동작은 "무엇을 하느냐"가 아니라 **"그 경로가 캐시를 채웠느냐"** 로 갈린다. 한 번 재보고 표로 적으면 안 되는 종류의 측정이다.
docs/evidence/b1-redis-session-store/README.md
+7
View File
@@ -16,3 +16,10 @@
1. **세션은 옮겨졌고 토큰은 안 옮겨졌다.** 빈 81개가 늘었는데 authorized client 관련은 하나도 안 바뀌었다. 1. **세션은 옮겨졌고 토큰은 안 옮겨졌다.** 빈 81개가 늘었는데 authorized client 관련은 하나도 안 바뀌었다.
2. **refresh token 은 Redis 에 평문으로 있는 게 아니라 아예 없다.** 암호화를 고민하기 전에 이걸 알아야 한다. 2. **refresh token 은 Redis 에 평문으로 있는 게 아니라 아예 없다.** 암호화를 고민하기 전에 이걸 알아야 한다.
3. **"로그인은 되어 있는데 아무것도 못 하는" 상태가 만들어진다** — 완전 로그아웃보다 나쁘다. 3. **"로그인은 되어 있는데 아무것도 못 하는" 상태가 만들어진다** — 완전 로그아웃보다 나쁘다.
## 스크린샷 주의
`b1-login-works-two-replicas.png` 과 `b1-token-boundary-after-redis.png` 는
**동일 파일**이며 `b2-before-relogin.png` 와도 같다 (md5 `6de826a7…`).
세 시점 모두 `accessTokenStoredOnServer: false` 인 같은 화면이었다.
**시점 구별은 터미널 출력과 Redis/DB 조회가 한다.**
docs/evidence/b2-multi-instance-session/README.md
+11
View File
@@ -19,3 +19,14 @@
2. **refresh token 은 평문이다.** DB 읽기 권한이면 작동하는 토큰을 얻는다. 2. **refresh token 은 평문이다.** DB 읽기 권한이면 작동하는 토큰을 얻는다.
3. **같은 사용자의 두 번째 로그인이 첫 번째를 덮어쓴다.** 기본키에 session id 가 없어 구조적으로 그렇다. 3. **같은 사용자의 두 번째 로그인이 첫 번째를 덮어쓴다.** 기본키에 session id 가 없어 구조적으로 그렇다.
4. **로그아웃은 셋 중 하나만 지운다.** 평문 토큰과 Keycloak SSO 세션이 남는다. 4. **로그아웃은 셋 중 하나만 지운다.** 평문 토큰과 Keycloak SSO 세션이 남는다.
## 스크린샷 주의
`b2-tokens-shared-across-instances.png` 는 **B-0 의
`b0-bff-token-boundary.png` 와 동일 파일**이다 (md5 `9ed00537…`).
두 시점 모두 `accessTokenStoredOnServer: true` 인 같은 화면이라 바이트가 같다.
**그래서 이 png 는 "JDBC 전환으로 토큰이 공유된다" 를 단독으로 증명하지
못한다.** 그 증명은 `01-jdbc-store-deploy.txt`(테이블 생성)과
`03-plaintext-tokens.txt`(행에 토큰이 들어 있음)가 한다.
`b2-before-relogin.png` 는 B-1 의 캡처와 동일 파일이다.
docs/evidence/b7-cookie-secret/01-deploy.txt
+13
View File
@@ -0,0 +1,13 @@
=== Grafana ingress 를 잠시 내린다 (app2 를 빌린다) ===
grafana ingress 삭제
secret/oauth2-proxy-secrets created
deployment.apps/oauth2-proxy created
service/oauth2-proxy created
ingress.networking.k8s.io/oauth2-proxy created
deployment "oauth2-proxy" successfully rolled out
oauth2-proxy-c76b49c59-8p5hl true kc-lab-1
oauth2-proxy-c76b49c59-b9928 true kc-lab-2
=== 진입점 확인 ===
https://app2.hyeonworks.com/ HTTP 302
/ping HTTP 200

Some files were not shown because too many files have changed in this diff Show More