donghyeon.kang/keycloak-pattern
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Compare commits

merge into: donghyeon.kang/keycloak-pattern:develop-keycloak-session-store
Branches Tags
donghyeon.kang/keycloak-pattern:main donghyeon.kang/keycloak-pattern:feature/keycloak-d4a-deploy-hook donghyeon.kang/keycloak-pattern:feature/keycloak-followup-untested-items donghyeon.kang/keycloak-pattern:feature/keycloak-d4-certificate-renewal donghyeon.kang/keycloak-pattern:feature/keycloak-d3-secret-management donghyeon.kang/keycloak-pattern:feature/keycloak-d2-version-upgrade donghyeon.kang/keycloak-pattern:feature/keycloak-d1-backup-restore donghyeon.kang/keycloak-pattern:feature/keycloak-c2-backchannel-logout donghyeon.kang/keycloak-pattern:feature/keycloak-c1-multi-app-sso donghyeon.kang/keycloak-pattern:feature/keycloak-b7-cookie-secret-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-b6-key-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-b5-redis-loss-persistence donghyeon.kang/keycloak-pattern:feature/keycloak-b4-edge-authorization-scope donghyeon.kang/keycloak-pattern:feature/keycloak-b3-refresh-token-contention donghyeon.kang/keycloak-pattern:feature/keycloak-b2-multi-instance-session donghyeon.kang/keycloak-pattern:feature/keycloak-b1-redis-session-store donghyeon.kang/keycloak-pattern:feature/keycloak-b0-bff-redis-deploy donghyeon.kang/keycloak-pattern:feature/keycloak-a7-volatile-comparison donghyeon.kang/keycloak-pattern:feature/keycloak-a8-rolling-restart donghyeon.kang/keycloak-pattern:feature/keycloak-a6-latency-injection donghyeon.kang/keycloak-pattern:feature/keycloak-a5-asymmetric-partition donghyeon.kang/keycloak-pattern:feature/keycloak-a4-node-loss donghyeon.kang/keycloak-pattern:feature/keycloak-a3-database-crash donghyeon.kang/keycloak-pattern:feature/keycloak-a2-database-loss donghyeon.kang/keycloak-pattern:feature/keycloak-a1-jgroups-transport-block donghyeon.kang/keycloak-pattern:feature/keycloak-multinode-cluster-jdbc-ping donghyeon.kang/keycloak-pattern:develop-keycloak-session-store donghyeon.kang/keycloak-pattern:develop donghyeon.kang/keycloak-pattern:develop-keycloak-pattern1 donghyeon.kang/keycloak-pattern:develop-keycloak-pattern4 donghyeon.kang/keycloak-pattern:feature/keycloak-traefik-forwardauth-alternative donghyeon.kang/keycloak-pattern:feature/keycloak-edge-forwardauth-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-edge-forwardauth-no-google donghyeon.kang/keycloak-pattern:develop-keycloak-pattern3 donghyeon.kang/keycloak-pattern:feature/keycloak-bff-vs-spa-direct donghyeon.kang/keycloak-pattern:feature/keycloak-single-ec2-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-single-ec2-no-google donghyeon.kang/keycloak-pattern:feature/keycloak-three-leg-trust-chain donghyeon.kang/keycloak-pattern:feature/keycloak-internal-spa-direct-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-internal-spa-direct-no-google donghyeon.kang/keycloak-pattern:feature/keycloak-federation-spa-zero-change donghyeon.kang/keycloak-pattern:feature/keycloak-account-linking-spa-ux donghyeon.kang/keycloak-pattern:feature/keycloak-refresh-token-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-pkce-flow-stages donghyeon.kang/keycloak-pattern:feature/keycloak-spring-rs-role-mapping donghyeon.kang/keycloak-pattern:develop-keycloak-pattern2 donghyeon.kang/keycloak-pattern:feature/keycloak-four-pattern-tradeoff-matrix donghyeon.kang/keycloak-pattern:feature/keycloak-public-domain-tunneling donghyeon.kang/keycloak-pattern:feature/keycloak-https-termination-caddy-nginx donghyeon.kang/keycloak-pattern:feature/keycloak-reverse-proxy-headers donghyeon.kang/keycloak-pattern:feature/keycloak-google-redirect-uri-policy donghyeon.kang/keycloak-pattern:feature/keycloak-account-linking-sub-vs-email donghyeon.kang/keycloak-pattern:feature/keycloak-idp-mappers-claim-to-role donghyeon.kang/keycloak-pattern:feature/keycloak-google-claim-attribute-mapping donghyeon.kang/keycloak-pattern:feature/keycloak-first-broker-login-flow donghyeon.kang/keycloak-pattern:feature/keycloak-idp-brokering-google-client donghyeon.kang/keycloak-pattern:feature/keycloak-patterns donghyeon.kang/keycloak-pattern:feature/keycloak-header-spoofing-defense donghyeon.kang/keycloak-pattern:feature/keycloak-nginx-auth-request-integration donghyeon.kang/keycloak-pattern:feature/keycloak-oauth2-proxy-oidc-flow donghyeon.kang/keycloak-pattern:feature/keycloak-bff-csrf-samesite-defense donghyeon.kang/keycloak-pattern:feature/keycloak-bff-oauth2login-session donghyeon.kang/keycloak-pattern:feature/keycloak-token-mediating-access-handoff donghyeon.kang/keycloak-pattern:feature/keycloak-token-mediating-confidential-client donghyeon.kang/keycloak-pattern:feature/keycloak-refresh-rotation-and-logout donghyeon.kang/keycloak-pattern:feature/keycloak-spa-token-storage-tradeoff donghyeon.kang/keycloak-pattern:feature/keycloak-iss-claim-hostname-mismatch donghyeon.kang/keycloak-pattern:feature/keycloak-spring-rs-audience-validator donghyeon.kang/keycloak-pattern:feature/keycloak-vanilla-js-spa-pkce donghyeon.kang/keycloak-pattern:feature/keycloak-realm-client-export donghyeon.kang/keycloak-pattern:feature/keycloak-docker-compose-stack
..
pull from: donghyeon.kang/keycloak-pattern:feature/keycloak-a8-rolling-restart
Branches Tags
donghyeon.kang/keycloak-pattern:feature/keycloak-d4a-deploy-hook donghyeon.kang/keycloak-pattern:feature/keycloak-followup-untested-items donghyeon.kang/keycloak-pattern:feature/keycloak-d4-certificate-renewal donghyeon.kang/keycloak-pattern:feature/keycloak-d3-secret-management donghyeon.kang/keycloak-pattern:feature/keycloak-d2-version-upgrade donghyeon.kang/keycloak-pattern:feature/keycloak-d1-backup-restore donghyeon.kang/keycloak-pattern:feature/keycloak-c2-backchannel-logout donghyeon.kang/keycloak-pattern:feature/keycloak-c1-multi-app-sso donghyeon.kang/keycloak-pattern:feature/keycloak-b7-cookie-secret-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-b6-key-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-b5-redis-loss-persistence donghyeon.kang/keycloak-pattern:feature/keycloak-b4-edge-authorization-scope donghyeon.kang/keycloak-pattern:feature/keycloak-b3-refresh-token-contention donghyeon.kang/keycloak-pattern:feature/keycloak-b2-multi-instance-session donghyeon.kang/keycloak-pattern:feature/keycloak-b1-redis-session-store donghyeon.kang/keycloak-pattern:feature/keycloak-b0-bff-redis-deploy donghyeon.kang/keycloak-pattern:feature/keycloak-a7-volatile-comparison donghyeon.kang/keycloak-pattern:feature/keycloak-a8-rolling-restart donghyeon.kang/keycloak-pattern:feature/keycloak-a6-latency-injection donghyeon.kang/keycloak-pattern:feature/keycloak-a5-asymmetric-partition donghyeon.kang/keycloak-pattern:feature/keycloak-a4-node-loss donghyeon.kang/keycloak-pattern:feature/keycloak-a3-database-crash donghyeon.kang/keycloak-pattern:feature/keycloak-a2-database-loss donghyeon.kang/keycloak-pattern:feature/keycloak-a1-jgroups-transport-block donghyeon.kang/keycloak-pattern:feature/keycloak-multinode-cluster-jdbc-ping donghyeon.kang/keycloak-pattern:develop-keycloak-session-store donghyeon.kang/keycloak-pattern:develop donghyeon.kang/keycloak-pattern:develop-keycloak-pattern1 donghyeon.kang/keycloak-pattern:develop-keycloak-pattern4 donghyeon.kang/keycloak-pattern:feature/keycloak-traefik-forwardauth-alternative donghyeon.kang/keycloak-pattern:feature/keycloak-edge-forwardauth-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-edge-forwardauth-no-google donghyeon.kang/keycloak-pattern:develop-keycloak-pattern3 donghyeon.kang/keycloak-pattern:feature/keycloak-bff-vs-spa-direct donghyeon.kang/keycloak-pattern:feature/keycloak-single-ec2-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-single-ec2-no-google donghyeon.kang/keycloak-pattern:feature/keycloak-three-leg-trust-chain donghyeon.kang/keycloak-pattern:feature/keycloak-internal-spa-direct-google-federation donghyeon.kang/keycloak-pattern:feature/keycloak-internal-spa-direct-no-google donghyeon.kang/keycloak-pattern:feature/keycloak-federation-spa-zero-change donghyeon.kang/keycloak-pattern:feature/keycloak-account-linking-spa-ux donghyeon.kang/keycloak-pattern:feature/keycloak-refresh-token-rotation donghyeon.kang/keycloak-pattern:feature/keycloak-pkce-flow-stages donghyeon.kang/keycloak-pattern:feature/keycloak-spring-rs-role-mapping donghyeon.kang/keycloak-pattern:develop-keycloak-pattern2 donghyeon.kang/keycloak-pattern:feature/keycloak-four-pattern-tradeoff-matrix donghyeon.kang/keycloak-pattern:feature/keycloak-public-domain-tunneling donghyeon.kang/keycloak-pattern:feature/keycloak-https-termination-caddy-nginx donghyeon.kang/keycloak-pattern:feature/keycloak-reverse-proxy-headers donghyeon.kang/keycloak-pattern:feature/keycloak-google-redirect-uri-policy donghyeon.kang/keycloak-pattern:feature/keycloak-account-linking-sub-vs-email donghyeon.kang/keycloak-pattern:feature/keycloak-idp-mappers-claim-to-role donghyeon.kang/keycloak-pattern:feature/keycloak-google-claim-attribute-mapping donghyeon.kang/keycloak-pattern:feature/keycloak-first-broker-login-flow donghyeon.kang/keycloak-pattern:feature/keycloak-idp-brokering-google-client donghyeon.kang/keycloak-pattern:feature/keycloak-patterns donghyeon.kang/keycloak-pattern:feature/keycloak-header-spoofing-defense donghyeon.kang/keycloak-pattern:feature/keycloak-nginx-auth-request-integration donghyeon.kang/keycloak-pattern:feature/keycloak-oauth2-proxy-oidc-flow donghyeon.kang/keycloak-pattern:feature/keycloak-bff-csrf-samesite-defense donghyeon.kang/keycloak-pattern:feature/keycloak-bff-oauth2login-session donghyeon.kang/keycloak-pattern:feature/keycloak-token-mediating-access-handoff donghyeon.kang/keycloak-pattern:feature/keycloak-token-mediating-confidential-client donghyeon.kang/keycloak-pattern:feature/keycloak-refresh-rotation-and-logout donghyeon.kang/keycloak-pattern:feature/keycloak-spa-token-storage-tradeoff donghyeon.kang/keycloak-pattern:feature/keycloak-iss-claim-hostname-mismatch donghyeon.kang/keycloak-pattern:feature/keycloak-spring-rs-audience-validator donghyeon.kang/keycloak-pattern:feature/keycloak-vanilla-js-spa-pkce donghyeon.kang/keycloak-pattern:feature/keycloak-realm-client-export donghyeon.kang/keycloak-pattern:feature/keycloak-docker-compose-stack donghyeon.kang/keycloak-pattern:main
17 Commits
develop-keycloak-session-store .. feature/keycloak-a8-rolling-restart
Author SHA1 Message Date
DongHyeonkaandClaude Opus 5 114d21aebe docs: A-8 — a rolling restart keeps every session and drops only the cache 
Nine samples through the restart all returned 200, the refresh token issued beforehand still works, and the session count is unchanged at 151 while both caches reset to zero. The updated last_session_refresh proves the write path recovered too, not just the response code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 13:20:56 +09:00
DongHyeonkaandClaude Opus 5 dba0c3975c docs: A-6 — 200ms of network delay becomes 22 seconds of user latency 
Nine database round trips per login multiply the injected delay to 1.9 seconds, and connection pool queueing multiplies it again under twenty concurrent requests. The readiness probe joins the same queue and times out, so the node leaves the load balancer and pushes its load onto the one still standing.

Two injections missed first: the guest interface is enp1s0 rather than eth0, and a filter on it can never match a pod IP because flannel has already encapsulated the packet. The delay has to go on flannel.1, before encapsulation.

The predicted rise in optimistic lock conflicts did not happen, because logins insert new rows rather than contending for one. That belongs to B-3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 13:18:10 +09:00
DongHyeonkaandClaude Opus 5 2fa77a899e docs: A-5 — a one-way block heals itself and a full split still leaves one node serving 
Three injections failed first: kube-router keeps reinserting its chain above a hand-placed FORWARD rule, the JGroups connection direction had reversed since A-1, and only the raw table runs ahead of conntrack. Each failure looked like nothing happening.

Blocking one direction never partitioned the cluster because JGroups reconnected the other way before failure detection fired. Blocking both produced a real split brain with two coordinators in JGROUPS_PING, yet only the non-coordinator node reported DOWN, so the Service kept an endpoint and the front door stayed at 200. That answers the question A-1 left open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 13:12:04 +09:00
DongHyeonkaandClaude Opus 5 d0666c5ba0 docs: A-4 — a dead pod reports healthier than a live one 
Kubernetes keeps calling the node Ready for forty seconds while users already see failures, and the pod on the powered-off machine stays ready=true because its kubelet can no longer contradict itself. Eviction waits another five minutes, then the StatefulSet refuses to recreate its pod and the replacement Deployment pod cannot schedule because the local-path volume is pinned to the dead node.

Killing the server node instead shows the opposite shape: containerd keeps the workload running while the API server, Traefik and the observability stack disappear, so the outage is the missing path rather than the missing application. Traefik at one replica is the ingress single point of failure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 12:25:30 +09:00
DongHyeonkaandClaude Opus 5 99b689e715 docs: A-3 — four logins returned tokens for sessions the crash erased 
Keycloak commits the login INSERT with synchronous_commit off, so a crash loses whole sessions and not just refresh timestamps. Measured 4 of 153 lost, matching the default wal_writer_delay window.

Two injections failed silently first: --grace-period=0 --force lets the container runtime send SIGTERM so PostgreSQL flushes and shuts down cleanly, and SIGKILL to PID 1 from inside its own namespace is ignored by the kernel. Killing a backend makes the postmaster reinitialize, which is a real crash recovery.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 12:05:08 +09:00
DongHyeonkaandClaude Opus 5 4177fb6a48 docs: A-2 — losing the database takes every node down while up stays 1 
Both pods go NotReady, the Service endpoint list empties and the front door returns 503, so adding Keycloak replicas buys nothing against database loss. The node holding the session in cache fails too, because a refresh writes LAST_SESSION_REFRESH. Recovery was automatic in about fifteen seconds with no restart, which is what readiness rather than liveness buys.

The observability finding matters as much: up stayed at 1 through a total outage, so alerting on it would have caught nothing. kube-state-metrics is missing and pod readiness is therefore not recorded as a metric.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 11:57:55 +09:00
DongHyeonkaandClaude Opus 5 2a98ef1090 docs: A-1 — sessions survive a JGroups partition but logout invalidation does not 
Cutting TCP 7800 leaves cross-node refresh working (200), confirming sessions travel through PostgreSQL rather than the cluster transport. Logout is the opposite: the database row is deleted but the other node answers from its stale local cache, so the A-0 conclusion that invalidation rides the database is corrected here.

Two things the plan did not anticipate: a NetworkPolicy cannot sever an established connection because conntrack accepts it before policy evaluation, and Keycloak reports the partition through its readiness probe so the split node removes itself from the Service.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 11:52:49 +09:00
DongHyeonkaandClaude Opus 5 9c3cde457e docs: close nine gaps found by re-reading every open question in full 
Cross-checked each question's 남은 미지수, 다음 검증 and 제약 against the plan item by item. Adds B-0 (autoconfiguration actually chosen), B-6 (encryption key rotation) and B-7 (oauth2-proxy cookie secret rotation) as new experiments, plus lock-holder death, rotation-disabled comparison, partial-logout recovery, store latency and the Q4 design checklist. Restores the Redis persistence comparison and records the correct index URL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 11:16:59 +09:00
DongHyeonkaandClaude Opus 5 f1e8c35805 docs: plan all remaining experiments with architecture, injection points and predictions 
Twenty experiments across four layers, each with a topology diagram marking where the fault goes in, the metrics to watch, a falsifiable prediction written before the run, and a pass/fail rule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 10:50:23 +09:00
DongHyeonkaandClaude Opus 5 1de6108157 docs: add the prerequisite knowledge this lab assumes 
Builds up from HTTP statelessness to why session storage location determines the operational response, so the measurements have context to land in.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 10:24:21 +09:00
DongHyeonkaandClaude Opus 5 22d873eb4f docs: capture the SQL the other node actually runs, and correct the replication claim 
PostgreSQL statement logging shows keycloak-1 reading and updating the session created on keycloak-0. The same transaction reveals optimistic locking via VERSION, SKIP LOCKED, and synchronous_commit turned off. Fixes the earlier concept note that credited Infinispan with cross-node propagation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 10:14:45 +09:00
DongHyeonkaandClaude Opus 5 e5ebaeb623 docs: prove sessions are shared by PostgreSQL, not Infinispan replication 
Experiment 0 with three probes: cross-node refresh/logout, cache counter deltas around a single login, and cache entry ownership. Each node caches only what it handled; cache totals sum exactly to the database count.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 09:58:10 +09:00
DongHyeonkaandClaude Opus 5 006da7d490 docs: record observability setup and add concept layers 10-13 
Adds Kubernetes resources (StatefulSet, PVC, Secret, RBAC, placement), Keycloak clustering internals (Infinispan, JGroups), Prometheus concepts and virtualization operations.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 09:21:45 +09:00
DongHyeonkaandClaude Opus 5 d5cc2b55a9 fix: grant nodes/proxy so kubelet metrics can be scraped 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 09:14:51 +09:00
DongHyeonkaandClaude Opus 5 0bb0e0ac49 feat: add Prometheus, node-exporter and Grafana for fault-injection observability 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 09:13:41 +09:00
DongHyeonkaandClaude Opus 5 33878e8880 docs: record multi-node cluster setup, rationale and formation evidence 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-03 17:26:45 +09:00
DongHyeonkaandClaude Opus 5 6dce35ec83 feat: deploy Keycloak multi-node cluster with PostgreSQL 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-03 17:17:25 +09:00
131 changed files with 9796 additions and 40 deletions
Expand all files Collapse all files
.playwright-mcp/console-2026-09-04T00-53-39-252Z.log
+22
View File
@@ -0,0 +1,22 @@
[ 1289ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/login:0
[ 1417ms] [VERBOSE] [DOM] Input elements should have autocomplete attributes (suggested: "username"): (More info: https://goo.gl/9p2vKq) %o @ https://app2.hyeonworks.com/login:0
[ 9024ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 9130ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/:0
[ 9456ms] [WARNING] Deprecation warning: value provided is not in a recognized RFC2822 or ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions. Non RFC2822/ISO date formats are discouraged. Please refer to http://momentjs.com/guides/#/warnings/js-date/ for more info.
Arguments:
[0] _isAMomentObject: true, _isUTC: false, _useUTC: false, _l: undefined, _i: Thu, 27 Aug 2026 13:03:49, _f: undefined, _strict: undefined, _locale: [object Object]
Error
at a.createFromInputFallback (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:3)
at an (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:25647)
at un (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:29355)
at aa (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:29221)
at on (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:28938)
at sa (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:29715)
at A (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:624:29748)
at a (https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:621:89)
at f (https://app2.hyeonworks.com/public/build/3719.c065b2e146c4c8347d51.js:1:4635)
at u (https://app2.hyeonworks.com/public/build/322.177b4bb01c5d74f9b28f.js:2473:47448) @ https://app2.hyeonworks.com/public/build/6029.0549a3fcb50e73c4b256.js:620
[ 9605ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 10620ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 11527ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 13875ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T00-53-54-407Z.log
+7
View File
@@ -0,0 +1,7 @@
[ 144ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 153ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore:0
[ 1077ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2101ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2922ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 7323ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 9370ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T00-54-13-939Z.log
+8
View File
@@ -0,0 +1,8 @@
[ 271ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&orgId=1&panes=%7B%22a%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_statistics_approximate_entries_unique%7Bcache%3D%5C%22sessions%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bpod%7D%7D%20on%20%7B%7Bnode%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-15m%22%2C%22to%22%3A%22now%22%7D%7D%7D:0
[ 346ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1512ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2433ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 6941ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 13188ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 21578ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 25998ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T00-54-40-326Z.log
+7
View File
@@ -0,0 +1,7 @@
[ 766ms] [WARNING] An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing. @ https://auth.hyeonworks.com/realms/master/protocol/openid-connect/3p-cookies/step1.html:0
[ 781ms] [WARNING] An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing. @ https://auth.hyeonworks.com/realms/master/protocol/openid-connect/3p-cookies/step2.html:0
[ 17929ms] [WARNING] An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing. @ https://auth.hyeonworks.com/realms/master/protocol/openid-connect/3p-cookies/step1.html:0
[ 17949ms] [WARNING] An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing. @ https://auth.hyeonworks.com/realms/master/protocol/openid-connect/3p-cookies/step2.html:0
[ 17981ms] [WARNING] An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing. @ https://auth.hyeonworks.com/realms/master/protocol/openid-connect/login-status-iframe.html:0
[ 18447ms] [WARNING] For accessibility reasons an aria-label should be specified on nav groups if a title isn't @ https://auth.hyeonworks.com/resources/9v5yc/admin/keycloak.v2/assets/main-BbID33M6.js:7
[ 18462ms] [WARNING] For accessibility reasons an aria-label should be specified on nav groups if a title isn't @ https://auth.hyeonworks.com/resources/9v5yc/admin/keycloak.v2/assets/main-BbID33M6.js:7
.playwright-mcp/console-2026-09-04T02-10-44-749Z.log
+2
View File
@@ -0,0 +1,2 @@
[ 75ms] [ERROR] Failed to load resource: the server responded with a status of 404 (Not Found) @ https://hyeonworks.com/questions:0
[ 100ms] [ERROR] Failed to load resource: the server responded with a status of 404 (Not Found) @ https://hyeonworks.com/favicon.ico:0
.playwright-mcp/console-2026-09-03T07-41-09-675Z.log → .playwright-mcp/console-2026-09-04T02-10-49-897Z.log
+1 -1
View File
@@ -1 +1 @@
[ 295ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
[ 149ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
.playwright-mcp/console-2026-09-03T07-41-27-913Z.log → .playwright-mcp/console-2026-09-04T02-10-53-778Z.log
+1 -1
View File
@@ -1 +1 @@
[ 125ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
[ 103ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
.playwright-mcp/console-2026-09-03T07-41-40-247Z.log → .playwright-mcp/console-2026-09-04T02-11-03-259Z.log
View File
.playwright-mcp/console-2026-09-03T07-41-50-414Z.log → .playwright-mcp/console-2026-09-04T02-12-52-735Z.log
+1 -1
View File
@@ -1 +1 @@
[ 91ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
[ 132ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
.playwright-mcp/console-2026-09-03T07-42-00-948Z.log → .playwright-mcp/console-2026-09-04T02-13-10-808Z.log
+1 -1
View File
@@ -1 +1 @@
[ 88ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
[ 239ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
.playwright-mcp/console-2026-09-04T02-13-18-948Z.log
+1
View File
@@ -0,0 +1 @@
[ 105ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
.playwright-mcp/console-2026-09-04T02-13-27-001Z.log
+1
View File
@@ -0,0 +1 @@
[ 108ms] [ERROR] Failed to load resource: the server responded with a status of 401 (Unauthorized) @ https://hyeonworks.com/api/v1/studio/session:0
.playwright-mcp/console-2026-09-04T02-50-02-217Z.log
+41
View File
@@ -0,0 +1,41 @@
[ 344ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 869ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&panes=%7B%22cf1%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_cluster_size%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bpod%7D%7D+on+%7B%7Bnode%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-45m%22%2C%22to%22%3A%22now%22%7D%7D%7D&orgId=1:0
[ 1014ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2039ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 4085ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 7289ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 15556ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 26818ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 32659ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 35930ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 53849ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 61722ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 79450ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 84571ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 87233ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 92665ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 113244ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 119181ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 137390ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 157071ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 166091ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 182011ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 188613ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 206228ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 218561ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 229607ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 235818ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 237158ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 252095ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 261118ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 273809ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 280227ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 292650ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 306477ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 309957ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 311984ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 325683ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 344586ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 357576ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 359294ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 364740ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T02-56-16-987Z.log
+129
View File
@@ -0,0 +1,129 @@
[ 615ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&panes=%7B%22te0%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22up%7Bjob%3D%5C%22keycloak%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22up+%E2%80%94+%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-20m%22%2C%22to%22%3A%22now%22%7D%7D%7D&orgId=1:0
[ 929ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2566ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 5541ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 7990ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 12402ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 17640ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 20285ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 28277ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 34341ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 34985ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 50081ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 65649ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 75046ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 75890ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 96067ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 113467ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 121972ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 139681ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 150971ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 161211ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 180744ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 191401ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 210668ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 229082ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 239345ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 252119ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 266045ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 276705ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 289911ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 296875ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 315099ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 325431ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 328418ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 343774ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 347054ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 352405ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 361291ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 370639ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 374286ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 377413ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 394362ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 405423ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 425476ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 434299ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 441775ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 450786ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 456829ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 468812ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 478913ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 491241ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 495228ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 502295ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 511625ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 524619ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 533050ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 553189ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 571699ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 575413ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 588005ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 599161ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 608816ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 619031ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 626612ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 639003ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 655077ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 674532ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 677197ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 696027ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 709512ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 727505ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 747139ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 754565ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 773682ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 784674ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 787399ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 802845ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 825589ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 832768ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 842043ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 852717ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 871748ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 876571ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 894491ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 895515ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 904310ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 916681ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 934752ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 941175ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 946004ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 960849ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 962180ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 984446ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 997579ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1010105ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1027190ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1043388ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1061311ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1066040ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1080349ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1092336ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1104312ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1109362ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1127047ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1134728ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1154899ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1161253ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1174971ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1184491ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1185340ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1189111ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1196984ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1204794ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1247961ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1286322ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1319329ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1362771ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1396881ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1434184ms] [ERROR] Failed to load resource: the server responded with a status of 502 () @ https://app2.hyeonworks.com/api/user/auth-tokens/rotate:0
[ 1443506ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1465016ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 502 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1513337ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1556906ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1562963ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 502 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1572896ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 502 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1576390ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 503 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1593794ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1600651ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1616424ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T03-23-15-961Z.log
+125
View File
@@ -0,0 +1,125 @@
[ 268ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/login:0
[ 379ms] [VERBOSE] [DOM] Input elements should have autocomplete attributes (suggested: "username"): (More info: https://goo.gl/9p2vKq) %o @ https://app2.hyeonworks.com/login:0
[ 10453ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 10532ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&panes=%7B%22ich%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22up%7Bjob%3D%7E%5C%22keycloak%7Cnode-exporter%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bjob%7D%7D+%E2%80%94+%7B%7Bpod%7D%7D%7B%7Bnode%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-55m%22%2C%22to%22%3A%22now%22%7D%7D%7D&orgId=1:0
[ 11687ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 13737ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 15113ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 20185ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 27259ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 40154ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 41870ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 50493ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 54082ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 60121ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 74362ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 83269ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 95349ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 98008ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 104458ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 123816ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 142764ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 157198ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 175220ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 182187ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 183411ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 190581ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 199499ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 209995ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 214849ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 233084ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 242910ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 257975ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 277220ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 290806ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 292674ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 300013ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 309267ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 323998ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 328620ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 339787ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 355959ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 361079ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 369272ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 381866ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 397120ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 400712ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 412579ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 431630ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 440746ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 447809ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 461741ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 475459ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 482723ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 494299ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 513174ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 521541ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 532145ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 551129ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 556250ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 574381ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 586190ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 595782ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 610555ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 630695ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 633257ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 652194ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 671361ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 677606ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 692444ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 696949ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 700122ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 709954ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 723992ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 743173ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 745075ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 753299ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 766683ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 767855ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 787261ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 789725ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 800680ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 811324ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 814096ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 829045ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 848506ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 857716ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 866212ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 873174ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 876033ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 895198ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 898884ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 918642ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 930133ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 934008ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 948541ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 956860ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 959397ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 968406ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 971788ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 982746ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 997394ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1005078ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1009269ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1021453ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1026674ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1042137ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1057497ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1059851ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1061855ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1071317ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1078179ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1084940ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1093950ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1112279ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1124360ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1139110ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1157981ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1163071ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1179032ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1181195ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1191955ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1201472ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1218261ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1221169ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1229430ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T03-43-55-141Z.log
+50
View File
@@ -0,0 +1,50 @@
[ 199ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 297ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&orgId=1&panes=%7B%22a%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_cluster_size%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-25m%22%2C%22to%22%3A%22now%22%7D%7D%7D:0
[ 1000ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 2023ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 3559ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 7749ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 16364ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 25781ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 42158ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 59672ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 77090ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 87426ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 89952ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 104743ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error in connection establishment: net::ERR_NAME_NOT_RESOLVED @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1573121ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1584038ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1601626ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1613280ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1630173ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1634776ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1654205ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1661576ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1665269ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1682669ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1689435ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1704385ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1706314ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1724551ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1743701ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1758339ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1769299ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1789478ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1795415ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1815382ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1820392ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1822445ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1839035ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1840162ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1847739ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1861419ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1880381ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1893315ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1911434ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1922701ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1933868ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1944951ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1948001ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1952503ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1953320ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1955369ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T04-16-32-330Z.log
+20
View File
@@ -0,0 +1,20 @@
[ 585ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1132ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&panes=%7B%22o68%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22agroal_blocking_time_max_milliseconds%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22blocking+max+-+%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-30m%22%2C%22to%22%3A%22now%22%7D%7D%7D&orgId=1:0
[ 1555ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 3604ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 6373ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 9135ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 19066ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 24595ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 47125ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 52154ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 67608ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 86965ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 108668ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 129158ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 137544ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 152699ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 159898ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 173491ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 187925ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 197824ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/console-2026-09-04T04-19-58-373Z.log
+11
View File
@@ -0,0 +1,11 @@
[ 635ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&orgId=1&panes=%7B%22a%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_statistics_approximate_entries_unique%7Bcache%3D%5C%22sessions%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%2C%7B%22refId%22%3A%22B%22%2C%22expr%22%3A%22vendor_cluster_size%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22cluster_size%20%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-15m%22%2C%22to%22%3A%22now%22%7D%7D%7D:0
[ 686ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 1451ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 3030ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 4308ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 5296ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 7691ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 18337ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 20184ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 32473ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
[ 50697ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
.playwright-mcp/page-2026-09-03T07-41-09-984Z.yml
-25
View File
@@ -1,25 +0,0 @@
- generic [ref=f7e3]:
- link "본문으로 건너뛰기" [ref=f7e4] [cursor=pointer]:
- /url: "#main-content"
- banner [ref=f7e5]:
- generic [ref=f7e6]:
- link "TechLog 홈" [ref=f7e8] [cursor=pointer]:
- /url: /
- text: TechLog
- generic [ref=f7e9]:
- button "TechLog 검색 열기" [ref=f7e11] [cursor=pointer]: 검색
- group [ref=f7e12]:
- generic "메뉴" [ref=f7e13] [cursor=pointer]
- generic [ref=f7e14]:
- paragraph [ref=f7e15]: 화면을 준비하고 있습니다.
- generic [ref=f7e16]: TechLog 로딩 중
- contentinfo [ref=f7e17]:
- generic [ref=f7e18]:
- generic [ref=f7e19]:
- paragraph [ref=f7e20]: 동현
- paragraph [ref=f7e21]: 문제를 재현하고 검증해 실제 운영에 적용할 수 있는 형태로 정리합니다.
- generic [ref=f7e22]:
- link "프로필" [ref=f7e23] [cursor=pointer]:
- /url: /profile
- link "변경 기록" [ref=f7e24] [cursor=pointer]:
- /url: /releases
.playwright-mcp/page-2026-09-04T00-53-40-561Z.yml
+2
View File
@@ -0,0 +1,2 @@
- main [ref=e7]:
- status "Loading" [ref=e10]
.playwright-mcp/page-2026-09-04T00-53-48-279Z.yml
+1
View File
@@ -0,0 +1 @@
- main [ref=f3e7]
.playwright-mcp/page-2026-09-04T00-53-54-595Z.yml
+2
View File
@@ -0,0 +1,2 @@
- main [ref=f6e7]:
- status "Loading" [ref=f6e10]
.playwright-mcp/page-2026-09-04T00-54-14-322Z.yml
+104
View File
@@ -0,0 +1,104 @@
- generic [ref=f9e4]:
- link "Skip to main content" [ref=f9e5] [cursor=pointer]:
- /url: "#pageContent"
- banner [ref=f9e7]:
- generic [ref=f9e8]:
- link [ref=f9e10] [cursor=pointer]:
- /url: /
- img "Grafana" [ref=f9e11]
- generic [ref=f9e14]:
- button "Search or jump to..." [ref=f9e18] [cursor=pointer]
- generic [ref=f9e19]: ctrl+k
- generic [ref=f9e23]:
- button "New" [ref=f9e24] [cursor=pointer]
- button "Help" [ref=f9e30] [cursor=pointer]
- button "News" [ref=f9e33] [cursor=pointer]
- button "Profile" [ref=f9e36] [cursor=pointer]:
- img "User avatar" [ref=f9e37]
- generic [ref=f9e38]:
- button "Open menu" [ref=f9e40] [cursor=pointer]
- navigation "Breadcrumbs" [ref=f9e43]:
- list [ref=f9e44]:
- listitem [ref=f9e45]:
- link "Home" [ref=f9e46] [cursor=pointer]:
- /url: /
- listitem [ref=f9e50]:
- link "Explore" [ref=f9e51] [cursor=pointer]:
- /url: /explore
- listitem [ref=f9e55]:
- generic "Prometheus" [ref=f9e56]
- generic [ref=f9e57]:
- button "Show more items" [ref=f9e60] [cursor=pointer]
- button "Toggle top search bar" [ref=f9e64] [cursor=pointer]
- main [ref=f9e70]:
- generic [ref=f9e72]:
- heading "Explore" [level=1] [ref=f9e73]
- generic [ref=f9e78]:
- navigation "Explore toolbar" [ref=f9e80]:
- navigation "Search links" [ref=f9e82]:
- generic [ref=f9e83]:
- button "Content outline" [expanded] [ref=f9e85] [cursor=pointer]:
- generic [ref=f9e88]: Outline
- generic [ref=f9e93] [cursor=pointer]:
- img "Prometheus logo" [ref=f9e95]
- textbox "Select a data source" [ref=f9e96]:
- /placeholder: ""
- button "Show more items" [ref=f9e102] [cursor=pointer]
- generic [ref=f9e106]:
- generic [ref=f9e110]:
- button "Collapse outline" [expanded] [ref=f9e112] [cursor=pointer]:
- img "arrow-from-right" [ref=f9e113]
- button "Queries" [ref=f9e116] [cursor=pointer]:
- img "arrow" [ref=f9e117]
- generic [ref=f9e124]:
- generic [ref=f9e126]:
- generic "Query editor row" [ref=f9e129]:
- generic [ref=f9e130]:
- generic [ref=f9e132]:
- generic [ref=f9e133]:
- button "Collapse query row" [expanded] [ref=f9e134] [cursor=pointer]
- generic [ref=f9e137]:
- button "Query editor row title A" [ref=f9e138] [cursor=pointer]:
- generic [ref=f9e139]: A
- emphasis [ref=f9e140]: (Prometheus)
- generic [ref=f9e141]:
- button "Show data source help" [ref=f9e143] [cursor=pointer]
- button "Duplicate query" [ref=f9e147] [cursor=pointer]
- button "Hide response" [ref=f9e151] [cursor=pointer]
- button "Remove query" [ref=f9e155] [cursor=pointer]
- button "Drag and drop to reorder" [ref=f9e158]:
- img "Drag and drop to reorder" [ref=f9e159]
- generic [ref=f9e162]:
- generic [ref=f9e163]:
- button "Kick start your query" [ref=f9e164] [cursor=pointer]
- generic [ref=f9e167]:
- generic [ref=f9e168] [cursor=pointer]: Explain
- generic [ref=f9e169]:
- checkbox "Explain Toggle switch" [ref=f9e170]
- generic "Toggle switch" [ref=f9e171] [cursor=pointer]
- radiogroup [ref=f9e176]:
- generic [ref=f9e177]:
- radio "Builder" [ref=f9e178] [cursor=pointer]
- generic [ref=f9e179] [cursor=pointer]: Builder
- generic [ref=f9e180]:
- radio "Code" [checked] [ref=f9e181] [cursor=pointer]
- generic [ref=f9e182] [cursor=pointer]: Code
- generic [ref=f9e184]:
- generic [ref=f9e186]:
- button "Loading metrics..." [disabled] [ref=f9e187] [cursor=pointer]
- generic [ref=f9e190]: Loading editor
- 'button "Options Legend: {{pod}} on {{node}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f9e198] [cursor=pointer]':
- generic [ref=f9e202]:
- heading "Options" [level=6] [ref=f9e203]
- generic [ref=f9e204]:
- generic [ref=f9e205]: "Legend: {{pod}} on {{node}}"
- generic [ref=f9e206]: "Format: Time series"
- generic [ref=f9e207]: "Step: auto"
- generic [ref=f9e208]: "Type: Range"
- generic [ref=f9e209]: "Exemplars: false"
- generic [ref=f9e210]:
- button "Add query" [ref=f9e211] [cursor=pointer]
- button "Query history" [ref=f9e215] [cursor=pointer]
- button "Query inspector" [ref=f9e219] [cursor=pointer]
- generic:
- main
.playwright-mcp/page-2026-09-04T00-54-40-837Z.yml
+4
View File
@@ -0,0 +1,4 @@
- main [ref=f12e3]:
- generic [ref=f12e4]:
- progressbar "Contents" [ref=f12e5]
- paragraph [ref=f12e8]: Loading the Administration Console
.playwright-mcp/page-2026-09-04T00-54-58-719Z.yml
+4
View File
@@ -0,0 +1,4 @@
- generic [active] [ref=f15e1]:
- progressbar "Loading" [ref=f15e4]
- generic:
- list
.playwright-mcp/page-2026-09-04T02-10-44-851Z.yml
+1
View File
@@ -0,0 +1 @@
- generic [active] [ref=f18e1]: Not Found
.playwright-mcp/page-2026-09-03T07-41-27-980Z.yml → .playwright-mcp/page-2026-09-04T02-10-50-012Z.yml
View File
.playwright-mcp/page-2026-09-03T07-41-40-313Z.yml → .playwright-mcp/page-2026-09-04T02-10-53-850Z.yml
View File
.playwright-mcp/page-2026-09-03T07-41-50-480Z.yml → .playwright-mcp/page-2026-09-04T02-11-03-323Z.yml
View File
.playwright-mcp/page-2026-09-03T07-42-01-011Z.yml → .playwright-mcp/page-2026-09-04T02-12-52-835Z.yml
View File
.playwright-mcp/page-2026-09-04T02-13-10-875Z.yml
View File
.playwright-mcp/page-2026-09-04T02-13-19-024Z.yml
View File
.playwright-mcp/page-2026-09-04T02-13-27-084Z.yml
View File
.playwright-mcp/page-2026-09-04T02-50-03-122Z.yml
+151
View File
@@ -0,0 +1,151 @@
- generic [active] [ref=e1]:
- generic [ref=e4]:
- link "Skip to main content" [ref=e5] [cursor=pointer]:
- /url: "#pageContent"
- banner [ref=e7]:
- generic [ref=e8]:
- link [ref=e10] [cursor=pointer]:
- /url: /
- img "Grafana" [ref=e11]
- generic [ref=e14]:
- button "Search or jump to..." [ref=e18] [cursor=pointer]
- generic [ref=e19]: ctrl+k
- generic [ref=e23]:
- button "New" [ref=e24] [cursor=pointer]
- button "Help" [ref=e30] [cursor=pointer]
- button "News" [ref=e33] [cursor=pointer]
- button "Profile" [ref=e36] [cursor=pointer]:
- img "User avatar" [ref=e37]
- generic [ref=e38]:
- button "Open menu" [ref=e40] [cursor=pointer]
- navigation "Breadcrumbs" [ref=e43]:
- list [ref=e44]:
- listitem [ref=e45]:
- link "Home" [ref=e46] [cursor=pointer]:
- /url: /
- listitem [ref=e50]:
- link "Explore" [ref=e51] [cursor=pointer]:
- /url: /explore
- listitem [ref=e55]:
- generic "Prometheus" [ref=e56]
- generic [ref=e57]:
- generic [ref=e60]:
- button "Copy shortened URL" [ref=e61] [cursor=pointer]
- button "Open copy link options" [ref=e64] [cursor=pointer]
- button "Toggle top search bar" [ref=e68] [cursor=pointer]
- main [ref=e74]:
- generic [ref=e76]:
- heading "Explore" [level=1] [ref=e77]
- generic [ref=e82]:
- navigation "Explore toolbar" [ref=e84]:
- navigation "Search links" [ref=e86]:
- generic [ref=e87]:
- button "Content outline" [expanded] [ref=e89] [cursor=pointer]:
- generic [ref=e92]: Outline
- generic [ref=e97] [cursor=pointer]:
- img "Prometheus logo" [ref=e99]
- textbox "Select a data source" [ref=e100]:
- /placeholder: ""
- generic [ref=e104]:
- button "Split the pane" [ref=e106] [cursor=pointer]:
- generic [ref=e109]: Split
- button "Add" [ref=e111] [cursor=pointer]
- generic [ref=e116]:
- 'button "Time range selected: Last 45 minutes" [ref=e117] [cursor=pointer]'
- button "Zoom out time range" [ref=e122] [cursor=pointer]
- generic [ref=e126]:
- button "Run query" [ref=e127] [cursor=pointer]
- button "Auto refresh turned off. Choose refresh time interval" [ref=e131] [cursor=pointer]
- generic [ref=e135]:
- generic [ref=e139]:
- button "Collapse outline" [expanded] [ref=e141] [cursor=pointer]:
- img "arrow-from-right" [ref=e142]
- button "Queries" [ref=e145] [cursor=pointer]:
- img "arrow" [ref=e146]
- button "Graph" [ref=e150] [cursor=pointer]:
- img "graph-bar" [ref=e151]
- generic [ref=e158]:
- generic [ref=e160]:
- generic "Query editor row" [ref=e163]:
- generic [ref=e164]:
- generic [ref=e166]:
- generic [ref=e167]:
- button "Collapse query row" [expanded] [ref=e168] [cursor=pointer]
- generic [ref=e171]:
- button "Query editor row title A" [ref=e172] [cursor=pointer]:
- generic [ref=e173]: A
- emphasis [ref=e174]: (Prometheus)
- generic [ref=e175]:
- button "Show data source help" [ref=e177] [cursor=pointer]
- button "Duplicate query" [ref=e181] [cursor=pointer]
- button "Hide response" [ref=e185] [cursor=pointer]
- button "Remove query" [ref=e189] [cursor=pointer]
- button "Drag and drop to reorder" [ref=e192]:
- img "Drag and drop to reorder" [ref=e193]
- generic [ref=e196]:
- generic [ref=e197]:
- button "Kick start your query" [ref=e198] [cursor=pointer]
- generic [ref=e201]:
- generic [ref=e202] [cursor=pointer]: Explain
- generic [ref=e203]:
- checkbox "Explain Toggle switch" [ref=e204]
- generic "Toggle switch" [ref=e205] [cursor=pointer]
- radiogroup [ref=e210]:
- generic [ref=e211]:
- radio "Builder" [ref=e212] [cursor=pointer]
- generic [ref=e213] [cursor=pointer]: Builder
- generic [ref=e214]:
- radio "Code" [checked] [ref=e215] [cursor=pointer]
- generic [ref=e216] [cursor=pointer]: Code
- generic [ref=e218]:
- generic [ref=e220]:
- button "Metrics browser" [ref=e221] [cursor=pointer]
- code [ref=e228]:
- generic [ref=e229]:
- generic [ref=e234]: vendor_cluster_size
- textbox "Editor content;Press Alt+F1 for Accessibility Options." [ref=e239]: vendor_cluster_size
- 'button "Options Legend: {{pod}} on {{node}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=e245] [cursor=pointer]':
- generic [ref=e249]:
- heading "Options" [level=6] [ref=e250]
- generic [ref=e251]:
- generic [ref=e252]: "Legend: {{pod}} on {{node}}"
- generic [ref=e253]: "Format: Time series"
- generic [ref=e254]: "Step: auto"
- generic [ref=e255]: "Type: Range"
- generic [ref=e256]: "Exemplars: false"
- generic [ref=e257]:
- button "Add query" [ref=e258] [cursor=pointer]
- button "Query history" [ref=e262] [cursor=pointer]
- button "Query inspector" [ref=e266] [cursor=pointer]
- main [ref=e270]:
- region [ref=e272]:
- generic [ref=e273]:
- heading "Graph" [level=2] [ref=e275]
- radiogroup [ref=e278]:
- generic [ref=e279]:
- radio "Lines" [checked] [ref=e280] [cursor=pointer]
- generic [ref=e281] [cursor=pointer]: Lines
- generic [ref=e282]:
- radio "Bars" [ref=e283] [cursor=pointer]
- generic [ref=e284] [cursor=pointer]: Bars
- generic [ref=e285]:
- radio "Points" [ref=e286] [cursor=pointer]
- generic [ref=e287] [cursor=pointer]: Points
- generic [ref=e288]:
- radio "Stacked lines" [ref=e289] [cursor=pointer]
- generic [ref=e290] [cursor=pointer]: Stacked lines
- generic [ref=e291]:
- radio "Stacked bars" [ref=e292] [cursor=pointer]
- generic [ref=e293] [cursor=pointer]: Stacked bars
- list [ref=e302]:
- listitem [ref=e303]:
- button "keycloak-0 on kc-lab-2" [ref=e307] [cursor=pointer]
- listitem [ref=e308]:
- button "keycloak-0 on kc-lab-2" [ref=e312] [cursor=pointer]
- listitem [ref=e313]:
- button "keycloak-1 on kc-lab-1" [ref=e317] [cursor=pointer]
- generic [ref=e322]:
- alert
- alert
- complementary
- complementary
.playwright-mcp/page-2026-09-04T02-56-17-648Z.yml
+145
View File
@@ -0,0 +1,145 @@
- generic [active] [ref=f3e1]:
- generic [ref=f3e4]:
- link "Skip to main content" [ref=f3e5] [cursor=pointer]:
- /url: "#pageContent"
- banner [ref=f3e7]:
- generic [ref=f3e8]:
- link [ref=f3e10] [cursor=pointer]:
- /url: /
- img "Grafana" [ref=f3e11]
- generic [ref=f3e14]:
- button "Search or jump to..." [ref=f3e18] [cursor=pointer]
- generic [ref=f3e19]: ctrl+k
- generic [ref=f3e23]:
- button "New" [ref=f3e24] [cursor=pointer]
- button "Help" [ref=f3e30] [cursor=pointer]
- button "News" [ref=f3e33] [cursor=pointer]
- button "Profile" [ref=f3e36] [cursor=pointer]:
- img "User avatar" [ref=f3e37]
- generic [ref=f3e38]:
- button "Open menu" [ref=f3e40] [cursor=pointer]
- navigation "Breadcrumbs" [ref=f3e43]:
- list [ref=f3e44]:
- listitem [ref=f3e45]:
- link "Home" [ref=f3e46] [cursor=pointer]:
- /url: /
- listitem [ref=f3e50]:
- link "Explore" [ref=f3e51] [cursor=pointer]:
- /url: /explore
- listitem [ref=f3e55]:
- generic "Prometheus" [ref=f3e56]
- generic [ref=f3e57]:
- generic [ref=f3e60]:
- button "Copy shortened URL" [ref=f3e61] [cursor=pointer]
- button "Open copy link options" [ref=f3e64] [cursor=pointer]
- button "Toggle top search bar" [ref=f3e68] [cursor=pointer]
- main [ref=f3e74]:
- generic [ref=f3e76]:
- heading "Explore" [level=1] [ref=f3e77]
- generic [ref=f3e82]:
- navigation "Explore toolbar" [ref=f3e84]:
- navigation "Search links" [ref=f3e86]:
- generic [ref=f3e87]:
- button "Content outline" [expanded] [ref=f3e89] [cursor=pointer]:
- generic [ref=f3e92]: Outline
- generic [ref=f3e97] [cursor=pointer]:
- img "Prometheus logo" [ref=f3e99]
- textbox "Select a data source" [ref=f3e100]:
- /placeholder: ""
- generic [ref=f3e104]:
- button "Split the pane" [ref=f3e106] [cursor=pointer]:
- generic [ref=f3e109]: Split
- button "Add" [ref=f3e111] [cursor=pointer]
- generic [ref=f3e116]:
- 'button "Time range selected: Last 20 minutes" [ref=f3e117] [cursor=pointer]'
- button "Zoom out time range" [ref=f3e122] [cursor=pointer]
- generic [ref=f3e126]:
- button "Run query" [ref=f3e127] [cursor=pointer]
- button "Auto refresh turned off. Choose refresh time interval" [ref=f3e131] [cursor=pointer]
- generic [ref=f3e135]:
- generic [ref=f3e139]:
- button "Collapse outline" [expanded] [ref=f3e141] [cursor=pointer]:
- img "arrow-from-right" [ref=f3e142]
- button "Queries" [ref=f3e145] [cursor=pointer]:
- img "arrow" [ref=f3e146]
- button "Graph" [ref=f3e150] [cursor=pointer]:
- img "graph-bar" [ref=f3e151]
- generic [ref=f3e158]:
- generic [ref=f3e160]:
- generic "Query editor row" [ref=f3e163]:
- generic [ref=f3e164]:
- generic [ref=f3e166]:
- generic [ref=f3e167]:
- button "Collapse query row" [expanded] [ref=f3e168] [cursor=pointer]
- generic [ref=f3e171]:
- button "Query editor row title A" [ref=f3e172] [cursor=pointer]:
- generic [ref=f3e173]: A
- emphasis [ref=f3e174]: (Prometheus)
- generic [ref=f3e175]:
- button "Show data source help" [ref=f3e177] [cursor=pointer]
- button "Duplicate query" [ref=f3e181] [cursor=pointer]
- button "Hide response" [ref=f3e185] [cursor=pointer]
- button "Remove query" [ref=f3e189] [cursor=pointer]
- button "Drag and drop to reorder" [ref=f3e192]:
- img "Drag and drop to reorder" [ref=f3e193]
- generic [ref=f3e196]:
- generic [ref=f3e197]:
- button "Kick start your query" [ref=f3e198] [cursor=pointer]
- generic [ref=f3e201]:
- generic [ref=f3e202] [cursor=pointer]: Explain
- generic [ref=f3e203]:
- checkbox "Explain Toggle switch" [ref=f3e204]
- generic "Toggle switch" [ref=f3e205] [cursor=pointer]
- radiogroup [ref=f3e210]:
- generic [ref=f3e211]:
- radio "Builder" [ref=f3e212] [cursor=pointer]
- generic [ref=f3e213] [cursor=pointer]: Builder
- generic [ref=f3e214]:
- radio "Code" [checked] [ref=f3e215] [cursor=pointer]
- generic [ref=f3e216] [cursor=pointer]: Code
- generic [ref=f3e218]:
- generic [ref=f3e220]:
- button "Loading metrics..." [disabled] [ref=f3e221] [cursor=pointer]
- code [ref=f3e228]:
- generic [ref=f3e229]:
- generic [ref=f3e234]: "up{job=\"keycloak\"}"
- textbox "Editor content;Press Alt+F1 for Accessibility Options." [ref=f3e239]: "up{job=\"keycloak\"}"
- 'button "Options Legend: up — {{pod}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f3e245] [cursor=pointer]':
- generic [ref=f3e249]:
- heading "Options" [level=6] [ref=f3e250]
- generic [ref=f3e251]:
- generic [ref=f3e252]: "Legend: up — {{pod}}"
- generic [ref=f3e253]: "Format: Time series"
- generic [ref=f3e254]: "Step: auto"
- generic [ref=f3e255]: "Type: Range"
- generic [ref=f3e256]: "Exemplars: false"
- generic [ref=f3e257]:
- button "Add query" [ref=f3e258] [cursor=pointer]
- button "Query history" [ref=f3e262] [cursor=pointer]
- button "Query inspector" [ref=f3e266] [cursor=pointer]
- main [ref=f3e270]:
- region [ref=f3e272]:
- generic [ref=f3e273]:
- heading "Graph" [level=2] [ref=f3e275]
- radiogroup [ref=f3e278]:
- generic [ref=f3e279]:
- radio "Lines" [checked] [ref=f3e280] [cursor=pointer]
- generic [ref=f3e281] [cursor=pointer]: Lines
- generic [ref=f3e282]:
- radio "Bars" [ref=f3e283] [cursor=pointer]
- generic [ref=f3e284] [cursor=pointer]: Bars
- generic [ref=f3e285]:
- radio "Points" [ref=f3e286] [cursor=pointer]
- generic [ref=f3e287] [cursor=pointer]: Points
- generic [ref=f3e288]:
- radio "Stacked lines" [ref=f3e289] [cursor=pointer]
- generic [ref=f3e290] [cursor=pointer]: Stacked lines
- generic [ref=f3e291]:
- radio "Stacked bars" [ref=f3e292] [cursor=pointer]
- generic [ref=f3e293] [cursor=pointer]: Stacked bars
- generic [ref=f3e294]: Loading plugin panel...
- generic [ref=f3e299]:
- alert
- alert
- complementary
- complementary
.playwright-mcp/page-2026-09-04T03-23-16-267Z.yml
+44
View File
@@ -0,0 +1,44 @@
- main [ref=f6e7]:
- generic [ref=f6e9]:
- generic [ref=f6e11]:
- generic [ref=f6e12]:
- img "Grafana" [ref=f6e13]
- heading "Welcome to Grafana" [level=1] [ref=f6e15]
- generic [ref=f6e19]:
- generic [ref=f6e20]:
- generic [ref=f6e21]: Email or username
- textbox "Email or username" [active] [ref=f6e28]:
- /placeholder: email or username
- generic [ref=f6e29]:
- generic [ref=f6e30]: Password
- generic [ref=f6e36]:
- textbox "Password" [ref=f6e37]:
- /placeholder: password
- switch "Show password" [ref=f6e39] [cursor=pointer]
- button "Log in" [ref=f6e42] [cursor=pointer]
- link "Forgot your password?" [ref=f6e45] [cursor=pointer]:
- /url: /user/password/send-reset-email
- list [ref=f6e49]:
- listitem [ref=f6e50]:
- link "Documentation" [ref=f6e53] [cursor=pointer]:
- /url: https://grafana.com/docs/grafana/latest/?utm_source=grafana_footer
- text: "|"
- listitem [ref=f6e54]:
- link "Support" [ref=f6e57] [cursor=pointer]:
- /url: https://grafana.com/products/enterprise/?utm_source=grafana_footer
- text: "|"
- listitem [ref=f6e58]:
- link "Community" [ref=f6e61] [cursor=pointer]:
- /url: https://community.grafana.com/?utm_source=grafana_footer
- text: "|"
- listitem [ref=f6e62]:
- link "Open Source" [ref=f6e63] [cursor=pointer]:
- /url: https://grafana.com/oss/grafana?utm_source=grafana_footer
- text: "|"
- listitem [ref=f6e64]:
- link "Grafana v11.4.0 (b58701869e)" [ref=f6e65] [cursor=pointer]:
- /url: https://github.com/grafana/grafana/blob/main/CHANGELOG.md
- text: "|"
- listitem [ref=f6e66]:
- link "New version available!" [ref=f6e69] [cursor=pointer]:
- /url: https://grafana.com/grafana/download?utm_source=grafana_footer
.playwright-mcp/page-2026-09-04T03-23-27-151Z.yml
+159
View File
@@ -0,0 +1,159 @@
- generic [active] [ref=f9e1]:
- generic [ref=f9e4]:
- link "Skip to main content" [ref=f9e5] [cursor=pointer]:
- /url: "#pageContent"
- banner [ref=f9e7]:
- generic [ref=f9e8]:
- link [ref=f9e10] [cursor=pointer]:
- /url: /
- img "Grafana" [ref=f9e11]
- generic [ref=f9e14]:
- button "Search or jump to..." [ref=f9e18] [cursor=pointer]
- generic [ref=f9e19]: ctrl+k
- generic [ref=f9e23]:
- button "New" [ref=f9e24] [cursor=pointer]
- button "Help" [ref=f9e30] [cursor=pointer]
- button "News" [ref=f9e33] [cursor=pointer]
- button "Profile" [ref=f9e36] [cursor=pointer]:
- img "User avatar" [ref=f9e37]
- generic [ref=f9e38]:
- button "Open menu" [ref=f9e40] [cursor=pointer]
- navigation "Breadcrumbs" [ref=f9e43]:
- list [ref=f9e44]:
- listitem [ref=f9e45]:
- link "Home" [ref=f9e46] [cursor=pointer]:
- /url: /
- listitem [ref=f9e50]:
- link "Explore" [ref=f9e51] [cursor=pointer]:
- /url: /explore
- listitem [ref=f9e55]:
- generic "Prometheus" [ref=f9e56]
- generic [ref=f9e57]:
- generic [ref=f9e60]:
- button "Copy shortened URL" [ref=f9e61] [cursor=pointer]
- button "Open copy link options" [ref=f9e64] [cursor=pointer]
- button "Toggle top search bar" [ref=f9e68] [cursor=pointer]
- main [ref=f9e74]:
- generic [ref=f9e76]:
- heading "Explore" [level=1] [ref=f9e77]
- generic [ref=f9e82]:
- navigation "Explore toolbar" [ref=f9e84]:
- navigation "Search links" [ref=f9e86]:
- generic [ref=f9e87]:
- button "Content outline" [expanded] [ref=f9e89] [cursor=pointer]:
- generic [ref=f9e92]: Outline
- generic [ref=f9e97] [cursor=pointer]:
- img "Prometheus logo" [ref=f9e99]
- textbox "Select a data source" [ref=f9e100]:
- /placeholder: ""
- generic [ref=f9e104]:
- button "Split the pane" [ref=f9e106] [cursor=pointer]:
- generic [ref=f9e109]: Split
- button "Add" [ref=f9e111] [cursor=pointer]
- generic [ref=f9e116]:
- 'button "Time range selected: Last 55 minutes" [ref=f9e117] [cursor=pointer]'
- button "Zoom out time range" [ref=f9e122] [cursor=pointer]
- generic [ref=f9e126]:
- button "Run query" [ref=f9e127] [cursor=pointer]
- button "Auto refresh turned off. Choose refresh time interval" [ref=f9e131] [cursor=pointer]
- generic [ref=f9e135]:
- generic [ref=f9e139]:
- button "Collapse outline" [expanded] [ref=f9e141] [cursor=pointer]:
- img "arrow-from-right" [ref=f9e142]
- button "Queries" [ref=f9e145] [cursor=pointer]:
- img "arrow" [ref=f9e146]
- button "Graph" [ref=f9e150] [cursor=pointer]:
- img "graph-bar" [ref=f9e151]
- generic [ref=f9e158]:
- generic [ref=f9e160]:
- generic "Query editor row" [ref=f9e163]:
- generic [ref=f9e164]:
- generic [ref=f9e166]:
- generic [ref=f9e167]:
- button "Collapse query row" [expanded] [ref=f9e168] [cursor=pointer]
- generic [ref=f9e171]:
- button "Query editor row title A" [ref=f9e172] [cursor=pointer]:
- generic [ref=f9e173]: A
- emphasis [ref=f9e174]: (Prometheus)
- generic [ref=f9e175]:
- button "Show data source help" [ref=f9e177] [cursor=pointer]
- button "Duplicate query" [ref=f9e181] [cursor=pointer]
- button "Hide response" [ref=f9e185] [cursor=pointer]
- button "Remove query" [ref=f9e189] [cursor=pointer]
- button "Drag and drop to reorder" [ref=f9e192]:
- img "Drag and drop to reorder" [ref=f9e193]
- generic [ref=f9e196]:
- generic [ref=f9e197]:
- button "Kick start your query" [ref=f9e198] [cursor=pointer]
- generic [ref=f9e201]:
- generic [ref=f9e202] [cursor=pointer]: Explain
- generic [ref=f9e203]:
- checkbox "Explain Toggle switch" [ref=f9e204]
- generic "Toggle switch" [ref=f9e205] [cursor=pointer]
- radiogroup [ref=f9e210]:
- generic [ref=f9e211]:
- radio "Builder" [ref=f9e212] [cursor=pointer]
- generic [ref=f9e213] [cursor=pointer]: Builder
- generic [ref=f9e214]:
- radio "Code" [checked] [ref=f9e215] [cursor=pointer]
- generic [ref=f9e216] [cursor=pointer]: Code
- generic [ref=f9e218]:
- generic [ref=f9e220]:
- button "Metrics browser" [ref=f9e221] [cursor=pointer]
- code [ref=f9e228]:
- generic [ref=f9e229]:
- generic [ref=f9e234]: "up{job=~\"keycloak|node-exporter\"}"
- textbox "Editor content;Press Alt+F1 for Accessibility Options." [ref=f9e239]: "up{job=~\"keycloak|node-exporter\"}"
- 'button "Options Legend: {{job}} — {{pod}}{{node}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f9e245] [cursor=pointer]':
- generic [ref=f9e249]:
- heading "Options" [level=6] [ref=f9e250]
- generic [ref=f9e251]:
- generic [ref=f9e252]: "Legend: {{job}} — {{pod}}{{node}}"
- generic [ref=f9e253]: "Format: Time series"
- generic [ref=f9e254]: "Step: auto"
- generic [ref=f9e255]: "Type: Range"
- generic [ref=f9e256]: "Exemplars: false"
- generic [ref=f9e257]:
- button "Add query" [ref=f9e258] [cursor=pointer]
- button "Query history" [ref=f9e262] [cursor=pointer]
- button "Query inspector" [ref=f9e266] [cursor=pointer]
- main [ref=f9e270]:
- region [ref=f9e272]:
- generic [ref=f9e273]:
- heading "Graph" [level=2] [ref=f9e275]
- radiogroup [ref=f9e278]:
- generic [ref=f9e279]:
- radio "Lines" [checked] [ref=f9e280] [cursor=pointer]
- generic [ref=f9e281] [cursor=pointer]: Lines
- generic [ref=f9e282]:
- radio "Bars" [ref=f9e283] [cursor=pointer]
- generic [ref=f9e284] [cursor=pointer]: Bars
- generic [ref=f9e285]:
- radio "Points" [ref=f9e286] [cursor=pointer]
- generic [ref=f9e287] [cursor=pointer]: Points
- generic [ref=f9e288]:
- radio "Stacked lines" [ref=f9e289] [cursor=pointer]
- generic [ref=f9e290] [cursor=pointer]: Stacked lines
- generic [ref=f9e291]:
- radio "Stacked bars" [ref=f9e292] [cursor=pointer]
- generic [ref=f9e293] [cursor=pointer]: Stacked bars
- list [ref=f9e311]:
- listitem [ref=f9e312]:
- button "keycloak — keycloak-1kc-lab-1" [ref=f9e316] [cursor=pointer]
- listitem [ref=f9e317]:
- button "keycloak — keycloak-1kc-lab-1" [ref=f9e321] [cursor=pointer]
- listitem [ref=f9e322]:
- button "keycloak — keycloak-0kc-lab-2" [ref=f9e326] [cursor=pointer]
- listitem [ref=f9e327]:
- button "keycloak — keycloak-0kc-lab-2" [ref=f9e331] [cursor=pointer]
- listitem [ref=f9e332]:
- button "keycloak — keycloak-0kc-lab-2" [ref=f9e336] [cursor=pointer]
- listitem [ref=f9e337]:
- button "node-exporter — kc-lab-1" [ref=f9e341] [cursor=pointer]
- listitem [ref=f9e342]:
- button "node-exporter — kc-lab-2" [ref=f9e346] [cursor=pointer]
- generic [ref=f9e351]:
- alert
- alert
- complementary
- complementary
.playwright-mcp/page-2026-09-04T03-43-55-616Z.yml
+104
View File
@@ -0,0 +1,104 @@
- generic [ref=f12e4]:
- link "Skip to main content" [ref=f12e5] [cursor=pointer]:
- /url: "#pageContent"
- banner [ref=f12e7]:
- generic [ref=f12e8]:
- link [ref=f12e10] [cursor=pointer]:
- /url: /
- img "Grafana" [ref=f12e11]
- generic [ref=f12e14]:
- button "Search or jump to..." [ref=f12e18] [cursor=pointer]
- generic [ref=f12e19]: ctrl+k
- generic [ref=f12e23]:
- button "New" [ref=f12e24] [cursor=pointer]
- button "Help" [ref=f12e30] [cursor=pointer]
- button "News" [ref=f12e33] [cursor=pointer]
- button "Profile" [ref=f12e36] [cursor=pointer]:
- img "User avatar" [ref=f12e37]
- generic [ref=f12e38]:
- button "Open menu" [ref=f12e40] [cursor=pointer]
- navigation "Breadcrumbs" [ref=f12e43]:
- list [ref=f12e44]:
- listitem [ref=f12e45]:
- link "Home" [ref=f12e46] [cursor=pointer]:
- /url: /
- listitem [ref=f12e50]:
- link "Explore" [ref=f12e51] [cursor=pointer]:
- /url: /explore
- listitem [ref=f12e55]:
- generic "Prometheus" [ref=f12e56]
- generic [ref=f12e57]:
- button "Show more items" [ref=f12e60] [cursor=pointer]
- button "Toggle top search bar" [ref=f12e64] [cursor=pointer]
- main [ref=f12e70]:
- generic [ref=f12e72]:
- heading "Explore" [level=1] [ref=f12e73]
- generic [ref=f12e78]:
- navigation "Explore toolbar" [ref=f12e80]:
- navigation "Search links" [ref=f12e82]:
- generic [ref=f12e83]:
- button "Content outline" [expanded] [ref=f12e85] [cursor=pointer]:
- generic [ref=f12e88]: Outline
- generic [ref=f12e93] [cursor=pointer]:
- img "Prometheus logo" [ref=f12e95]
- textbox "Select a data source" [ref=f12e96]:
- /placeholder: ""
- button "Show more items" [ref=f12e102] [cursor=pointer]
- generic [ref=f12e106]:
- generic [ref=f12e110]:
- button "Collapse outline" [expanded] [ref=f12e112] [cursor=pointer]:
- img "arrow-from-right" [ref=f12e113]
- button "Queries" [ref=f12e116] [cursor=pointer]:
- img "arrow" [ref=f12e117]
- generic [ref=f12e124]:
- generic [ref=f12e126]:
- generic "Query editor row" [ref=f12e129]:
- generic [ref=f12e130]:
- generic [ref=f12e132]:
- generic [ref=f12e133]:
- button "Collapse query row" [expanded] [ref=f12e134] [cursor=pointer]
- generic [ref=f12e137]:
- button "Query editor row title A" [ref=f12e138] [cursor=pointer]:
- generic [ref=f12e139]: A
- emphasis [ref=f12e140]: (Prometheus)
- generic [ref=f12e141]:
- button "Show data source help" [ref=f12e143] [cursor=pointer]
- button "Duplicate query" [ref=f12e147] [cursor=pointer]
- button "Hide response" [ref=f12e151] [cursor=pointer]
- button "Remove query" [ref=f12e155] [cursor=pointer]
- button "Drag and drop to reorder" [ref=f12e158]:
- img "Drag and drop to reorder" [ref=f12e159]
- generic [ref=f12e162]:
- generic [ref=f12e163]:
- button "Kick start your query" [ref=f12e164] [cursor=pointer]
- generic [ref=f12e167]:
- generic [ref=f12e168] [cursor=pointer]: Explain
- generic [ref=f12e169]:
- checkbox "Explain Toggle switch" [ref=f12e170]
- generic "Toggle switch" [ref=f12e171] [cursor=pointer]
- radiogroup [ref=f12e176]:
- generic [ref=f12e177]:
- radio "Builder" [ref=f12e178] [cursor=pointer]
- generic [ref=f12e179] [cursor=pointer]: Builder
- generic [ref=f12e180]:
- radio "Code" [checked] [ref=f12e181] [cursor=pointer]
- generic [ref=f12e182] [cursor=pointer]: Code
- generic [ref=f12e184]:
- generic [ref=f12e186]:
- button "Loading metrics..." [disabled] [ref=f12e187] [cursor=pointer]
- generic [ref=f12e190]: Loading editor
- 'button "Options Legend: {{pod}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f12e198] [cursor=pointer]':
- generic [ref=f12e202]:
- heading "Options" [level=6] [ref=f12e203]
- generic [ref=f12e204]:
- generic [ref=f12e205]: "Legend: {{pod}}"
- generic [ref=f12e206]: "Format: Time series"
- generic [ref=f12e207]: "Step: auto"
- generic [ref=f12e208]: "Type: Range"
- generic [ref=f12e209]: "Exemplars: false"
- generic [ref=f12e210]:
- button "Add query" [ref=f12e211] [cursor=pointer]
- button "Query history" [ref=f12e215] [cursor=pointer]
- button "Query inspector" [ref=f12e219] [cursor=pointer]
- generic:
- main
.playwright-mcp/page-2026-09-04T04-16-33-535Z.yml
+145
View File
@@ -0,0 +1,145 @@
- generic [active] [ref=f15e1]:
- generic [ref=f15e4]:
- link "Skip to main content" [ref=f15e5] [cursor=pointer]:
- /url: "#pageContent"
- banner [ref=f15e7]:
- generic [ref=f15e8]:
- link [ref=f15e10] [cursor=pointer]:
- /url: /
- img "Grafana" [ref=f15e11]
- generic [ref=f15e14]:
- button "Search or jump to..." [ref=f15e18] [cursor=pointer]
- generic [ref=f15e19]: ctrl+k
- generic [ref=f15e23]:
- button "New" [ref=f15e24] [cursor=pointer]
- button "Help" [ref=f15e30] [cursor=pointer]
- button "News" [ref=f15e33] [cursor=pointer]
- button "Profile" [ref=f15e36] [cursor=pointer]:
- img "User avatar" [ref=f15e37]
- generic [ref=f15e38]:
- button "Open menu" [ref=f15e40] [cursor=pointer]
- navigation "Breadcrumbs" [ref=f15e43]:
- list [ref=f15e44]:
- listitem [ref=f15e45]:
- link "Home" [ref=f15e46] [cursor=pointer]:
- /url: /
- listitem [ref=f15e50]:
- link "Explore" [ref=f15e51] [cursor=pointer]:
- /url: /explore
- listitem [ref=f15e55]:
- generic "Prometheus" [ref=f15e56]
- generic [ref=f15e57]:
- generic [ref=f15e60]:
- button "Copy shortened URL" [ref=f15e61] [cursor=pointer]
- button "Open copy link options" [ref=f15e64] [cursor=pointer]
- button "Toggle top search bar" [ref=f15e68] [cursor=pointer]
- main [ref=f15e74]:
- generic [ref=f15e76]:
- heading "Explore" [level=1] [ref=f15e77]
- generic [ref=f15e82]:
- navigation "Explore toolbar" [ref=f15e84]:
- navigation "Search links" [ref=f15e86]:
- generic [ref=f15e87]:
- button "Content outline" [expanded] [ref=f15e89] [cursor=pointer]:
- generic [ref=f15e92]: Outline
- generic [ref=f15e97] [cursor=pointer]:
- img "Prometheus logo" [ref=f15e99]
- textbox "Select a data source" [ref=f15e100]:
- /placeholder: ""
- generic [ref=f15e104]:
- button "Split the pane" [ref=f15e106] [cursor=pointer]:
- generic [ref=f15e109]: Split
- button "Add" [ref=f15e111] [cursor=pointer]
- generic [ref=f15e116]:
- 'button "Time range selected: Last 30 minutes" [ref=f15e117] [cursor=pointer]'
- button "Zoom out time range" [ref=f15e122] [cursor=pointer]
- generic [ref=f15e126]:
- button "Run query" [ref=f15e127] [cursor=pointer]
- button "Auto refresh turned off. Choose refresh time interval" [ref=f15e131] [cursor=pointer]
- generic [ref=f15e135]:
- generic [ref=f15e139]:
- button "Collapse outline" [expanded] [ref=f15e141] [cursor=pointer]:
- img "arrow-from-right" [ref=f15e142]
- button "Queries" [ref=f15e145] [cursor=pointer]:
- img "arrow" [ref=f15e146]
- button "Graph" [ref=f15e150] [cursor=pointer]:
- img "graph-bar" [ref=f15e151]
- generic [ref=f15e158]:
- generic [ref=f15e160]:
- generic "Query editor row" [ref=f15e163]:
- generic [ref=f15e164]:
- generic [ref=f15e166]:
- generic [ref=f15e167]:
- button "Collapse query row" [expanded] [ref=f15e168] [cursor=pointer]
- generic [ref=f15e171]:
- button "Query editor row title A" [ref=f15e172] [cursor=pointer]:
- generic [ref=f15e173]: A
- emphasis [ref=f15e174]: (Prometheus)
- generic [ref=f15e175]:
- button "Show data source help" [ref=f15e177] [cursor=pointer]
- button "Duplicate query" [ref=f15e181] [cursor=pointer]
- button "Hide response" [ref=f15e185] [cursor=pointer]
- button "Remove query" [ref=f15e189] [cursor=pointer]
- button "Drag and drop to reorder" [ref=f15e192]:
- img "Drag and drop to reorder" [ref=f15e193]
- generic [ref=f15e196]:
- generic [ref=f15e197]:
- button "Kick start your query" [ref=f15e198] [cursor=pointer]
- generic [ref=f15e201]:
- generic [ref=f15e202] [cursor=pointer]: Explain
- generic [ref=f15e203]:
- checkbox "Explain Toggle switch" [ref=f15e204]
- generic "Toggle switch" [ref=f15e205] [cursor=pointer]
- radiogroup [ref=f15e210]:
- generic [ref=f15e211]:
- radio "Builder" [ref=f15e212] [cursor=pointer]
- generic [ref=f15e213] [cursor=pointer]: Builder
- generic [ref=f15e214]:
- radio "Code" [checked] [ref=f15e215] [cursor=pointer]
- generic [ref=f15e216] [cursor=pointer]: Code
- generic [ref=f15e218]:
- generic [ref=f15e220]:
- button "Loading metrics..." [disabled] [ref=f15e221] [cursor=pointer]
- code [ref=f15e228]:
- generic [ref=f15e229]:
- generic [ref=f15e234]: agroal_blocking_time_max_milliseconds
- textbox "Editor content;Press Alt+F1 for Accessibility Options." [ref=f15e239]: agroal_blocking_time_max_milliseconds
- 'button "Options Legend: blocking max - {{pod}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f15e245] [cursor=pointer]':
- generic [ref=f15e249]:
- heading "Options" [level=6] [ref=f15e250]
- generic [ref=f15e251]:
- generic [ref=f15e252]: "Legend: blocking max - {{pod}}"
- generic [ref=f15e253]: "Format: Time series"
- generic [ref=f15e254]: "Step: auto"
- generic [ref=f15e255]: "Type: Range"
- generic [ref=f15e256]: "Exemplars: false"
- generic [ref=f15e257]:
- button "Add query" [ref=f15e258] [cursor=pointer]
- button "Query history" [ref=f15e262] [cursor=pointer]
- button "Query inspector" [ref=f15e266] [cursor=pointer]
- main [ref=f15e270]:
- region [ref=f15e272]:
- generic [ref=f15e273]:
- heading "Graph" [level=2] [ref=f15e275]
- radiogroup [ref=f15e278]:
- generic [ref=f15e279]:
- radio "Lines" [checked] [ref=f15e280] [cursor=pointer]
- generic [ref=f15e281] [cursor=pointer]: Lines
- generic [ref=f15e282]:
- radio "Bars" [ref=f15e283] [cursor=pointer]
- generic [ref=f15e284] [cursor=pointer]: Bars
- generic [ref=f15e285]:
- radio "Points" [ref=f15e286] [cursor=pointer]
- generic [ref=f15e287] [cursor=pointer]: Points
- generic [ref=f15e288]:
- radio "Stacked lines" [ref=f15e289] [cursor=pointer]
- generic [ref=f15e290] [cursor=pointer]: Stacked lines
- generic [ref=f15e291]:
- radio "Stacked bars" [ref=f15e292] [cursor=pointer]
- generic [ref=f15e293] [cursor=pointer]: Stacked bars
- generic [ref=f15e294]: Loading plugin panel...
- generic [ref=f15e299]:
- alert
- alert
- complementary
- complementary
.playwright-mcp/page-2026-09-04T04-19-59-064Z.yml
+56
View File
@@ -0,0 +1,56 @@
- generic [ref=f18e4]:
- link "Skip to main content" [ref=f18e5] [cursor=pointer]:
- /url: "#pageContent"
- banner [ref=f18e7]:
- generic [ref=f18e8]:
- link [ref=f18e10] [cursor=pointer]:
- /url: /
- img "Grafana" [ref=f18e11]
- generic [ref=f18e14]:
- button "Search or jump to..." [ref=f18e18] [cursor=pointer]
- generic [ref=f18e19]: ctrl+k
- generic [ref=f18e23]:
- button "New" [ref=f18e24] [cursor=pointer]
- button "Help" [ref=f18e30] [cursor=pointer]
- button "News" [ref=f18e33] [cursor=pointer]
- button "Profile" [ref=f18e36] [cursor=pointer]:
- img "User avatar" [ref=f18e37]
- generic [ref=f18e38]:
- button "Open menu" [ref=f18e40] [cursor=pointer]
- navigation "Breadcrumbs" [ref=f18e43]:
- list [ref=f18e44]:
- listitem [ref=f18e45]:
- link "Home" [ref=f18e46] [cursor=pointer]:
- /url: /
- listitem [ref=f18e50]:
- link "Explore" [ref=f18e51] [cursor=pointer]:
- /url: /explore
- listitem [ref=f18e55]:
- generic "Prometheus" [ref=f18e56]
- generic [ref=f18e57]:
- button "Show more items" [ref=f18e60] [cursor=pointer]
- button "Toggle top search bar" [ref=f18e64] [cursor=pointer]
- main [ref=f18e70]:
- generic [ref=f18e72]:
- heading "Explore" [level=1] [ref=f18e73]
- generic [ref=f18e78]:
- navigation "Explore toolbar" [ref=f18e80]:
- navigation "Search links" [ref=f18e82]:
- generic [ref=f18e83]:
- button "Content outline" [expanded] [ref=f18e85] [cursor=pointer]:
- generic [ref=f18e88]: Outline
- generic [ref=f18e93] [cursor=pointer]:
- img "Prometheus logo" [ref=f18e95]
- textbox "Select a data source" [ref=f18e96]:
- /placeholder: ""
- button "Show more items" [ref=f18e102] [cursor=pointer]
- generic [ref=f18e106]:
- button "Collapse outline" [expanded] [ref=f18e112] [cursor=pointer]:
- img "arrow-from-right" [ref=f18e113]
- generic [ref=f18e120]:
- generic [ref=f18e123]:
- button "Add query" [ref=f18e124] [cursor=pointer]
- button "Query history" [ref=f18e128] [cursor=pointer]
- button "Query inspector" [ref=f18e132] [cursor=pointer]
- generic:
- main
deploy/lab/k8s/a1-block-jgroups-transport.yaml
+46
View File
@@ -0,0 +1,46 @@
# Experiment A-1 — cut the JGroups transport (TCP 7800) while leaving discovery alone.
#
# The point is to separate two things that are easy to conflate:
#
# discovery how the nodes FIND each other -> PostgreSQL JGROUPS_PING table
# transport how they actually TALK -> TCP 7800
#
# Blocking only the transport produces a state that cannot happen on a single
# node: both members stay registered in the database, so each believes the other
# exists, yet no message gets through.
#
# kubectl apply -f deploy/lab/k8s/a1-block-jgroups-transport.yaml
# kubectl -n keycloak-lab delete networkpolicy a1-block-jgroups-transport
#
# NetworkPolicy is an ALLOWLIST, not a firewall with deny rules. There is no way
# to write "deny 7800". The moment a pod is selected by a policy carrying
# policyTypes: [Ingress], every inbound port is denied unless a rule permits it.
# So 7800 is blocked by *omission*: 8080 and 9000 are listed, 7800 is not.
#
# That makes the two allow rules load-bearing — get them wrong and the experiment
# measures a dead Keycloak instead of a partitioned cluster:
#
# 8080 the HTTP endpoint. Traefik, the other pod's REST calls, and the probe
# traffic all arrive here.
# 9000 the management port: /health/started, /health/ready, /health/live and
# /metrics. Losing it means the kubelet fails the readiness probe and
# kills the pod — the cluster would break for the wrong reason.
#
# Both rules deliberately omit `from:`, which allows those ports from any source.
# Narrowing the source is not the subject here; the 2-hop experiment already
# established how to do that by label when it matters.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: a1-block-jgroups-transport
namespace: keycloak-lab
spec:
podSelector:
matchLabels:
app: keycloak
policyTypes: [Ingress]
ingress:
- ports:
- { port: 8080, protocol: TCP } # HTTP — must stay open
- { port: 9000, protocol: TCP } # health + metrics — must stay open
# 7800 is absent on purpose. That is the whole experiment.
deploy/lab/k8s/keycloak-cluster.yaml
+277
View File
@@ -0,0 +1,277 @@
# Keycloak multi-node cluster with PostgreSQL.
#
# Goal of this manifest: two Keycloak pods on two different nodes must discover
# each other and form one Infinispan cluster. Keycloak 26 discovers peers through
# the database (jdbc-ping) rather than multicast, writing to a JGROUPS_PING table,
# but the cluster traffic itself runs over TCP 7800 between the pods. Those are
# two separate mechanisms, which is why "registered in the DB but not clustered"
# is a real failure mode — and one that a single node cannot reproduce.
#
# kubectl apply -f deploy/lab/k8s/keycloak-cluster.yaml
# kubectl -n keycloak-lab rollout status statefulset/keycloak --timeout=600s
#
# Secrets are plain here. Proper secret handling is roadmap item 11; keeping it
# visible for now is deliberate so the gap is obvious rather than forgotten.
apiVersion: v1
kind: Namespace
metadata:
name: keycloak-lab
---
apiVersion: v1
kind: Secret
metadata:
name: keycloak-lab-secrets
namespace: keycloak-lab
type: Opaque
stringData:
POSTGRES_PASSWORD: lab-postgres-change-me
KC_BOOTSTRAP_ADMIN_PASSWORD: lab-admin-change-me
---
# PostgreSQL. local-path binds the volume to whichever node the pod lands on, so
# the database is effectively pinned to one node. That is not a flaw here: it is
# what makes "the database node dies" a meaningful experiment later.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: postgres-data
namespace: keycloak-lab
spec:
accessModes: [ReadWriteOnce]
storageClassName: local-path
resources:
requests:
storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
namespace: keycloak-lab
spec:
replicas: 1
strategy:
type: Recreate # RWO volume cannot be mounted by two pods at once
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:16-alpine
ports:
- containerPort: 5432
name: postgres
env:
- name: POSTGRES_DB
value: keycloak
- name: POSTGRES_USER
value: keycloak
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: keycloak-lab-secrets
key: POSTGRES_PASSWORD
# The image refuses to initialise into a non-empty mount, and
# local-path volumes are clean, but this keeps the data one level
# down so a lost+found or similar never blocks initdb.
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
readinessProbe:
exec:
command: ["sh", "-c", "pg_isready -U keycloak -d keycloak"]
initialDelaySeconds: 10
periodSeconds: 5
resources:
requests:
memory: 192Mi
cpu: 50m
limits:
memory: 512Mi
volumes:
- name: data
persistentVolumeClaim:
claimName: postgres-data
---
apiVersion: v1
kind: Service
metadata:
name: postgres
namespace: keycloak-lab
spec:
selector:
app: postgres
ports:
- port: 5432
targetPort: postgres
---
# Keycloak. A StatefulSet rather than a Deployment so each pod keeps a stable
# name (keycloak-0, keycloak-1); cluster membership is far easier to read in
# logs and in the JGROUPS_PING table when the identities do not churn.
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: keycloak
namespace: keycloak-lab
spec:
serviceName: keycloak-headless
replicas: 2
podManagementPolicy: Parallel # both pods start together, so they race to
# register — which is the interesting case
selector:
matchLabels:
app: keycloak
template:
metadata:
labels:
app: keycloak
spec:
# One pod per node. Two pods on one node would share a kernel and make the
# 7800 blocking experiment meaningless.
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
app: keycloak
containers:
- name: keycloak
image: quay.io/keycloak/keycloak:26.7.0
# "start", not "start-dev". Dev mode forces cache=local and there is
# no cluster to form at all.
args: ["start"]
ports:
- containerPort: 8080
name: http
- containerPort: 9000
name: management
- containerPort: 7800
name: jgroups
env:
- name: KC_DB
value: postgres
- name: KC_DB_URL
value: jdbc:postgresql://postgres:5432/keycloak
- name: KC_DB_USERNAME
value: keycloak
- name: KC_DB_PASSWORD
valueFrom:
secretKeyRef:
name: keycloak-lab-secrets
key: POSTGRES_PASSWORD
# Settings confirmed by the two-hop header measurement.
# KC_HOSTNAME carries the full external URL, which pins scheme and
# host for issuer and redirect URLs regardless of headers.
# KC_PROXY_HEADERS is the separate opt-in that lets the forwarded
# client address through — the same kind of switch as Spring's
# forward-headers-strategy. See docs/two-hop-proxy-header-contract.md.
- name: KC_HOSTNAME
value: https://auth.hyeonworks.com
- name: KC_HOSTNAME_STRICT
value: "true"
- name: KC_PROXY_HEADERS
value: xforwarded
- name: KC_HTTP_ENABLED
value: "true"
- name: KC_HEALTH_ENABLED
value: "true"
- name: KC_METRICS_ENABLED
value: "true"
# Without an explicit cap the JVM sizes its heap from the container
# limit and this lab has roughly 3.8GB of guest headroom in total.
- name: JAVA_OPTS_KC_HEAP
value: "-Xms256m -Xmx512m"
- name: KC_BOOTSTRAP_ADMIN_USERNAME
value: admin
- name: KC_BOOTSTRAP_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: keycloak-lab-secrets
key: KC_BOOTSTRAP_ADMIN_PASSWORD
# Keycloak serves health and metrics on the management port (9000),
# not on 8080, since version 25.
startupProbe:
httpGet:
path: /health/started
port: management
periodSeconds: 10
failureThreshold: 60 # first boot runs an implicit build
readinessProbe:
httpGet:
path: /health/ready
port: management
periodSeconds: 10
livenessProbe:
httpGet:
path: /health/live
port: management
periodSeconds: 30
resources:
requests:
memory: 640Mi
cpu: 100m
limits:
memory: 900Mi
---
# Headless service. Not required for jdbc-ping discovery, which goes through the
# database, but it gives each pod a stable DNS name for direct inspection.
apiVersion: v1
kind: Service
metadata:
name: keycloak-headless
namespace: keycloak-lab
spec:
clusterIP: None
selector:
app: keycloak
ports:
- port: 8080
targetPort: http
name: http
- port: 9000
targetPort: management
name: management
---
apiVersion: v1
kind: Service
metadata:
name: keycloak
namespace: keycloak-lab
spec:
selector:
app: keycloak
ports:
- port: 8080
targetPort: http
name: http
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: keycloak
namespace: keycloak-lab
spec:
ingressClassName: traefik
rules:
- host: auth.hyeonworks.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: keycloak
port:
number: 8080
deploy/lab/k8s/observability.yaml
+373
View File
@@ -0,0 +1,373 @@
# Prometheus + node-exporter + Grafana.
#
# Purpose: during a fault-injection experiment, know *which signal moved first*.
# Without a metrics store the only record is whatever scrolled past in a terminal,
# and "the cluster recovered in about a minute" is not a measurement.
#
# kubectl apply -f deploy/lab/k8s/observability.yaml
# kubectl -n observability rollout status deployment/prometheus --timeout=300s
#
# Placement decision — Prometheus and Grafana are pinned to the control-plane
# node (kc-lab-1). An observability stack must not share a failure domain with
# the thing it observes. With only two nodes that cannot be fully avoided, so the
# rule here is: the node that gets killed in experiments is the *agent*
# (kc-lab-2, holding keycloak-0 and postgres), and everything needed to watch
# that happen lives on the server node.
apiVersion: v1
kind: Namespace
metadata:
name: observability
---
# Prometheus discovers scrape targets by querying the Kubernetes API, so it
# needs read access to nodes, services, endpoints and pods. Without this the
# kubernetes_sd_configs below silently return no targets.
apiVersion: v1
kind: ServiceAccount
metadata:
name: prometheus
namespace: observability
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: prometheus
rules:
- apiGroups: [""]
# nodes/proxy is required in addition to nodes/metrics: the kubelet job
# reaches each node through the API server's proxy subresource
# (/api/v1/nodes/<name>/proxy/metrics). Without it every kubelet target
# fails with 403 Forbidden while the other jobs stay green — a partial
# failure that is easy to miss unless the target list is checked.
resources: [nodes, nodes/metrics, nodes/proxy, services, endpoints, pods]
verbs: [get, list, watch]
- nonResourceURLs: ["/metrics"]
verbs: [get]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: prometheus
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: prometheus
subjects:
- kind: ServiceAccount
name: prometheus
namespace: observability
---
apiVersion: v1
kind: ConfigMap
metadata:
name: prometheus-config
namespace: observability
data:
prometheus.yml: |
global:
# 15s is short for production but right here: a node loss should show up
# within a couple of samples, not a minute later.
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
# Prometheus scraping itself. Useful as a control: if this target is down,
# the problem is Prometheus, not the thing being measured.
- job_name: prometheus
static_configs:
- targets: ['localhost:9090']
# Keycloak. Metrics live on the management port 9000, not 8080 — the same
# split that the health probes use. KC_METRICS_ENABLED=true is already set
# on the StatefulSet.
#
# Discovery is by endpoints rather than a static list because pod IPs
# change on every restart; that was observed directly when the lab was
# power-cycled and every pod came back with a new address.
- job_name: keycloak
kubernetes_sd_configs:
- role: endpoints
namespaces:
names: [keycloak-lab]
relabel_configs:
- source_labels: [__meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
action: keep
regex: keycloak-headless;management
- source_labels: [__meta_kubernetes_pod_name]
target_label: pod
- source_labels: [__meta_kubernetes_pod_node_name]
target_label: node
# node-exporter, one per node via DaemonSet. This is what answers
# "did the machine die or did the process die".
- job_name: node-exporter
kubernetes_sd_configs:
- role: endpoints
namespaces:
names: [observability]
relabel_configs:
- source_labels: [__meta_kubernetes_service_name]
action: keep
regex: node-exporter
- source_labels: [__meta_kubernetes_pod_node_name]
target_label: node
# The kubelet's own metrics, reached through the API server proxy so no
# extra port needs opening.
- job_name: kubelet
scheme: https
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
insecure_skip_verify: true
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
kubernetes_sd_configs:
- role: node
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [__meta_kubernetes_node_name]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/${1}/proxy/metrics
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: prometheus-data
namespace: observability
spec:
accessModes: [ReadWriteOnce]
storageClassName: local-path
resources:
requests:
storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus
namespace: observability
spec:
replicas: 1
strategy:
type: Recreate # RWO volume; two pods cannot mount it at once
selector:
matchLabels:
app: prometheus
template:
metadata:
labels:
app: prometheus
spec:
serviceAccountName: prometheus
# See the placement note at the top of this file.
nodeSelector:
node-role.kubernetes.io/control-plane: "true"
securityContext:
fsGroup: 65534 # the image runs as nobody and must own the volume
containers:
- name: prometheus
image: prom/prometheus:v3.1.0
args:
- --config.file=/etc/prometheus/prometheus.yml
- --storage.tsdb.path=/prometheus
# 7 days is far more than an experiment needs and keeps the volume
# small enough that it never becomes the reason a node fills up.
- --storage.tsdb.retention.time=7d
- --web.enable-lifecycle
ports:
- containerPort: 9090
name: http
volumeMounts:
- name: config
mountPath: /etc/prometheus
- name: data
mountPath: /prometheus
readinessProbe:
httpGet: { path: /-/ready, port: http }
initialDelaySeconds: 10
livenessProbe:
httpGet: { path: /-/healthy, port: http }
initialDelaySeconds: 30
resources:
requests: { memory: 256Mi, cpu: 50m }
limits: { memory: 640Mi }
volumes:
- name: config
configMap:
name: prometheus-config
- name: data
persistentVolumeClaim:
claimName: prometheus-data
---
apiVersion: v1
kind: Service
metadata:
name: prometheus
namespace: observability
spec:
selector:
app: prometheus
ports:
- port: 9090
targetPort: http
---
# node-exporter. A DaemonSet so every node reports, including one that is about
# to be killed — the last samples before it goes silent are the interesting part.
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: node-exporter
namespace: observability
spec:
selector:
matchLabels:
app: node-exporter
template:
metadata:
labels:
app: node-exporter
spec:
# Host namespaces: the point is to measure the machine, not the container.
hostNetwork: true
hostPID: true
tolerations:
- operator: Exists # must also run on tainted nodes
containers:
- name: node-exporter
image: prom/node-exporter:v1.8.2
args:
- --path.procfs=/host/proc
- --path.sysfs=/host/sys
- --path.rootfs=/host/root
- --collector.filesystem.mount-points-exclude=^/(dev|proc|sys|var/lib/docker/.+|var/lib/kubelet/.+)($|/)
ports:
- containerPort: 9100
name: metrics
hostPort: 9100
volumeMounts:
- { name: proc, mountPath: /host/proc, readOnly: true }
- { name: sys, mountPath: /host/sys, readOnly: true }
- { name: rootfs, mountPath: /host/root, readOnly: true, mountPropagation: HostToContainer }
resources:
requests: { memory: 32Mi, cpu: 20m }
limits: { memory: 96Mi }
volumes:
- { name: proc, hostPath: { path: /proc } }
- { name: sys, hostPath: { path: /sys } }
- { name: rootfs, hostPath: { path: / } }
---
apiVersion: v1
kind: Service
metadata:
name: node-exporter
namespace: observability
spec:
clusterIP: None # headless: Prometheus wants each pod, not a VIP
selector:
app: node-exporter
ports:
- port: 9100
targetPort: metrics
name: metrics
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: grafana
namespace: observability
spec:
replicas: 1
selector:
matchLabels:
app: grafana
template:
metadata:
labels:
app: grafana
spec:
nodeSelector:
node-role.kubernetes.io/control-plane: "true"
containers:
- name: grafana
image: grafana/grafana:11.4.0
ports:
- containerPort: 3000
name: http
env:
- name: GF_SECURITY_ADMIN_USER
value: admin
- name: GF_SECURITY_ADMIN_PASSWORD
value: lab-grafana-change-me
# Grafana builds absolute URLs for redirects and asset paths. Behind
# the nginx -> Traefik chain it must be told the external address,
# for exactly the reason Keycloak needs KC_HOSTNAME. Without it,
# login redirects come back as http://<pod-ip>:3000.
- name: GF_SERVER_ROOT_URL
value: https://app2.hyeonworks.com
volumeMounts:
- name: datasources
mountPath: /etc/grafana/provisioning/datasources
readinessProbe:
httpGet: { path: /api/health, port: http }
initialDelaySeconds: 15
resources:
requests: { memory: 128Mi, cpu: 50m }
limits: { memory: 320Mi }
volumes:
- name: datasources
configMap:
name: grafana-datasources
---
# Provisioning the datasource as a file means Grafana comes up already wired to
# Prometheus. Clicking through the UI would leave the configuration only in
# Grafana's own database, which is emptyDir here and disappears on restart.
apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-datasources
namespace: observability
data:
prometheus.yaml: |
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
access: proxy
url: http://prometheus.observability.svc:9090
isDefault: true
---
apiVersion: v1
kind: Service
metadata:
name: grafana
namespace: observability
spec:
selector:
app: grafana
ports:
- port: 3000
targetPort: http
---
# Grafana is published on app2.hyeonworks.com because that name is already in
# the wildcard-free certificate (auth / app1 / app2) and is otherwise unused.
# It moves when app2 is needed for the SSO experiment.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: grafana
namespace: observability
spec:
ingressClassName: traefik
rules:
- host: app2.hyeonworks.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: grafana
port:
number: 3000
deploy/lab/scripts/experiment-cache-ownership.sh
Executable
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env bash
# Experiment 0c — where does a session entry actually live?
#
# Experiment 0b showed keycloak-1's session cache never moved when keycloak-0
# handled a login. That leaves two explanations:
#
# (a) a DISTRIBUTED cache with owners=1 — entries are spread across nodes by
# consistent hashing, and this one happened to land on keycloak-0;
# (b) a LOCAL cache — each node only ever caches what it handled itself.
#
# They are distinguished by driving logins at the OTHER node. Under (a) the
# entries would keep landing on both nodes regardless of who was asked. Under
# (b) the count rises only on the node that received the request.
set -uo pipefail
NS="${NS:-keycloak-lab}"
N="${N:-5}"
K0_IP=$(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.status.podIP}')
K1_IP=$(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.status.podIP}')
ADMIN_PW=$(kubectl -n "$NS" get secret keycloak-lab-secrets \
-o jsonpath='{.data.KC_BOOTSTRAP_ADMIN_PASSWORD}' | base64 -d)
echo "수집 시각: $(date '+%Y-%m-%d %H:%M:%S %Z')"
echo " keycloak-0 = $K0_IP ($(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.spec.nodeName}'))"
echo " keycloak-1 = $K1_IP ($(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.spec.nodeName}'))"
echo
kubectl -n "$NS" run kc-own --rm -i --restart=Never \
--image=curlimages/curl:8.11.1 --quiet --command -- sh -c "
O=/tmp/o; : > \$O
ent() {
curl -s --retry 3 --max-time 20 http://\$1:9000/metrics \
| grep -E '^vendor_statistics_approximate_entries_unique.cache=.sessions' \
| awk '{print \$NF}'
}
login() { i=0; while [ \$i -lt $N ]; do
curl -s -o /dev/null -X POST http://\$1:8080/realms/master/protocol/openid-connect/token \
-d grant_type=password -d client_id=admin-cli \
-d username=admin -d 'password=$ADMIN_PW'
i=\$((i+1)); done; sleep 5; }
{
printf '%-32s %12s %12s\n' '단계' 'k0 entries' 'k1 entries'
printf '%-32s %12s %12s\n' '시작' \"\$(ent $K0_IP)\" \"\$(ent $K1_IP)\"
login $K1_IP
printf '%-32s %12s %12s\n' 'keycloak-1 에 로그인 ${N}회' \"\$(ent $K0_IP)\" \"\$(ent $K1_IP)\"
login $K0_IP
printf '%-32s %12s %12s\n' 'keycloak-0 에 로그인 ${N}회' \"\$(ent $K0_IP)\" \"\$(ent $K1_IP)\"
} >> \$O
cat \$O
" 2>&1 | grep -v '^pod .* deleted$'
echo
echo "=== 대조: PostgreSQL 에는 몇 건인가 ==="
kubectl -n "$NS" exec deploy/postgres -- psql -U keycloak -d keycloak -tAc \
"select count(*) from offline_user_session where offline_flag='0'" 2>/dev/null | sed 's/^/ online 세션 /'
deploy/lab/scripts/experiment-cache-replication-delta.sh
Executable
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env bash
# Experiment 0b — does the Infinispan cache itself replicate, or do both nodes
# merely agree because they read the same database?
#
# Experiment 0 proved the two nodes give the same answers. That alone does NOT
# prove Infinispan replicated anything: with persistent-user-sessions (the
# Keycloak 26 default) the session is written to PostgreSQL, so two nodes reading
# one database would agree even with the cache disabled entirely.
#
# This script separates the two by measuring the cache counters on BOTH nodes
# around a single login. If the write on keycloak-0 shows up as cache activity
# on keycloak-1, the replication is real and not a database artifact.
set -uo pipefail
NS="${NS:-keycloak-lab}"
K0_IP=$(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.status.podIP}')
K1_IP=$(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.status.podIP}')
ADMIN_PW=$(kubectl -n "$NS" get secret keycloak-lab-secrets \
-o jsonpath='{.data.KC_BOOTSTRAP_ADMIN_PASSWORD}' | base64 -d)
echo "수집 시각: $(date '+%Y-%m-%d %H:%M:%S %Z')"
echo
# 파드 출력을 스트리밍으로 받으면 조각이 유실된다. 실제로 첫 시도에서
# keycloak-1 의 스냅샷과 그 다음 마커가 통째로 사라져 델타가 0 으로 보였다.
# 파드 안에서 파일로 모았다가 마지막에 한 번만 내보낸다.
kubectl -n "$NS" run kc-delta --rm -i --restart=Never \
--image=curlimages/curl:8.11.1 --quiet --command -- sh -c "
set -u
K0='http://$K0_IP'; K1='http://$K1_IP'
O=/tmp/o.txt; : > \$O
snap() {
curl -s --retry 3 --retry-connrefused --max-time 20 \$1:9000/metrics \
| grep -E '^vendor_(statistics_(stores|hits|misses|approximate_entries_unique)|rpc_manager_replication_count)\{cache=\"(sessions|clientSessions)\"' \
| sed 's/,cache_manager=\"keycloak\"//; s/,node=\"[^\"]*\"//' >> \$O
}
echo '###BEFORE_K0' >> \$O; snap \$K0
echo '###BEFORE_K1' >> \$O; snap \$K1
echo '###LOGIN' >> \$O
curl -s -o /dev/null -w 'http_code=%{http_code}\n' -X POST \
\"\$K0:8080/realms/master/protocol/openid-connect/token\" \
-d grant_type=password -d client_id=admin-cli \
-d username=admin -d 'password=$ADMIN_PW' >> \$O
sleep 5
echo '###AFTER_K0' >> \$O; snap \$K0
echo '###AFTER_K1' >> \$O; snap \$K1
echo '###END' >> \$O
cat \$O
" 2>&1 | grep -v '^pod .* deleted$' > /tmp/cache-delta.txt
python3 - /tmp/cache-delta.txt <<'PY'
import re, sys
raw = open(sys.argv[1]).read()
blocks, cur = {}, None
for line in raw.splitlines():
if line.startswith('###'):
cur = line[3:]; blocks[cur] = {}
elif cur and '{' in line:
m = re.match(r'(\S+?)\{cache="(\w+)"\}\s+(\S+)', line)
if m:
blocks[cur][(m.group(1), m.group(2))] = float(m.group(3))
print('=== 로그인은 keycloak-0 에만 보냈다 ===')
code = [l for l in raw.splitlines() if l.startswith('http_code=')]
print(' 로그인 응답: ' + (code[0] if code else '없음'))
for n in ('BEFORE_K0','BEFORE_K1','AFTER_K0','AFTER_K1'):
if not blocks.get(n):
print(f' !! {n} 스냅샷이 비었다 — 델타를 신뢰할 수 없다')
print()
hdr = f" {'계수기':<42} {'캐시':<15} {'전':>8} {'후':>8} {'증가':>7}"
for node in ('K0', 'K1'):
who = 'keycloak-0 (로그인을 받은 노드)' if node == 'K0' else 'keycloak-1 (아무 요청도 받지 않은 노드)'
print(f'=== {who} ===')
print(hdr)
b, a = blocks.get(f'BEFORE_{node}', {}), blocks.get(f'AFTER_{node}', {})
for k in sorted(set(b) | set(a)):
before, after = b.get(k[0:2], 0.0), a.get(k[0:2], 0.0)
d = after - before
mark = ' ←' if d else ''
name = k[0].replace('vendor_statistics_', '').replace('vendor_rpc_manager_', 'rpc.')
print(f" {name:<42} {k[1]:<15} {before:>8.0f} {after:>8.0f} {d:>+7.0f}{mark}")
print()
PY
deploy/lab/scripts/experiment-session-read-path.sh
Executable
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env bash
# Experiment 0d — capture the actual SQL that the OTHER node runs.
#
# Experiments 0b/0c showed that session entries never appear in keycloak-1's
# memory, yet keycloak-1 can use a session keycloak-0 created. The conclusion
# "keycloak-1 reads it from PostgreSQL" was an inference, not an observation.
#
# This script turns on statement logging in PostgreSQL for a few seconds, sends
# ONE refresh request to keycloak-1 for a session born on keycloak-0, and greps
# the database log for that session id. If the inference is right, the SQL is
# there, issued from keycloak-1's pod IP.
#
# It also checks whether serving that request makes keycloak-1 cache the session
# — which sharpens "each node caches what it handled" from "what it logged in"
# to "what it touched".
set -uo pipefail
NS="${NS:-keycloak-lab}"
PSQL="kubectl -n $NS exec deploy/postgres -- psql -U keycloak -d keycloak -tAc"
K0_IP=$(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.status.podIP}')
K1_IP=$(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.status.podIP}')
ADMIN_PW=$(kubectl -n "$NS" get secret keycloak-lab-secrets \
-o jsonpath='{.data.KC_BOOTSTRAP_ADMIN_PASSWORD}' | base64 -d)
echo "수집 시각: $(date '+%Y-%m-%d %H:%M:%S %Z')"
echo " keycloak-0 = $K0_IP (세션을 만드는 노드)"
echo " keycloak-1 = $K1_IP (읽기만 하는 노드)"
echo
# %h 를 넣어야 어느 파드가 보낸 질의인지 로그에서 구분된다.
echo "=== PostgreSQL 문장 로깅을 켠다 ==="
$PSQL "alter system set log_statement='all'" >/dev/null 2>&1
$PSQL "alter system set log_line_prefix='%m [%p] %h '" >/dev/null 2>&1
$PSQL "select pg_reload_conf()" >/dev/null 2>&1
echo " log_statement = $($PSQL 'show log_statement' 2>/dev/null)"
echo " log_line_prefix = $($PSQL 'show log_line_prefix' 2>/dev/null)"
echo
# 로그 커서를 잡아둔다. 이 줄 수 이후만 본다.
LOG_BEFORE=$(kubectl -n "$NS" logs deploy/postgres --tail=-1 2>/dev/null | wc -l)
RESULT=$(kubectl -n "$NS" run kc-readpath --rm -i --restart=Never \
--image=curlimages/curl:8.11.1 --quiet --command -- sh -c "
O=/tmp/o; : > \$O
TOKEN_EP='/realms/master/protocol/openid-connect/token'
jget() { sed -n \"s/.*\\\"\$1\\\":\\\"\\([^\\\"]*\\)\\\".*/\\1/p\"; }
ent() {
curl -s --retry 3 --max-time 20 http://\$1:9000/metrics \
| grep -E '^vendor_statistics_approximate_entries_unique.cache=.sessions' | awk '{print \$NF}'
}
# keycloak-0 에서 로그인한다
L=\$(curl -s -X POST \"http://$K0_IP:8080\$TOKEN_EP\" -d grant_type=password \
-d client_id=admin-cli -d username=admin -d 'password=$ADMIN_PW')
SID=\$(echo \"\$L\" | jget access_token | cut -d. -f2 | sed 's/\$/==/' | base64 -d 2>/dev/null | jget sid)
RT=\$(echo \"\$L\" | jget refresh_token)
echo \"SID=\$SID\" >> \$O
echo \"K1_ENTRIES_BEFORE=\$(ent $K1_IP)\" >> \$O
sleep 2
# 반대편 노드에 refresh 를 딱 한 번 보낸다
# 인용을 한 겹 더 쌓으면 curl 이 URL 을 통째로 못 읽는다. 실제로 000 이 나왔다.
CODE=\$(curl -s -o /dev/null -w '%{http_code}' -X POST \
\"http://$K1_IP:8080\$TOKEN_EP\" \
-d grant_type=refresh_token -d client_id=admin-cli -d \"refresh_token=\$RT\")
echo \"REFRESH_ON_K1=\$CODE\" >> \$O
sleep 3
echo \"K1_ENTRIES_AFTER=\$(ent $K1_IP)\" >> \$O
cat \$O
" 2>&1 | grep -v '^pod .* deleted$')
echo "=== 요청 ==="
echo "$RESULT" | sed 's/^/ /'
SID=$(echo "$RESULT" | sed -n 's/^SID=//p')
echo
echo "=== PostgreSQL 문장 로깅을 끈다 ==="
$PSQL "alter system reset log_statement" >/dev/null 2>&1
$PSQL "alter system reset log_line_prefix" >/dev/null 2>&1
$PSQL "select pg_reload_conf()" >/dev/null 2>&1
echo " log_statement = $($PSQL 'show log_statement' 2>/dev/null)"
echo
echo "=== keycloak-1 이 실제로 보낸 SQL 문장 ==="
echo " (파라미터가 \$1 로 묶여 있어, sid 는 바로 아래 DETAIL 줄에 있다)"
echo
kubectl -n "$NS" logs deploy/postgres --tail=-1 2>/dev/null \
| tail -n +$((LOG_BEFORE + 1)) \
| grep -F "$K1_IP" | grep -E "LOG: execute" \
| sed 's/.*execute [^:]*: //' | sed 's/^/ /' | head -12
echo
echo "=== 그 sid 를 언급한 SQL — 누가 보냈는가 ==="
echo " 찾는 sid: $SID"
echo
kubectl -n "$NS" logs deploy/postgres --tail=-1 2>/dev/null \
| tail -n +$((LOG_BEFORE + 1)) \
| grep -F "$SID" \
| sed -e "s/$K0_IP/[keycloak-0]/g" -e "s/$K1_IP/[keycloak-1]/g" \
| cut -c1-220 \
| head -20
echo
echo "=== 요약: 파드별 질의 건수 ==="
kubectl -n "$NS" logs deploy/postgres --tail=-1 2>/dev/null \
| tail -n +$((LOG_BEFORE + 1)) \
| grep -F "$SID" \
| grep -oE "^[0-9-]+ [0-9:.]+ [A-Z]+ \[[0-9]+\] [0-9.]+" \
| awk '{print $NF}' | sort | uniq -c \
| sed -e "s/$K0_IP/[keycloak-0]/" -e "s/$K1_IP/[keycloak-1]/" -e 's/^/ /'
deploy/lab/scripts/experiment-session-replication.sh
Executable
+207
View File
@@ -0,0 +1,207 @@
#!/usr/bin/env bash
# Experiment 0 — is a session created on one Keycloak node usable on the other?
#
# Forming a cluster is not the same as sharing session state. The Infinispan log
# says "cluster view (2)", but that only proves the members found each other.
#
# Design notes, learned the hard way:
#
# * Every probe has a CONTROL. A result from the far node means nothing unless
# the same call against the issuing node is also measured. The first version
# of this script reported "403 on keycloak-1" as if it were a replication
# failure; the issuing node returned 403 too, and the cause was a missing
# openid scope. Measure both, always.
#
# * Sessions are tracked by SID, not by count. Both the test login and the
# admin API calls create sessions for the same user, so counts are noisy.
# A specific session id either appears in a node's answer or it does not.
#
# * The probe is the REFRESH TOKEN grant, not userinfo. userinfo only validates
# a signature and can succeed on a node that knows nothing about the session.
# Refreshing requires the node to find the session, check it is alive, and
# write back a new refresh time — it actually touches the session store.
#
# Talks to pod IPs directly: going through nginx/Traefik would hide which node
# handled each request, which is the entire question.
#
# ./deploy/lab/scripts/experiment-session-replication.sh
set -uo pipefail
NS="${NS:-keycloak-lab}"
OUT="${OUT:-/tmp/session-replication}"
mkdir -p "$OUT"
PSQL="kubectl -n $NS exec deploy/postgres -- psql -U keycloak -d keycloak -tAc"
echo "수집 시각: $(date '+%Y-%m-%d %H:%M:%S %Z')"
echo
K0_IP=$(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.status.podIP}')
K1_IP=$(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.status.podIP}')
K0_NODE=$(kubectl -n "$NS" get pod keycloak-0 -o jsonpath='{.spec.nodeName}')
K1_NODE=$(kubectl -n "$NS" get pod keycloak-1 -o jsonpath='{.spec.nodeName}')
ADMIN_PW=$(kubectl -n "$NS" get secret keycloak-lab-secrets \
-o jsonpath='{.data.KC_BOOTSTRAP_ADMIN_PASSWORD}' | base64 -d)
echo "=== 대상 ==="
printf ' keycloak-0 %-14s %s\n' "$K0_IP" "$K0_NODE"
printf ' keycloak-1 %-14s %s\n' "$K1_IP" "$K1_NODE"
echo
echo "=== [0] 실험 전 DB 세션 ==="
$PSQL "select offline_flag, count(*) from offline_user_session group by offline_flag" 2>/dev/null \
| sed 's/^/ offline_flag=/' || echo " (없음)"
echo
# 파드 하나 안에서 전 단계를 실행한다. 단계마다 파드를 새로 띄우면 토큰을
# 단계 사이로 넘길 수 없다.
kubectl -n "$NS" run kc-probe --rm -i --restart=Never \
--image=curlimages/curl:8.11.1 --quiet --command -- sh -c "
set -u
K0='http://$K0_IP:8080'; K1='http://$K1_IP:8080'
TOKEN_EP='/realms/master/protocol/openid-connect/token'
jget() { sed -n \"s/.*\\\"\$1\\\":\\\"\\([^\\\"]*\\)\\\".*/\\1/p\"; }
# ── [1] keycloak-0 에서 로그인. 이 노드가 세션의 출생지다 ──────────────────
LOGIN=\$(curl -s -X POST \"\$K0\$TOKEN_EP\" \
-d grant_type=password -d client_id=admin-cli \
-d username=admin -d 'password=$ADMIN_PW')
echo '###STEP1_LOGIN'; echo \"\$LOGIN\"
AT=\$(echo \"\$LOGIN\" | jget access_token)
RT=\$(echo \"\$LOGIN\" | jget refresh_token)
# ── [2] 관리 API 조회용 토큰. 세션 오염을 피하려고 따로 하나만 더 만든다 ──
ADMTOK=\$(curl -s -X POST \"\$K0\$TOKEN_EP\" \
-d grant_type=password -d client_id=admin-cli \
-d username=admin -d 'password=$ADMIN_PW' | jget access_token)
CID=\$(curl -s -H \"Authorization: Bearer \$ADMTOK\" \
\"\$K0/admin/realms/master/clients?clientId=admin-cli\" | jget id | head -1)
# ── [3] 두 노드에 같은 질문을 한다: admin-cli 의 세션 목록 ────────────────
echo '###STEP3_SESSIONS_K0'
curl -s -H \"Authorization: Bearer \$ADMTOK\" \
\"\$K0/admin/realms/master/clients/\$CID/user-sessions?max=100\"
echo
echo '###STEP3_SESSIONS_K1'
curl -s -H \"Authorization: Bearer \$ADMTOK\" \
\"\$K1/admin/realms/master/clients/\$CID/user-sessions?max=100\"
echo
# ── [4] 대조군: keycloak-0 이 발급한 refresh token 을 keycloak-0 에 쓴다 ──
# 먼저 반대편에 써야 하므로 여기서는 쓰지 않고, 순서를 [5] 뒤로 미룬다.
# refresh token 은 회전(rotation)되므로 한 번 쓰면 옛 것이 무효가 된다.
# 따라서 '반대편 먼저'가 유일하게 의미 있는 순서다.
# ── [5] 시험군: keycloak-0 이 발급한 refresh token 을 keycloak-1 에 쓴다 ──
echo '###STEP5_REFRESH_ON_K1'
curl -s -w '\nhttp_code=%{http_code}\n' -X POST \"\$K1\$TOKEN_EP\" \
-d grant_type=refresh_token -d client_id=admin-cli -d \"refresh_token=\$RT\"
RT2=\$(curl -s -X POST \"\$K1\$TOKEN_EP\" \
-d grant_type=refresh_token -d client_id=admin-cli -d \"refresh_token=\$RT\" \
| jget refresh_token)
# ── [6] 무효화가 반대 방향으로도 전파되는가 ───────────────────────────────
# keycloak-1 에서 로그아웃시키고, keycloak-0 에서 갱신을 시도한다.
echo '###STEP6_LOGOUT_VIA_K1'
curl -s -o /dev/null -w 'http_code=%{http_code}\n' -X POST \"\$K1/realms/master/protocol/openid-connect/logout\" \
-d client_id=admin-cli -d \"refresh_token=\$RT2\"
echo '###STEP7_REFRESH_ON_K0_AFTER_LOGOUT'
curl -s -w '\nhttp_code=%{http_code}\n' -X POST \"\$K0\$TOKEN_EP\" \
-d grant_type=refresh_token -d client_id=admin-cli -d \"refresh_token=\$RT2\"
echo '###END'
" > "$OUT/raw.txt" 2>&1
sed -i '/^pod .* deleted$/d' "$OUT/raw.txt"
python3 - "$OUT/raw.txt" <<'PY' | tee "$OUT/report.txt"
import base64, json, sys
raw = open(sys.argv[1]).read()
blocks, cur = {}, None
for line in raw.splitlines():
if line.startswith('###'):
cur = line[3:]; blocks[cur] = []
elif cur is not None:
blocks[cur].append(line)
get = lambda k: '\n'.join(blocks.get(k, [])).strip()
def j(s):
try: return json.JSONDecoder().raw_decode(s.strip())[0]
except Exception: return None
def claims(tok):
p = tok.split('.')[1]; p += '=' * (-len(p) % 4)
return json.loads(base64.urlsafe_b64decode(p))
login = j(get('STEP1_LOGIN'))
if not login or 'access_token' not in login:
print('로그인 실패:', get('STEP1_LOGIN')[:300]); sys.exit(1)
ac = claims(login['access_token'])
rc = claims(login['refresh_token'])
SID = ac['sid']
print('=== [1] keycloak-0 에서 로그인 ===')
print(f" sid {SID}")
print(f" sub {ac.get('sub')}")
print(f" iss {ac.get('iss')}")
print(f" access 수명 {ac['exp']-ac['iat']}초")
print(f" refresh 수명 {rc['exp']-rc['iat']}초 typ={rc.get('typ')}")
print(f" refresh jti {rc.get('jti')}")
print()
print('=== [3] 같은 sid 가 두 노드 모두에서 보이는가 ===')
for step, who in (('STEP3_SESSIONS_K0', 'keycloak-0 (발급 노드)'),
('STEP3_SESSIONS_K1', 'keycloak-1 (반대편)')):
d = j(get(step))
if d is None:
print(f' {who:24} 파싱 실패: {get(step)[:120]}'); continue
ids = [s.get('id') for s in d]
mark = '보임 ✔' if SID in ids else '없음 ✘'
print(f' {who:24} 세션 {len(ids)}개 중 대상 sid → {mark}')
for s in d:
if s.get('id') == SID:
print(f" ipAddress={s.get('ipAddress')} start={s.get('start')} lastAccess={s.get('lastAccess')}")
def show(step, title, expect):
print(); print(f'=== {title} ===')
body = get(step)
code = [l for l in body.splitlines() if l.startswith('http_code=')]
code = code[0].split('=')[1] if code else '?'
d = j(body)
ok = '기대대로' if code == expect else f'기대({expect})와 다름'
print(f' HTTP {code} ← {ok}')
if d and 'access_token' in d:
c = claims(d['access_token'])
same = '동일 ✔' if c.get('sid') == SID else f"다름 ✘ ({c.get('sid')})"
print(f' 새 토큰의 sid → {same}')
elif d:
print(f" error {d.get('error')}")
print(f" error_description {d.get('error_description')}")
show('STEP5_REFRESH_ON_K1',
'[5] keycloak-0 이 발급한 refresh token 을 keycloak-1 에 사용', '200')
print(); print('=== [6] keycloak-1 을 통해 로그아웃 ===')
print(' ' + get('STEP6_LOGOUT_VIA_K1').strip())
show('STEP7_REFRESH_ON_K0_AFTER_LOGOUT',
'[7] 로그아웃 후 keycloak-0 에서 갱신 시도 (무효화 전파)', '400')
open('/tmp/session-replication/sid.txt','w').write(SID)
PY
SID=$(cat /tmp/session-replication/sid.txt 2>/dev/null)
echo
echo "=== [8] PostgreSQL 에서 그 sid 를 직접 확인 ==="
echo " 대상 sid: $SID"
$PSQL "select user_session_id, offline_flag, created_on, last_session_refresh
from offline_user_session where user_session_id='$SID'" 2>/dev/null \
| sed 's/^/ /' | grep -q . \
&& $PSQL "select user_session_id||' | flag='||offline_flag||' | created='||created_on||' | refresh='||last_session_refresh
from offline_user_session where user_session_id='$SID'" 2>/dev/null | sed 's/^/ /' \
|| echo " 행 없음 — 로그아웃으로 삭제되었다"
echo
echo " 전체 세션 수: $($PSQL 'select count(*) from offline_user_session' 2>/dev/null)"
docs/evidence/a1-jgroups-transport-block/01-baseline-cluster.txt
+13
View File
@@ -0,0 +1,13 @@
=== [기준선 1] 클러스터 뷰 — 양쪽 파드의 마지막 ISPN000094 ===
keycloak-0: [keycloak-1-48749(v=16.0.12)|5] (2) [keycloak-1-48749(v=16.0.12), keycloak-0-30843(v=16.0.12)]
keycloak-1: [keycloak-1-48749(v=16.0.12)|5] (2) [keycloak-1-48749(v=16.0.12), keycloak-0-30843(v=16.0.12)]
=== [기준선 2] JGROUPS_PING — 디스커버리 등록 ===
name | ip | coord | coordinated_by
------------------+-----------------+-------+---------------------------------------------
keycloak-0-30843 | 10.42.1.43:7800 | f | uuid://00000000-0000-0000-0000-000000000007
keycloak-1-48749 | 10.42.0.35:7800 | t | uuid://00000000-0000-0000-0000-000000000007
(2 rows)
=== [기준선 3] 기존 NetworkPolicy ===
No resources found in keycloak-lab namespace.
docs/evidence/a1-jgroups-transport-block/02-control-before-block.txt
+17
View File
@@ -0,0 +1,17 @@
=== [대조군] 차단 전 — keycloak-0 로그인 → keycloak-1 에서 refresh ===
sid tAWs2gCPr6SOcD4jDR9-_CzB
keycloak-1 에서 refresh: 200
=== [기준선 4] JGroups 지표 — 양쪽 노드 ===
--- K0 (10.42.1.43) ---
vendor_jgroups_stats_bytes_sent_total 31476.0
vendor_jgroups_merge3_get_num_merge_events 0.0
vendor_jgroups_merge3_get_views 0.0
vendor_jgroups_fd_sock2_get_num_suspected_members 0.0
vendor_jgroups_nakack2_get_xmit_table_missing_messages 0.0
--- K1 (10.42.0.35) ---
vendor_jgroups_merge3_get_views 0.0
vendor_jgroups_stats_bytes_sent_total 126765.0
vendor_jgroups_nakack2_get_xmit_table_missing_messages 0.0
vendor_jgroups_fd_sock2_get_num_suspected_members 0.0
vendor_jgroups_merge3_get_num_merge_events 0.0
docs/evidence/a1-jgroups-transport-block/03-block-applied.txt
+8
View File
@@ -0,0 +1,8 @@
=== 차단 적용 ===
networkpolicy.networking.k8s.io/a1-block-jgroups-transport created
a1-block-jgroups-transport map[app:keycloak]
적용 시각: 11:38:08
=== FD_SOCK2 가 상대를 의심하기까지 기다린다 (15초 간격, 최대 3분) ===
+15초 suspected(k0 k1) =
→ 변화 감지
docs/evidence/a1-jgroups-transport-block/04-after-block-state.txt
+26
View File
@@ -0,0 +1,26 @@
차단 경과: 11:38:51 (적용 11:38:08)
=== [차단 후 1] JGroups 지표 ===
--- keycloak-0 ---
vendor_jgroups_stats_bytes_sent_total 32857.0
vendor_jgroups_merge3_get_num_merge_events 0.0
vendor_jgroups_merge3_get_views 0.0
vendor_jgroups_fd_sock2_get_num_suspected_members 0.0
vendor_jgroups_nakack2_get_xmit_table_missing_messages 0.0
--- keycloak-1 ---
vendor_jgroups_merge3_get_views 0.0
vendor_jgroups_stats_bytes_sent_total 129103.0
vendor_jgroups_nakack2_get_xmit_table_missing_messages 0.0
vendor_jgroups_fd_sock2_get_num_suspected_members 0.0
vendor_jgroups_merge3_get_num_merge_events 0.0
=== [차단 후 2] 클러스터 뷰 — 갈라졌는가 ===
keycloak-0:
keycloak-1:
=== [차단 후 3] JGROUPS_PING — 디스커버리는 살아 있는가 ===
name | ip | coord
------------------+-----------------+-------
keycloak-0-30843 | 10.42.1.43:7800 | f
keycloak-1-48749 | 10.42.0.35:7800 | t
(2 rows)
docs/evidence/a1-jgroups-transport-block/05-conntrack-problem.txt
+16
View File
@@ -0,0 +1,16 @@
=== [문제 확정] NetworkPolicy 적용 후에도 기존 연결이 conntrack 에 살아 있다 ===
--- kc-lab-1 ---
tcp 6 86398 ESTABLISHED src=10.42.0.35 dst=10.42.1.43 sport=40023 dport=7800 src=10.42.1.43 dst=10.42.0.35 sport=7800 dport=40023 [ASSURED] mark=0 use=1
tcp 6 79982 ESTABLISHED src=10.42.0.35 dst=10.42.1.43 sport=50477 dport=57800 src=10.42.1.43 dst=10.42.0.35 sport=57800 dport=50477 [ASSURED] mark=0 use=1
--- kc-lab-2 ---
tcp 6 86398 ESTABLISHED src=10.42.0.35 dst=10.42.1.43 sport=40023 dport=7800 src=10.42.1.43 dst=10.42.0.35 sport=7800 dport=40023 [ASSURED] mark=0 use=1
tcp 6 33 SYN_SENT src=10.42.1.58 dst=10.42.0.35 sport=34824 dport=7800 [UNREPLIED] src=10.42.0.35 dst=10.42.1.58 sport=7800 dport=34824 mark=0 use=1
tcp 6 79982 ESTABLISHED src=10.42.0.35 dst=10.42.1.43 sport=50477 dport=57800 src=10.42.1.43 dst=10.42.0.35 sport=57800 dport=50477 [ASSURED] mark=0 use=1
=== [조치] 7800 흐름의 conntrack 항목을 지운다 → 다음 패킷이 정책을 다시 탄다 ===
kc-lab-1: tcp 6 86398 ESTABLISHED src=10.42.0.35 dst=10.42.1.43 sport=40023 dport=7800 src=10.42.1.43 dst=10.42.0.35 sport=7800 dport=40023 [ASSURED] mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 0 flow entries have been deleted.
kc-lab-2: tcp 6 33 SYN_SENT src=10.42.1.58 dst=10.42.0.35 sport=34824 dport=7800 [UNREPLIED] src=10.42.0.35 dst=10.42.1.58 sport=7800 dport=34824 mark=0 use=1 conntrack v1.4.7 (conntrack-tools): 0 flow entries have been deleted.
=== 삭제 후 7800 conntrack ===
kc-lab-1: 2 건
kc-lab-2: 2 건
docs/evidence/a1-jgroups-transport-block/06-partition-observed.txt
+13
View File
@@ -0,0 +1,13 @@
관찰 시작: 11:42:03
+20초 suspected(k0 k1) = []
+40초 suspected(k0 k1) = []
+60초 suspected(k0 k1) = [0.0 0.0 0.0 0.0 ]
+80초 suspected(k0 k1) = []
+100초 suspected(k0 k1) = []
+120초 suspected(k0 k1) = []
+140초 suspected(k0 k1) = [0.0 ]
+160초 suspected(k0 k1) = [0.0 0.0 0.0 0.0 ]
=== 클러스터 뷰 변화 (최근 8분) ===
--- keycloak-0 ---
--- keycloak-1 ---
docs/evidence/a1-jgroups-transport-block/07-cluster-size.txt
+16
View File
@@ -0,0 +1,16 @@
=== vendor_cluster_size — 지난 25분 (차단 11:38:08, conntrack 삭제 11:41) ===
keycloak-0:
11:20=2 11:21=2 11:22=2 11:23=2 11:24=2 11:25=2 11:26=2 11:27=2 11:28=2 11:29=2 11:30=2 11:31=2 11:32=2 11:33=2 11:34=2 11:35=2 11:36=2 11:37=2 11:38=2 11:39=2 11:40=2 11:41=2 11:42=2 11:43=2 11:44=2 11:45=2
keycloak-1:
11:20=2 11:21=2 11:22=2 11:23=2 11:24=2 11:25=2 11:26=2 11:27=2 11:28=2 11:29=2 11:30=2 11:31=2 11:32=2 11:33=2 11:34=2 11:35=2 11:36=2 11:37=2 11:38=2 11:39=2 11:40=2 11:41=2 11:42=2 11:43=2 11:44=2 11:45=2
=== 현재 값 ===
keycloak-1 = 2 멤버
keycloak-0 = 2 멤버
=== 7800 소켓 상태 (파드 내부) ===
keycloak-0 2
keycloak-1 2
=== conntrack ===
kc-lab-1 1 건
kc-lab-2 1 건
docs/evidence/a1-jgroups-transport-block/08-restart-forced-partition.txt
+9
View File
@@ -0,0 +1,9 @@
=== 정책이 걸린 상태에서 keycloak-0 을 재시작한다 → 재연결이 막힌다 ===
재시작 시각: 11:46:07
pod "keycloak-0" deleted from keycloak-lab namespace
keycloak-0 false 10.42.1.67 2026-09-04T02:44:23Z
=== cluster_size 추이 ===
keycloak-0: 11:45:27=1 11:45:57=1 11:46:27=1 11:46:57=1 11:47:27=1
keycloak-0: 11:40:57=2 11:41:27=2 11:41:57=2 11:42:27=2 11:42:57=2 11:43:27=2 11:43:57=2
keycloak-1: 11:40:57=2 11:41:27=2 11:41:57=2 11:42:27=2 11:42:57=2 11:43:27=2 11:43:57=2 11:44:27=1 11:44:57=1 11:45:27=1 11:45:57=1 11:46:27=1 11:46:57=1 11:47:27=1
docs/evidence/a1-jgroups-transport-block/09-cross-node-under-partition.txt
+16
View File
@@ -0,0 +1,16 @@
=== keycloak-0 헬스 상태 ===
keycloak-0 = 10.42.1.67 keycloak-1 = 10.42.0.35
PodReadyToStartContainers=True
Initialized=True
Ready=False ContainersNotReady
ContainersReady=False ContainersNotReady
PodScheduled=True
=== ★ 본 시험 — 분단 상태에서 교차 노드 세션이 되는가 ===
[1] keycloak-0 로그인 sid=nShl5TaBrZnKStDqaspjgmJB
[2] keycloak-1 에서 refresh HTTP 200
[3] keycloak-1 에서 로그아웃 HTTP 204
[4] keycloak-0 에서 재갱신 시도 HTTP 200
(400 이면 무효화가 전파된 것)
=== DB 세션 수 ===
docs/evidence/a1-jgroups-transport-block/10-logout-not-propagated.txt
+33
View File
@@ -0,0 +1,33 @@
=== 그 sid 가 DB 에 남아 있는가 ===
user_session_id | offline_flag | last_session_refresh
-----------------+--------------+----------------------
(0 rows)
=== 전체 온라인 세션 수 ===
1
=== 노드별 세션 캐시 엔트리 (Prometheus) ===
keycloak-1 kc-lab-1 = 0
keycloak-0 kc-lab-2 = 1
=== keycloak-0 이 Ready 가 아닌 이유 — 헬스 응답 ===
{
"status": "DOWN",
"checks": [
{
"name": "Graceful Shutdown",
"status": "UP"
},
{
"name": "Keycloak cluster health check",
"status": "DOWN",
"data": {
"Failing since": "2026-09-04 02:45:14,251"
}
},
{
"name": "Keycloak database connections async health check",
"status": "UP"
},
{
"name": "Keycloak Initialized",
docs/evidence/a1-jgroups-transport-block/11-service-impact.txt
+25
View File
@@ -0,0 +1,25 @@
=== 양쪽 노드의 readiness — 둘 다 DOWN 이면 전면 장애다 ===
Traceback (most recent call last):
File "<string>", line 3, in <module>
d=json.load(sys.stdin)
File "/usr/lib/python3.14/json/__init__.py", line 298, in load
return loads(fp.read(),
cls=cls, object_hook=object_hook,
parse_float=parse_float, parse_int=parse_int,
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib/python3.14/json/__init__.py", line 352, in loads
return _default_decoder.decode(s)
~~~~~~~~~~~~~~~~~~~~~~~^^^
File "/usr/lib/python3.14/json/decoder.py", line 348, in decode
raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 21 column 2 (char 446)
=== 파드 Ready 상태 ===
keycloak-0 false 0
keycloak-1 true 0
=== ★ Service 엔드포인트 — 트래픽을 받는 파드가 남아 있는가 ===
ready 주소: [10.42.0.35] notReady : [10.42.1.67]
=== ★ 외부 진입점으로 실제 로그인이 되는가 (nginx→Traefik→Service) ===
https://auth.hyeonworks.com/realms/master HTTP 200
토큰 발급 HTTP 200
docs/evidence/a1-jgroups-transport-block/12-recovery.txt
+24
View File
@@ -0,0 +1,24 @@
=== 차단 해제 ===
해제 시각: 11:49:58
networkpolicy.networking.k8s.io "a1-block-jgroups-transport" deleted from keycloak-lab namespace
=== 자동으로 다시 붙는가 (30초 간격, 최대 4분) ===
+30초 keycloak-0=1 keycloak-1=1 | Ready 파드 2 개
+60초 keycloak-0=1 keycloak-1=1 | Ready 파드 2 개
+90초 keycloak-0=2 keycloak-1=2 | Ready 파드 3 개
→ 클러스터 재형성
=== 복구 로그 ===
keycloak-0: [keycloak-0-26403(v=16.0.12)|0] (1) [keycloak-0-26403(v=16.0.12)]
keycloak-1: [keycloak-1-48749(v=16.0.12)|6] (1) [keycloak-1-48749(v=16.0.12)]
=== MERGE3 가 합쳤는가 ===
merge_events keycloak-1 = 1
merge_events keycloak-0 = 1
=== JGROUPS_PING — 코디네이터가 하나로 돌아왔는가 ===
name | ip | coord
------------------+-----------------+-------
keycloak-0-26403 | 10.42.1.67:7800 | t
keycloak-1-48749 | 10.42.0.35:7800 | f
(2 rows)
docs/evidence/a1-jgroups-transport-block/README.md
+26
View File
@@ -0,0 +1,26 @@
# A-1 — JGroups 트랜스포트(7800) 차단 증거
2026-09-04 11:38–11:52 KST · Keycloak 26.7.0 / Infinispan 16.0.12
해설: [`docs/experiment-a1-jgroups-transport-block.md`](../../experiment-a1-jgroups-transport-block.md)
| 파일 | 무엇을 보여주는가 |
|---|---|
| `01-baseline-cluster.txt` | 차단 전 — 양쪽이 뷰 ID 5·멤버 2로 일치, `JGROUPS_PING` 코디네이터 1명 |
| `02-control-before-block.txt` | **대조군** — 차단 전 교차 노드 refresh `200`, JGroups 지표 전부 0 |
| `03-block-applied.txt` | NetworkPolicy 적용. **빈 측정값을 "변화 감지"로 오판한 기록** |
| `04-after-block-state.txt` | 차단 43초 후 — 지표 무변화, `JGROUPS_PING` 그대로 |
| `05-conntrack-problem.txt` | **핵심 문제** — `ESTABLISHED [ASSURED]` 로 기존 연결이 살아 있음. FD_SOCK2 의 **57800** 포트도 함께 드러남 |
| `06-partition-observed.txt` | 임시 curl 파드 폴링의 실패 — 빈 값·개수 불일치 |
| `07-cluster-size.txt` | **`vendor_cluster_size` 가 25분 내내 2** — 분단이 일어나지 않았다는 결정적 증거 |
| `08-restart-forced-partition.txt` | 재연결 강제 후 `2 → 1` |
| `09-cross-node-under-partition.txt` | **본 시험** — 교차 refresh `200`(예측 적중), **로그아웃 후 재갱신 `200`(예측 빗나감)** |
| `10-logout-not-propagated.txt` | 기제 확정 — **DB 행 0건인데 keycloak-0 캐시에 1건**, 헬스체크 `cluster health: DOWN` |
| `11-service-impact.txt` | **분단 노드가 Service 에서 빠짐.** `ready=[10.42.0.35] notReady=[10.42.1.67]`, 외부 로그인 `200` |
| `12-recovery.txt` | 90초 만에 자동 재형성, `merge3_get_num_merge_events = 1`, 코디네이터 재선출 |
| `a1-cluster-size-partition-recovery.png` | Grafana — `vendor_cluster_size` 가 `2 → 1 → 2` 로 움직이는 전 구간 |
## 핵심 세 줄
1. **NetworkPolicy 만으로는 이미 붙어 있는 클러스터를 못 끊는다.** conntrack 의 ESTABLISHED 가 먼저 통과시킨다.
2. **세션 공유는 분단을 견딘다(200).** 통념이 틀렸고 A-0 모델이 맞다.
3. **로그아웃 무효화는 7800 을 탄다.** DB 행이 지워져도 반대편은 낡은 캐시로 200 을 준다 — A-0 의 인과 해석을 정정한다.
docs/evidence/a1-jgroups-transport-block/a1-cluster-size-partition-recovery.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 66 KiB

docs/evidence/a2-database-loss/01-baseline.txt
+14
View File
@@ -0,0 +1,14 @@
=== A-2 기준선 — 클러스터가 정상으로 돌아왔는가 ===
keycloak-0 true 10.42.1.67 kc-lab-2
keycloak-1 true 10.42.0.35 kc-lab-1
postgres-7b474b88c8-sn9ff true 10.42.1.24 kc-lab-2
cluster_size keycloak-1 = 2
cluster_size keycloak-0 = 2
=== 노드별 세션 캐시 (실험 설계에 필요) ===
keycloak-1 kc-lab-1 = 0 건
keycloak-0 kc-lab-2 = 0 건
=== DB 온라인 세션 ===
2
docs/evidence/a2-database-loss/02-setup-sessions.txt
+10
View File
@@ -0,0 +1,10 @@
pod/a2-probe condition met
keycloak-0=10.42.1.67 keycloak-1=10.42.0.35
=== [준비] 양쪽 노드에 세션을 하나씩 만든다 ===
keycloak-0 에서 로그인 sid=EAXV5HcG2J1BZ3vnwONf64AQ 토큰길이=613
keycloak-1 에서 로그인 sid=McyTj5lj3n_JqApCXeuAHExc 토큰길이=613
=== [확인] 세션이 각자 노드에만 캐시되었는가 ===
keycloak-1 = 0 건
keycloak-0 = 1 건
docs/evidence/a2-database-loss/03-four-paths.txt
+16
View File
@@ -0,0 +1,16 @@
=== [1] 토큰을 새로 발급 (access 수명 60초) ===
발급 완료 sid=RKXQGAgkuLtouFMVPTFmp_0_
=== [2] PostgreSQL 정지 ===
정지 시각: 11:56:04
deployment.apps/postgres scaled
pod/postgres-7b474b88c8-sn9ff condition met
삭제 완료: 11:56:04
=== [3] 네 경로를 즉시 시험 ===
④ 이미 발급된 access token 으로 관리 API HTTP 000000{"error":"HTTP 401 Unauthorized"}401
① 캐시를 가진 노드(keycloak-0)에서 refresh HTTP 500
② 캐시가 없는 노드(keycloak-1)에서 refresh HTTP 500
③ 새 로그인 HTTP 500
--- 오류 본문 (새 로그인) ---
{"error":"unknown_error","error_description":"For more on this error consult the server log."}
docs/evidence/a2-database-loss/04-health-and-service.txt
+29
View File
@@ -0,0 +1,29 @@
=== 파드 Ready 상태 — DB 가 없으면 어떻게 되는가 ===
keycloak-0 false 0
keycloak-1 false 0
=== Service 엔드포인트 ===
Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
notReady: [10.42.0.35 10.42.1.67]
=== health/ready 상세 ===
전체: DOWN
Graceful Shutdown UP
Keycloak cluster health check UP
Keycloak database connections async health check DOWN
Keycloak Initialized UP
=== ④ 다시 — 서명 검증만 필요한 경로는 살아 있는가 ===
JWKS 엔드포인트(realm 공개키) HTTP 200
realm 메타데이터(.well-known) HTTP 200
관리 API(세션 조회 필요) HTTP 500
=== 외부 진입점 ===
https://auth.hyeonworks.com/realms/master HTTP 503
=== Keycloak 로그 — 실제 오류 ===
at io.agroal.pool.ConnectionPool$CreateConnectionTask.call(ConnectionPool.java:664)
at io.agroal.pool.ConnectionPool$CreateConnectionTask.call(ConnectionPool.java:645)
Caused by: java.net.ConnectException: Connection refused
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:219)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:365)
docs/evidence/a2-database-loss/05-recovery.txt
+21
View File
@@ -0,0 +1,21 @@
=== ★ up 지표는 무엇을 말하는가 (프로세스는 살아 있다) ===
up{pod=keycloak-1} = 1 ← 1 인데 서비스는 503 이다
up{pod=keycloak-0} = 1 ← 1 인데 서비스는 503 이다
=== 복구 — PostgreSQL 재기동 ===
재기동 시각: 11:57:09
deployment.apps/postgres scaled
Waiting for deployment "postgres" rollout to finish: 0 out of 1 new replicas have been updated...
Waiting for deployment "postgres" rollout to finish: 0 of 1 updated replicas are available...
deployment "postgres" successfully rolled out
=== Keycloak 이 스스로 회복하는가 (재시작 없이) ===
+15초 keycloak-0 true keycloak-1 true | 외부 HTTP 200
→ 서비스 복귀
=== 재시작 횟수 — 파드가 죽었다 살아난 것인가, 그대로 회복한 것인가 ===
keycloak-0 0
keycloak-1 0
=== 정지 전 세션이 살아남았는가 ===
online 세션 5
docs/evidence/a2-database-loss/README.md
+19
View File
@@ -0,0 +1,19 @@
# A-2 — PostgreSQL 정지 증거
2026-09-04 11:56–11:58 KST · Keycloak 26.7.0
해설: [`docs/experiment-a2-database-loss.md`](../../experiment-a2-database-loss.md)
| 파일 | 무엇을 보여주는가 |
|---|---|
| `01-baseline.txt` | 정지 전 — 양쪽 Ready, `cluster_size=2` |
| `02-setup-sessions.txt` | 양쪽 노드에 세션 하나씩. 캐시는 각자 노드에만 |
| `03-four-paths.txt` | **네 경로 전부 `500`.** 캐시를 가진 노드도 실패 — refresh 는 쓰기다 |
| `04-health-and-service.txt` | **전면 장애 증거** — Ready 파드 0개, `ready 주소=[]`, 외부 **503**, `database connections: DOWN`. JWKS·.well-known 은 `200` |
| `05-recovery.txt` | **`up=1` 인 채로 503.** DB 복귀 15초 후 재시작 0회로 자동 회복, 세션 5건 생존 |
| `a2-up-stayed-1-during-outage.png` | Grafana — `up{job="keycloak"}` 이 전면 장애 내내 **1에 평평** |
## 핵심 세 줄
1. **DB 는 단일 장애점이다.** Keycloak 을 몇 대로 늘려도 같이 죽는다 — Ready 파드 0개, 외부 503.
2. **캐시는 읽기를 대신할 뿐 쓰기를 못 한다.** refresh 는 `UPDATE LAST_SESSION_REFRESH` 를 하므로 캐시가 있어도 실패한다.
3. **`up` 은 이 장애를 못 잡는다.** 알림은 readiness 와 외부 응답 코드에 걸어야 한다.
docs/evidence/a2-database-loss/a2-up-stayed-1-during-outage.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 60 KiB

docs/evidence/a3-database-crash/01-crash-injection.txt
+19
View File
@@ -0,0 +1,19 @@
=== [준비] 손실 측정 설계 확인 ===
LAST_SESSION_REFRESH 는 integer(초) — 200ms 손실은 보이지 않는다
created_on | integer | | not null |
last_session_refresh | integer | | not null | 0
"idx_user_session_expiration_created" btree (realm_id, offline_flag, remember_me, created_on, user_session_id, user_id)
"idx_user_session_expiration_last_refresh" btree (realm_id, offline_flag, remember_me, last_session_refresh, user_session_id, user_id)
→ 대신 행 존재 여부로 잰다. 로그인 하나 = 행 하나 = 이진 판정
전역 synchronous_commit: on
=== [1] 빠른 연속 로그인을 백그라운드로 시작 ===
루프 시작
6초 경과 — 지금까지 성공한 로그인: 0
=== [2] PostgreSQL 강제 종료 (SIGKILL) ===
종료 시각: 12:00:26.511
pod "postgres-7b474b88c8-xc2vt" force deleted from keycloak-lab namespace
삭제 반환: 12:00:26.586
클라이언트가 200 을 받은 로그인 수: 0
docs/evidence/a3-database-crash/02-design-check.txt
+14
View File
@@ -0,0 +1,14 @@
deployment "postgres" successfully rolled out
=== crash recovery 가 실행되었는가 (강제 종료의 흔적) ===
2026-09-04 02:58:41.036 UTC [1] LOG: database system is ready to accept connections
=== [설계 확인] 로그인 트랜잭션도 synchronous_commit 을 끄는가 ===
--- 로그인 트랜잭션 (INSERT 가 있는 것) ---
2:BEGIN
5:COMMIT
6:BEGIN
9:insert into OFFLINE_USER_SESSION (BROKER_SESSION_ID,CREATED_ON,DATA,LAST_SESSION_REFRESH,REALM_ID,REMEMBER_ME,USER_ID,VERSION,OFFLINE_FLAG,USER_SESSION_ID) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)
10:insert into OFFLINE_CLIENT_SESSION (DATA,REALM_ID,TIMESTAMP,VERSION,CLIENT_ID,CLIENT_STORAGE_PROVIDER,EXTERNAL_CLIENT_ID,OFFLINE_FLAG,USER_SESSION_ID) values ($1,$2,$3,$4,$5,$6,$7,$8,$9)
11:SET LOCAL synchronous_commit TO OFF
12:COMMIT
docs/evidence/a3-database-crash/03-loss-measurement.txt
+17
View File
@@ -0,0 +1,17 @@
=== [1] 로그인 루프 시작 (호스트에서 백그라운드로 exec — 세션이 살아 있어야 한다) ===
8초 동안 클라이언트가 200 을 받은 로그인: 106 건
=== [2] SIGKILL ===
종료: 12:01:32.981
반환: 12:01:33.236
최종 성공 로그인 수: 110 건
마지막 sid: FimM-krSybBACP2qIvshLWwU
마지막 sid: EwFfFwOIfqiv8N5GQ5OjtsVq
마지막 sid: CJX-PxFkS7rUc_9FQgB7iw1f
마지막 sid: 1EFK7SgUA4M7tq_SkC_BD2er
마지막 sid: _LiqTczuyxlpOs3T3xs25SLv
=== [3] PostgreSQL 재기동 후 crash recovery 확인 ===
Waiting for deployment "postgres" rollout to finish: 0 of 1 updated replicas are available...
deployment "postgres" successfully rolled out
2026-09-04 02:59:48.427 UTC [1] LOG: database system is ready to accept connections
docs/evidence/a3-database-crash/04-comparison.txt
+27
View File
@@ -0,0 +1,27 @@
=== [4] 클라이언트가 받은 sid 가 DB 에 있는가 ===
클라이언트가 200 을 받은 sid: 291 건
DB 온라인 세션 총계: 375
--- 마지막 15건을 하나씩 조회 ---
TCCOYnVlN30Y2fGyJEsVJ_Lq 있음
vBcllIkKWovh-FXN9tSzmxAe 있음
eMpN_ywUdRTks9uBE9aTumPK 있음
aPC_T0yrlMjrlskcLAp0AvY6 있음
UBPxmduB-ahGHBg636sY3AEz 있음
HLnBloNX9R3qQJSkpOnkNqL4 있음
_KPJS30IAqVhkTJGHcCxKvrM 있음
rq3caZ9MkyMYlFSSzRjQLygD 있음
ivvQm70hjl55DPpF7vpYmz_E 있음
81mx-rmi-tAeogHx3-su3z2q 있음
WnvNDH93uzcz1XNFbaMSXk1A 있음
DXPAIhjO5sGpCUlcD8IEoS8L 있음
aZMvl4IwPdK-rC_7bG005Z5C 있음
eIuBCprfWA5x0glcgSKYrrX0 있음
ozES5kEeu2IFf_cfcC_jFlbF 있음
마지막 15건 중 유실: 0 건
=== [5] 전체 대조 — 몇 건이나 사라졌는가 ===
클라이언트 성공: 291 건
DB 에 존재: 291 건
★ 유실: 0 건
docs/evidence/a3-database-crash/05-true-crash.txt
+12
View File
@@ -0,0 +1,12 @@
=== [정리] 세션 테이블 비우고 루프 잔여 확인 ===
DELETE 375
남은 세션: 0
=== [재주입] postmaster(PID 1)에 SIGKILL — 진짜 크래시 ===
8초 후 성공 로그인: 110 건
SIGKILL: 12:03:21.441
최종 성공 로그인: 139 건
=== [검증] 이번엔 crash recovery 가 돌았는가 ===
deployment "postgres" successfully rolled out
2026-09-04 02:59:48.427 UTC [1] LOG: database system is ready to accept connections
docs/evidence/a3-database-crash/06-backend-kill-crash.txt
+17
View File
@@ -0,0 +1,17 @@
=== 로그인 루프 시작 ===
8초 후: 112 건
=== 백엔드 프로세스에 SIGKILL → postmaster 가 재초기화한다 ===
시각: 12:04:22.063
최종 성공 로그인: 153 건
=== [검증] crash recovery 가 돌았는가 ===
2026-09-04 02:59:48.427 UTC [1] LOG: database system is ready to accept connections
2026-09-04 03:02:35.807 UTC [1] LOG: server process (PID 40) was terminated by signal 9: Killed
2026-09-04 03:02:35.807 UTC [1] LOG: terminating any other active server processes
2026-09-04 03:02:35.814 UTC [1] LOG: all server processes terminated; reinitializing
2026-09-04 03:02:35.896 UTC [2585] LOG: database system was not properly shut down; automatic recovery in progress
2026-09-04 03:02:35.899 UTC [2585] LOG: redo starts at 0/23CAB68
2026-09-04 03:02:35.904 UTC [2585] LOG: redo done at 0/2529E40 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
2026-09-04 03:02:35.923 UTC [2586] LOG: checkpoint complete: wrote 113 buffers (0.7%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.004 s, sync=0.004 s, total=0.015 s; sync files=27, longest=0.003 s, average=0.001 s; distance=1405 kB, estimate=1405 kB; lsn=0/252A048, redo lsn=0/252A048
2026-09-04 03:02:35.926 UTC [1] LOG: database system is ready to accept connections
docs/evidence/a3-database-crash/07-loss-result.txt
+19
View File
@@ -0,0 +1,19 @@
=== 크래시 전후 대조 ===
클라이언트가 200 과 토큰을 받은 로그인 : 153 건
그중 DB 에 실제로 존재 : 149 건
★ 유실 : 4 건
DB 전체 온라인 세션 : 150 건
=== 유실된 sid 목록 ===
★ CQUfg9HLH29xvhiu6pVlfWOo ← 토큰은 발급됐는데 세션이 없다
★ 5gLP4fqmpZBbjhH_d-0TPMMr ← 토큰은 발급됐는데 세션이 없다
★ hkcOv1QskUFmYveMLB6Hljra ← 토큰은 발급됐는데 세션이 없다
★ p5XybeQIYmAs818gO4Vl_5ea ← 토큰은 발급됐는데 세션이 없다
=== 그 토큰이 지금 실제로 쓰이는가 (마지막 sid 로 확인) ===
마지막 sid: 8do0Bw6tkVLDVxgxotE7GosH
user_session_id | created_on | last_session_refresh
--------------------------+------------+----------------------
8do0Bw6tkVLDVxgxotE7GosH | 1788490958 | 1788490958
(1 row)
docs/evidence/a3-database-crash/README.md
+20
View File
@@ -0,0 +1,20 @@
# A-3 — DB 강제 종료와 데이터 손실 증거
2026-09-04 12:00–12:05 KST · Keycloak 26.7.0 / PostgreSQL 16
해설: [`docs/experiment-a3-database-crash.md`](../../experiment-a3-database-crash.md)
| 파일 | 무엇을 보여주는가 |
|---|---|
| `01-crash-injection.txt` | 첫 시도 실패 — 파드 안 백그라운드 루프가 `exec` 종료와 함께 죽어 0건 수집 |
| `02-design-check.txt` | **핵심 설계 확인** — 로그인 트랜잭션도 `SET LOCAL synchronous_commit TO OFF` 로 커밋한다 |
| `03-loss-measurement.txt` | `--grace-period=0 --force` 주입 |
| `04-comparison.txt` | **유실 0건** — 그러나 crash recovery 가 안 돌았다. 죽인 적이 없는 것 |
| `05-true-crash.txt` | `kill -9 1` 시도 — **컨테이너 안에서 PID 1 은 SIGKILL 을 무시한다** |
| `06-backend-kill-crash.txt` | **성공한 주입** — 백엔드에 SIGKILL → `not properly shut down` / `redo starts` / `redo done` |
| `07-loss-result.txt` | **결과: 153건 중 4건 유실.** 토큰은 발급됐는데 세션 행이 없는 sid 목록 |
## 핵심 세 줄
1. **로그인도 비동기 커밋이다.** refresh 시각뿐 아니라 **로그인 자체**가 사라질 수 있다.
2. **153건 중 4건(약 2.6%) 유실** — 초당 19건 기준 마지막 0.2초 분량, `wal_writer_delay` 기본값과 일치.
3. **주입을 세 번 시도해 세 번째에 성공했다.** 앞의 둘은 "손실 0"으로 보였지만 실제로는 크래시가 아니었다.
docs/evidence/a4-node-loss/01-baseline.txt
+20
View File
@@ -0,0 +1,20 @@
=== A-4 기준선 ===
kc-lab-1 Ready true
kc-lab-2 Ready <none>
a2-probe true kc-lab-2
keycloak-0 true kc-lab-2
keycloak-1 true kc-lab-1
postgres-7b474b88c8-2gf27 true kc-lab-2
=== PVC 가 어느 노드에 묶여 있는가 (재배치 가능성) ===
persistentvolumeclaim/postgres-data → kc-lab-2
=== 서비스 정상 확인 ===
https://auth.hyeonworks.com/realms/master HTTP 200
=== VM 상태 ===
--------------------------
1 kc-lab-1 running
2 kc-lab-2 running
docs/evidence/a4-node-loss/02-worker-node-killed.txt
+17
View File
@@ -0,0 +1,17 @@
=== 워커 노드(kc-lab-2) 전원 차단 — virsh destroy 는 종료 신호가 없다 ===
차단 시각: 12:07:43
Domain 'kc-lab-2' destroyed
+15초 node=Ready | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 000
+30초 node=Ready | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 000
+45초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+60초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+75초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+90초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+105초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+120초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+135초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+150초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+165초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
+180초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
docs/evidence/a4-node-loss/03-state-during-loss.txt
+28
View File
@@ -0,0 +1,28 @@
=== 파드 상태의 진실 — Running 인데 노드가 없다 ===
a2-probe Running true kc-lab-2 <none>
keycloak-0 Running true kc-lab-2 <none>
keycloak-1 Running false kc-lab-1 <none>
postgres-7b474b88c8-2gf27 Running true kc-lab-2 <none>
=== 재배치가 시도되었는가 ===
10m Warning Unhealthy pod/keycloak-0 Readiness probe failed: Get "http://10.42.1.67:9000/health/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
3m15s Warning NodeNotReady pod/postgres-7b474b88c8-2gf27 Node is not ready
3m15s Warning NodeNotReady pod/keycloak-0 Node is not ready
3m15s Warning NodeNotReady pod/a2-probe Node is not ready
2m27s Warning Unhealthy pod/keycloak-1 Readiness probe failed: Get "http://10.42.0.35:9000/health/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2s Warning Unhealthy pod/keycloak-1 Readiness probe failed: HTTP probe failed with statuscode: 503
=== 노드 taint — 쿠버네티스가 붙인 것 ===
node.kubernetes.io/unreachable=:NoSchedule
node.kubernetes.io/unreachable=:NoExecute
=== Prometheus 가 본 것 (kc-lab-1 에 있어 살아남았다) ===
up{job=keycloak pod=keycloak-1 } = 1
up{job=keycloak pod=keycloak-0 } = 0
up{job=kubelet pod=- } = 1
up{job=kubelet pod=- } = 0
up{job=node-exporter pod=kc-lab-1 } = 1
up{job=node-exporter pod=kc-lab-2 } = 0
up{job=prometheus pod=- } = 1
=== 진입점이 처음 40초간 000 이었던 이유 — nginx upstream ===
docs/evidence/a4-node-loss/04-eviction-timing.txt
+15
View File
@@ -0,0 +1,15 @@
=== nginx 설정 위치 찾기 ===
=== NoExecute taint 의 tolerationSeconds — 언제 축출되는가 ===
node.kubernetes.io/not-ready NoExecute tolerationSeconds=300
node.kubernetes.io/unreachable NoExecute tolerationSeconds=300
=== 5분 축출 시점까지 관찰 ===
+210초 a2-probe:Running keycloak-0:Running keycloak-1:Running postgres-7b474b88c8-2gf27:Running
+240초 a2-probe:Running keycloak-0:Running keycloak-1:Running postgres-7b474b88c8-2gf27:Running
+270초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
+300초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
+330초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
+360초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
+390초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
+420초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
docs/evidence/a4-node-loss/05-recovery.txt
+18
View File
@@ -0,0 +1,18 @@
=== 새 postgres 가 Pending 인 이유 ===
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 4m45s default-scheduler 0/2 nodes are available: 1 node(s) didn't match PersistentVolume's node affinity, 1 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.
=== keycloak-0 대체 파드가 안 생기는 이유 (StatefulSet) ===
keycloak 2 <none> 1
keycloak-0 1/1 Terminating 0 30m
keycloak-1 0/1 Running 0 143m
=== 복구 — 노드 재기동 ===
재기동 시각: 12:16:31
Domain 'kc-lab-2' started
+30초 node=Ready | Running 파드 3 개 | 외부 HTTP 503
+60초 node=Ready | Running 파드 3 개 | 외부 HTTP 200
→ 서비스 복귀
docs/evidence/a4-node-loss/06-control-plane-inventory.txt
+19
View File
@@ -0,0 +1,19 @@
=== 복구 확인 ===
keycloak-0 1/1 Running 0 68s
keycloak-1 1/1 Running 0 144m
postgres-7b474b88c8-9cmsv 1/1 Running 0 4m20s
=== kc-lab-1(k3s server)에 무엇이 있는가 — 이게 곧 영향 범위다 ===
keycloak-lab keycloak-1
kube-system coredns-54996dc9b4-8k8fj
kube-system helm-install-traefik-crd-q29b5
kube-system local-path-provisioner-77b9867795-g27z8
kube-system metrics-server-6dc596dfb8-7xxq4
kube-system svclb-traefik-5eb6a9a1-qwwk5
kube-system traefik-5d6fcf895-wpfhr
observability grafana-845b5678cf-b6gvc
observability node-exporter-9qk9w
observability prometheus-6774f94f7c-pzr2t
=== Traefik replica 수 (진입점의 단일 장애점인가) ===
traefik 1 1
docs/evidence/a4-node-loss/07-control-plane-loss.txt
+19
View File
@@ -0,0 +1,19 @@
=== 컨트롤 플레인 노드(kc-lab-1) 전원 차단 ===
차단 시각: 12:18:08
Domain 'kc-lab-1' destroyed
+20초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
+40초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
+60초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
+80초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
+100초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
+120초 외부 auth=000 grafana=502 | kubectl: Unable to connect to the server: dial tcp
+140초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
+160초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
=== 살아 있는 노드에서 직접 확인 — 워크로드는 도는가 ===
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD NAMESPACE
e5f777900b762 60e153026e8f5 4 minutes ago Running keycloak 0 640d4dafaefb3 keycloak-0 keycloak-lab
6
docs/evidence/a4-node-loss/08-control-plane-recovery.txt
+13
View File
@@ -0,0 +1,13 @@
=== 컨트롤 플레인 노드 복구 ===
재기동: 12:23:39
Domain 'kc-lab-1' started
+30초 외부=502 | kc-lab-1=Ready kc-lab-2=Ready
+60초 외부=200 | kc-lab-1=Ready kc-lab-2=Ready
→ 서비스 복귀 (총 60초)
=== 최종 상태 ===
keycloak-0 1/1 Running 0 7m57s
keycloak-1 1/1 Running 1 (<invalid> ago) 151m
postgres-7b474b88c8-9cmsv 1/1 Running 0 11m
(prometheus port-forward 재연결 필요)
docs/evidence/a4-node-loss/README.md
+23
View File
@@ -0,0 +1,23 @@
# A-4 — 노드 전원 차단 증거
2026-09-04 12:07–12:24 KST
해설: [`docs/experiment-a4-node-loss.md`](../../experiment-a4-node-loss.md)
| 파일 | 무엇을 보여주는가 |
|---|---|
| `01-baseline.txt` | 차단 전 — 양쪽 Ready, **PVC 가 kc-lab-2 에 못박혀 있음**(재배치 불가의 원인), 외부 200 |
| `02-worker-node-killed.txt` | `virsh destroy kc-lab-2` — **40초간 노드가 Ready 로 남아 있고** 외부는 이미 `000`. 이후 `503` |
| `03-state-during-loss.txt` | **죽은 파드가 `ready=true`, 산 파드가 `ready=false`.** `up` 은 정확히 0. `unreachable` taint |
| `04-eviction-timing.txt` | `tolerationSeconds=300` — **5분 뒤** 축출, 새 postgres 는 `Pending` |
| `05-recovery.txt` | `FailedScheduling: didn't match PersistentVolume's node affinity`, StatefulSet `DESIRED=2 CURRENT=1`. 노드 복귀 후 **60초** |
| `06-control-plane-inventory.txt` | kc-lab-1 에 있는 것 목록 — **Traefik `replicas=1`** |
| `07-control-plane-loss.txt` | `virsh destroy kc-lab-1` — 외부 `000`, `kubectl` 불통. **그런데 `crictl ps` 로 보면 keycloak-0 은 Running** |
| `08-control-plane-recovery.txt` | 60초 만에 복귀 |
| `a4-up-dropped-per-node.png` | Grafana — `up` 이 노드별로 0 으로 떨어지는 구간. 12:18–12:23 은 **0 이 아니라 데이터 없음**(관측자가 같이 죽음) |
## 핵심 네 줄
1. **쿠버네티스는 40초 동안 노드가 살아 있다고 말한다.** 사용자는 이미 장애를 겪는 중이다.
2. **죽은 파드의 상태는 화석이다.** `ready=true` 인 파드가 꺼진 기계 위에 있다.
3. **StatefulSet 은 대체 파드를 만들지 않고, PVC 는 재배치를 막는다.** 사람이 개입해야 한다.
4. **컨트롤 플레인 상실 ≠ 워크로드 상실.** 컨테이너는 계속 돌고, 들어갈 문만 사라진다.
docs/evidence/a4-node-loss/a4-up-dropped-per-node.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 79 KiB

docs/evidence/a5-asymmetric-partition/01-injection.txt
+42
View File
@@ -0,0 +1,42 @@
=== A-5 기준선 ===
keycloak-0=10.42.1.77 (kc-lab-2) keycloak-1=10.42.0.42 (kc-lab-1)
keycloak-0 1/1 Running 0 11m
keycloak-1 1/1 Running 1 (2m48s ago) 155m
postgres-7b474b88c8-9cmsv 1/1 Running 0 14m
Traceback (most recent call last):
File "<string>", line 3, in <module>
for r in json.load(sys.stdin)["data"]["result"]: print(f" cluster_size {r["metric"].get("pod"):12} = {r["value"][1]}")
~~~~~~~~~^^^^^^^^^^^
File "/usr/lib/python3.14/json/__init__.py", line 298, in load
return loads(fp.read(),
cls=cls, object_hook=object_hook,
parse_float=parse_float, parse_int=parse_int,
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib/python3.14/json/__init__.py", line 352, in loads
return _default_decoder.decode(s)
~~~~~~~~~~~~~~~~~~~~~~~^^^
File "/usr/lib/python3.14/json/decoder.py", line 345, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.14/json/decoder.py", line 363, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
=== 주입: keycloak-0 으로 들어가는 7800/57800 만 DROP (한 방향) ===
A-1 의 NetworkPolicy 는 conntrack ESTABLISHED 에 막혀 기존 연결을 못 끊었다.
FORWARD 최상단에 넣으면 conntrack 승인보다 먼저 평가된다.
Chain FORWARD (policy ACCEPT)
num target prot opt source destination
1 DROP 6 -- 0.0.0.0/0 10.42.1.77 tcp dpt:57800
2 DROP 6 -- 0.0.0.0/0 10.42.1.77 tcp dpt:7800
주입 시각: 12:28:23
=== 관찰 — 어느 쪽이 먼저 상대를 의심하는가 ===
+25초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+50초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+75초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+100초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+125초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+150초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+175초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+200초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
docs/evidence/a5-asymmetric-partition/02-injection-verify.txt
View File
docs/evidence/a5-asymmetric-partition/03-raw-table-injection.txt
+16
View File
@@ -0,0 +1,16 @@
=== 실패한 규칙 제거 ===
제거완료
=== raw 테이블 PREROUTING 에 넣는다 — conntrack 보다 먼저 실행된다 ===
netfilter 순서: raw PREROUTING → conntrack → mangle → nat → filter
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 0 0 DROP 6 -- * * 0.0.0.0/0 10.42.1.77 tcp dpt:57800
2 0 0 DROP 6 -- * * 0.0.0.0/0 10.42.1.77 tcp dpt:7800
주입: 12:32:35
=== [검증] 이번엔 패킷이 걸렸는가 ===
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DROP 6 -- * * 0.0.0.0/0 10.42.1.77 tcp dpt:57800
0 0 DROP 6 -- * * 0.0.0.0/0 10.42.1.77 tcp dpt:7800
docs/evidence/a5-asymmetric-partition/04-correct-direction.txt
+18
View File
@@ -0,0 +1,18 @@
=== keycloak-1(수신측)으로 들어가는 7800/57800 만 DROP — kc-lab-1 에 넣는다 ===
주입: 12:33:58
=== [검증] 패킷이 걸리는가 ===
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DROP 6 -- * * 0.0.0.0/0 10.42.0.42 tcp dpt:57800
19 2938 DROP 6 -- * * 0.0.0.0/0 10.42.0.42 tcp dpt:7800
=== 관찰 ===
+25초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+50초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+75초 - | keycloak-0:1/1 keycloak-1:0/1 | 외부 200
+100초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+125초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+150초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+175초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
+200초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200
docs/evidence/a5-asymmetric-partition/05-reconnect-observed.txt
+19
View File
@@ -0,0 +1,19 @@
=== 지금 연결 방향은? (차단은 → 10.42.0.42:7800 만) ===
tcp 6 299 ESTABLISHED src=10.42.0.42 dst=10.42.1.77 sport=48473 dport=7800 src=10.42.1.77 dst=10.42.0.42 sport=7800 dport=48473
tcp 6 86232 ESTABLISHED src=10.42.0.42 dst=10.42.1.77 sport=44205 dport=57800 src=10.42.1.77 dst=10.42.0.42 sport=57800 dport=44205
=== 차단 규칙 누적 카운터 ===
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
19 1096 DROP 6 -- * * 0.0.0.0/0 10.42.0.42 tcp dpt:57800
21 3058 DROP 6 -- * * 0.0.0.0/0 10.42.0.42 tcp dpt:7800
=== 클러스터 멤버 (파드에서 직접) ===
keycloak-0 cluster_size=2.0
keycloak-1 cluster_size=2.0
=== 최근 뷰 로그 ===
keycloak-0: [keycloak-0-24309(v=16.0.12)|12] (1) [keycloak-0-24309(v=16.0.12)]
keycloak-1: [keycloak-1-45480(v=16.0.12)|12] (1) [keycloak-1-45480(v=16.0.12)]
docs/evidence/a5-asymmetric-partition/06-view-history-and-cleanup.txt
+28
View File
@@ -0,0 +1,28 @@
=== 뷰 변화 전체 이력 (차단 12:33:58 전후) ===
--- keycloak-0 ---
2026-09-04 03:33:49 | during merge: CacheTopology{id=37, phase=NO_REBALANCE, rebalanceId=11, currentCH=DefaultConsistentHash{ns=25
2026-09-04 03:33:49,398 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t1) [Context=offlineSessions] ISPN100007: After me
2026-09-04 03:33:49 | during merge: CacheTopology{id=37, phase=NO_REBALANCE, rebalanceId=11, currentCH=DefaultConsistentHash{ns=25
2026-09-04 03:33:49,399 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t1) [Context=loginFailures] ISPN100007: After merg
2026-09-04 03:33:49 | during merge: CacheTopology{id=37, phase=NO_REBALANCE, rebalanceId=11, currentCH=DefaultConsistentHash{ns=25
2026-09-04 03:33:49,402 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t1) [Context=actionTokens] ISPN100007: After merge
--- keycloak-1 ---
2026-09-04 03:32:59,874 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t2) [Context=work] ISPN100007: After merge (or coo
2026-09-04 03:32:59,879 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t2) [Context=clientSessions] ISPN100007: After mer
2026-09-04 03:32:59,879 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t1) [Context=offlineSessions] ISPN100007: After me
2026-09-04 03:32:59,883 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t1) [Context=loginFailures] ISPN100007: After merg
2026-09-04 03:32:59,884 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t2) [Context=actionTokens] ISPN100007: After merge
2026-09-04 03:33:49 | MergeView::[keycloak-0-24309(v=16.0.12)|13] (2) [keycloak-0-24309(v=16.0.12), keycloak-1-45480(v=16.0.12)],
=== MERGE3 병합 이벤트 ===
keycloak-0 merge_events=1.0 suspected=0.0
keycloak-1 merge_events=1.0 suspected=0.0
=== 차단 해제 ===
해제완료
keycloak-0 cluster_size=2.0
warning: couldn't attach to pod/a5-f, falling back to streaming logs: unable to upgrade connection: container a5-f not found in pod a5-f_keycloak-lab
keycloak-0 cluster_size=2.0
keycloak-1 cluster_size=2.0
keycloak-0 1/1 Running 0 22m
keycloak-1 1/1 Running 1 (13m ago) 166m
docs/evidence/a5-asymmetric-partition/07-bidirectional-block.txt
+17
View File
@@ -0,0 +1,17 @@
=== 양방향 차단 — 두 노드 모두에 raw DROP ===
주입: 12:40:25
+25초 keycloak-0:1/1 keycloak-1:1/1 | ready=[10.42.0.42 10.42.1.77] 외부 200
+50초 keycloak-0:1/1 keycloak-1:1/1 | ready=[10.42.0.42 10.42.1.77] 외부 200
+75초 keycloak-0:1/1 keycloak-1:1/1 | ready=[10.42.0.42 10.42.1.77] 외부 200
+100초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200
+125초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200
+150초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200
+175초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200
+200초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200
+225초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200
=== 뷰 상태 ===
keycloak-0 [keycloak-0-24309(v=16.0.12)|14] (1) [keycloak-0-24309(v=16.0.12)]
keycloak-1 [keycloak-1-45480(v=16.0.12)|14] (1) [keycloak-1-45480(v=16.0.12)]
docs/evidence/a5-asymmetric-partition/08-coordinator-and-recovery.txt
+39
View File
@@ -0,0 +1,39 @@
=== 왜 한쪽만 DOWN 인가 — 코디네이터 여부 확인 ===
name | ip | coord
------------------+-----------------+-------
keycloak-0-24309 | 10.42.1.77:7800 | t
keycloak-1-45480 | 10.42.0.42:7800 | t
(2 rows)
=== 양쪽 헬스체크 상세 ===
--- keycloak-0 ---
{"status":"UP","checks":[{"name":"GracefulShutdown","status":"UP"}
{"name":"KeycloakInitialized","status":"UP"}
{"name":"Keycloakclusterhealthcheck","status":"UP"}
--- keycloak-1 ---
{"status":"DOWN","checks":[{"name":"GracefulShutdown","status":"UP"}
{"name":"Keycloakdatabaseconnectionsasynchealthcheck","status":"UP"}
{"name":"KeycloakInitialized","status":"UP"}
warning: couldn't attach to pod/a5-h, falling back to streaming logs: Internal error occurred: Internal error occurred: error attaching to container: container is in CONTAINER_EXITED state
--- keycloak-0 ---
{"status":"UP","checks":[{"name":"GracefulShutdown","status":"UP"}
{"name":"KeycloakInitialized","status":"UP"}
{"name":"Keycloakclusterhealthcheck","status":"UP"}
--- keycloak-1 ---
{"status":"DOWN","checks":[{"name":"GracefulShutdown","status":"UP"}
{"name":"Keycloakdatabaseconnectionsasynchealthcheck","status":"UP"}
{"name":"KeycloakInitialized","status":"UP"}
=== 차단 해제 ===
해제: 12:44:37
+25초 keycloak-0:1/1 keycloak-1:0/1
+50초 keycloak-0:1/1 keycloak-1:1/1
→ 복구 완료
keycloak-0 MergeView::[keycloak-0-24309(v=16.0.12)|15] (2) [keycloak-0-24309(v=16.0.12), keycloak-1-45480(
keycloak-1 MergeView::[keycloak-0-24309(v=16.0.12)|15] (2) [keycloak-0-24309(v=16.0.12), keycloak-1-45480(
docs/evidence/a5-asymmetric-partition/README.md
+23
View File
@@ -0,0 +1,23 @@
# A-5 — 비대칭 파티션 증거
2026-09-04 12:28–12:46 KST
해설: [`docs/experiment-a5-asymmetric-partition.md`](../../experiment-a5-asymmetric-partition.md)
| 파일 | 무엇을 보여주는가 |
|---|---|
| `01-injection.txt` | 첫 시도 — `iptables -I FORWARD 1` |
| `02-injection-verify.txt` | **실패 확인** — kube-router 가 자기 체인을 위로 재삽입, DROP 규칙 **0 패킷** |
| `03-raw-table-injection.txt` | `raw` 테이블로 이동. 그래도 0 패킷 |
| `04-correct-direction.txt` | **연결 방향이 반대였다.** 수신측 노드로 옮기니 **19/21 패킷 차단** |
| `05-reconnect-observed.txt` | **핵심** — 연결이 **열린 방향으로 뒤집혀 재연결**. 뷰 변화 없음, `suspected=0` |
| `06-view-history-and-cleanup.txt` | 뷰 ID 이력. 주입 이후 변화 없음 확인 |
| `07-bidirectional-block.txt` | 양방향 차단 → **뷰 14, 멤버 1개씩.** 그런데 `ready=[10.42.1.77]` 로 **한쪽은 남고 외부 200** |
| `08-coordinator-and-recovery.txt` | **`coord=t` 가 둘**(split brain). keycloak-0 `UP` / keycloak-1 `DOWN`. 해제 후 50초, `MergeView |15] (2)` |
| `a5-cluster-size-bidirectional-block.png` | Grafana — `vendor_cluster_size` 가 갈라졌다 합쳐지는 구간 |
## 핵심 네 줄
1. **주입을 세 번 실패했다** — CNI 체인 경쟁, 연결 방향 오판, conntrack. 셋 다 "아무 일 없음"으로 보였다.
2. **단방향 차단으로는 못 가른다.** JGroups 가 열린 방향으로 재연결한다 (자가 치유).
3. **양방향이면 갈라진다** — `coord=t` 가 둘.
4. **그래도 전면 장애가 아니다.** 코디네이터 쪽만 UP 을 유지해 서비스가 계속된다 — A-1 의 열린 질문에 대한 답.
docs/evidence/a5-asymmetric-partition/a5-cluster-size-bidirectional-block.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 65 KiB

docs/evidence/a6-latency-injection/01-baseline.txt
+22
View File
@@ -0,0 +1,22 @@
=== 배치 — 왜 A/B 가 되는가 ===
postgres 10.42.1.76 (kc-lab-2)
keycloak-0 10.42.1.77 (kc-lab-2) → DB 와 같은 노드, cni0 로 직행
keycloak-1 10.42.0.42 (kc-lab-1) → DB 와 다른 노드, VXLAN 을 건넌다 ← 여기에 지연을 건다
=== 사용 가능한 커넥션 풀 지표 ===
agroal_acquire_count_total
agroal_active_count
agroal_available_count
agroal_awaiting_count
agroal_blocking_time_average_milliseconds
agroal_blocking_time_max_milliseconds
agroal_blocking_time_total_milliseconds
agroal_creation_count_total
agroal_creation_time_average_milliseconds
agroal_creation_time_max_milliseconds
agroal_creation_time_total_milliseconds
agroal_destroy_count_total
=== 기준선 지연 — 각 노드에서 로그인 20회 ===
keycloak-0 평균 70 ms
keycloak-1 평균 66 ms
docs/evidence/a6-latency-injection/02-delay-injected.txt
+25
View File
@@ -0,0 +1,25 @@
=== 주입: postgres(10.42.1.76) 가 보내는 패킷만 200ms 지연 (kc-lab-2 eth0) ===
prio qdisc 로 밴드를 나누고, u32 필터로 출발지 IP 가 postgres 인 것만 3번 밴드로 보낸다
Cannot find device "eth0"
Cannot find device "eth0"
적용완료
Cannot find device "eth0"
Cannot find device "eth0"
주입: 13:14:55
=== [검증] 지연이 실제로 걸렸는가 — 두 노드 비교 ===
keycloak-0 평균 43 ms 최대 64 ms
keycloak-1 평균 47 ms 최대 70 ms
=== 커넥션 풀 상태 ===
--- keycloak-0 ---
agroal_blocking_time_max_milliseconds 102.0
agroal_active_count 0.0
agroal_awaiting_count 0.0
agroal_blocking_time_average_milliseconds 0.0
agroal_available_count 2.0
agroal_blocking_time_max_milliseconds 164.0
agroal_active_count 0.0
agroal_awaiting_count 0.0
agroal_blocking_time_average_milliseconds 0.0
agroal_available_count 2.0
docs/evidence/a6-latency-injection/03-flannel-injection.txt
+20
View File
@@ -0,0 +1,20 @@
=== 오버레이 인터페이스 확인 ===
flannel.1 UNKNOWN a6:b2:62:04:c1:a4 <BROADCAST,MULTICAST,UP,LOWER_UP>
cni0 UP 5a:77:1a:e2:b0:a4 <BROADCAST,MULTICAST,UP,LOWER_UP>
=== flannel.1 에 주입 — 여기서는 파드 IP 가 보인다 (캡슐화 전) ===
qdisc prio 1: root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc netem 30: parent 1:3 limit 1000 delay 200ms
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
=== [검증] 필터에 패킷이 걸리는가 ===
qdisc netem 30: parent 1:3 limit 1000 delay 200ms
Sent 18388 bytes 150 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
=== 두 노드 지연 비교 (기준선: k0=70ms k1=66ms) ===
keycloak-0 평균 41 ms 최대 57 ms
keycloak-1 평균 1872 ms 최대 1887 ms
docs/evidence/a6-latency-injection/04-pool-under-load.txt
+37
View File
@@ -0,0 +1,37 @@
=== 동시 부하 20건을 keycloak-1 에 — 커넥션 풀이 견디는가 ===
1 200 1.911191
1 200 1.913766
1 200 1.958374
1 200 1.981620
1 200 10.539402
1 200 11.951943
1 200 13.351102
1 200 14.785832
1 200 16.189533
1 200 17.625166
1 200 19.053724
1 200 20.495883
1 200 21.905932
1 200 22.228466
1 200 22.230871
1 200 3.441366
1 200 4.841075
1 200 6.257489
1 200 7.704608
1 200 9.104792
=== 부하 직후 커넥션 풀 ===
agroal_blocking_time_max_milliseconds 20000.0
agroal_max_used_count 19.0
agroal_acquire_count_total 672.0
agroal_active_count 0.0
agroal_awaiting_count 0.0
agroal_blocking_time_average_milliseconds 281.0
agroal_available_count 19.0
=== readiness 가 흔들렸는가 ===
keycloak-0 1/1 Running 0 60m
keycloak-1 1/1 Running 1 (51m ago) 3h24m
52m Normal TaintManagerEviction pod/keycloak-1 Cancelling deletion of Pod keycloak-lab/keycloak-1
32m Warning Unhealthy pod/keycloak-1 Readiness probe failed: HTTP probe failed with statuscode: 503
89s Warning Unhealthy pod/keycloak-1 Readiness probe failed: Get "http://10.42.0.42:9000/health/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Some files were not shown because too many files have changed in this diff Show More