diff --git a/.playwright-mcp/console-2026-09-04T03-23-15-961Z.log b/.playwright-mcp/console-2026-09-04T03-23-15-961Z.log index 102ab83..3adf2b1 100644 --- a/.playwright-mcp/console-2026-09-04T03-23-15-961Z.log +++ b/.playwright-mcp/console-2026-09-04T03-23-15-961Z.log @@ -18,3 +18,108 @@ [ 98008ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 [ 104458ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 [ 123816ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 142764ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 157198ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 175220ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 182187ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 183411ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 190581ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 199499ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 209995ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 214849ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 233084ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 242910ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 257975ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 277220ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 290806ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 292674ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 300013ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 309267ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 323998ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 328620ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 339787ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 355959ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 361079ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 369272ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 381866ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 397120ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 400712ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 412579ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 431630ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 440746ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 447809ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 461741ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 475459ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 482723ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 494299ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 513174ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 521541ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 532145ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 551129ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 556250ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 574381ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 586190ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 595782ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 610555ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 630695ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 633257ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 652194ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 671361ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 677606ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 692444ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 696949ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 700122ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 709954ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 723992ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 743173ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 745075ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 753299ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 766683ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 767855ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 787261ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 789725ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 800680ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 811324ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 814096ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 829045ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 848506ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 857716ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 866212ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 873174ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 876033ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 895198ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 898884ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 918642ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 930133ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 934008ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 948541ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 956860ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 959397ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 968406ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 971788ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 982746ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 997394ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1005078ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1009269ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1021453ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1026674ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1042137ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1057497ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1059851ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1061855ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1071317ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1078179ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1084940ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1093950ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1112279ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1124360ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1139110ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1157981ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1163071ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1179032ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1181195ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1191955ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1201472ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1218261ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1221169ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1229430ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 diff --git a/.playwright-mcp/console-2026-09-04T03-43-55-141Z.log b/.playwright-mcp/console-2026-09-04T03-43-55-141Z.log new file mode 100644 index 0000000..8b28483 --- /dev/null +++ b/.playwright-mcp/console-2026-09-04T03-43-55-141Z.log @@ -0,0 +1,24 @@ +[ 199ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 297ms] [WARNING] is deprecated. Please include @ https://app2.hyeonworks.com/explore?schemaVersion=1&orgId=1&panes=%7B%22a%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22vendor_cluster_size%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bpod%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-25m%22%2C%22to%22%3A%22now%22%7D%7D%7D:0 +[ 1000ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 2023ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 3559ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 7749ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 16364ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 25781ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 42158ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 59672ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 77090ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 87426ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 89952ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 104743ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error in connection establishment: net::ERR_NAME_NOT_RESOLVED @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1573121ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1584038ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1601626ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1613280ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1630173ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1634776ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1654205ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1661576ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1665269ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 +[ 1682669ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362 diff --git a/.playwright-mcp/page-2026-09-04T03-43-55-616Z.yml b/.playwright-mcp/page-2026-09-04T03-43-55-616Z.yml new file mode 100644 index 0000000..3b566a8 --- /dev/null +++ b/.playwright-mcp/page-2026-09-04T03-43-55-616Z.yml @@ -0,0 +1,104 @@ +- generic [ref=f12e4]: + - link "Skip to main content" [ref=f12e5] [cursor=pointer]: + - /url: "#pageContent" + - banner [ref=f12e7]: + - generic [ref=f12e8]: + - link [ref=f12e10] [cursor=pointer]: + - /url: / + - img "Grafana" [ref=f12e11] + - generic [ref=f12e14]: + - button "Search or jump to..." [ref=f12e18] [cursor=pointer] + - generic [ref=f12e19]: ctrl+k + - generic [ref=f12e23]: + - button "New" [ref=f12e24] [cursor=pointer] + - button "Help" [ref=f12e30] [cursor=pointer] + - button "News" [ref=f12e33] [cursor=pointer] + - button "Profile" [ref=f12e36] [cursor=pointer]: + - img "User avatar" [ref=f12e37] + - generic [ref=f12e38]: + - button "Open menu" [ref=f12e40] [cursor=pointer] + - navigation "Breadcrumbs" [ref=f12e43]: + - list [ref=f12e44]: + - listitem [ref=f12e45]: + - link "Home" [ref=f12e46] [cursor=pointer]: + - /url: / + - listitem [ref=f12e50]: + - link "Explore" [ref=f12e51] [cursor=pointer]: + - /url: /explore + - listitem [ref=f12e55]: + - generic "Prometheus" [ref=f12e56] + - generic [ref=f12e57]: + - button "Show more items" [ref=f12e60] [cursor=pointer] + - button "Toggle top search bar" [ref=f12e64] [cursor=pointer] + - main [ref=f12e70]: + - generic [ref=f12e72]: + - heading "Explore" [level=1] [ref=f12e73] + - generic [ref=f12e78]: + - navigation "Explore toolbar" [ref=f12e80]: + - navigation "Search links" [ref=f12e82]: + - generic [ref=f12e83]: + - button "Content outline" [expanded] [ref=f12e85] [cursor=pointer]: + - generic [ref=f12e88]: Outline + - generic [ref=f12e93] [cursor=pointer]: + - img "Prometheus logo" [ref=f12e95] + - textbox "Select a data source" [ref=f12e96]: + - /placeholder: "" + - button "Show more items" [ref=f12e102] [cursor=pointer] + - generic [ref=f12e106]: + - generic [ref=f12e110]: + - button "Collapse outline" [expanded] [ref=f12e112] [cursor=pointer]: + - img "arrow-from-right" [ref=f12e113] + - button "Queries" [ref=f12e116] [cursor=pointer]: + - img "arrow" [ref=f12e117] + - generic [ref=f12e124]: + - generic [ref=f12e126]: + - generic "Query editor row" [ref=f12e129]: + - generic [ref=f12e130]: + - generic [ref=f12e132]: + - generic [ref=f12e133]: + - button "Collapse query row" [expanded] [ref=f12e134] [cursor=pointer] + - generic [ref=f12e137]: + - button "Query editor row title A" [ref=f12e138] [cursor=pointer]: + - generic [ref=f12e139]: A + - emphasis [ref=f12e140]: (Prometheus) + - generic [ref=f12e141]: + - button "Show data source help" [ref=f12e143] [cursor=pointer] + - button "Duplicate query" [ref=f12e147] [cursor=pointer] + - button "Hide response" [ref=f12e151] [cursor=pointer] + - button "Remove query" [ref=f12e155] [cursor=pointer] + - button "Drag and drop to reorder" [ref=f12e158]: + - img "Drag and drop to reorder" [ref=f12e159] + - generic [ref=f12e162]: + - generic [ref=f12e163]: + - button "Kick start your query" [ref=f12e164] [cursor=pointer] + - generic [ref=f12e167]: + - generic [ref=f12e168] [cursor=pointer]: Explain + - generic [ref=f12e169]: + - checkbox "Explain Toggle switch" [ref=f12e170] + - generic "Toggle switch" [ref=f12e171] [cursor=pointer] + - radiogroup [ref=f12e176]: + - generic [ref=f12e177]: + - radio "Builder" [ref=f12e178] [cursor=pointer] + - generic [ref=f12e179] [cursor=pointer]: Builder + - generic [ref=f12e180]: + - radio "Code" [checked] [ref=f12e181] [cursor=pointer] + - generic [ref=f12e182] [cursor=pointer]: Code + - generic [ref=f12e184]: + - generic [ref=f12e186]: + - button "Loading metrics..." [disabled] [ref=f12e187] [cursor=pointer] + - generic [ref=f12e190]: Loading editor + - 'button "Options Legend: {{pod}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f12e198] [cursor=pointer]': + - generic [ref=f12e202]: + - heading "Options" [level=6] [ref=f12e203] + - generic [ref=f12e204]: + - generic [ref=f12e205]: "Legend: {{pod}}" + - generic [ref=f12e206]: "Format: Time series" + - generic [ref=f12e207]: "Step: auto" + - generic [ref=f12e208]: "Type: Range" + - generic [ref=f12e209]: "Exemplars: false" + - generic [ref=f12e210]: + - button "Add query" [ref=f12e211] [cursor=pointer] + - button "Query history" [ref=f12e215] [cursor=pointer] + - button "Query inspector" [ref=f12e219] [cursor=pointer] + - generic: + - main \ No newline at end of file diff --git a/docs/evidence/a5-asymmetric-partition/01-injection.txt b/docs/evidence/a5-asymmetric-partition/01-injection.txt new file mode 100644 index 0000000..3eca4b9 --- /dev/null +++ b/docs/evidence/a5-asymmetric-partition/01-injection.txt @@ -0,0 +1,42 @@ +=== A-5 기준선 === + keycloak-0=10.42.1.77 (kc-lab-2) keycloak-1=10.42.0.42 (kc-lab-1) +keycloak-0 1/1 Running 0 11m +keycloak-1 1/1 Running 1 (2m48s ago) 155m +postgres-7b474b88c8-9cmsv 1/1 Running 0 14m +Traceback (most recent call last): + File "", line 3, in + for r in json.load(sys.stdin)["data"]["result"]: print(f" cluster_size {r["metric"].get("pod"):12} = {r["value"][1]}") + ~~~~~~~~~^^^^^^^^^^^ + File "/usr/lib/python3.14/json/__init__.py", line 298, in load + return loads(fp.read(), + cls=cls, object_hook=object_hook, + parse_float=parse_float, parse_int=parse_int, + parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) + File "/usr/lib/python3.14/json/__init__.py", line 352, in loads + return _default_decoder.decode(s) + ~~~~~~~~~~~~~~~~~~~~~~~^^^ + File "/usr/lib/python3.14/json/decoder.py", line 345, in decode + obj, end = self.raw_decode(s, idx=_w(s, 0).end()) + ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ + File "/usr/lib/python3.14/json/decoder.py", line 363, in raw_decode + raise JSONDecodeError("Expecting value", s, err.value) from None +json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) + +=== 주입: keycloak-0 으로 들어가는 7800/57800 만 DROP (한 방향) === + A-1 의 NetworkPolicy 는 conntrack ESTABLISHED 에 막혀 기존 연결을 못 끊었다. + FORWARD 최상단에 넣으면 conntrack 승인보다 먼저 평가된다. +Chain FORWARD (policy ACCEPT) +num target prot opt source destination +1 DROP 6 -- 0.0.0.0/0 10.42.1.77 tcp dpt:57800 +2 DROP 6 -- 0.0.0.0/0 10.42.1.77 tcp dpt:7800 + 주입 시각: 12:28:23 + +=== 관찰 — 어느 쪽이 먼저 상대를 의심하는가 === + +25초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +50초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +75초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +100초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +125초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +150초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +175초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +200초 | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 diff --git a/docs/evidence/a5-asymmetric-partition/02-injection-verify.txt b/docs/evidence/a5-asymmetric-partition/02-injection-verify.txt new file mode 100644 index 0000000..e69de29 diff --git a/docs/evidence/a5-asymmetric-partition/03-raw-table-injection.txt b/docs/evidence/a5-asymmetric-partition/03-raw-table-injection.txt new file mode 100644 index 0000000..5aa87ed --- /dev/null +++ b/docs/evidence/a5-asymmetric-partition/03-raw-table-injection.txt @@ -0,0 +1,16 @@ +=== 실패한 규칙 제거 === +제거완료 + +=== raw 테이블 PREROUTING 에 넣는다 — conntrack 보다 먼저 실행된다 === + netfilter 순서: raw PREROUTING → conntrack → mangle → nat → filter +Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) +num pkts bytes target prot opt in out source destination +1 0 0 DROP 6 -- * * 0.0.0.0/0 10.42.1.77 tcp dpt:57800 +2 0 0 DROP 6 -- * * 0.0.0.0/0 10.42.1.77 tcp dpt:7800 + 주입: 12:32:35 + +=== [검증] 이번엔 패킷이 걸렸는가 === + Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) + pkts bytes target prot opt in out source destination + 0 0 DROP 6 -- * * 0.0.0.0/0 10.42.1.77 tcp dpt:57800 + 0 0 DROP 6 -- * * 0.0.0.0/0 10.42.1.77 tcp dpt:7800 diff --git a/docs/evidence/a5-asymmetric-partition/04-correct-direction.txt b/docs/evidence/a5-asymmetric-partition/04-correct-direction.txt new file mode 100644 index 0000000..f322dc7 --- /dev/null +++ b/docs/evidence/a5-asymmetric-partition/04-correct-direction.txt @@ -0,0 +1,18 @@ +=== keycloak-1(수신측)으로 들어가는 7800/57800 만 DROP — kc-lab-1 에 넣는다 === + 주입: 12:33:58 + +=== [검증] 패킷이 걸리는가 === + Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) + pkts bytes target prot opt in out source destination + 0 0 DROP 6 -- * * 0.0.0.0/0 10.42.0.42 tcp dpt:57800 + 19 2938 DROP 6 -- * * 0.0.0.0/0 10.42.0.42 tcp dpt:7800 + +=== 관찰 === + +25초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +50초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +75초 - | keycloak-0:1/1 keycloak-1:0/1 | 외부 200 + +100초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +125초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +150초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +175초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +200초 - | keycloak-0:1/1 keycloak-1:1/1 | 외부 200 diff --git a/docs/evidence/a5-asymmetric-partition/05-reconnect-observed.txt b/docs/evidence/a5-asymmetric-partition/05-reconnect-observed.txt new file mode 100644 index 0000000..db18390 --- /dev/null +++ b/docs/evidence/a5-asymmetric-partition/05-reconnect-observed.txt @@ -0,0 +1,19 @@ +=== 지금 연결 방향은? (차단은 → 10.42.0.42:7800 만) === + tcp 6 299 ESTABLISHED src=10.42.0.42 dst=10.42.1.77 sport=48473 dport=7800 src=10.42.1.77 dst=10.42.0.42 sport=7800 dport=48473 + tcp 6 86232 ESTABLISHED src=10.42.0.42 dst=10.42.1.77 sport=44205 dport=57800 src=10.42.1.77 dst=10.42.0.42 sport=57800 dport=44205 + +=== 차단 규칙 누적 카운터 === + Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) + pkts bytes target prot opt in out source destination + 19 1096 DROP 6 -- * * 0.0.0.0/0 10.42.0.42 tcp dpt:57800 + 21 3058 DROP 6 -- * * 0.0.0.0/0 10.42.0.42 tcp dpt:7800 + +=== 클러스터 멤버 (파드에서 직접) === + keycloak-0 cluster_size=2.0 + keycloak-1 cluster_size=2.0 + +=== 최근 뷰 로그 === + keycloak-0: [keycloak-0-24309(v=16.0.12)|12] (1) [keycloak-0-24309(v=16.0.12)] + + keycloak-1: [keycloak-1-45480(v=16.0.12)|12] (1) [keycloak-1-45480(v=16.0.12)] + diff --git a/docs/evidence/a5-asymmetric-partition/06-view-history-and-cleanup.txt b/docs/evidence/a5-asymmetric-partition/06-view-history-and-cleanup.txt new file mode 100644 index 0000000..170eafe --- /dev/null +++ b/docs/evidence/a5-asymmetric-partition/06-view-history-and-cleanup.txt @@ -0,0 +1,28 @@ +=== 뷰 변화 전체 이력 (차단 12:33:58 전후) === +--- keycloak-0 --- + 2026-09-04 03:33:49 | during merge: CacheTopology{id=37, phase=NO_REBALANCE, rebalanceId=11, currentCH=DefaultConsistentHash{ns=25 + 2026-09-04 03:33:49,398 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t1) [Context=offlineSessions] ISPN100007: After me + 2026-09-04 03:33:49 | during merge: CacheTopology{id=37, phase=NO_REBALANCE, rebalanceId=11, currentCH=DefaultConsistentHash{ns=25 + 2026-09-04 03:33:49,399 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t1) [Context=loginFailures] ISPN100007: After merg + 2026-09-04 03:33:49 | during merge: CacheTopology{id=37, phase=NO_REBALANCE, rebalanceId=11, currentCH=DefaultConsistentHash{ns=25 + 2026-09-04 03:33:49,402 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t1) [Context=actionTokens] ISPN100007: After merge +--- keycloak-1 --- + 2026-09-04 03:32:59,874 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t2) [Context=work] ISPN100007: After merge (or coo + 2026-09-04 03:32:59,879 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t2) [Context=clientSessions] ISPN100007: After mer + 2026-09-04 03:32:59,879 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t1) [Context=offlineSessions] ISPN100007: After me + 2026-09-04 03:32:59,883 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t1) [Context=loginFailures] ISPN100007: After merg + 2026-09-04 03:32:59,884 INFO [org.infinispan.CLUSTER] (non-blocking-thread--p2-t2) [Context=actionTokens] ISPN100007: After merge + 2026-09-04 03:33:49 | MergeView::[keycloak-0-24309(v=16.0.12)|13] (2) [keycloak-0-24309(v=16.0.12), keycloak-1-45480(v=16.0.12)], + +=== MERGE3 병합 이벤트 === + keycloak-0 merge_events=1.0 suspected=0.0 + keycloak-1 merge_events=1.0 suspected=0.0 + +=== 차단 해제 === +해제완료 + keycloak-0 cluster_size=2.0 +warning: couldn't attach to pod/a5-f, falling back to streaming logs: unable to upgrade connection: container a5-f not found in pod a5-f_keycloak-lab + keycloak-0 cluster_size=2.0 + keycloak-1 cluster_size=2.0 +keycloak-0 1/1 Running 0 22m +keycloak-1 1/1 Running 1 (13m ago) 166m diff --git a/docs/evidence/a5-asymmetric-partition/07-bidirectional-block.txt b/docs/evidence/a5-asymmetric-partition/07-bidirectional-block.txt new file mode 100644 index 0000000..88daf92 --- /dev/null +++ b/docs/evidence/a5-asymmetric-partition/07-bidirectional-block.txt @@ -0,0 +1,17 @@ +=== 양방향 차단 — 두 노드 모두에 raw DROP === + 주입: 12:40:25 + +25초 keycloak-0:1/1 keycloak-1:1/1 | ready=[10.42.0.42 10.42.1.77] 외부 200 + +50초 keycloak-0:1/1 keycloak-1:1/1 | ready=[10.42.0.42 10.42.1.77] 외부 200 + +75초 keycloak-0:1/1 keycloak-1:1/1 | ready=[10.42.0.42 10.42.1.77] 외부 200 + +100초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200 + +125초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200 + +150초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200 + +175초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200 + +200초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200 + +225초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200 + +=== 뷰 상태 === + keycloak-0 [keycloak-0-24309(v=16.0.12)|14] (1) [keycloak-0-24309(v=16.0.12)] + + keycloak-1 [keycloak-1-45480(v=16.0.12)|14] (1) [keycloak-1-45480(v=16.0.12)] + diff --git a/docs/evidence/a5-asymmetric-partition/08-coordinator-and-recovery.txt b/docs/evidence/a5-asymmetric-partition/08-coordinator-and-recovery.txt new file mode 100644 index 0000000..484261b --- /dev/null +++ b/docs/evidence/a5-asymmetric-partition/08-coordinator-and-recovery.txt @@ -0,0 +1,39 @@ +=== 왜 한쪽만 DOWN 인가 — 코디네이터 여부 확인 === + name | ip | coord +------------------+-----------------+------- + keycloak-0-24309 | 10.42.1.77:7800 | t + keycloak-1-45480 | 10.42.0.42:7800 | t +(2 rows) + +=== 양쪽 헬스체크 상세 === +--- keycloak-0 --- +{"status":"UP","checks":[{"name":"GracefulShutdown","status":"UP"} +{"name":"KeycloakInitialized","status":"UP"} +{"name":"Keycloakclusterhealthcheck","status":"UP"} + +--- keycloak-1 --- +{"status":"DOWN","checks":[{"name":"GracefulShutdown","status":"UP"} +{"name":"Keycloakdatabaseconnectionsasynchealthcheck","status":"UP"} +{"name":"KeycloakInitialized","status":"UP"} + +warning: couldn't attach to pod/a5-h, falling back to streaming logs: Internal error occurred: Internal error occurred: error attaching to container: container is in CONTAINER_EXITED state +--- keycloak-0 --- +{"status":"UP","checks":[{"name":"GracefulShutdown","status":"UP"} +{"name":"KeycloakInitialized","status":"UP"} +{"name":"Keycloakclusterhealthcheck","status":"UP"} + +--- keycloak-1 --- +{"status":"DOWN","checks":[{"name":"GracefulShutdown","status":"UP"} +{"name":"Keycloakdatabaseconnectionsasynchealthcheck","status":"UP"} +{"name":"KeycloakInitialized","status":"UP"} + + +=== 차단 해제 === + 해제: 12:44:37 + +25초 keycloak-0:1/1 keycloak-1:0/1 + +50초 keycloak-0:1/1 keycloak-1:1/1 + → 복구 완료 + keycloak-0 MergeView::[keycloak-0-24309(v=16.0.12)|15] (2) [keycloak-0-24309(v=16.0.12), keycloak-1-45480( + + keycloak-1 MergeView::[keycloak-0-24309(v=16.0.12)|15] (2) [keycloak-0-24309(v=16.0.12), keycloak-1-45480( + diff --git a/docs/evidence/a5-asymmetric-partition/README.md b/docs/evidence/a5-asymmetric-partition/README.md new file mode 100644 index 0000000..73c8bea --- /dev/null +++ b/docs/evidence/a5-asymmetric-partition/README.md @@ -0,0 +1,23 @@ +# A-5 — 비대칭 파티션 증거 + +2026-09-04 12:28–12:46 KST +해설: [`docs/experiment-a5-asymmetric-partition.md`](../../experiment-a5-asymmetric-partition.md) + +| 파일 | 무엇을 보여주는가 | +|---|---| +| `01-injection.txt` | 첫 시도 — `iptables -I FORWARD 1` | +| `02-injection-verify.txt` | **실패 확인** — kube-router 가 자기 체인을 위로 재삽입, DROP 규칙 **0 패킷** | +| `03-raw-table-injection.txt` | `raw` 테이블로 이동. 그래도 0 패킷 | +| `04-correct-direction.txt` | **연결 방향이 반대였다.** 수신측 노드로 옮기니 **19/21 패킷 차단** | +| `05-reconnect-observed.txt` | **핵심** — 연결이 **열린 방향으로 뒤집혀 재연결**. 뷰 변화 없음, `suspected=0` | +| `06-view-history-and-cleanup.txt` | 뷰 ID 이력. 주입 이후 변화 없음 확인 | +| `07-bidirectional-block.txt` | 양방향 차단 → **뷰 14, 멤버 1개씩.** 그런데 `ready=[10.42.1.77]` 로 **한쪽은 남고 외부 200** | +| `08-coordinator-and-recovery.txt` | **`coord=t` 가 둘**(split brain). keycloak-0 `UP` / keycloak-1 `DOWN`. 해제 후 50초, `MergeView |15] (2)` | +| `a5-cluster-size-bidirectional-block.png` | Grafana — `vendor_cluster_size` 가 갈라졌다 합쳐지는 구간 | + +## 핵심 네 줄 + +1. **주입을 세 번 실패했다** — CNI 체인 경쟁, 연결 방향 오판, conntrack. 셋 다 "아무 일 없음"으로 보였다. +2. **단방향 차단으로는 못 가른다.** JGroups 가 열린 방향으로 재연결한다 (자가 치유). +3. **양방향이면 갈라진다** — `coord=t` 가 둘. +4. **그래도 전면 장애가 아니다.** 코디네이터 쪽만 UP 을 유지해 서비스가 계속된다 — A-1 의 열린 질문에 대한 답. diff --git a/docs/evidence/a5-asymmetric-partition/a5-cluster-size-bidirectional-block.png b/docs/evidence/a5-asymmetric-partition/a5-cluster-size-bidirectional-block.png new file mode 100644 index 0000000..e9c6c87 Binary files /dev/null and b/docs/evidence/a5-asymmetric-partition/a5-cluster-size-bidirectional-block.png differ diff --git a/docs/experiment-a5-asymmetric-partition.md b/docs/experiment-a5-asymmetric-partition.md new file mode 100644 index 0000000..6420ece --- /dev/null +++ b/docs/experiment-a5-asymmetric-partition.md @@ -0,0 +1,319 @@ +# A-5 — 한쪽 방향만 끊기면 어떻게 되는가 (split brain) + +브랜치 `feature/keycloak-a5-asymmetric-partition` · +증거 [`docs/evidence/a5-asymmetric-partition/`](evidence/a5-asymmetric-partition/) · +2026-09-04 12:28–12:46 KST + +선행: [`A-1`](experiment-a1-jgroups-transport-block.md) — 이 실험은 A-1 이 +열어둔 질문에 답한다. + +> **A-1 의 열린 질문** — *"양쪽이 동시에 NotReady 가 되는 경로가 있다면 +> 전면 장애다. A-5 에서 이어서 본다."* + +--- + +## 0. 결론부터 + +| 물음 | 답 | +|---|---| +| 비대칭(한 방향) 차단이 클러스터를 가르는가 | **아니다.** 열린 방향으로 재연결한다 | +| 양방향 완전 차단은 가르는가 | **그렇다.** 양쪽 모두 멤버 1개 | +| **그때 양쪽이 다 NotReady 가 되는가** | **아니다. 한쪽만 DOWN 이다** | +| 그래서 서비스는 | **계속된다.** 외부 200 유지 | + +**전면 장애 경로는 없었다.** 코디네이터 쪽이 항상 살아남는다. + +그리고 주입을 **세 번 실패**했다. 세 번 모두 다른 이유였고, 셋 다 +**"아무 일도 없었다"로 보였다.** + +--- + +## 1. 주입을 세 번 실패했다 + +A-1 에서 NetworkPolicy 가 conntrack 의 `ESTABLISHED` 승인에 막혀 기존 연결을 +못 끊는다는 것을 배웠다. 그래서 이번에는 iptables 로 갔는데, **또 다른 벽이 +세 개** 있었다. + +### 실패 ① — `iptables -I FORWARD 1` 은 최상단에 남지 않는다 + +```bash +ssh kc-lab-2 'sudo iptables -I FORWARD 1 -p tcp -d 10.42.1.77 --dport 7800 -j DROP' +``` + +``` +num pkts bytes target +1 232 377K KUBE-ROUTER-FORWARD /* kube-router netpol */ ← 다시 1번이 되었다 +2 0 0 DROP tcp dpt:57800 +3 0 0 DROP tcp dpt:7800 ← 0 패킷 +``` + +**kube-router 가 주기적으로 자기 체인을 최상단에 다시 삽입한다.** +내가 1번에 넣어도 곧 2번, 3번으로 밀려난다. + +> **직접 넣은 iptables 규칙은 CNI 가 관리하는 체인과 경쟁한다.** +> 넣는 것으로 끝이 아니라 **패킷 카운터로 확인해야 한다.** + +### 실패 ② — 그래도 0 패킷: **연결 방향을 잘못 짚었다** + +`raw` 테이블로 옮겼는데도 0 패킷이었다. + +```bash +ssh kc-lab-2 'sudo iptables -t raw -I PREROUTING 1 -p tcp -d 10.42.1.77 --dport 7800 -j DROP' +``` + +실제 연결을 봤더니 + +``` +ESTABLISHED src=10.42.1.77 dst=10.42.0.42 sport=60485 dport=7800 + ──────────── ──────────────────── + keycloak-0 가 클라이언트 keycloak-1 이 서버 +``` + +**A-1 때와 방향이 반대였다.** 파드가 재시작되면서 누가 먼저 연결을 걸었는지가 +바뀌었다. + +> **JGroups 의 TCP 연결 방향은 고정이 아니다.** 먼저 뜬 쪽, 먼저 JOIN 을 건 +> 쪽에 따라 달라진다. **가정하지 말고 `conntrack -L` 로 봐야 한다.** + +### 성공 — `raw` 테이블 PREROUTING, 수신측 노드에 + +```bash +ssh kc-lab-1 'sudo iptables -t raw -I PREROUTING 1 -p tcp -d 10.42.0.42 --dport 7800 -j DROP + sudo iptables -t raw -I PREROUTING 1 -p tcp -d 10.42.0.42 --dport 57800 -j DROP' +``` + +``` + pkts bytes target + 19 1096 DROP tcp dpt:57800 + 21 3058 DROP tcp dpt:7800 ← 걸린다 +``` + +### 개념 — netfilter 처리 순서 + +``` + 패킷 도착 + │ + ├─▶ raw PREROUTING ← conntrack 보다 먼저. NOTRACK·DROP 용 + │ + ├─▶ conntrack 조회/생성 ← 여기서 ESTABLISHED 가 결정된다 + │ + ├─▶ mangle PREROUTING + ├─▶ nat PREROUTING + ├─▶ filter FORWARD ← NetworkPolicy·kube-router 가 여기 있다 + └─▶ 목적지 파드 +``` + +| 어디에 넣는가 | 기존 연결을 끊는가 | CNI 와 경쟁하는가 | +|---|---|---| +| NetworkPolicy (filter) | **못 끊는다** — conntrack 이 먼저 통과시킨다 | 없음 | +| filter FORWARD 직접 | 순서에 따라 | **경쟁한다** (kube-router 가 밀어낸다) | +| **raw PREROUTING** | **끊는다** | **없다** — CNI 가 안 쓰는 테이블 | + +**진짜 네트워크 분단을 흉내내려면 `raw` 테이블이 맞다.** + +--- + +## 2. 비대칭 차단 — 클러스터가 갈라지지 않았다 + +한 방향(`→ keycloak-1:7800`)만 막고 관찰했다. + +``` + +25초 keycloak-0:1/1 keycloak-1:1/1 | 외부 200 + +75초 keycloak-0:1/1 keycloak-1:0/1 | 외부 200 ← 잠깐 흔들림 + +100초 keycloak-0:1/1 keycloak-1:1/1 | 외부 200 ← 회복 + ... + +200초 keycloak-0:1/1 keycloak-1:1/1 | 외부 200 +``` + +**주입 이후 뷰 변화가 하나도 없었다.** + +```bash +kubectl -n keycloak-lab logs keycloak-0 --since=20m | grep ISPN000094 | awk '$2 >= "03:33:58"' +# (아무것도 안 나온다) +``` + +``` + 뷰 ID 이력: |13] (2) ← 주입 전부터 지금까지 그대로 + suspected = 0 +``` + +### 왜 안 갈라졌는가 — **연결 방향이 뒤집혔다** + +``` +차단 전: keycloak-0 → keycloak-1:7800 ← 내가 막은 방향 +차단 후: keycloak-1 → keycloak-0:7800 ← 열린 방향으로 다시 붙었다 +``` + +``` +ESTABLISHED src=10.42.0.42 dst=10.42.1.77 sport=48473 dport=7800 + ──────────── ───────────── + keycloak-1 이 클라이언트 keycloak-0 이 서버 +``` + +**JGroups 는 막힌 연결이 죽자 반대 방향으로 새로 연결했고, 그 사이 FD_SOCK2 가 +상대를 의심하기 전에 복구가 끝났다.** `suspected = 0` 이 그 증거다. + +> **한 방향만 막는 것으로는 JGroups 를 가를 수 없다.** +> 두 노드는 서로에게 연결을 걸 수 있으므로, **한쪽 길이 막히면 다른 길로 간다.** +> +> 운영적으로는 좋은 소식이다 — 단방향 방화벽 오설정은 **자가 치유**된다. +> 반대로 **분단을 재현하려는 실험자에게는 함정**이다. + +--- + +## 3. 양방향 차단 — 갈라졌지만 전면 장애는 아니다 + +양쪽 노드 모두에 규칙을 넣었다. + +``` + +75초 keycloak-0:1/1 keycloak-1:1/1 | ready=[10.42.0.42 10.42.1.77] 외부 200 + +100초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200 + ... + +225초 keycloak-0:1/1 keycloak-1:0/1 | ready=[10.42.1.77] 외부 200 +``` + +``` +=== 뷰 === + keycloak-0 [keycloak-0-24309|14] (1) [keycloak-0-24309] + keycloak-1 [keycloak-1-45480|14] (1) [keycloak-1-45480] + +=== JGROUPS_PING === + keycloak-0-24309 | 10.42.1.77:7800 | t ← 코디네이터 + keycloak-1-45480 | 10.42.0.42:7800 | t ← 코디네이터 +``` + +**`coord = t` 가 둘.** 완전한 split brain 이다. + +![cluster_size 가 갈라졌다 합쳐진다](evidence/a5-asymmetric-partition/a5-cluster-size-bidirectional-block.png) + +### 그런데 **한쪽만 DOWN 이다** + +``` +--- keycloak-0 --- +{"status":"UP", ... {"name":"Keycloak cluster health check","status":"UP"} + +--- keycloak-1 --- +{"status":"DOWN", ... (cluster health check 가 DOWN) +``` + +| | keycloak-0 | keycloak-1 | +|---|---|---| +| 분단 전 역할 | **코디네이터** (뷰 13 의 발행자) | 일반 멤버 | +| 분단 후 자기 인식 | "멤버가 하나 나갔다" — **정상 사건** | "코디네이터를 잃었다" — **비정상** | +| 헬스체크 | **UP** | **DOWN** | +| Service 엔드포인트 | **남는다** | 빠진다 | + +> **A-1 의 열린 질문에 대한 답 — 양쪽이 동시에 NotReady 가 되는 경로는 없었다.** +> +> Keycloak 의 클러스터 헬스체크는 **비대칭**이다. 코디네이터였던 쪽은 +> 자기가 정상이라고 보고, 잃은 쪽만 DOWN 이 된다. 그래서 완전 분단조차 +> **용량 저하로 끝나고 전면 장애가 되지 않는다.** +> +> A-2(DB 상실)에서 양쪽이 동시에 DOWN 이 된 것과 대조된다. **DB 는 모두가 +> 의존하는 하나지만, 클러스터 멤버십은 서로 상대적**이기 때문이다. + +--- + +## 4. 복구 + +```bash +ssh kc-lab-1 'sudo iptables -t raw -F PREROUTING' +ssh kc-lab-2 'sudo iptables -t raw -F PREROUTING' +``` + +``` + +25초 keycloak-0:1/1 keycloak-1:0/1 + +50초 keycloak-0:1/1 keycloak-1:1/1 + → 복구 완료 + +MergeView::[keycloak-0-24309|15] (2) [keycloak-0-24309, keycloak-1-45480] +``` + +**50초, 사람 개입 없음.** `MergeView` 로 뷰 15 가 발행되며 병합됐다. + +각 캐시마다 재분배 로그가 남는다. + +``` +[Context=work] ISPN100007: After merge (or coordinator change) ... +[Context=clientSessions] ISPN100007: After merge ... +[Context=offlineSessions] ISPN100007: After merge ... +[Context=loginFailures] ISPN100007: After merge ... +[Context=actionTokens] ISPN100007: After merge ... +``` + +**`ISPN100007` 은 병합 후 캐시별 토폴로지 재계산**이다. 캐시가 여럿이므로 +로그도 캐시 수만큼 나온다. + +--- + +## 5. 개념 + +### `MergeView` 와 뷰 ID + +``` +[keycloak-0-24309|13] (2) ← 정상 +[keycloak-0-24309|14] (1) ← 분단. 양쪽이 각자 14 를 발행 +MergeView::[...|15] (2) ← 병합. 뷰 ID 는 계속 증가한다 +``` + +**뷰 ID 는 단조 증가**하므로 "언제 몇 번 갈라졌는지"를 로그만으로 셀 수 있다. + +### 코디네이터의 비대칭성 + +JGroups 코디네이터는 **가장 오래된 멤버**다. 분단이 나면 + +``` + 코디네이터 쪽: "멤버가 나갔다" → 정상 처리, 계속 코디네이터 + 나머지 쪽: "코디네이터가 사라졌다" → 새 코디네이터를 자기로 선출 +``` + +**둘 다 자기가 코디네이터라고 믿는 상태**가 split brain 이고, +`JGROUPS_PING.coord` 컬럼에 그대로 드러난다. + +### 실험자를 위한 규칙 + +| 상황 | 확인 방법 | +|---|---| +| 규칙을 넣었는데 안 걸린다 | `iptables -L -n -v` 의 **패킷 카운터** | +| 방향을 모르겠다 | `conntrack -L \| grep <포트>` | +| CNI 가 밀어낸다 | `raw` 테이블을 쓴다 | +| 갈라졌는지 알고 싶다 | `vendor_cluster_size`, `ISPN000094`, `JGROUPS_PING.coord` | + +--- + +## 6. 재현 절차 (명령어) + +```bash +# 1. 지금 연결이 어느 방향인지 먼저 본다 — 가정하면 실패한다 +ssh kc-lab-1 'sudo conntrack -L | grep 7800' + +# 2. 수신측 노드의 raw PREROUTING 에 넣는다 (filter 는 CNI 와 경쟁한다) +ssh kc-lab-1 'sudo iptables -t raw -I PREROUTING 1 -p tcp -d <수신 파드IP> --dport 7800 -j DROP' +ssh kc-lab-1 'sudo iptables -t raw -I PREROUTING 1 -p tcp -d <수신 파드IP> --dport 57800 -j DROP' + +# 3. 걸렸는지 카운터로 확인 — 0 이면 해석 금지 +ssh kc-lab-1 'sudo iptables -t raw -L PREROUTING -n -v' + +# 4. 양방향으로 하려면 반대 노드에도 (한 방향만으로는 자가 치유된다) +ssh kc-lab-2 'sudo iptables -t raw -I PREROUTING 1 -p tcp -d <반대 파드IP> --dport 7800 -j DROP' + +# 5. 분단 확인 +kubectl -n keycloak-lab exec deploy/postgres -- psql -U keycloak -d keycloak \ + -c "select name, coord from jgroups_ping" # coord=t 가 둘이면 split brain +kubectl -n keycloak-lab get endpoints keycloak -o jsonpath='{.subsets[*].addresses[*].ip}' + +# 6. 해제 +ssh kc-lab-1 'sudo iptables -t raw -F PREROUTING' +ssh kc-lab-2 'sudo iptables -t raw -F PREROUTING' +``` + +--- + +## 7. 다음 실험에 남기는 것 + +| 실험 | 이 실험이 준 것 | +|---|---| +| **A-6** 지연 주입 | `tc netem` 도 같은 함정 — **주입이 걸렸는지 먼저 확인** | +| **A-7** volatile 비교 | 여기서는 분단에도 서비스가 계속됐다. volatile 이면 **세션이 갈라진다** | +| 운영 | **단방향 방화벽 오설정은 자가 치유된다.** 양방향이어야 사고가 된다 | +| 운영 | **완전 분단조차 전면 장애가 아니다** — 코디네이터 쪽이 살아남는다 |