Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 DongHyeonkaandClaude Opus 5
|
d0666c5ba0 |
@@ -53,3 +53,77 @@
|
||||
[ 502295ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 511625ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 524619ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 533050ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 553189ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 571699ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 575413ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 588005ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 599161ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 608816ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 619031ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 626612ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 639003ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 655077ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 674532ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 677197ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 696027ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 709512ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 727505ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 747139ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 754565ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 773682ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 784674ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 787399ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 802845ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 825589ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 832768ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 842043ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 852717ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 871748ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 876571ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 894491ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 895515ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 904310ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 916681ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 934752ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 941175ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 946004ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 960849ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 962180ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 984446ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 997579ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1010105ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1027190ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1043388ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1061311ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1066040ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1080349ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1092336ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1104312ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1109362ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1127047ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1134728ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1154899ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1161253ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1174971ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1184491ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1185340ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1189111ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1196984ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1204794ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1247961ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1286322ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1319329ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1362771ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1396881ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1434184ms] [ERROR] Failed to load resource: the server responded with a status of 502 () @ https://app2.hyeonworks.com/api/user/auth-tokens/rotate:0
|
||||
[ 1443506ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1465016ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 502 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1513337ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1556906ms] [WARNING] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: WebSocket is closed before the connection is established. @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1562963ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 502 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1572896ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 502 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1576390ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 503 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1593794ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1600651ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 1616424ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: HTTP Authentication failed; no valid credentials available @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
[ 268ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/login:0
|
||||
[ 379ms] [VERBOSE] [DOM] Input elements should have autocomplete attributes (suggested: "username"): (More info: https://goo.gl/9p2vKq) %o @ https://app2.hyeonworks.com/login:0
|
||||
[ 10453ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 10532ms] [WARNING] <meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes"> @ https://app2.hyeonworks.com/explore?schemaVersion=1&panes=%7B%22ich%22%3A%7B%22datasource%22%3A%22PBFA97CFB590B2093%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22up%7Bjob%3D%7E%5C%22keycloak%7Cnode-exporter%5C%22%7D%22%2C%22range%22%3Atrue%2C%22instant%22%3Afalse%2C%22editorMode%22%3A%22code%22%2C%22legendFormat%22%3A%22%7B%7Bjob%7D%7D+%E2%80%94+%7B%7Bpod%7D%7D%7B%7Bnode%7D%7D%22%2C%22datasource%22%3A%7B%22type%22%3A%22prometheus%22%2C%22uid%22%3A%22PBFA97CFB590B2093%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-55m%22%2C%22to%22%3A%22now%22%7D%7D%7D&orgId=1:0
|
||||
[ 11687ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 13737ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 15113ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 20185ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 27259ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 40154ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 41870ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 50493ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 54082ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 60121ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 74362ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 83269ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 95349ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 98008ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 104458ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
[ 123816ms] [ERROR] WebSocket connection to 'wss://app2.hyeonworks.com/api/live/ws' failed: Error during WebSocket handshake: Unexpected response code: 400 @ https://app2.hyeonworks.com/public/build/1518.a3f1f690c084a37f01c7.js:362
|
||||
@@ -0,0 +1,44 @@
|
||||
- main [ref=f6e7]:
|
||||
- generic [ref=f6e9]:
|
||||
- generic [ref=f6e11]:
|
||||
- generic [ref=f6e12]:
|
||||
- img "Grafana" [ref=f6e13]
|
||||
- heading "Welcome to Grafana" [level=1] [ref=f6e15]
|
||||
- generic [ref=f6e19]:
|
||||
- generic [ref=f6e20]:
|
||||
- generic [ref=f6e21]: Email or username
|
||||
- textbox "Email or username" [active] [ref=f6e28]:
|
||||
- /placeholder: email or username
|
||||
- generic [ref=f6e29]:
|
||||
- generic [ref=f6e30]: Password
|
||||
- generic [ref=f6e36]:
|
||||
- textbox "Password" [ref=f6e37]:
|
||||
- /placeholder: password
|
||||
- switch "Show password" [ref=f6e39] [cursor=pointer]
|
||||
- button "Log in" [ref=f6e42] [cursor=pointer]
|
||||
- link "Forgot your password?" [ref=f6e45] [cursor=pointer]:
|
||||
- /url: /user/password/send-reset-email
|
||||
- list [ref=f6e49]:
|
||||
- listitem [ref=f6e50]:
|
||||
- link "Documentation" [ref=f6e53] [cursor=pointer]:
|
||||
- /url: https://grafana.com/docs/grafana/latest/?utm_source=grafana_footer
|
||||
- text: "|"
|
||||
- listitem [ref=f6e54]:
|
||||
- link "Support" [ref=f6e57] [cursor=pointer]:
|
||||
- /url: https://grafana.com/products/enterprise/?utm_source=grafana_footer
|
||||
- text: "|"
|
||||
- listitem [ref=f6e58]:
|
||||
- link "Community" [ref=f6e61] [cursor=pointer]:
|
||||
- /url: https://community.grafana.com/?utm_source=grafana_footer
|
||||
- text: "|"
|
||||
- listitem [ref=f6e62]:
|
||||
- link "Open Source" [ref=f6e63] [cursor=pointer]:
|
||||
- /url: https://grafana.com/oss/grafana?utm_source=grafana_footer
|
||||
- text: "|"
|
||||
- listitem [ref=f6e64]:
|
||||
- link "Grafana v11.4.0 (b58701869e)" [ref=f6e65] [cursor=pointer]:
|
||||
- /url: https://github.com/grafana/grafana/blob/main/CHANGELOG.md
|
||||
- text: "|"
|
||||
- listitem [ref=f6e66]:
|
||||
- link "New version available!" [ref=f6e69] [cursor=pointer]:
|
||||
- /url: https://grafana.com/grafana/download?utm_source=grafana_footer
|
||||
@@ -0,0 +1,159 @@
|
||||
- generic [active] [ref=f9e1]:
|
||||
- generic [ref=f9e4]:
|
||||
- link "Skip to main content" [ref=f9e5] [cursor=pointer]:
|
||||
- /url: "#pageContent"
|
||||
- banner [ref=f9e7]:
|
||||
- generic [ref=f9e8]:
|
||||
- link [ref=f9e10] [cursor=pointer]:
|
||||
- /url: /
|
||||
- img "Grafana" [ref=f9e11]
|
||||
- generic [ref=f9e14]:
|
||||
- button "Search or jump to..." [ref=f9e18] [cursor=pointer]
|
||||
- generic [ref=f9e19]: ctrl+k
|
||||
- generic [ref=f9e23]:
|
||||
- button "New" [ref=f9e24] [cursor=pointer]
|
||||
- button "Help" [ref=f9e30] [cursor=pointer]
|
||||
- button "News" [ref=f9e33] [cursor=pointer]
|
||||
- button "Profile" [ref=f9e36] [cursor=pointer]:
|
||||
- img "User avatar" [ref=f9e37]
|
||||
- generic [ref=f9e38]:
|
||||
- button "Open menu" [ref=f9e40] [cursor=pointer]
|
||||
- navigation "Breadcrumbs" [ref=f9e43]:
|
||||
- list [ref=f9e44]:
|
||||
- listitem [ref=f9e45]:
|
||||
- link "Home" [ref=f9e46] [cursor=pointer]:
|
||||
- /url: /
|
||||
- listitem [ref=f9e50]:
|
||||
- link "Explore" [ref=f9e51] [cursor=pointer]:
|
||||
- /url: /explore
|
||||
- listitem [ref=f9e55]:
|
||||
- generic "Prometheus" [ref=f9e56]
|
||||
- generic [ref=f9e57]:
|
||||
- generic [ref=f9e60]:
|
||||
- button "Copy shortened URL" [ref=f9e61] [cursor=pointer]
|
||||
- button "Open copy link options" [ref=f9e64] [cursor=pointer]
|
||||
- button "Toggle top search bar" [ref=f9e68] [cursor=pointer]
|
||||
- main [ref=f9e74]:
|
||||
- generic [ref=f9e76]:
|
||||
- heading "Explore" [level=1] [ref=f9e77]
|
||||
- generic [ref=f9e82]:
|
||||
- navigation "Explore toolbar" [ref=f9e84]:
|
||||
- navigation "Search links" [ref=f9e86]:
|
||||
- generic [ref=f9e87]:
|
||||
- button "Content outline" [expanded] [ref=f9e89] [cursor=pointer]:
|
||||
- generic [ref=f9e92]: Outline
|
||||
- generic [ref=f9e97] [cursor=pointer]:
|
||||
- img "Prometheus logo" [ref=f9e99]
|
||||
- textbox "Select a data source" [ref=f9e100]:
|
||||
- /placeholder: ""
|
||||
- generic [ref=f9e104]:
|
||||
- button "Split the pane" [ref=f9e106] [cursor=pointer]:
|
||||
- generic [ref=f9e109]: Split
|
||||
- button "Add" [ref=f9e111] [cursor=pointer]
|
||||
- generic [ref=f9e116]:
|
||||
- 'button "Time range selected: Last 55 minutes" [ref=f9e117] [cursor=pointer]'
|
||||
- button "Zoom out time range" [ref=f9e122] [cursor=pointer]
|
||||
- generic [ref=f9e126]:
|
||||
- button "Run query" [ref=f9e127] [cursor=pointer]
|
||||
- button "Auto refresh turned off. Choose refresh time interval" [ref=f9e131] [cursor=pointer]
|
||||
- generic [ref=f9e135]:
|
||||
- generic [ref=f9e139]:
|
||||
- button "Collapse outline" [expanded] [ref=f9e141] [cursor=pointer]:
|
||||
- img "arrow-from-right" [ref=f9e142]
|
||||
- button "Queries" [ref=f9e145] [cursor=pointer]:
|
||||
- img "arrow" [ref=f9e146]
|
||||
- button "Graph" [ref=f9e150] [cursor=pointer]:
|
||||
- img "graph-bar" [ref=f9e151]
|
||||
- generic [ref=f9e158]:
|
||||
- generic [ref=f9e160]:
|
||||
- generic "Query editor row" [ref=f9e163]:
|
||||
- generic [ref=f9e164]:
|
||||
- generic [ref=f9e166]:
|
||||
- generic [ref=f9e167]:
|
||||
- button "Collapse query row" [expanded] [ref=f9e168] [cursor=pointer]
|
||||
- generic [ref=f9e171]:
|
||||
- button "Query editor row title A" [ref=f9e172] [cursor=pointer]:
|
||||
- generic [ref=f9e173]: A
|
||||
- emphasis [ref=f9e174]: (Prometheus)
|
||||
- generic [ref=f9e175]:
|
||||
- button "Show data source help" [ref=f9e177] [cursor=pointer]
|
||||
- button "Duplicate query" [ref=f9e181] [cursor=pointer]
|
||||
- button "Hide response" [ref=f9e185] [cursor=pointer]
|
||||
- button "Remove query" [ref=f9e189] [cursor=pointer]
|
||||
- button "Drag and drop to reorder" [ref=f9e192]:
|
||||
- img "Drag and drop to reorder" [ref=f9e193]
|
||||
- generic [ref=f9e196]:
|
||||
- generic [ref=f9e197]:
|
||||
- button "Kick start your query" [ref=f9e198] [cursor=pointer]
|
||||
- generic [ref=f9e201]:
|
||||
- generic [ref=f9e202] [cursor=pointer]: Explain
|
||||
- generic [ref=f9e203]:
|
||||
- checkbox "Explain Toggle switch" [ref=f9e204]
|
||||
- generic "Toggle switch" [ref=f9e205] [cursor=pointer]
|
||||
- radiogroup [ref=f9e210]:
|
||||
- generic [ref=f9e211]:
|
||||
- radio "Builder" [ref=f9e212] [cursor=pointer]
|
||||
- generic [ref=f9e213] [cursor=pointer]: Builder
|
||||
- generic [ref=f9e214]:
|
||||
- radio "Code" [checked] [ref=f9e215] [cursor=pointer]
|
||||
- generic [ref=f9e216] [cursor=pointer]: Code
|
||||
- generic [ref=f9e218]:
|
||||
- generic [ref=f9e220]:
|
||||
- button "Metrics browser" [ref=f9e221] [cursor=pointer]
|
||||
- code [ref=f9e228]:
|
||||
- generic [ref=f9e229]:
|
||||
- generic [ref=f9e234]: "up{job=~\"keycloak|node-exporter\"}"
|
||||
- textbox "Editor content;Press Alt+F1 for Accessibility Options." [ref=f9e239]: "up{job=~\"keycloak|node-exporter\"}"
|
||||
- 'button "Options Legend: {{job}} — {{pod}}{{node}} Format: Time series Step: auto Type: Range Exemplars: false" [ref=f9e245] [cursor=pointer]':
|
||||
- generic [ref=f9e249]:
|
||||
- heading "Options" [level=6] [ref=f9e250]
|
||||
- generic [ref=f9e251]:
|
||||
- generic [ref=f9e252]: "Legend: {{job}} — {{pod}}{{node}}"
|
||||
- generic [ref=f9e253]: "Format: Time series"
|
||||
- generic [ref=f9e254]: "Step: auto"
|
||||
- generic [ref=f9e255]: "Type: Range"
|
||||
- generic [ref=f9e256]: "Exemplars: false"
|
||||
- generic [ref=f9e257]:
|
||||
- button "Add query" [ref=f9e258] [cursor=pointer]
|
||||
- button "Query history" [ref=f9e262] [cursor=pointer]
|
||||
- button "Query inspector" [ref=f9e266] [cursor=pointer]
|
||||
- main [ref=f9e270]:
|
||||
- region [ref=f9e272]:
|
||||
- generic [ref=f9e273]:
|
||||
- heading "Graph" [level=2] [ref=f9e275]
|
||||
- radiogroup [ref=f9e278]:
|
||||
- generic [ref=f9e279]:
|
||||
- radio "Lines" [checked] [ref=f9e280] [cursor=pointer]
|
||||
- generic [ref=f9e281] [cursor=pointer]: Lines
|
||||
- generic [ref=f9e282]:
|
||||
- radio "Bars" [ref=f9e283] [cursor=pointer]
|
||||
- generic [ref=f9e284] [cursor=pointer]: Bars
|
||||
- generic [ref=f9e285]:
|
||||
- radio "Points" [ref=f9e286] [cursor=pointer]
|
||||
- generic [ref=f9e287] [cursor=pointer]: Points
|
||||
- generic [ref=f9e288]:
|
||||
- radio "Stacked lines" [ref=f9e289] [cursor=pointer]
|
||||
- generic [ref=f9e290] [cursor=pointer]: Stacked lines
|
||||
- generic [ref=f9e291]:
|
||||
- radio "Stacked bars" [ref=f9e292] [cursor=pointer]
|
||||
- generic [ref=f9e293] [cursor=pointer]: Stacked bars
|
||||
- list [ref=f9e311]:
|
||||
- listitem [ref=f9e312]:
|
||||
- button "keycloak — keycloak-1kc-lab-1" [ref=f9e316] [cursor=pointer]
|
||||
- listitem [ref=f9e317]:
|
||||
- button "keycloak — keycloak-1kc-lab-1" [ref=f9e321] [cursor=pointer]
|
||||
- listitem [ref=f9e322]:
|
||||
- button "keycloak — keycloak-0kc-lab-2" [ref=f9e326] [cursor=pointer]
|
||||
- listitem [ref=f9e327]:
|
||||
- button "keycloak — keycloak-0kc-lab-2" [ref=f9e331] [cursor=pointer]
|
||||
- listitem [ref=f9e332]:
|
||||
- button "keycloak — keycloak-0kc-lab-2" [ref=f9e336] [cursor=pointer]
|
||||
- listitem [ref=f9e337]:
|
||||
- button "node-exporter — kc-lab-1" [ref=f9e341] [cursor=pointer]
|
||||
- listitem [ref=f9e342]:
|
||||
- button "node-exporter — kc-lab-2" [ref=f9e346] [cursor=pointer]
|
||||
- generic [ref=f9e351]:
|
||||
- alert
|
||||
- alert
|
||||
- complementary
|
||||
- complementary
|
||||
@@ -0,0 +1,20 @@
|
||||
=== A-4 기준선 ===
|
||||
kc-lab-1 Ready true
|
||||
kc-lab-2 Ready <none>
|
||||
|
||||
a2-probe true kc-lab-2
|
||||
keycloak-0 true kc-lab-2
|
||||
keycloak-1 true kc-lab-1
|
||||
postgres-7b474b88c8-2gf27 true kc-lab-2
|
||||
|
||||
=== PVC 가 어느 노드에 묶여 있는가 (재배치 가능성) ===
|
||||
persistentvolumeclaim/postgres-data → kc-lab-2
|
||||
|
||||
=== 서비스 정상 확인 ===
|
||||
https://auth.hyeonworks.com/realms/master HTTP 200
|
||||
|
||||
=== VM 상태 ===
|
||||
--------------------------
|
||||
1 kc-lab-1 running
|
||||
2 kc-lab-2 running
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
=== 워커 노드(kc-lab-2) 전원 차단 — virsh destroy 는 종료 신호가 없다 ===
|
||||
차단 시각: 12:07:43
|
||||
Domain 'kc-lab-2' destroyed
|
||||
|
||||
|
||||
+15초 node=Ready | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 000
|
||||
+30초 node=Ready | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 000
|
||||
+45초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
|
||||
+60초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
|
||||
+75초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
|
||||
+90초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
|
||||
+105초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
|
||||
+120초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
|
||||
+135초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
|
||||
+150초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
|
||||
+165초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
|
||||
+180초 node=NotReady | keycloak-0=Running postgres-7b474b88c8-2gf27=Running | 외부 HTTP 503
|
||||
@@ -0,0 +1,28 @@
|
||||
=== 파드 상태의 진실 — Running 인데 노드가 없다 ===
|
||||
a2-probe Running true kc-lab-2 <none>
|
||||
keycloak-0 Running true kc-lab-2 <none>
|
||||
keycloak-1 Running false kc-lab-1 <none>
|
||||
postgres-7b474b88c8-2gf27 Running true kc-lab-2 <none>
|
||||
|
||||
=== 재배치가 시도되었는가 ===
|
||||
10m Warning Unhealthy pod/keycloak-0 Readiness probe failed: Get "http://10.42.1.67:9000/health/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
|
||||
3m15s Warning NodeNotReady pod/postgres-7b474b88c8-2gf27 Node is not ready
|
||||
3m15s Warning NodeNotReady pod/keycloak-0 Node is not ready
|
||||
3m15s Warning NodeNotReady pod/a2-probe Node is not ready
|
||||
2m27s Warning Unhealthy pod/keycloak-1 Readiness probe failed: Get "http://10.42.0.35:9000/health/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
|
||||
2s Warning Unhealthy pod/keycloak-1 Readiness probe failed: HTTP probe failed with statuscode: 503
|
||||
|
||||
=== 노드 taint — 쿠버네티스가 붙인 것 ===
|
||||
node.kubernetes.io/unreachable=:NoSchedule
|
||||
node.kubernetes.io/unreachable=:NoExecute
|
||||
|
||||
=== Prometheus 가 본 것 (kc-lab-1 에 있어 살아남았다) ===
|
||||
up{job=keycloak pod=keycloak-1 } = 1
|
||||
up{job=keycloak pod=keycloak-0 } = 0
|
||||
up{job=kubelet pod=- } = 1
|
||||
up{job=kubelet pod=- } = 0
|
||||
up{job=node-exporter pod=kc-lab-1 } = 1
|
||||
up{job=node-exporter pod=kc-lab-2 } = 0
|
||||
up{job=prometheus pod=- } = 1
|
||||
|
||||
=== 진입점이 처음 40초간 000 이었던 이유 — nginx upstream ===
|
||||
@@ -0,0 +1,15 @@
|
||||
=== nginx 설정 위치 찾기 ===
|
||||
|
||||
=== NoExecute taint 의 tolerationSeconds — 언제 축출되는가 ===
|
||||
node.kubernetes.io/not-ready NoExecute tolerationSeconds=300
|
||||
node.kubernetes.io/unreachable NoExecute tolerationSeconds=300
|
||||
|
||||
=== 5분 축출 시점까지 관찰 ===
|
||||
+210초 a2-probe:Running keycloak-0:Running keycloak-1:Running postgres-7b474b88c8-2gf27:Running
|
||||
+240초 a2-probe:Running keycloak-0:Running keycloak-1:Running postgres-7b474b88c8-2gf27:Running
|
||||
+270초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
|
||||
+300초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
|
||||
+330초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
|
||||
+360초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
|
||||
+390초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
|
||||
+420초 a2-probe:Terminating keycloak-0:Terminating keycloak-1:Running postgres-7b474b88c8-2gf27:Terminating postgres-7b474b88c8-9cmsv:Pending
|
||||
@@ -0,0 +1,18 @@
|
||||
=== 새 postgres 가 Pending 인 이유 ===
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Warning FailedScheduling 4m45s default-scheduler 0/2 nodes are available: 1 node(s) didn't match PersistentVolume's node affinity, 1 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.
|
||||
|
||||
=== keycloak-0 대체 파드가 안 생기는 이유 (StatefulSet) ===
|
||||
keycloak 2 <none> 1
|
||||
keycloak-0 1/1 Terminating 0 30m
|
||||
keycloak-1 0/1 Running 0 143m
|
||||
|
||||
=== 복구 — 노드 재기동 ===
|
||||
재기동 시각: 12:16:31
|
||||
Domain 'kc-lab-2' started
|
||||
|
||||
+30초 node=Ready | Running 파드 3 개 | 외부 HTTP 503
|
||||
+60초 node=Ready | Running 파드 3 개 | 외부 HTTP 200
|
||||
→ 서비스 복귀
|
||||
@@ -0,0 +1,19 @@
|
||||
=== 복구 확인 ===
|
||||
keycloak-0 1/1 Running 0 68s
|
||||
keycloak-1 1/1 Running 0 144m
|
||||
postgres-7b474b88c8-9cmsv 1/1 Running 0 4m20s
|
||||
|
||||
=== kc-lab-1(k3s server)에 무엇이 있는가 — 이게 곧 영향 범위다 ===
|
||||
keycloak-lab keycloak-1
|
||||
kube-system coredns-54996dc9b4-8k8fj
|
||||
kube-system helm-install-traefik-crd-q29b5
|
||||
kube-system local-path-provisioner-77b9867795-g27z8
|
||||
kube-system metrics-server-6dc596dfb8-7xxq4
|
||||
kube-system svclb-traefik-5eb6a9a1-qwwk5
|
||||
kube-system traefik-5d6fcf895-wpfhr
|
||||
observability grafana-845b5678cf-b6gvc
|
||||
observability node-exporter-9qk9w
|
||||
observability prometheus-6774f94f7c-pzr2t
|
||||
|
||||
=== Traefik replica 수 (진입점의 단일 장애점인가) ===
|
||||
traefik 1 1
|
||||
@@ -0,0 +1,19 @@
|
||||
=== 컨트롤 플레인 노드(kc-lab-1) 전원 차단 ===
|
||||
차단 시각: 12:18:08
|
||||
Domain 'kc-lab-1' destroyed
|
||||
|
||||
|
||||
+20초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
|
||||
+40초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
|
||||
+60초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
|
||||
+80초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
|
||||
+100초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
|
||||
+120초 외부 auth=000 grafana=502 | kubectl: Unable to connect to the server: dial tcp
|
||||
+140초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
|
||||
+160초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server: dial tcp
|
||||
|
||||
=== 살아 있는 노드에서 직접 확인 — 워크로드는 도는가 ===
|
||||
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD NAMESPACE
|
||||
e5f777900b762 60e153026e8f5 4 minutes ago Running keycloak 0 640d4dafaefb3 keycloak-0 keycloak-lab
|
||||
|
||||
6
|
||||
@@ -0,0 +1,13 @@
|
||||
=== 컨트롤 플레인 노드 복구 ===
|
||||
재기동: 12:23:39
|
||||
Domain 'kc-lab-1' started
|
||||
|
||||
+30초 외부=502 | kc-lab-1=Ready kc-lab-2=Ready
|
||||
+60초 외부=200 | kc-lab-1=Ready kc-lab-2=Ready
|
||||
→ 서비스 복귀 (총 60초)
|
||||
|
||||
=== 최종 상태 ===
|
||||
keycloak-0 1/1 Running 0 7m57s
|
||||
keycloak-1 1/1 Running 1 (<invalid> ago) 151m
|
||||
postgres-7b474b88c8-9cmsv 1/1 Running 0 11m
|
||||
(prometheus port-forward 재연결 필요)
|
||||
@@ -0,0 +1,23 @@
|
||||
# A-4 — 노드 전원 차단 증거
|
||||
|
||||
2026-09-04 12:07–12:24 KST
|
||||
해설: [`docs/experiment-a4-node-loss.md`](../../experiment-a4-node-loss.md)
|
||||
|
||||
| 파일 | 무엇을 보여주는가 |
|
||||
|---|---|
|
||||
| `01-baseline.txt` | 차단 전 — 양쪽 Ready, **PVC 가 kc-lab-2 에 못박혀 있음**(재배치 불가의 원인), 외부 200 |
|
||||
| `02-worker-node-killed.txt` | `virsh destroy kc-lab-2` — **40초간 노드가 Ready 로 남아 있고** 외부는 이미 `000`. 이후 `503` |
|
||||
| `03-state-during-loss.txt` | **죽은 파드가 `ready=true`, 산 파드가 `ready=false`.** `up` 은 정확히 0. `unreachable` taint |
|
||||
| `04-eviction-timing.txt` | `tolerationSeconds=300` — **5분 뒤** 축출, 새 postgres 는 `Pending` |
|
||||
| `05-recovery.txt` | `FailedScheduling: didn't match PersistentVolume's node affinity`, StatefulSet `DESIRED=2 CURRENT=1`. 노드 복귀 후 **60초** |
|
||||
| `06-control-plane-inventory.txt` | kc-lab-1 에 있는 것 목록 — **Traefik `replicas=1`** |
|
||||
| `07-control-plane-loss.txt` | `virsh destroy kc-lab-1` — 외부 `000`, `kubectl` 불통. **그런데 `crictl ps` 로 보면 keycloak-0 은 Running** |
|
||||
| `08-control-plane-recovery.txt` | 60초 만에 복귀 |
|
||||
| `a4-up-dropped-per-node.png` | Grafana — `up` 이 노드별로 0 으로 떨어지는 구간. 12:18–12:23 은 **0 이 아니라 데이터 없음**(관측자가 같이 죽음) |
|
||||
|
||||
## 핵심 네 줄
|
||||
|
||||
1. **쿠버네티스는 40초 동안 노드가 살아 있다고 말한다.** 사용자는 이미 장애를 겪는 중이다.
|
||||
2. **죽은 파드의 상태는 화석이다.** `ready=true` 인 파드가 꺼진 기계 위에 있다.
|
||||
3. **StatefulSet 은 대체 파드를 만들지 않고, PVC 는 재배치를 막는다.** 사람이 개입해야 한다.
|
||||
4. **컨트롤 플레인 상실 ≠ 워크로드 상실.** 컨테이너는 계속 돌고, 들어갈 문만 사라진다.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 79 KiB |
@@ -0,0 +1,397 @@
|
||||
# A-4 — 노드가 통째로 죽으면 어떻게 되는가
|
||||
|
||||
브랜치 `feature/keycloak-a4-node-loss` ·
|
||||
증거 [`docs/evidence/a4-node-loss/`](evidence/a4-node-loss/) ·
|
||||
2026-09-04 12:07–12:24 KST
|
||||
|
||||
선행: [`A-2`](experiment-a2-database-loss.md) · [`A-3`](experiment-a3-database-crash.md)
|
||||
|
||||
**두 판본으로 나눠 실행했다.**
|
||||
|
||||
| | 죽인 노드 | 성격 |
|
||||
|---|---|---|
|
||||
| **4a** | `kc-lab-2` (워커) | Keycloak + **DB 동시 상실** |
|
||||
| **4b** | `kc-lab-1` (k3s 서버) | **컨트롤 플레인 + 진입점 상실** |
|
||||
|
||||
`virsh destroy` 는 종료 신호를 보내지 않는다. **전원을 뽑는 것과 같다.**
|
||||
|
||||
---
|
||||
|
||||
## 0. 결론부터
|
||||
|
||||
| | 4a 워커 상실 | 4b 컨트롤 플레인 상실 |
|
||||
|---|---|---|
|
||||
| 외부 응답 | **503** | **000** (연결 자체가 안 됨) |
|
||||
| `kubectl` | 정상 | **불통** |
|
||||
| 살아 있는 워크로드 | keycloak-1 (하지만 DB 없음) | **keycloak-0 은 계속 돌고 있다** |
|
||||
| 복구 시간 | **60초** | **60초** |
|
||||
|
||||
**둘 다 전면 장애**지만 이유가 다르다. 4a 는 **DB 가 같이 죽어서**, 4b 는
|
||||
**들어갈 길이 없어서**다.
|
||||
|
||||
그리고 예상하지 못한 것 셋 —
|
||||
**죽은 파드가 산 파드보다 건강해 보이고**, **StatefulSet 은 대체 파드를 만들지
|
||||
않으며**, **PVC 때문에 재배치가 불가능하다.**
|
||||
|
||||
---
|
||||
|
||||
## 4a. 워커 노드 상실 (`kc-lab-2`)
|
||||
|
||||
### 기준선
|
||||
|
||||
```
|
||||
kc-lab-1 Ready control-plane=true
|
||||
kc-lab-2 Ready <none>
|
||||
|
||||
keycloak-0 ready=true kc-lab-2
|
||||
keycloak-1 ready=true kc-lab-1
|
||||
postgres ready=true kc-lab-2
|
||||
|
||||
PVC postgres-data → kc-lab-2 ← 재배치 가능성을 여기서 이미 알 수 있다
|
||||
외부 HTTP 200
|
||||
```
|
||||
|
||||
### 주입
|
||||
|
||||
```bash
|
||||
virsh destroy kc-lab-2
|
||||
```
|
||||
|
||||
### 관찰
|
||||
|
||||
```
|
||||
+15초 node=Ready | keycloak-0=Running postgres=Running | 외부 HTTP 000
|
||||
+30초 node=Ready | keycloak-0=Running postgres=Running | 외부 HTTP 000
|
||||
+45초 node=NotReady | keycloak-0=Running postgres=Running | 외부 HTTP 503
|
||||
...
|
||||
+180초 node=NotReady | keycloak-0=Running postgres=Running | 외부 HTTP 503
|
||||
```
|
||||
|
||||
### 발견 ① — 쿠버네티스가 알아채는 데 **40초** 걸린다
|
||||
|
||||
`+15초`, `+30초` 에도 노드는 여전히 `Ready` 다. **기계는 이미 없는데.**
|
||||
|
||||
kube-controller-manager 의 `node-monitor-grace-period` (기본 40초) 동안
|
||||
kubelet 의 하트비트가 없어야 `NotReady` 로 바꾼다.
|
||||
|
||||
> **그 40초 동안 쿠버네티스는 거짓말을 한다.** 사용자는 이미 장애를 겪고 있다
|
||||
> (`000`). **노드 상태를 알림 근거로 삼으면 항상 늦는다.**
|
||||
|
||||
### 발견 ② — **죽은 파드가 산 파드보다 건강해 보인다**
|
||||
|
||||
```
|
||||
POD PHASE READY NODE
|
||||
keycloak-0 Running true kc-lab-2 ← 기계가 꺼져 있다
|
||||
keycloak-1 Running false kc-lab-1 ← 살아 있다
|
||||
```
|
||||
|
||||
**`keycloak-0` 은 `ready=true`, `keycloak-1` 은 `ready=false`.**
|
||||
|
||||
| 왜 | |
|
||||
|---|---|
|
||||
| keycloak-0 | kubelet 이 없어 **상태를 갱신할 수 없다.** 마지막으로 보고한 값이 그대로 얼어 있다 |
|
||||
| keycloak-1 | 살아서 **정직하게 보고한다** — DB 가 없으니 readiness 실패 |
|
||||
|
||||
> **파드 상태는 "지금 어떤가"가 아니라 "마지막으로 그렇게 들었다"이다.**
|
||||
> 노드가 죽으면 그 노드 파드의 상태는 **화석**이 된다.
|
||||
|
||||
Prometheus 는 정확했다.
|
||||
|
||||
```
|
||||
up{job=keycloak pod=keycloak-1} = 1
|
||||
up{job=keycloak pod=keycloak-0} = 0 ← 긁기에 실패했다 = 사실
|
||||
up{job=kubelet } = 0
|
||||
up{job=node-exporter node=kc-lab-2 } = 0
|
||||
```
|
||||
|
||||
**A-2 와 정반대다.** A-2 에서는 `up=1` 인데 서비스가 죽었고, 여기서는 `up=0`
|
||||
이 정확했다.
|
||||
|
||||
| | `up` 이 잡는가 |
|
||||
|---|---|
|
||||
| **대상이 사라짐** (노드 상실) | **잡는다** |
|
||||
| **대상이 살아서 못 씀** (DB 상실) | **못 잡는다** |
|
||||
|
||||
|
||||
|
||||
### 발견 ③ — 축출은 **5분** 뒤에 시작된다
|
||||
|
||||
쿠버네티스가 노드에 taint 를 붙인다.
|
||||
|
||||
```
|
||||
node.kubernetes.io/unreachable=:NoSchedule
|
||||
node.kubernetes.io/unreachable=:NoExecute
|
||||
```
|
||||
|
||||
그런데 모든 파드에는 기본 관용이 붙어 있다.
|
||||
|
||||
```
|
||||
node.kubernetes.io/not-ready NoExecute tolerationSeconds=300
|
||||
node.kubernetes.io/unreachable NoExecute tolerationSeconds=300
|
||||
```
|
||||
|
||||
```
|
||||
+240초 전부 Running
|
||||
+270초 a2-probe:Terminating keycloak-0:Terminating postgres:Terminating
|
||||
postgres-...-9cmsv:Pending ← 새 파드가 생겼다
|
||||
```
|
||||
|
||||
**노드가 죽고 약 5분이 지나야 파드가 축출된다.** 잠깐 끊긴 노드가 돌아올 수도
|
||||
있으니 성급하게 옮기지 않겠다는 설계다. 대신 **그동안은 아무 일도 일어나지
|
||||
않는다.**
|
||||
|
||||
### 발견 ④ — 축출돼도 **갈 곳이 없다**
|
||||
|
||||
```
|
||||
Warning FailedScheduling default-scheduler
|
||||
0/2 nodes are available:
|
||||
1 node(s) didn't match PersistentVolume's node affinity,
|
||||
1 node(s) had untolerated taint(s).
|
||||
```
|
||||
|
||||
```
|
||||
kc-lab-2 → taint 때문에 못 간다 (죽은 노드)
|
||||
kc-lab-1 → PVC 가 kc-lab-2 에 묶여 있어 못 간다
|
||||
```
|
||||
|
||||
**`local-path` PVC 는 그 노드의 로컬 디스크 디렉터리다.** 노드가 죽으면
|
||||
볼륨도 죽는다. 새 파드는 **영원히 Pending** 이다.
|
||||
|
||||
> 결함이 아니라 **조건**이다. 이 실험대는 그걸 알고 `local-path` 를 골랐다.
|
||||
> 운영이라면 네트워크 스토리지나 DB 복제가 이 자리를 메워야 한다.
|
||||
|
||||
### 발견 ⑤ — StatefulSet 은 **대체 파드를 만들지 않는다**
|
||||
|
||||
```
|
||||
NAME DESIRED READY CURRENT
|
||||
keycloak 2 <none> 1
|
||||
|
||||
keycloak-0 1/1 Terminating 30m ← 30분째
|
||||
keycloak-1 0/1 Running
|
||||
```
|
||||
|
||||
**`keycloak-0` 이 `Terminating` 에서 영원히 멈춰 있고, 대체 파드가 안 생긴다.**
|
||||
|
||||
| 왜 | |
|
||||
|---|---|
|
||||
| StatefulSet 의 계약 | **같은 이름의 파드는 클러스터에 하나뿐**이어야 한다 |
|
||||
| 컨트롤 플레인이 아는 것 | 노드가 안 보인다 = **파드가 죽었는지 확신할 수 없다** |
|
||||
| 그래서 | 옛 파드를 확실히 지우기 전엔 새 `keycloak-0` 을 못 만든다 |
|
||||
|
||||
**Deployment 였다면 즉시 새 파드를 만든다.** 이름이 아무래도 되기 때문이다.
|
||||
StatefulSet 의 "안정된 이름"이라는 이득의 **반대편 비용**이 여기다.
|
||||
|
||||
강제로 진행시키려면 사람이 개입해야 한다.
|
||||
|
||||
```bash
|
||||
kubectl -n keycloak-lab delete pod keycloak-0 --grace-period=0 --force
|
||||
```
|
||||
|
||||
**위험한 명령이다.** 노드가 사실은 살아 있고 네트워크만 끊긴 것이라면
|
||||
**같은 이름의 파드 둘이 동시에 존재**하게 된다 (split brain).
|
||||
|
||||
### 복구
|
||||
|
||||
```bash
|
||||
virsh start kc-lab-2
|
||||
```
|
||||
|
||||
```
|
||||
+30초 node=Ready | Running 파드 3 개 | 외부 HTTP 503
|
||||
+60초 node=Ready | Running 파드 3 개 | 외부 HTTP 200
|
||||
→ 서비스 복귀
|
||||
```
|
||||
|
||||
**60초.** 사람 개입 없이 전부 제자리로 돌아왔다.
|
||||
|
||||
---
|
||||
|
||||
## 4b. 컨트롤 플레인 노드 상실 (`kc-lab-1`)
|
||||
|
||||
### 이 노드에 무엇이 있었나 — 그게 곧 영향 범위다
|
||||
|
||||
```
|
||||
keycloak-lab keycloak-1
|
||||
kube-system coredns
|
||||
kube-system local-path-provisioner
|
||||
kube-system metrics-server
|
||||
kube-system svclb-traefik
|
||||
kube-system traefik ← replicas=1
|
||||
observability grafana
|
||||
observability prometheus
|
||||
observability node-exporter
|
||||
```
|
||||
|
||||
```
|
||||
NAME REPLICAS READY
|
||||
traefik 1 1 ← 진입점이 한 개다
|
||||
```
|
||||
|
||||
### 주입과 관찰
|
||||
|
||||
```bash
|
||||
virsh destroy kc-lab-1
|
||||
```
|
||||
|
||||
```
|
||||
+20초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server
|
||||
+60초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server
|
||||
+120초 외부 auth=000 grafana=502 | kubectl: Unable to connect to the server
|
||||
+160초 외부 auth=000 grafana=000 | kubectl: Unable to connect to the server
|
||||
```
|
||||
|
||||
### 발견 ⑥ — 워크로드는 살아 있는데 **길이 없다**
|
||||
|
||||
살아남은 노드에서 직접 확인했다.
|
||||
|
||||
```bash
|
||||
ssh kc-lab-2 'sudo crictl ps --name keycloak'
|
||||
```
|
||||
|
||||
```
|
||||
CONTAINER STATE NAME POD NAMESPACE
|
||||
e5f777900b76 Running keycloak keycloak-0 keycloak-lab
|
||||
```
|
||||
|
||||
**`keycloak-0` 은 멀쩡히 돌고 있다.** 컨테이너 런타임(containerd)은 API 서버
|
||||
없이도 이미 떠 있는 파드를 계속 굴린다.
|
||||
|
||||
```
|
||||
죽은 것: API 서버 · 스케줄러 · coredns · Traefik · Prometheus · Grafana
|
||||
산 것: keycloak-0 · postgres · containerd
|
||||
문제: 들어갈 문(Traefik)이 없다
|
||||
```
|
||||
|
||||
> **컨트롤 플레인 상실은 워크로드 상실이 아니다.**
|
||||
> 이미 떠 있는 것은 계속 돈다. **새로 뜨거나 옮기거나 고치는 것이 안 될 뿐.**
|
||||
|
||||
### 발견 ⑦ — `000` 과 `503` 의 차이
|
||||
|
||||
| 코드 | 뜻 |
|
||||
|---|---|
|
||||
| **`503`** (4a) | nginx·Traefik 은 살아 있고 **뒤에 보낼 파드가 없다** |
|
||||
| **`000`** (4b) | curl 이 응답 자체를 못 받았다 — **Traefik 이 없어 nginx 가 죽은 주소를 기다리다 타임아웃** |
|
||||
|
||||
`grafana=502` 가 한 번 찍힌 것이 증거다 — nginx 는 살아서 502 를 만들 수
|
||||
있었지만, 대부분은 타임아웃(`--max-time 8`)에 걸렸다.
|
||||
|
||||
**두 코드가 다른 층의 고장을 가리킨다.**
|
||||
|
||||
### 발견 ⑧ — 관측 시스템이 같이 죽으면 **0 이 아니라 구멍이 남는다**
|
||||
|
||||
Prometheus 가 `kc-lab-1` 에 있었다. 그 노드가 죽은 12:18–12:23 구간은
|
||||
그래프에 **0 이 아니라 데이터 없음**으로 남는다.
|
||||
|
||||
```
|
||||
대상이 죽음 → up = 0 → "언제 죽었는지" 알 수 있다
|
||||
관측자가 죽음 → 데이터 없음 → "그때 무슨 일이 있었는지" 모른다
|
||||
```
|
||||
|
||||
**계획 단계에서 "관측 스택은 죽이지 않는 노드에 둔다"고 정했던 이유**를
|
||||
실제로 확인한 셈이다. 다만 노드가 둘뿐이라 완전히 피할 수는 없다.
|
||||
|
||||
### 복구
|
||||
|
||||
```bash
|
||||
virsh start kc-lab-1
|
||||
```
|
||||
|
||||
```
|
||||
+30초 외부=502 | kc-lab-1=Ready kc-lab-2=Ready
|
||||
+60초 외부=200 | kc-lab-1=Ready kc-lab-2=Ready
|
||||
→ 서비스 복귀 (총 60초)
|
||||
```
|
||||
|
||||
**Grafana 는 로그인 세션을 잃었다** — 데이터가 `emptyDir` 이라 파드 재시작에
|
||||
사라진다. Prometheus 는 PVC 라 지표가 남았다. **의도한 설계대로 동작했다.**
|
||||
|
||||
---
|
||||
|
||||
## 5. 개념
|
||||
|
||||
### `node-monitor-grace-period` 와 `tolerationSeconds`
|
||||
|
||||
```
|
||||
기계 정지
|
||||
│
|
||||
│ 40초 node-monitor-grace-period → 노드 NotReady
|
||||
│
|
||||
│ +300초 tolerationSeconds (NoExecute) → 파드 축출 시작
|
||||
│
|
||||
▼
|
||||
총 약 5분 40초 동안 쿠버네티스는 아무것도 하지 않는다
|
||||
```
|
||||
|
||||
**빠른 장애 대응이 필요하면 이 값을 줄여야 하지만**, 짧게 하면 일시적
|
||||
네트워크 끊김에도 파드를 옮기게 되어 **불안정해진다.** 맞바꿈이다.
|
||||
|
||||
### `Terminating` 이 안 끝나는 이유
|
||||
|
||||
```
|
||||
파드 삭제 요청
|
||||
└─ kubelet 이 컨테이너를 멈추고 "지웠다"고 보고해야 끝난다
|
||||
└─ kubelet 이 없다 → 보고가 없다 → 영원히 Terminating
|
||||
```
|
||||
|
||||
`--grace-period=0 --force` 는 **보고를 기다리지 않고 API 에서 지우는 것**이다.
|
||||
컨테이너가 실제로 죽었는지는 **모른 채** 진행한다.
|
||||
|
||||
### StatefulSet vs Deployment — 노드 상실에서
|
||||
|
||||
| | Deployment | StatefulSet |
|
||||
|---|---|---|
|
||||
| 대체 파드 | **즉시 만든다** (이름 무관) | **안 만든다** (이름이 유일해야) |
|
||||
| 노드 상실 시 | 자동 회복 | **사람이 강제 삭제해야** |
|
||||
| 위험 | 없음 | 강제 삭제 시 **중복 실행 가능** |
|
||||
|
||||
**A-0 에서 "로그 대조가 쉬워서" StatefulSet 을 골랐는데, 그 대가가 여기 있다.**
|
||||
|
||||
### `local-path` PVC 와 재배치
|
||||
|
||||
```
|
||||
PVC → PV → nodeAffinity: kubernetes.io/hostname = kc-lab-2
|
||||
└─ 그 노드의 /var/lib/rancher/k3s/storage/... 디렉터리
|
||||
```
|
||||
|
||||
**볼륨이 노드에 못박히면 파드도 못박힌다.** 노드가 죽으면 데이터도 함께
|
||||
접근 불가가 된다.
|
||||
|
||||
---
|
||||
|
||||
## 6. 재현 절차 (명령어)
|
||||
|
||||
```bash
|
||||
export LIBVIRT_DEFAULT_URI=qemu:///system
|
||||
|
||||
# 사전 — PVC 가 어디 묶여 있는지 먼저 본다 (재배치 가능성이 여기서 정해진다)
|
||||
kubectl -n keycloak-lab get pvc -o name | while read p; do
|
||||
V=$(kubectl -n keycloak-lab get $p -o jsonpath='{.spec.volumeName}')
|
||||
kubectl get pv $V -o jsonpath='{.spec.nodeAffinity.required.nodeSelectorTerms[0].matchExpressions[0].values[0]}'
|
||||
done
|
||||
|
||||
# 4a 워커 상실
|
||||
virsh destroy kc-lab-2
|
||||
kubectl get node kc-lab-2 # 40초 뒤 NotReady
|
||||
kubectl -n keycloak-lab get pods -o wide # Running 인 채로 얼어 있다
|
||||
kubectl get node kc-lab-2 -o jsonpath='{.spec.taints}'
|
||||
# 5분 뒤 Terminating + 새 파드 Pending
|
||||
kubectl -n keycloak-lab describe pod <new-pod> | grep -A4 Events
|
||||
|
||||
# 4b 컨트롤 플레인 상실 — kubectl 이 죽으므로 노드에서 직접 본다
|
||||
virsh destroy kc-lab-1
|
||||
ssh kc-lab-2 'sudo crictl ps' # 워크로드는 살아 있다
|
||||
|
||||
# 복구
|
||||
virsh start kc-lab-2 && virsh start kc-lab-1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. 다음 실험에 남기는 것
|
||||
|
||||
| 실험 | 이 실험이 준 것 |
|
||||
|---|---|
|
||||
| **A-5** 비대칭 파티션 | 여기서는 노드가 **완전히** 사라졌다. 부분 단절은 더 고약하다 |
|
||||
| **D-1** 백업·복구 | **PVC 가 노드에 묶여 있다** — 노드가 영영 안 돌아오면 백업이 유일한 길 |
|
||||
| 구성 개선 | **Traefik `replicas=1` 은 진입점 단일 장애점** — 2로 늘리거나 DaemonSet 으로 |
|
||||
| 구성 개선 | **Grafana 가 `emptyDir`** — 재시작마다 로그인 세션이 사라진다 |
|
||||
Reference in New Issue
Block a user