docs(d4): the renewal succeeded and was not served for 36 minutes 39 seconds

The forced renewal did work. What did not work was anything making nginx
notice.

  disk     cert2.pem written 2026-09-04 17:22:13 KST
  network  old serial through 08:58:47 UTC, new serial from 08:58:52 UTC
  gap      2199 seconds, and the old certificate was observed 428 times in it

The gap closed only because a person ran `nginx -s reload`. Nothing else
would have: certbot-renew.service is just `ExecStart=/usr/bin/certbot -q
renew` with no ExecStartPost, all three of renewal-hooks/{deploy,post,pre}
are empty, and certbot 5.7.0 here has no nginx plugin (dns-cloudflare,
manual, null, standalone, webroot). Three paths, all empty. Any one of them
would have been enough.

nginx holds the certificate in memory from startup, and certbot swaps the
live/ symlink rather than the path — so the config still looks right while
the served certificate is stale. Master 585 and worker 586 sharing a start
time 22.4 hours old is what proved no reload had happened.

This is invisible for 88 days. The timer ran twice today and exited
0/SUCCESS both times, because a renewal is not due yet. The day it becomes
due, the symptom is an expired certificate and the log still says SUCCESS.

The other half of the plan's question now has a measurement rather than an
expectation. The reload is genuinely graceful: 8856 polled connections, zero
non-200, p95 205.7ms before against 204.3ms after. And an in-flight request
— 845KB pulled at 20k/s, so still transferring 12 seconds in when the reload
landed — completed with all 845361 bytes over a single connection.

The earlier note that nginx reload "is graceful by design but must not be
written that way without checking" was right to hold back. The neighbouring
belief, that renewing gets the new certificate served, was the one that was
false, and there was no way to tell in advance which it would be.

Adds diagrams/d4-renewal-gap.svg and evidence 09-13.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
DongHyeonka
2026-09-04 18:04:44 +09:00
co-authored by Claude Opus 5
parent 0d84921a56
commit faf55dc1be
9 changed files with 1295 additions and 104 deletions
+2 -2
View File
@@ -26,7 +26,7 @@
| **D-1** | 백업·복구 | `...d1-backup-restore` | **빈 데이터베이스가 `200` 을 냈다** |
| **D-2** | 버전 업그레이드 | `...d2-version-upgrade` | **이미지를 되돌려도 스키마는 안 돌아온다** |
| **D-3** | 비밀 관리 | `...d3-secret-management` | **RBAC 만 실제로 감춘다** |
| **D-4** | 인증서 갱신 | `...d4-certificate-renewal` | 구성은 정상. **강제 갱신은 사람 손이 필요** |
| **D-4** | 인증서 갱신 | `...d4-certificate-renewal` | **갱신은 됐는데 36분 39초 반영 안 됨** (훅 3경로 전부 비었음). reload 자체는 **무중단**(8856건 0실패) |
| **후속** | 미측정 항목 채우기 | `...followup-untested-items` | 정방향 업그레이드 무중단 · **롤백 불가는 조건부였다** · role 변경은 요청으로 반영 안 됨 |
## 시각 자료
@@ -85,7 +85,7 @@
| A-6 | `tc ... dev eth0` | 인터페이스가 `enp1s0` 이다 |
| A-6 | `enp1s0` 에 파드 IP 필터 | VXLAN 캡슐화로 **안 보인다** |
| B-2 | `spring.sql.init` 스키마 | PostgreSQL 에 없는 `blob` 타입 + `continue-on-error` |
| B-7·D-4 | `sudo nginx -T` 등 | **호스트 sudo 가 비밀번호를 요구**한다 |
| B-7·D-4 | `sudo nginx -T` 등 | **호스트 sudo 가 비밀번호를 요구**한다. D-4 는 사용자가 직접 실행해 해소했다 |
**그래서 실험마다 "주입 성공 신호" 를 먼저 정하게 됐다**
`cluster_size` 하락, `not properly shut down` 로그, iptables 패킷 카운터,