문제: High·P1을 고친 뒤에도 P2·P3 약 90건이 남았다. 연결·구독·메시지 크기 상한이 없는 전송(WebSocket·gRPC·GraphQL deadline), 낙관적 충돌의 500, upsert 경쟁, Mongo 무제한 timeout, 샘플 목록 N+1, 갈라진 복사본 (TenantId 3벌, 이벤트 ID 문법 5곳, SQLState 표 2벌), 문서를 읽는 PR 테스트와 캐시를 막는 lane 설정 등이다. 새 기능을 어디에 어떤 순서로 만드는지 알려 주는 문서와 도구가 없었다. MinIO 공식 이미지가 사라져 객체 저장소 lane이 돌지 않았다. 방법: 영역별 묶음으로 실패 테스트를 먼저 쓰고 고쳤다. 고치는 중에 찾은 결함도 함께 고쳤다(durable operation id가 UUIDv4라 이벤트가 전부 DEAD, S3 어댑터 생성·작은 chunk 읽기 실패, Redis 조건부 기록의 EVALSHA 권한). MinIO는 pgsty/minio를 digest로 고정했다(Chainguard는 태그·실행 사용자· mc 이미지 문제로 버림). docs/manual/에 작업 순서와 판단 지점을 쓰고, scripts/new-feature.sh가 5개 leaf에 뼈대를 만든다. 생성→빌드→통과 왕복을 verifyFeatureScaffold가 mainCheck에서 본다. 묶음별 결과와 결정은 리뷰 README 14절에 있다. 위험·한계: 파생 서비스 동작이 바뀐다. JDBC batch 50 기본, 운영 jar에서 H2 제거, 트랜잭션 30초 제한, 격리 수준을 Hikari에서 고정, gRPC· WebSocket·GraphQL 새 한도, correlation ID 규칙 밖이면 서버가 교체, 샘플 키 오류 코드 IDEMPOTENCY_KEY_INVALID. 새 leaf 두 개 (composition:durable-operation, adapter:inbound:messaging)와 새 마이그레이션(postgresql V20·V21, notification V11·V12, sample V8_1, grpc-ledger V002)이 생긴다. V21 유일 인덱스는 기존 중복이 있으면 실패한다(runbook에 확인 쿼리). 검증: ci, mainCheck(생성 명령 왕복 포함), integrationCheck(MinIO 포함 전 컨테이너 lane), -p optional-platforms ci integrationCheck 통과. Refs: docs/reviews/2026-11-01-maturity-review/README.md 14절 BREAKING CHANGE: JDBC batch 기본 켜짐, H2 운영 의존 제거, 트랜잭션 기본 제한 30초, gRPC·WebSocket·GraphQL 서버 한도 기본값 추가, 샘플 Idempotency-Key 오류 코드 변경.
4.5 KiB
Retry and Ambiguity
The platform never decides a retry from the HTTP method alone (design D-09). A second attempt happens only when idempotency, body replayability, execution evidence, deadline, and retry budget all permit it.
Execution evidence
| Evidence | Meaning | Typical cause |
|---|---|---|
NOT_SENT |
Proven that the server never received the request | profile rejection, pool timeout, DNS failure, connect failure, pre-request TLS failure, HTTP/2 REFUSED_STREAM |
SENT_NO_RESPONSE |
Some or all of the request was written, no final header arrived | partial write, response-header timeout, connection reset |
RESPONSE_RECEIVED |
Final headers arrived, whatever the status | 2xx, 4xx, 5xx, redirect |
PARTIAL_RESPONSE |
Headers and part of the body arrived | reset during decode, interrupted stream |
NOT_SENT is only produced by a stage failure that proves it. A generic engine I/O error is never
upgraded to NOT_SENT, because that is exactly how a timeout becomes a duplicate payment.
Body replayability
| Body | Replayability |
|---|---|
immutable byte[] |
REPLAYABLE |
| DTO plus a deterministic codec | REPLAYABLE |
| reopenable file or resource supplier | REOPENABLE |
a single InputStream instance |
ONE_SHOT |
| publisher factory | as declared |
| publisher instance | ONE_SHOT |
| multipart | the weakest part |
Decision order
DefaultRetryEligibilityEngine evaluates in this order, and a later rule can never re-enable
something an earlier one forbade:
SENT_NO_RESPONSEon an operation that is not safely idempotent →AmbiguousFailure- attempts exhausted →
RetryDenied.maxAttempts() - retry budget empty →
RetryDenied.budgetExhausted() - body not replayable →
RetryDenied.bodyNotReplayable() - first byte already delivered →
RetryDenied.responseAlreadyDelivered() - runtime draining →
RetryDenied.runtimeDraining() - remaining deadline below the minimum attempt budget →
RetryDenied.deadline() - permanent failure category →
RetryDenied.permanentFailure(...) PARTIAL_RESPONSEon an operation that is not safely idempotent →AmbiguousFailure- status- and failure-specific rules
Ambiguity comes first because it describes the outcome, not whether another attempt is allowed. It
used to sit below the attempt check, so with the default max-attempts: 1 a POST that timed out
after it was sent reached the caller as an ordinary timeout (review 2026-11-01 05-14).
Status rules
| Status | Decision |
|---|---|
| 408 | retry inside deadline and budget |
| 425 | at most one retry, first attempt only |
| 429 | retry inside Retry-After, deadline, and budget |
| 401 | one refresh-and-replay, safe replayable operations only. The credential is refreshed only once the retry is granted; a 401 that will not be replayed (one attempt, no budget, no deadline) leaves the token cache alone |
| 500 | denied unless the upstream registered it as transient and the operation is safely idempotent |
| 502, 503, 504 | retry for safely idempotent operations; ambiguous otherwise |
| other 4xx | denied |
Ambiguity
A non-idempotent request that reached SENT_NO_RESPONSE raises
HttpAmbiguousExecutionException. It is a third answer on purpose: retrying may duplicate a side
effect, and reporting a plain failure would tell the caller the request did not happen, which may
be false. The caller reconciles, usually by querying the upstream or replaying with an idempotency
key.
Where the evidence comes from
Whether a failed request was sent is read from the stage the attempt reached and the engine's
exception type. Apache reports the moment it starts writing the request onto an established
connection (RequestWriteSignal), so a failure while leasing, resolving, connecting or completing
the TLS handshake is NOT_SENT even when the engine throws a generic exception. Transports that
cannot report that moment (JDK, Jetty) are treated as writing from the moment the engine is called,
because guessing "not sent" is what turns a timeout into a duplicate payment. An HTTP/2
REFUSED_STREAM frame is proof the peer processed nothing and is kept as NOT_SENT through
exception mapping.
Budget and backoff
Retry tokens come from a per-upstream token bucket sized as a fraction of real traffic, so a failing
upstream cannot be flooded by retries from a healthy fleet. Backoff is exponential with full or
decorrelated jitter, bounded by max-backoff, by Retry-After, and by the remaining deadline. No
connection and no bulkhead permit is held while a backoff is waiting.