diff --git a/docs/testing/TESTING_STRATEGY.md b/docs/testing/TESTING_STRATEGY.md index f95dfa50..c224d85f 100644 --- a/docs/testing/TESTING_STRATEGY.md +++ b/docs/testing/TESTING_STRATEGY.md @@ -260,7 +260,13 @@ source set에서 태그 `jpa-contract`로 `jpaPlatformContractTest` lane이 돌 claim을 `TakenOverClaimed`(attempt 2)로, Redis는 `Acquired`로 답한다. executor는 둘을 같게 처리하므로 계약은 "다음 claim이 scope를 가진다"만 요구한다. 같은 이유로 release 재시도는 PostgreSQL이 `ALREADY_RELEASED_SAME_OPERATION`, Redis·fake가 `ABSENT`다. 기록이 없으면 셋 다 `ABSENT`다(Redis는 전에 - `ALREADY_RELEASED_SAME_OPERATION`으로 답해 확인할 수 없는 해제를 확인했다, 2026-11-06). + `ALREADY_RELEASED_SAME_OPERATION`으로 답해 확인할 수 없는 해제를 확인했다, 2026-11-06). 같은 까닭으로 + release한 핸들로 시작하면 PostgreSQL은 `NOT_OWNER`, Redis·fake는 `ABSENT`이고 계약은 둘 다 받는다. (4) 이미 + 적용된 전이를 "이미 처리됨"으로 확인받는 핸들이 셋 다 달랐다. Redis는 complete·markFailed를 그 operation의 + 어느 핸들에게나, PostgreSQL은 complete를 같은 owner token의 어느 핸들에게나 확인했고 다른 핸들의 + markFailed·renew·release는 `OPERATION_CONFLICT`로 답했다 — 주인이 renew가 돌려준 핸들로 다시 renew해도 + 그랬다. fake는 complete 말고는 적용한 핸들의 재전송도 `NOT_OWNER`였다. 셋 다 적용한 핸들(기록보다 + revision이 하나 낮은 핸들)에게만 확인하고 다른 핸들은 `NOT_OWNER`로 맞췄다(2026-11-06 소유자 결정). - `OutboxAppendContract`는 static helper 두 개였고 JPA readiness 테스트만 불렀다. 추상 계약으로 바꿔 readiness 테스트가 상속한다(같은 두 케이스 + 트랜잭션 밖·읽기 전용·중복 eventId). 실 PostgreSQL에서 `inRead` 안의 append가 예외 없이 사라지는 것을 찾았다(실측: 예외 없음, 행 없음) — JPA 트랜잭션 diff --git a/docs/upgrade/2026-11-maturity-review.md b/docs/upgrade/2026-11-maturity-review.md index 7a0f1509..11c4178b 100644 --- a/docs/upgrade/2026-11-maturity-review.md +++ b/docs/upgrade/2026-11-maturity-review.md @@ -241,6 +241,7 @@ V21 실패의 분류는 `docs/runbooks/migration-failed.md` "Classify the failur | 멱등성(lease가 끝난 시작, r9) | lease가 지난 `CLAIMED`는 주인도 시작할 수 없다(`LEASE_EXPIRED`). executor는 작업을 돌리지 않고 503 `IDEMPOTENCY_UNAVAILABLE`(실행 전 거부, 메시지 `the claim's processing lease ran out before the action started`)로 답하고, 다음 같은 키 요청이 그 claim을 넘겨받아 실행한다. 전에는 Redis가 시작해 작업을 돌렸고 그동안 다른 요청은 lease가 지난 실행으로 500을 받았다. PostgreSQL은 `NOT_OWNER`로 거절해 500 `IDEMPOTENCY_RECOVERY_REQUIRED`였다. | claim과 시작 사이가 lease(기본 30s)보다 길었던 드문 요청이 500 대신 503(같은 키 재전송)을 받는다. 클라이언트는 `error.code`로 이미 가르므로 바꿀 것이 없다. 운영자는 이 503에서 저장소 장애를 찾지 않는다(runbook) | r9(재리뷰 README 15 남음, RR5-01-02·03) · `IdempotencyExecutor.java`, `src/adapter/outbound/cache-redis/.../idempotency/IdempotencyScripts.java`, `src/adapter/outbound/persistence-jpa/.../postgresql/PostgreSqlIdempotencyStore.java` | | 멱등성(같은 attempt의 재시도, r10) | lease가 지나 시작을 거절당한 claim과 재시도 가능 실패 기록은 같은 attempt로 다시 claim해도 넘겨받는다(`TakenOverClaimed`, attempt+1). 전에는 두 저장소가 그 claim을 재생해(`ReplayedAcquire`) 시작이 다시 거절되거나(503, 기록이 사라질 때까지) 시작할 수 없는 기록이라 500 `IDEMPOTENCY_RECOVERY_REQUIRED`였다. 효과 불명으로 포기한 기록은 같은 attempt에도 `RecoveryRequired`(PostgreSQL은 전에 `ReplayedAcquire`). Redis는 이미 실행 중인 기록에 같은 attempt의 다른 핸들(넘겨받기 전 핸들, 재생받은 현재 핸들)로 온 시작을 `ALREADY_STARTED_SAME_OPERATION`으로 확인하지 않는다 — 전에는 executor가 작업을 한 번 더 돌렸다(PostgreSQL은 전부터 거절). in-memory fake는 재시도 가능 실패 기록에 다른 요청 본문이 오면 넘겨주지 않고 `FingerprintMismatch` | attempt를 재시도 사이에 들고 있다가 503 뒤 같은 attempt로 다시 부르는 호출자(재시도 래퍼, 메시지 소비자)가 이제 실행된다. 요청마다 새 attempt를 만드는 호출자(템플릿의 HTTP 경로)는 바뀌는 것이 없다. 같은 attempt를 두 호출이 동시에 쓰는 것은 여전히 가려내지 못한다 — 재시도는 앞 호출이 돌아온 뒤에. 직접 구현한 store는 `IdempotencyStoreContract`의 새 케이스로 확인한다 | r10(라운드6 RR6-02-01) · `src/adapter/outbound/cache-redis/.../idempotency/IdempotencyScripts.java`, `src/adapter/outbound/persistence-jpa/.../postgresql/PostgreSqlIdempotencyStore.java`, `src/application-core/src/testFixtures/.../IdempotencyStoreContract.java`, `InMemoryIdempotencyStore.java` | | 멱등성(전이의 operation id·release·inspect, 2026-11-06) | owner token이 같아도 호출이 claim과 다른 operation id를 대면 PostgreSQL·in-memory fake도 다섯 전이(start·renew·complete·fail·release)를 `OPERATION_CONFLICT`로 거절하고 기록을 바꾸지 않는다(전에는 적용했다. Redis는 전부터 거절). in-memory fake의 claim은 같은 token·다른 operation에 `OwnerOperationConflict`(전에는 `InProgress`), inspect는 `OPERATION_CONFLICT`(전에는 `IN_PROGRESS_OTHER`). PostgreSQL·fake의 inspect는 완료된 기록에도 `OPERATION_CONFLICT`(전에는 응답을 재생). PostgreSQL complete는 같은 operation id를 쓴 다른 attempt에 `NOT_OWNER`(전에는 `ALREADY_COMPLETED_SAME_RESULT`). Redis release는 기록이 없으면 `ABSENT`(전에는 `ALREADY_RELEASED_SAME_OPERATION`)이고 핸들 revision을 본다: 같은 attempt가 넘겨받기 전 핸들로 release하면 `NOT_OWNER`(전에는 지금의 claim을 지웠다) | 템플릿의 호출자(`IdempotencyExecutor`)는 늘 claim의 operation id를 넘기고 release를 부르지 않아 HTTP 동작은 같다. store 포트를 직접 부르며 전이마다 다른 operation id를 넘기던 코드는 이제 `OPERATION_CONFLICT`를 받는다 — `IdempotencyOwner.claimOperationId()`를 넘긴다. Redis release의 `ALREADY_RELEASED_SAME_OPERATION`을 기다리던 코드는 `ABSENT`도 "해제됨"으로 읽는다. 직접 구현한 store는 `IdempotencyStoreContract`의 새 케이스로 확인한다 | 2026-11-06(소유자 결정 "(a)로 진행", 라운드6 02 Declined) · `src/adapter/outbound/persistence-jpa/.../postgresql/PostgreSqlIdempotencyStore.java`, `src/adapter/outbound/cache-redis/.../idempotency/IdempotencyScripts.java`, `RedisIdempotencyStoreAdapter.java`, `src/application-core/src/testFixtures/.../IdempotencyStoreContract.java`, `InMemoryIdempotencyStore.java` | +| 멱등성(이미 적용된 전이의 재전송, 2026-11-06) | 이미 적용된 전이를 "이미 처리됨"으로 확인받는 것은 그 전이를 적용한 핸들(기록보다 revision이 하나 낮은 핸들)뿐이다. 같은 attempt가 넘겨받기 전 핸들로, 넘겨받은 뒤 적용된 complete·markFailed를 다시 보내면 `NOT_OWNER`(전에는 Redis가 `ALREADY_COMPLETED_SAME_RESULT`·`ALREADY_MARKED_SAME_OPERATION`, PostgreSQL이 `ALREADY_COMPLETED_SAME_RESULT`·`OPERATION_CONFLICT`). PostgreSQL은 renew가 돌려준 핸들로 다시 renew하면 한 번 더 renew한다(전에는 `OPERATION_CONFLICT`). 넘겨받기 전 핸들의 renew는 `NOT_OWNER`, release는 `NOT_OWNER`(전에는 둘 다 `OPERATION_CONFLICT`). in-memory fake는 적용한 핸들이 start·renew·markFailed를 다시 보내면 `ALREADY_STARTED_SAME_OPERATION`·`ALREADY_RENEWED_SAME_OPERATION`(다른 lease면 `OPERATION_CONFLICT`)·`ALREADY_MARKED_SAME_OPERATION`(전에는 `NOT_OWNER`). release한 핸들로 start하면 PostgreSQL은 `NOT_OWNER`, Redis·fake는 `ABSENT`이고 계약은 둘 다 받는다(동작 변경 없음, Javadoc만) | 템플릿의 호출자(`IdempotencyExecutor`)는 넘겨받기 전 핸들로 다시 보내지 않고 renew·release를 부르지 않아 HTTP 동작은 같다. store 포트를 직접 부르며 heartbeat로 renew를 이어 부르는 코드는 PostgreSQL에서 두 번째 renew부터 거절당하던 것이 이제 된다. 직접 구현한 store는 `IdempotencyStoreContract`의 새 케이스로 확인한다. fake로 lost reply를 흉내 내던 테스트는 이제 "이미 처리됨"을 받는다 | 2026-11-06(소유자 결정 「넘겨받기 전 핸들의 재전송」「해제된 핸들로 start」) · `src/adapter/outbound/cache-redis/.../idempotency/IdempotencyScripts.java`, `src/adapter/outbound/persistence-jpa/.../postgresql/PostgreSqlIdempotencyStore.java`, `src/application-core/src/testFixtures/.../IdempotencyStoreContract.java`, `InMemoryIdempotencyStore.java`, `IdempotencyStartOutcome.java` | | 빌드 단계(HTTP client) | `httpClientFailureInjectionTest`(Toxiproxy 장애 주입)가 `mainCheck`에서 빠졌다. `integrationCheck`에만 있고, main push·nightly의 `integration-main.yml` `integration-check` job이 돌린다 | main push마다 같은 lane이 `main-check`·`integration-check` 두 job에서 두 번 돌던 것이 한 번이 된다. 로컬에서 `mainCheck`만 돌리면 이 lane은 돌지 않는다(`integrationCheck`로 돌린다) | r9 · `src/adapter/outbound/httpclient/build.gradle` | | 서버 로그(GraphQL·gRPC·STOMP) | carrier로 답한 실패를 web과 같은 줄(shared-contract `ServerFailureLog`)로 ERROR에 남긴다: persistence 실패는 분류만, 5xx carrier는 예외와 함께, 4xx carrier는 남기지 않는다. gRPC는 ` (method=…)`, GraphQL은 ` (executionId=…)`를 붙인다. gRPC는 인식 못 한 실패(`INTERNAL_ERROR`)도 `unhandled failure on …`로 남긴다. 클라이언트가 취소한 호출은 남기지 않고, 서비스가 `CANCELLED` status로 끝낸 살아 있는 호출은 남긴다. raw status는 넘겼든 던졌든 설명을 남기지 않는다(원인 예외가 있으면 그것만 붙인다). `IllegalArgumentException`(`BAD_PARAMETER`)은 web과 같은 WARN 한 줄 `request refused as BAD_PARAMETER by <예외 타입>`(메시지 없음)으로 남긴다. STOMP `@MessageMapping` 실패는 Spring의 `Unhandled exception from message handler method`(예외 전체) 대신 같은 줄로 남는다 | ERROR 로그가 늘어난다(전에는 gRPC가 아무것도 남기지 않았다). GraphQL·gRPC·STOMP는 잘못된 입력마다 WARN 한 줄이 생긴다. STOMP에서 Spring 문구로 경보를 걸었다면 고친다. `IDEMPOTENCY_RECOVERY_REQUIRED`의 scope digest를 모든 전송에서 찾을 수 있다(`docs/runbooks/idempotency-unavailable.md`) | r9 · `src/shared-contract/.../error/ServerFailureLog.java`, `GrpcExceptionHandlingInterceptor.java`, `GraphQlWireErrorMapper.java`, `StompHandlerFailureLog.java` | | sample-portfolio | 익명 쓰기 401, 포스터 제목 대소문자 무시 유일(409 `POSTER_ALREADY_EXISTS`), 관리자도 남의 work log 수정 불가, 샘플 이벤트 tenant `global` | sample을 남긴 서비스만 | `34565130`, `1400dc15` | diff --git a/docs/work/reference/decisions.md b/docs/work/reference/decisions.md index d28c6a33..0a76dea5 100644 --- a/docs/work/reference/decisions.md +++ b/docs/work/reference/decisions.md @@ -100,11 +100,11 @@ | 다른 핸들의 시작 답 | PostgreSQL도 Redis와 같게 `NOT_OWNER`/`NOT_CLAIMED`로 답한다 | 기본값 | `1103` 17절 r10-02 | | 전이의 operation id | owner token이 같아도 호출의 operation id가 기록된 claim의 operation id와 다르면 다섯 전이(start·renew·complete·fail·release) 모두 `OPERATION_CONFLICT`로 거절하고 기록은 그대로 둔다. PostgreSQL·in-memory fake를 Redis에 맞췄다. 그래서 `OPERATION_CONFLICT`는 "claim과 다른 id"와 "같은 id·다른 인자" 두 뜻이다 | 소유자(2026-11-06) | 2026-11-06 총괄 세션 전달(소유자 원문 "(a)로 진행"), `IdempotencyStoreContract` | | 같은 token·다른 operation의 claim·inspect | claim은 진행 중인 기록에 `OwnerOperationConflict`, inspect는 상태와 상관없이(완료된 기록도) `OPERATION_CONFLICT`. 거절이 안전한 쪽을 골랐다 | 기본값(2026-11-06) | 위 결정과 같은 전달의 판단 원칙, `IdempotencyStoreContract` | -| complete 재생 판정 | 완료 확인은 같은 owner token에만 준다. 같은 operation id를 쓴 다른 attempt의 늦은 complete는 `NOT_OWNER` | 기본값(2026-11-06) | Redis TRANSITION에 맞춤, `IdempotencyStoreContract` | +| complete 재생 판정 | 완료 확인은 같은 owner token에만 준다. 같은 operation id를 쓴 다른 attempt의 늦은 complete는 `NOT_OWNER`. → 아래 「넘겨받기 전 핸들의 재전송」이 적용한 핸들 하나로 좁혔다 | 기본값(2026-11-06) | Redis TRANSITION에 맞춤, `IdempotencyStoreContract` | | 기록 없는 release | 저장소 셋 다 `ABSENT`로 답한다(확인할 수 없는 해제를 확인하지 않는다). Redis release도 핸들 revision을 본다. release한 기록을 지울지 남길지는 저장소가 정하므로 release 재시도 답은 `ABSENT`(Redis·fake)와 `ALREADY_RELEASED_SAME_OPERATION`(PostgreSQL)으로 갈린다 | 기본값(2026-11-06) | JPA 설계 22.2 "affected row가 0이면 성공으로 간주하지 않는다", `IdempotencyStoreContract` | | Redis 연결 예산 | 캐시 조건부 쓰기는 멱등성·lease와 다른 연결 예산(SCRIPT lane) | 위임 | `1103` 4절(RR-04-02) | -| 넘겨받기 전 핸들의 재전송 | 같은 attempt를 넘겨받기 전의 핸들이 이미 일어난 complete·markFailed를 다시 보내면, 그 전이를 실제로 적용한 핸들에게만 "이미 처리됨"으로 확인하고 그 밖의 핸들은 거절한다. Redis TRANSITION 주석의 원래 의도를 뒤집는다 | 소유자 | t5 보고 「남은 문제」, 2026-11-06 | -| 해제된 핸들로 start | 계약은 `ABSENT`와 `NOT_OWNER`를 둘 다 받는다. 포트 Javadoc에 그렇게 적는다 | 소유자 | t5 보고 「남은 문제」, 2026-11-06 | +| 넘겨받기 전 핸들의 재전송 | 같은 attempt를 넘겨받기 전의 핸들이 이미 일어난 complete·markFailed를 다시 보내면, 그 전이를 실제로 적용한 핸들에게만 "이미 처리됨"으로 확인하고 그 밖의 핸들은 거절한다. Redis TRANSITION 주석의 원래 의도를 뒤집는다. 적용: 적용한 핸들은 기록보다 revision이 하나 낮은 핸들이고, 거절은 `NOT_OWNER`다. 같은 규칙을 start·renew·release에도 맞췄다 | 소유자 | t5 보고 「남은 문제」, 2026-11-06, `IdempotencyStoreContract` | +| 해제된 핸들로 start | 계약은 `ABSENT`와 `NOT_OWNER`를 둘 다 받는다. 포트 Javadoc에 그렇게 적는다 | 소유자 | t5 보고 「남은 문제」, 2026-11-06, `IdempotencyStartOutcome`, `IdempotencyStoreContract` | ### 2.5 영속성 diff --git a/src/adapter/outbound/cache-redis/README.md b/src/adapter/outbound/cache-redis/README.md index 2e8f9902..c7fd0f6c 100644 --- a/src/adapter/outbound/cache-redis/README.md +++ b/src/adapter/outbound/cache-redis/README.md @@ -277,11 +277,12 @@ revision) come from app-bootstrap `IdempotencySettings`. This leaf reads only `k `renew` has its own program rather than the generic transition. Renewal keeps the record in `EXECUTING`, and the transition answers "already in the target state" before it reads the revision (that answer exists for a retry whose first reply was lost), so a renewal sent through it was always -`ALREADY_RENEWED_SAME_OPERATION` and never moved the lease. For a start that answer is given only to -the handle the start was applied from — the record one revision past it — because the executor runs -the action on it: another handle of the same attempt (one from before a takeover, or the current one -a replayed claim returned) used to be told the start was applied and ran the action a second time; -it is now `NOT_OWNER` or `NOT_CLAIMED`, as in the PostgreSQL store (round6 RR6-02-01). The renew program checks owner, +`ALREADY_RENEWED_SAME_OPERATION` and never moved the lease. That answer is given only to the handle +the transition was applied from — the record one revision past it. Another handle of the same +attempt (one from before a takeover, or the current one a replayed claim returned) used to be told a +start was applied and ran the action a second time (round6 RR6-02-01), and to be told a completion +or failure that followed a takeover was its own (owner decision 2026-11-06); it is now `NOT_OWNER` +or `NOT_CLAIMED`/`NOT_IN_PROGRESS`, as in the PostgreSQL store. The renew program checks owner, operation, revision and state, then extends `leaseUntil` from Redis `TIME` and bumps the revision. A lost renewal reply is recognised by what the renewal recorded — the revision it started from and the lease it asked for: the same lease is `ALREADY_RENEWED_SAME_OPERATION` with the current handle, diff --git a/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/IdempotencyScripts.java b/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/IdempotencyScripts.java index 2df2090d..6a942373 100644 --- a/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/IdempotencyScripts.java +++ b/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/IdempotencyScripts.java @@ -123,6 +123,11 @@ public final class IdempotencyScripts { * execution that had just begun; the PostgreSQL store refused it (re-review 2026-11-04 README 15, * {@code IdempotencyStoreContract}). Completion and failure read no lease: the holder's late * result is recorded (owner decision 2026-11-04). + * + *

A transition already applied is answered {@code ALREADY} only to the handle it was applied + * from. A completion or failure used to be reported back to any handle of its operation, so the + * same attempt's handle from before a takeover was told the completion that followed it was its + * own (owner decision 2026-11-06). */ private static final String TRANSITION = """ @@ -147,17 +152,16 @@ public final class IdempotencyScripts { -- — they never received the reply that would have replaced it — so that check first would -- turn every lost reply into NOT_OWNER and make the idempotent call non-idempotent. -- - -- A start is the one transition whose ALREADY runs the action, so for a start the owner and - -- operation are not proof enough: the same attempt also holds a handle from before a - -- takeover, and the current handle a replayed claim returned after the start. Either one + -- The owner and operation are not proof enough: the same attempt also holds a handle from + -- before a takeover, and the current handle a replayed claim returned. Either one -- presenting a start the record already shows was answered ALREADY, and the executor ran the - -- action a second time (round6 RR6-02-01). A start is confirmed only to the handle it was - -- applied from — the record one revision past it — and any other falls through to the - -- revision check: NOT_OWNER for a stale handle, WRONG_STATE for the current one. The - -- PostgreSQL store answers the same: its start replay digest covers the handle's attempt - -- and revision. A completion or failure already applied is still reported back to any - -- handle of its operation: nothing runs on that answer. - if ARGV[4] ~= 'CLAIMED' or rev == tonumber(ARGV[2]) + 1 then + -- action a second time (round6 RR6-02-01); either one presenting a completion or failure + -- applied after the takeover was told it was its own. A transition is confirmed only to + -- the handle it was applied from — the record one revision past it — and any other falls + -- through to the revision check: NOT_OWNER for a stale handle, WRONG_STATE for the current + -- one (owner decision 2026-11-06). The PostgreSQL store answers the same: it reads a + -- recorded transition as a replay only for the handle one revision behind the row. + if rev == tonumber(ARGV[2]) + 1 then return {'ALREADY', attempt, rev, owner, redis.call('HGET', KEYS[1], 'resp') or '', ''} end end diff --git a/src/adapter/outbound/persistence-jpa/README.md b/src/adapter/outbound/persistence-jpa/README.md index a13880a2..c33c0b29 100644 --- a/src/adapter/outbound/persistence-jpa/README.md +++ b/src/adapter/outbound/persistence-jpa/README.md @@ -442,10 +442,15 @@ Redis store·in-memory fake와 같은 계약(`IdempotencyStoreContract`)을 실 전이(start·renew·complete·fail·release) 모두 거절하고 행을 바꾸지 않는다. 같은 token·다른 operation의 inspect도 상태와 상관없이(완료된 행도) `OPERATION_CONFLICT`다(2026-11-06 소유자 결정 — 전에는 CAS가 핸들의 claim operation만 보고 호출의 id는 `last_transition_operation_id`에 적기만 해서 적용했다). 다른 하나는 같은 operation -id가 다른 인자로 이미 적용된 경우다. complete 재생은 같은 owner token에만 확인한다 — 같은 operation id를 쓴 다른 -attempt의 늦은 complete는 `NOT_OWNER`다(전에는 operation id만 보고 `ALREADY_COMPLETED_SAME_RESULT`). 시작은 핸들 말고 인자가 없어서, -이미 기록된 시작과 digest가 다르면 다른 핸들이다 — 넘겨받기 전 핸들은 `NOT_OWNER`, 재생받은 현재 핸들은 -`NOT_CLAIMED`(라운드6 RR6-02-01, 전에는 `OPERATION_CONFLICT`). renew는 `EXECUTING`의 +id가 다른 인자로 이미 적용된 경우다. 이미 기록된 전이는 그 전이를 적용한 핸들(owner identity가 같고 행보다 +revision이 하나 낮은 핸들)에게만 재생으로 읽는다(2026-11-06 소유자 결정). 그 핸들이면 digest가 같을 때 "이미 +처리됨", 다를 때 `OPERATION_CONFLICT`다. 다른 핸들은 재생이 아니라 owner 검사로 간다. 그래서 같은 operation id를 +쓴 다른 attempt의 늦은 complete와, 같은 attempt가 넘겨받기 전 핸들로 다시 보낸 complete·markFailed·renew·release는 +`NOT_OWNER`다(전에는 complete를 owner token만 보고 `ALREADY_COMPLETED_SAME_RESULT`로 확인했고, 나머지는 digest가 +달라 `OPERATION_CONFLICT`였다). renew가 돌려준 핸들로 다시 renew하면 한 번 더 renew된다(전에는 앞 renew의 digest와 +달라 `OPERATION_CONFLICT`였다). 시작은 핸들 말고 인자가 없어서, 넘겨받기 전 핸들은 `NOT_OWNER`, 재생받은 현재 +핸들은 `NOT_CLAIMED`다(라운드6 RR6-02-01, 전에는 `OPERATION_CONFLICT`). release한 핸들로 시작하면 행이 남아 있어 +`NOT_OWNER`다. 행을 지우는 Redis·fake는 `ABSENT`이고 계약은 둘 다 받는다. renew는 `EXECUTING`의 lease만 늘리고, 시작 전 `CLAIMED`는 `NOT_IN_PROGRESS`다. ### outbox V2 (삭제됨) diff --git a/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/PostgreSqlIdempotencyStore.java b/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/PostgreSqlIdempotencyStore.java index 0b83e9c5..dec82c02 100644 --- a/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/PostgreSqlIdempotencyStore.java +++ b/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/PostgreSqlIdempotencyStore.java @@ -193,7 +193,8 @@ public final class PostgreSqlIdempotencyStore implements IdempotencyStorePort { // handle, which the classification below names as the Redis store does (NOT_OWNER, // NOT_CLAIMED); it used to be OPERATION_CONFLICT (round6 RR6-02-01). It never reaches the // update: a recorded start leaves the row EXECUTING until a later transition replaces it. - if (replayVerdict(row, "START", operationId, startDigest) == ReplayVerdict.SAME_ARGUMENTS) { + if (replayVerdict(row, owner, "START", operationId, startDigest) + == ReplayVerdict.SAME_ARGUMENTS) { return startResult(IdempotencyStartOutcome.ALREADY_STARTED_SAME_OPERATION, owner(row)); } IdempotencyStartOutcome mismatch = classifyStartMismatch(row, owner); @@ -231,7 +232,7 @@ public final class PostgreSqlIdempotencyStore implements IdempotencyStorePort { // caller went on believing it held the record for longer than the row says it does. String renewDigest = transitionDigest("RENEW", operationId, owner, Long.toString(processingLeaseTtl.toMillis())); - switch (replayVerdict(row, "RENEW", operationId, renewDigest)) { + switch (replayVerdict(row, owner, "RENEW", operationId, renewDigest)) { case SAME_ARGUMENTS -> { return renewResult(IdempotencyRenewOutcome.ALREADY_RENEWED_SAME_OPERATION, owner(row)); } @@ -277,13 +278,14 @@ public final class PostgreSqlIdempotencyStore implements IdempotencyStorePort { return IdempotencyCompleteOutcome.OPERATION_CONFLICT; } String responseDigest = sha256(response.payload()); - // A completion is reported back to its own owner only. The operation id alone confirmed the - // late completion of another attempt that carried the same id, which the Redis store refuses - // as NOT_OWNER (IdempotencyStoreContract). + // A completion is reported back only to the handle it was applied from. The operation id alone + // confirmed the late completion of another attempt that carried the same id, and the owner + // token alone confirmed it to the same attempt's handle from before a takeover; the Redis + // store refuses both as NOT_OWNER (IdempotencyStoreContract, owner decision 2026-11-06). if (row.state() == IdempotencyState.COMPLETED && "COMPLETE".equals(row.lastTransitionKind()) - && row.ownerToken().equals(owner.ownerToken()) - && operationId.value().equals(row.lastTransitionOperationId())) { + && operationId.value().equals(row.lastTransitionOperationId()) + && appliedFrom(row, owner)) { return responseDigest.equals(row.responseDigest()) ? IdempotencyCompleteOutcome.ALREADY_COMPLETED_SAME_RESULT : IdempotencyCompleteOutcome.RESPONSE_CONFLICT; @@ -342,7 +344,7 @@ public final class PostgreSqlIdempotencyStore implements IdempotencyStorePort { owner, disposition.name(), Long.toString(retention.toMillis())); - switch (replayVerdict(row, transitionKind, operationId, failDigest)) { + switch (replayVerdict(row, owner, transitionKind, operationId, failDigest)) { case SAME_ARGUMENTS -> { return IdempotencyFailOutcome.ALREADY_MARKED_SAME_OPERATION; } @@ -392,7 +394,7 @@ public final class PostgreSqlIdempotencyStore implements IdempotencyStorePort { return IdempotencyReleaseOutcome.OPERATION_CONFLICT; } String releaseDigest = transitionDigest("RELEASE", operationId, owner); - switch (replayVerdict(row, "RELEASE", operationId, releaseDigest)) { + switch (replayVerdict(row, owner, "RELEASE", operationId, releaseDigest)) { case SAME_ARGUMENTS -> { return IdempotencyReleaseOutcome.ALREADY_RELEASED_SAME_OPERATION; } @@ -591,6 +593,15 @@ public final class PostgreSqlIdempotencyStore implements IdempotencyStorePort { return sameOwnerIdentity(row, owner) && row.stateRevision() == owner.stateRevision(); } + /** + * Whether the row's last transition was applied from this handle: the same owner identity, one + * revision behind the row, as every transition advances the revision by one. Only that handle may + * be told a recorded transition was its own (owner decision 2026-11-06). + */ + private static boolean appliedFrom(IdempotencyRecordRow row, IdempotencyOwner owner) { + return sameOwnerIdentity(row, owner) && row.stateRevision() == owner.stateRevision() + 1; + } + /** * Whether the recorded transition is this call again, and whether it carried these arguments. * @@ -605,14 +616,23 @@ public final class PostgreSqlIdempotencyStore implements IdempotencyStorePort { * the owner handle it presented the first time, so the revision, the owner and every semantic * argument reproduce the stored digest exactly; anything else is a different transition wearing * the same operation id, and belongs in a conflict rather than in a confirmation. + * + *

That holds only for the handle the transition was applied from. Any other handle recomputes + * a different digest from its own attempt and revision, and was answered {@code + * OPERATION_CONFLICT}: the holder's second renewal through the handle its first one returned, and + * the same attempt's handle from before a takeover resending a failure or a release that followed + * it. Neither is a replay — the first is a new transition, the second a stale owner — so both go + * on to the ownership checks, as in the Redis store (owner decision 2026-11-06). */ private static ReplayVerdict replayVerdict( IdempotencyRecordRow row, + IdempotencyOwner owner, String transitionKind, OperationId operationId, String expectedDigest) { if (!transitionKind.equals(row.lastTransitionKind()) - || !operationId.value().equals(row.lastTransitionOperationId())) { + || !operationId.value().equals(row.lastTransitionOperationId()) + || !appliedFrom(row, owner)) { return ReplayVerdict.NOT_A_REPLAY; } return expectedDigest.equals(row.lastTransitionResultDigest()) diff --git a/src/adapter/outbound/redis-runtime/src/testFixtures/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/command/InMemorySemanticRedisCommands.java b/src/adapter/outbound/redis-runtime/src/testFixtures/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/command/InMemorySemanticRedisCommands.java index f25a5a11..bd13f4c3 100644 --- a/src/adapter/outbound/redis-runtime/src/testFixtures/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/command/InMemorySemanticRedisCommands.java +++ b/src/adapter/outbound/redis-runtime/src/testFixtures/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/command/InMemorySemanticRedisCommands.java @@ -280,10 +280,11 @@ public final class InMemorySemanticRedisCommands implements SemanticRedisCommand return reply("OPERATION_CONFLICT", attempt, rev, owner, "", ""); } // ALREADY before the revision check, matching the program: a lost reply leaves the caller - // holding the pre-transition revision. A start is confirmed only to the handle it was applied - // from — the record one revision past it — because its ALREADY runs the action (round6 - // RR6-02-01); any other handle falls through to NOT_OWNER or WRONG_STATE. - if (state.equals(toState) && (!"CLAIMED".equals(fromState) || rev == expectedRev + 1)) { + // holding the pre-transition revision. A transition is confirmed only to the handle it was + // applied from — the record one revision past it (round6 RR6-02-01 for a start, owner decision + // 2026-11-06 for a completion or failure); any other handle falls through to NOT_OWNER or + // WRONG_STATE. + if (state.equals(toState) && rev == expectedRev + 1) { String stored = field(rendered, "resp"); return reply("ALREADY", attempt, rev, owner, stored == null ? "" : stored, ""); } diff --git a/src/application-core/CLAUDE.md b/src/application-core/CLAUDE.md index e69b39d1..368c81c1 100644 --- a/src/application-core/CLAUDE.md +++ b/src/application-core/CLAUDE.md @@ -291,6 +291,10 @@ generation; the V1 `IdempotencyScope`/`tryBegin` API is gone. - Every transition names the operation its record was claimed under (`IdempotencyOwner.claimOperationId()`). Another one is `OPERATION_CONFLICT` in every store and changes nothing; so is an inspection that brings the owner token under another operation (owner decision 2026-11-06). +- A transition already applied is confirmed ("already …") only to the handle it was applied from — + the record one revision past it. Any other handle is `NOT_OWNER`, the same attempt's handle from + before a takeover included; a start through a released handle is `ABSENT` or `NOT_OWNER` by + whether the store keeps the released record (owner decision 2026-11-06, `IdempotencyStoreContract`). The former ArchUnit freeze rule `inbound_port_implementations_do_not_declare_keyed_idempotency` and its fixture were removed when this branch merged. diff --git a/src/application-core/README.md b/src/application-core/README.md index 541e8917..14c805f2 100644 --- a/src/application-core/README.md +++ b/src/application-core/README.md @@ -270,8 +270,10 @@ application 계층이 claim·replay·복구 **정책** 을 소유하고, 저장 - processing lease 와 replay TTL 은 따로 간다. 넘겨받을 수 있는 것은 만료된 `CLAIMED` 와 재시도 가능한 실패뿐이고, 다음 claim 이 누구든 넘겨받는다 — 같은 attempt 를 다시 보낸 caller 도 새 attempt 번호로 넘겨받는다(라운드6 RR6-02-01: 전에는 같은 attempt 가 이 기록을 재생받아 시작을 다시 거절당했다). 넘겨받기 전 - 핸들은 같은 attempt 가 들고 있어도 낡은 핸들이라 시작이 확인되지 않는다. 한 attempt 를 두 호출이 동시에 쓰는 - 것은 가려내지 못하므로 재시도는 앞 호출이 돌아온 뒤에 한다. + 핸들은 같은 attempt 가 들고 있어도 낡은 핸들이라 시작이 확인되지 않는다. 이미 적용된 전이(start·renew· + complete·markFailed·release)를 "이미 처리됨"으로 확인받는 것은 그 전이를 적용한 핸들뿐이다. 넘겨받은 뒤에 + 일어난 complete·markFailed 를 넘겨받기 전 핸들로 다시 보내면 `NOT_OWNER` 다(2026-11-06 소유자 결정). 한 attempt 를 + 두 호출이 동시에 쓰는 것은 가려내지 못하므로 재시도는 앞 호출이 돌아온 뒤에 한다. 만료된 `EXECUTING` 은 다른 caller 에게 `RECOVERY_REQUIRED` 로 답하되 기록은 그대로 둔다. 주인이 늦게 보고한 결과·실패는 기록된다 — 먼저 온 보고 하나만 남고 뒤의 보고는 거절된다(2026-11-04 소유자 결정, 재리뷰 RR5-01-03). lease 가 이미 끝났으면 주인도 시작(`markExecutionStarted`)과 `renew` 를 할 수 없고 diff --git a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyCompleteOutcome.java b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyCompleteOutcome.java index 4f09a42c..205acb69 100644 --- a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyCompleteOutcome.java +++ b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyCompleteOutcome.java @@ -3,7 +3,14 @@ package dev.caskeleton.application.idempotency; /** Outcomes of an owner-safe EXECUTING to COMPLETED transition. */ public enum IdempotencyCompleteOutcome { COMPLETED, + /** + * This completion, from the handle presented, was already recorded with the same result (a lost + * reply). Only the handle the completion was applied from is told so; another handle of the same + * attempt — one from before a takeover — is {@link #NOT_OWNER} (owner decision 2026-11-06: the + * Redis and PostgreSQL stores confirmed it). + */ ALREADY_COMPLETED_SAME_RESULT, + /** The handle the completion was applied from resent it with another result. */ RESPONSE_CONFLICT, ABSENT, NOT_OWNER, diff --git a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyFailOutcome.java b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyFailOutcome.java index 9d65912d..aecee742 100644 --- a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyFailOutcome.java +++ b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyFailOutcome.java @@ -4,6 +4,13 @@ package dev.caskeleton.application.idempotency; public enum IdempotencyFailOutcome { MARKED_RETRYABLE, MARKED_ABANDONED, + /** + * This failure, from the handle presented, was already recorded (a lost reply). Only the handle + * the failure was applied from is told so; another handle of the same attempt — one from before a + * takeover — is {@link #NOT_OWNER} (owner decision 2026-11-06: the Redis store confirmed it and + * the PostgreSQL store answered {@link #OPERATION_CONFLICT}; the in-memory fake refused even the + * handle that applied it). + */ ALREADY_MARKED_SAME_OPERATION, ABSENT, NOT_OWNER, diff --git a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyRenewOutcome.java b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyRenewOutcome.java index 6ae5b541..a0093c4d 100644 --- a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyRenewOutcome.java +++ b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyRenewOutcome.java @@ -10,7 +10,12 @@ package dev.caskeleton.application.idempotency; public enum IdempotencyRenewOutcome { /** The lease was extended; the returned handle carries the advanced revision. */ RENEWED(true), - /** This exact renewal was already applied (a lost reply); the handle is the current one. */ + /** + * This exact renewal was already applied (a lost reply); the handle is the current one. Only the + * handle the renewal was applied from is told so; the handle that renewal returned renews again, + * and any other is {@link #NOT_OWNER} (owner decision 2026-11-06: the PostgreSQL store answered + * both {@link #OPERATION_CONFLICT}). + */ ALREADY_RENEWED_SAME_OPERATION(true), /** No record exists: it expired, or was never there. */ ABSENT(false), @@ -31,7 +36,8 @@ public enum IdempotencyRenewOutcome { LEASE_EXPIRED(false), /** * The call names an operation other than the one the record was claimed under (owner decision - * 2026-11-06), or this renewal was already applied asking for a different lease. Nothing changed. + * 2026-11-06), or this renewal was already applied, from this handle, asking for a different + * lease. Nothing changed. */ OPERATION_CONFLICT(false), /** The store may have applied the renewal; inspect before relying on either answer. */ diff --git a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyStartOutcome.java b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyStartOutcome.java index c35ea305..3806ad6e 100644 --- a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyStartOutcome.java +++ b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyStartOutcome.java @@ -13,11 +13,17 @@ public enum IdempotencyStartOutcome { * #OPERATION_CONFLICT}). */ ALREADY_STARTED_SAME_OPERATION(true), - /** No record exists: it expired, was released, or was never there. */ + /** + * No record exists: it expired, was released, or was never there. A start through a released + * handle is this or {@link #NOT_OWNER}, by whether the store forgets a released record (Redis, + * the in-memory fake) or keeps it one revision past the handle (PostgreSQL). Both refuse it and + * nothing starts; a caller treats them alike (owner decision 2026-11-06). + */ ABSENT(false), /** * The record belongs to somebody else — the claim was taken over — or this handle's revision is - * stale. + * stale, as after the handle released the claim in a store that keeps the released record (see + * {@link #ABSENT}). */ NOT_OWNER(false), /** The record is not a claim waiting to start: it is executing or has ended. */ diff --git a/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/IdempotencyStoreContract.java b/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/IdempotencyStoreContract.java index c378a743..d2cb5e34 100644 --- a/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/IdempotencyStoreContract.java +++ b/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/IdempotencyStoreContract.java @@ -752,6 +752,221 @@ public abstract class IdempotencyStoreContract { .isEqualTo(IdempotencyStartOutcome.STARTED); } + // A transition already applied is confirmed only to the handle it was applied from — the record + // one revision past it — and any other handle is refused as a stale owner, the same attempt's + // handle from before a takeover included (owner decision 2026-11-06). The stores used to differ: + // the Redis store confirmed an applied completion or failure to every handle of its operation; + // the PostgreSQL store confirmed a completion to every handle of its owner token and answered + // another handle's failure, renewal or release OPERATION_CONFLICT — the holder's own second + // renewal included; the in-memory fake confirmed a completion and nothing else. + + @Test + @DisplayName("같은 attempt가 넘겨받기 전 핸들로, 넘겨받은 뒤 적용된 complete를 다시 보내면 확인하지 않고 NOT_OWNER다") + void theHandleFromBeforeATakeoverIsNotConfirmedTheCompletionThatFollowedIt() { + // given + IdempotencyStorePort store = subject(); + IdempotencyClaimAttempt attempt = store.newClaimAttempt(OPERATION); + IdempotencyOwner before = started(store, ownerOf(store.claim(claim(attempt, FINGERPRINT)))); + store.markFailed( + before, IdempotencyFailureDisposition.NO_EFFECT_RETRYABLE, Duration.ofHours(1), OPERATION); + IdempotencyOwner after = started(store, ownerOf(store.claim(claim(attempt, FINGERPRINT)))); + store.complete(after, new StoredResponse("{\"v\":1}"), Duration.ofHours(1), OPERATION); + + // when + IdempotencyCompleteOutcome resent = + store.complete(before, new StoredResponse("{\"v\":1}"), Duration.ofHours(1), OPERATION); + + // then + assertThat(resent).isEqualTo(IdempotencyCompleteOutcome.NOT_OWNER); + } + + @Test + @DisplayName("같은 attempt가 넘겨받기 전 핸들로, 넘겨받은 뒤 적용된 markFailed를 다시 보내면 확인하지 않고 NOT_OWNER다") + void theHandleFromBeforeATakeoverIsNotConfirmedTheFailureThatFollowedIt() { + // given + IdempotencyStorePort store = subject(); + IdempotencyClaimAttempt attempt = store.newClaimAttempt(OPERATION); + IdempotencyOwner before = started(store, ownerOf(store.claim(claim(attempt, FINGERPRINT)))); + store.markFailed( + before, IdempotencyFailureDisposition.NO_EFFECT_RETRYABLE, Duration.ofHours(1), OPERATION); + IdempotencyOwner after = started(store, ownerOf(store.claim(claim(attempt, FINGERPRINT)))); + store.markFailed( + after, IdempotencyFailureDisposition.NO_EFFECT_RETRYABLE, Duration.ofHours(1), OPERATION); + + // when + IdempotencyFailOutcome resent = + store.markFailed( + before, + IdempotencyFailureDisposition.NO_EFFECT_RETRYABLE, + Duration.ofHours(1), + OPERATION); + + // then + assertThat(resent).isEqualTo(IdempotencyFailOutcome.NOT_OWNER); + } + + @Test + @DisplayName("markFailed를 적용한 핸들이 같은 인자로 다시 보내면 이미 처리됨으로 확인한다") + void aFailureResentThroughTheHandleItWasAppliedFromIsConfirmed() { + // given + // The reply was lost, so the caller still holds the handle it sent the failure with. + IdempotencyStorePort store = subject(); + IdempotencyOwner executing = executing(store); + store.markFailed( + executing, + IdempotencyFailureDisposition.NO_EFFECT_RETRYABLE, + Duration.ofHours(1), + OPERATION); + + // when + IdempotencyFailOutcome resent = + store.markFailed( + executing, + IdempotencyFailureDisposition.NO_EFFECT_RETRYABLE, + Duration.ofHours(1), + OPERATION); + + // then + assertThat(resent).isEqualTo(IdempotencyFailOutcome.ALREADY_MARKED_SAME_OPERATION); + } + + @Test + @DisplayName("release한 핸들로 start하면 ABSENT나 NOT_OWNER로 거절되고 아무것도 시작되지 않는다") + void startingThroughAReleasedHandleStartsNothing() { + // given + IdempotencyStorePort store = subject(); + IdempotencyOwner claimed = acquire(store); + store.releaseBeforeExecution(claimed, OPERATION); + + // when + IdempotencyMutationResult started = + store.markExecutionStarted(claimed, OPERATION); + + // then + // A store that forgets the released record finds none (Redis, the in-memory fake); one that + // keeps it finds the handle one revision behind it (PostgreSQL). Either way nothing started + // and the next claim owns the scope (owner decision 2026-11-06). + assertThat(started.outcome()) + .isIn(IdempotencyStartOutcome.ABSENT, IdempotencyStartOutcome.NOT_OWNER); + IdempotencyClaimOutcome next = + store.claim(claim(store.newClaimAttempt(OTHER_OPERATION), FINGERPRINT)); + assertThat(store.markExecutionStarted(ownerOf(next), OTHER_OPERATION).outcome()) + .isEqualTo(IdempotencyStartOutcome.STARTED); + } + + @Test + @DisplayName("start를 적용한 핸들이 다시 보내면 이미 시작됨과 현재 핸들을 받는다") + void aStartResentThroughTheHandleItWasAppliedFromIsConfirmedWithTheCurrentHandle() { + // given + IdempotencyStorePort store = subject(); + IdempotencyOwner claimed = acquire(store); + IdempotencyOwner started = started(store, claimed); + + // when + IdempotencyMutationResult resent = + store.markExecutionStarted(claimed, OPERATION); + + // then + assertThat(resent.outcome()).isEqualTo(IdempotencyStartOutcome.ALREADY_STARTED_SAME_OPERATION); + assertThat(resent.owner().orElseThrow().stateRevision()).isEqualTo(started.stateRevision()); + } + + @Test + @DisplayName("renew를 적용한 핸들이 같은 lease로 다시 보내면 이미 renew됨과 현재 핸들을 받는다") + void aRenewalResentThroughTheHandleItWasAppliedFromIsConfirmedWithTheCurrentHandle() { + // given + IdempotencyStorePort store = subject(); + IdempotencyOwner executing = executing(store); + IdempotencyOwner renewed = + store.renew(executing, Duration.ofSeconds(30), OPERATION).owner().orElseThrow(); + + // when + IdempotencyMutationResult resent = + store.renew(executing, Duration.ofSeconds(30), OPERATION); + + // then + assertThat(resent.outcome()).isEqualTo(IdempotencyRenewOutcome.ALREADY_RENEWED_SAME_OPERATION); + assertThat(resent.owner().orElseThrow().stateRevision()).isEqualTo(renewed.stateRevision()); + } + + @Test + @DisplayName("renew를 적용한 핸들이 다른 lease로 다시 보내면 OPERATION_CONFLICT다") + void aRenewalResentThroughTheHandleItWasAppliedFromWithAnotherLeaseIsAConflict() { + // given + IdempotencyStorePort store = subject(); + IdempotencyOwner executing = executing(store); + store.renew(executing, Duration.ofSeconds(30), OPERATION); + + // when + IdempotencyMutationResult resent = + store.renew(executing, Duration.ofSeconds(45), OPERATION); + + // then + assertThat(resent.outcome()).isEqualTo(IdempotencyRenewOutcome.OPERATION_CONFLICT); + } + + @Test + @DisplayName("renew가 돌려준 핸들로 다시 renew하면 한 번 더 renew된다") + void aHolderRenewsAgainThroughTheHandleItsRenewalReturned() { + // given + IdempotencyStorePort store = subject(); + IdempotencyOwner executing = executing(store); + IdempotencyOwner renewed = + store.renew(executing, Duration.ofSeconds(30), OPERATION).owner().orElseThrow(); + + // when + IdempotencyMutationResult again = + store.renew(renewed, Duration.ofSeconds(30), OPERATION); + + // then + assertThat(again.outcome()).isEqualTo(IdempotencyRenewOutcome.RENEWED); + assertThat(again.owner().orElseThrow().stateRevision()).isGreaterThan(renewed.stateRevision()); + } + + @Test + @DisplayName("같은 attempt가 넘겨받기 전 핸들로 renew하면, 넘겨받은 뒤 적용된 renew로 확인하지 않고 NOT_OWNER다") + void theHandleFromBeforeATakeoverIsNotConfirmedTheRenewalThatFollowedIt() { + // given + IdempotencyStorePort store = subject(); + IdempotencyClaimAttempt attempt = store.newClaimAttempt(OPERATION); + IdempotencyOwner before = started(store, ownerOf(store.claim(claim(attempt, FINGERPRINT)))); + store.markFailed( + before, IdempotencyFailureDisposition.NO_EFFECT_RETRYABLE, Duration.ofHours(1), OPERATION); + IdempotencyOwner after = started(store, ownerOf(store.claim(claim(attempt, FINGERPRINT)))); + store.renew(after, Duration.ofSeconds(30), OPERATION); + + // when + IdempotencyMutationResult resent = + store.renew(before, Duration.ofSeconds(30), OPERATION); + + // then + assertThat(resent.outcome()).isEqualTo(IdempotencyRenewOutcome.NOT_OWNER); + } + + @Test + @DisplayName("같은 attempt가 넘겨받기 전 핸들로 release하면, 넘겨받은 뒤 적용된 release로 확인하지 않는다") + void theHandleFromBeforeATakeoverIsNotConfirmedTheReleaseThatFollowedIt() { + // given + IdempotencyStorePort store = subject(); + IdempotencyClaimAttempt attempt = store.newClaimAttempt(OPERATION); + IdempotencyOwner before = ownerOf(store.claim(claim(attempt, FINGERPRINT))); + store.markFailed( + started(store, before), + IdempotencyFailureDisposition.NO_EFFECT_RETRYABLE, + Duration.ofHours(1), + OPERATION); + IdempotencyOwner after = ownerOf(store.claim(claim(attempt, FINGERPRINT))); + store.releaseBeforeExecution(after, OPERATION); + + // when + IdempotencyReleaseOutcome resent = store.releaseBeforeExecution(before, OPERATION); + + // then + // As for a start through a released handle: a store that forgets the released record finds + // none, one that keeps it finds the handle stale. + assertThat(resent).isIn(IdempotencyReleaseOutcome.ABSENT, IdempotencyReleaseOutcome.NOT_OWNER); + } + @Test @DisplayName("실행 중인 자기 기록을 inspect하면 기록 수명이 아니라 claim이 받은 처리 lease 만료 시각이 나온다") void inspectingAnOwnExecutionReportsItsProcessingLeaseNotTheRecordLifetime() { diff --git a/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/InMemoryIdempotencyStore.java b/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/InMemoryIdempotencyStore.java index d840a5d9..39c02f4d 100644 --- a/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/InMemoryIdempotencyStore.java +++ b/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/InMemoryIdempotencyStore.java @@ -34,8 +34,9 @@ import java.util.Random; * attempt against an in-flight record is always {@code InProgress}. Leases and replay windows are * reported as {@link Instant#MAX}. What it keeps exact is what a test of a caller relies on: one * claim wins, the owner handle's revision advances on every transition, a completed response - * replays, a different fingerprint is refused, a retryable failure can be claimed again, and an - * owner naming an operation other than its claim's is refused. + * replays, a different fingerprint is refused, a retryable failure can be claimed again, an owner + * naming an operation other than its claim's is refused, and a transition resent after a lost reply + * is confirmed to the handle it was applied from and to no other. */ public final class InMemoryIdempotencyStore implements IdempotencyStorePort { @@ -99,17 +100,18 @@ public final class InMemoryIdempotencyStore implements IdempotencyStorePort { ? IdempotencyStartOutcome.ABSENT : row.namesAnotherOperation(owner, operationId) ? IdempotencyStartOutcome.OPERATION_CONFLICT - : !row.owner.equals(owner) - ? IdempotencyStartOutcome.NOT_OWNER - : row.state != IdempotencyState.CLAIMED - ? IdempotencyStartOutcome.NOT_CLAIMED - : IdempotencyStartOutcome.STARTED; - if (outcome != IdempotencyStartOutcome.STARTED) { - return new IdempotencyMutationResult<>(outcome, null, IdempotencyStartOutcome::carriesOwner); + : row.repeats(owner, Transition.START) + ? IdempotencyStartOutcome.ALREADY_STARTED_SAME_OPERATION + : !row.owner.equals(owner) + ? IdempotencyStartOutcome.NOT_OWNER + : row.state != IdempotencyState.CLAIMED + ? IdempotencyStartOutcome.NOT_CLAIMED + : IdempotencyStartOutcome.STARTED; + if (outcome == IdempotencyStartOutcome.STARTED) { + row.advance(IdempotencyState.EXECUTING, owner, Transition.START, null); } - row.advance(IdempotencyState.EXECUTING); return new IdempotencyMutationResult<>( - outcome, row.owner, IdempotencyStartOutcome::carriesOwner); + outcome, outcome.carriesOwner() ? row.owner : null, IdempotencyStartOutcome::carriesOwner); } @Override @@ -118,22 +120,27 @@ public final class InMemoryIdempotencyStore implements IdempotencyStorePort { Row row = rows.get(owner.scope().digest()); // Only a running execution has a processing lease to extend; the stores answer a claim that // has not started with NOT_IN_PROGRESS, and IdempotencyStoreContract holds the fake to that. - IdempotencyRenewOutcome refused = + // A renewal resent from the handle it was applied from is confirmed for the same lease and a + // conflict for another, as the stores answer. + IdempotencyRenewOutcome outcome = row == null ? IdempotencyRenewOutcome.ABSENT : row.namesAnotherOperation(owner, operationId) ? IdempotencyRenewOutcome.OPERATION_CONFLICT - : !row.owner.equals(owner) - ? IdempotencyRenewOutcome.NOT_OWNER - : row.state != IdempotencyState.EXECUTING - ? IdempotencyRenewOutcome.NOT_IN_PROGRESS - : null; - if (refused != null) { - return new IdempotencyMutationResult<>(refused, null, IdempotencyRenewOutcome::carriesOwner); + : row.repeats(owner, Transition.RENEW) + ? processingLeaseTtl.equals(row.applied.argument()) + ? IdempotencyRenewOutcome.ALREADY_RENEWED_SAME_OPERATION + : IdempotencyRenewOutcome.OPERATION_CONFLICT + : !row.owner.equals(owner) + ? IdempotencyRenewOutcome.NOT_OWNER + : row.state != IdempotencyState.EXECUTING + ? IdempotencyRenewOutcome.NOT_IN_PROGRESS + : IdempotencyRenewOutcome.RENEWED; + if (outcome == IdempotencyRenewOutcome.RENEWED) { + row.advance(row.state, owner, Transition.RENEW, processingLeaseTtl); } - row.advance(row.state); return new IdempotencyMutationResult<>( - IdempotencyRenewOutcome.RENEWED, row.owner, IdempotencyRenewOutcome::carriesOwner); + outcome, outcome.carriesOwner() ? row.owner : null, IdempotencyRenewOutcome::carriesOwner); } @Override @@ -149,7 +156,7 @@ public final class InMemoryIdempotencyStore implements IdempotencyStorePort { if (row.namesAnotherOperation(owner, operationId)) { return IdempotencyCompleteOutcome.OPERATION_CONFLICT; } - if (row.state == IdempotencyState.COMPLETED && owner.equals(row.completedBy)) { + if (row.repeats(owner, Transition.COMPLETE)) { // The completing handle retried, as after a lost reply: the same result is a no-op, another // result is a conflict, exactly as the stores answer. return row.response.equals(response) @@ -163,8 +170,7 @@ public final class InMemoryIdempotencyStore implements IdempotencyStorePort { return IdempotencyCompleteOutcome.NOT_IN_PROGRESS; } row.response = response; - row.completedBy = owner; - row.advance(IdempotencyState.COMPLETED); + row.advance(IdempotencyState.COMPLETED, owner, Transition.COMPLETE, null); return IdempotencyCompleteOutcome.COMPLETED; } @@ -181,6 +187,11 @@ public final class InMemoryIdempotencyStore implements IdempotencyStorePort { if (row.namesAnotherOperation(owner, operationId)) { return IdempotencyFailOutcome.OPERATION_CONFLICT; } + // The failing handle retried with the same disposition, as after a lost reply. Another + // disposition is not this failure and falls through to NOT_OWNER, as in the stores. + if (row.repeats(owner, Transition.FAIL) && disposition.equals(row.applied.argument())) { + return IdempotencyFailOutcome.ALREADY_MARKED_SAME_OPERATION; + } if (!row.owner.equals(owner)) { return IdempotencyFailOutcome.NOT_OWNER; } @@ -188,7 +199,11 @@ public final class InMemoryIdempotencyStore implements IdempotencyStorePort { return IdempotencyFailOutcome.NOT_IN_PROGRESS; } boolean retryable = disposition == IdempotencyFailureDisposition.NO_EFFECT_RETRYABLE; - row.advance(retryable ? IdempotencyState.FAILED_RETRYABLE : IdempotencyState.ABANDONED); + row.advance( + retryable ? IdempotencyState.FAILED_RETRYABLE : IdempotencyState.ABANDONED, + owner, + Transition.FAIL, + disposition); return retryable ? IdempotencyFailOutcome.MARKED_RETRYABLE : IdempotencyFailOutcome.MARKED_ABANDONED; @@ -280,7 +295,7 @@ public final class InMemoryIdempotencyStore implements IdempotencyStorePort { private IdempotencyOwner owner; private IdempotencyState state = IdempotencyState.CLAIMED; private StoredResponse response; - private IdempotencyOwner completedBy; + private Applied applied; Row(RequestFingerprint fingerprint, IdempotencyClaimRequest request, long attempt, long rev) { this.fingerprint = fingerprint; @@ -293,9 +308,20 @@ public final class InMemoryIdempotencyStore implements IdempotencyStorePort { request.claimAttempt().operationId()); } - void advance(IdempotencyState next) { + /** Applies a transition from {@code from}, remembering it for a resend of the same call. */ + void advance(IdempotencyState next, IdempotencyOwner from, Transition kind, Object argument) { owner = owner.withStateRevision(owner.stateRevision() + 1); state = next; + applied = new Applied(from, kind, argument); + } + + /** + * Whether {@code caller} resends the last transition applied, from the handle it was applied + * from. Only that handle is told the transition was already applied; any other is not, the same + * attempt's handle from before a takeover included (owner decision 2026-11-06). + */ + boolean repeats(IdempotencyOwner caller, Transition kind) { + return applied != null && applied.kind() == kind && applied.from().equals(caller); } /** @@ -308,4 +334,22 @@ public final class InMemoryIdempotencyStore implements IdempotencyStorePort { && !owner.claimOperationId().equals(operationId); } } + + /** The transitions a handle can resend after a lost reply. */ + private enum Transition { + START, + RENEW, + COMPLETE, + FAIL + } + + /** + * The last transition applied to a record. + * + * @param from the handle it was applied from + * @param kind which transition + * @param argument what it decided besides the handle: the lease of a renewal, the disposition of + * a failure; {@code null} for the others + */ + private record Applied(IdempotencyOwner from, Transition kind, Object argument) {} }