From 1885bf8da7b142eac56cdbd30b376be350a17528 Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Wed, 4 Nov 2026 01:47:50 +0900 Subject: [PATCH] =?UTF-8?q?fix!:=202=EC=B0=A8=20=EC=9E=AC=EB=A6=AC?= =?UTF-8?q?=EB=B7=B0=EA=B0=80=20=EC=B0=BE=EC=9D=80=20=EA=B2=B0=ED=95=A8=20?= =?UTF-8?q?29=EA=B1=B4=EC=9D=84=20=EA=B3=A0=EC=B9=9C=EB=8B=A4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 문제: 재리뷰 수정 커밋(bbe944bd)만 다시 본 2차 재리뷰가 Important 7건, Minor 22건을 찾았다. 결과 불명 503 뒤 같은 키를 다시 보내면 Redis store가 lease 지난 실행을 넘겨받아 작업을 다시 돌렸다. replay TTL이 lease보다 짧으면 lease 도중 key가 사라져 같은 키가 두 번 실행됐다. @SystemInvoked 규칙이 매뉴얼 09장의 이벤트 소비 경로까지 막았다. V22가 outbox_event에 가장 강한 잠금을 lock_timeout 없이 걸었다. prod의 fixture- 거부가 HOSTNAME 같은 비밀 아닌 값까지 막았다. 데모 허용 목록의 test·demo 프로필로 sample이 뜨지 않았다. 기동 rule bean의 @Value 필드가 조용히 null이 됐다. 방법: 영역별 4묶음이 test-driven-development 절차로 실패 테스트를 먼저 쓰고 고쳤다. 결정은 재리뷰 README 9.1절이다. - 두 멱등성 store가 같은 계약을 따른다. lease가 지난 실행 기록은 다시 돌리지 않고 포기로 표시해 500 RECOVERY_REQUIRED로 답한다. 기록 수명은 lease + replay TTL이고, replay-ttl < processing-lease면 기동이 실패한다. - 사용자 transport만 @SystemInvoked use case에 닿지 못한다. 메시지 handler·scheduler는 새 표시 @SystemCaller(reason)를 달고 use case를 생성자에서 new로 만든다. 매뉴얼 경로를 테스트로 고정했다. - 아직 어디에도 적용되지 않은 마이그레이션(V20·V22·알림 V11~V13·sample V8_1·grpc V002·V003)에 lock_timeout 5s를 넣고, 텍스트 검사기와 실 PG lane을 더했다. 이미 배포된 24개는 동결 목록으로 둔다. - fixture- 검사는 비밀로 분류된 값만 본다. sample은 local·test·demo에서 object storage를 끈다. 기동 rule bean에 주입 멤버가 있으면 모든 프로필에서 기동이 실패한다. 리뷰어가 제안한 "static @Bean 선언 강제"는 문제없는 등록을 막고 @Value 필드는 통과시켜서 버렸다. - 이 수정 전에 move된 파일의 쿼터 차이를 찾는 쿼리와 재계산 SQL을 docs/fileserver/operations.md에 적었다. 위험·한계: lease 뒤 늦게 끝난 첫 시도의 결과는 기록되지 않는다(그 요청도 500). @SystemCaller가 없는 메시지 handler, 주입 멤버가 있는 기동 rule, working-directory 없이 ./gradlew를 부르는 workflow 명령은 파생 서비스에서 새로 실패하거나 세지지 않는다(업그레이드 안내 B-14~16·B-36·B-39). spotbugsTestFixtures는 게이트 밖이고 기준 커밋에서도 실패하며, 새 검사기의 정규식 경고 8건이 더해졌다. 실제 CI runner 실행은 확인하지 못했다(Gitea repo의 Actions가 꺼져 있다). 검증: ci, 두 빌드의 verifyStrictLaneStages --dry-run, mainCheck, :integrationCheck, -p optional-platforms ci integrationCheck performanceCheck, :performanceCheck, :releaseCheck(-PgitRevision), verify-compose-profile-contracts(18 lane) 첫 실행 통과. 쿼터 재계산 SQL은 실 PostgreSQL 16에서 차이 2행 → 0행을 확인했다. Refs: docs/reviews/2026-11-03-rereview/README.md 9·9.1·10절 BREAKING CHANGE: replay-ttl < processing-lease 설정은 기동 실패, ledger release가 claim token을 받는다, HttpBodyReadException(IOException 하위)이 본문 실패를 싣는다, 기동 rule·비밀 목록 bean의 주입 멤버 금지, verifyStrictLaneStages가 workflow 명령의 working-directory로 빌드를 정한다. --- .../object-storage-qualification.yml | 3 +- AGENTS.md | 5 +- CLAUDE.md | 6 +- ...retry-ownership-and-durable-idempotency.md | 11 + docs/capability-status.md | 2 +- docs/fileserver/operations.md | 69 +++++ docs/httpclient/configuration-reference.md | 18 +- docs/httpclient/operations.md | 5 + docs/jpa/entity-mapping-guide.md | 11 +- docs/jpa/migration-guide.md | 28 ++ docs/manual/04-persistence.md | 6 + docs/manual/05-idempotency.md | 5 + docs/manual/07-authorization.md | 35 ++- docs/manual/09-outbox-events.md | 47 ++- docs/messaging/outbox-inbox.md | 4 +- docs/registries/env-keys.yaml | 10 +- docs/registries/metrics.yaml | 10 +- docs/registries/secrets-classification.yaml | 8 +- docs/reviews/2026-11-03-rereview/README.md | 41 +++ docs/runbooks/migration-failed.md | 10 +- docs/runbooks/outbox-dead-letter.md | 27 +- docs/runbooks/outbox-publish-failed.md | 11 +- docs/runbooks/redis-capability-incident.md | 18 +- docs/upgrade/2026-11-maturity-review.md | 56 ++-- docs/web/http-contract.md | 31 +- infra/redis-sdk/acl/README.md | 9 +- src/.env.example | 4 +- src/Dockerfile.sample | 10 +- src/README.md | 3 +- .../graphql/policy/GraphQlClientPolicy.java | 19 +- .../GraphQlConnectionAssemblerTest.java | 12 + .../policy/GraphQlClientPolicyTest.java | 9 - src/adapter/inbound/messaging/CLAUDE.md | 3 + src/adapter/inbound/messaging/README.md | 25 +- src/adapter/inbound/messaging/build.gradle | 7 +- src/adapter/inbound/messaging/gradle.lockfile | 46 ++- .../InboxRedeliveryIntegrationTest.java | 44 ++- .../SystemInvokedConsumePathTest.java | 286 +++++++++++++++++ .../web/error/EnvelopeErrorController.java | 8 + .../error/EnvelopeErrorControllerTest.java | 58 ++++ src/adapter/outbound/cache-redis/README.md | 19 +- .../redis/idempotency/IdempotencyScripts.java | 61 +++- .../RedisIdempotencyStoreAdapter.java | 1 + .../RedisIdempotencyStoreAdapterTest.java | 25 +- .../RedisServerTimeScriptContractTest.java | 5 +- src/adapter/outbound/httpclient/README.md | 7 +- .../api/error/HttpBodyReadException.java | 42 +++ .../api/result/BlockingStreamingResponse.java | 10 + .../jdk/JdkStreamingClientHttpRequest.java | 21 +- .../httpclient/jdk/ReadIdleInputStream.java | 99 +++++- .../resilience/AttemptResiliencePipeline.java | 12 +- .../restclient/ClassifiedBodyInputStream.java | 13 +- .../JdkStreamingRestrictedHeadersTest.java | 69 +++++ .../jdk/ReadIdleInputStreamTest.java | 152 ++++++++++ .../restclient/StreamingBodyLifetimeTest.java | 12 +- .../provider/ses/SesEventNormalizer.java | 5 +- src/adapter/outbound/objectstorage/README.md | 6 +- ...itional-spring-configuration-metadata.json | 10 + .../outbound/persistence-jpa/CLAUDE.md | 5 +- .../outbound/persistence-jpa/README.md | 27 +- .../outbound/persistence-jpa/build.gradle | 1 + .../postgresql/IdempotencyClaimGateway.java | 21 +- .../IdempotencyTransitionGateway.java | 12 + .../outbox/jpa/JpaOutboxStore.java | 63 +++- .../outbox/jpa/entity/OutboxEventEntity.java | 22 +- ...0__outbox_event_deferred_attempt_count.sql | 14 + .../V22__outbox_event_last_failure_code.sql | 14 + ...ostgreSqlIdempotencyStoreContractTest.java | 49 ++- ...reSqlMigrationLockWaitIntegrationTest.java | 205 +++++++++++++ ...tboxDispositionRunbookIntegrationTest.java | 29 ++ .../AppliedMigrationImmutabilityTest.java | 45 +-- .../migration/MigrationLockWaitTest.java | 105 +++++++ .../persistence/migration/MigrationTrees.java | 70 +++++ .../outbox/jpa/JpaOutboxStoreTest.java | 75 +++++ .../arch/JpaArchitectureRulesTest.java | 25 ++ .../entity/MutualForeignKeyInvoiceEntity.java | 30 ++ .../entity/MutualForeignKeyOrderEntity.java | 37 +++ .../LockWaitMigrationInspectorTest.java | 89 ++++++ .../resources/db/applied-migrations.sha256 | 10 +- .../arch/SingleSidedAssociationCondition.java | 26 +- .../migration/LockWaitMigrationInspector.java | 164 ++++++++++ .../persistence-notification/CLAUDE.md | 5 +- ...otification_reconciliation_claim_lease.sql | 13 + ...notification_suppression_target_unique.sql | 13 + .../V13__notification_request_acceptance.sql | 13 + .../runtime/connection/RedisRuntimeOwner.java | 16 +- .../runtime/config/RedisLaneMetricsTest.java | 38 +++ .../InMemorySemanticRedisCommands.java | 10 +- .../outbound/support/startup/StartupRule.java | 6 +- .../support/startup/StartupRuleRunner.java | 111 ++++++- .../startup/StartupRuleRunnerTest.java | 138 +++++++++ src/app-bootstrap/CLAUDE.md | 5 +- src/app-bootstrap/README.md | 9 +- .../ApplicationBoundaryArchitectureTest.java | 101 +++++- .../ArchRuleDiscoveryContractTest.java | 3 +- .../ArchitectureViolationFixtureTest.java | 75 ++++- .../messaging/SystemCallerHandlerFixture.java | 18 ++ .../inbound/messaging/package-info.java | 9 + .../SystemCallerRestControllerFixture.java | 23 ++ .../UnmarkedSystemInvokedHandlerFixture.java | 17 ++ .../SystemCallerControllerFixture.java | 19 ++ ...SystemInvokedCallingControllerFixture.java | 6 +- .../SystemInvokedFixtureUseCase.java | 7 +- .../runtime/SecretSourceValidator.java | 188 ++++++++---- .../SecretsClassificationRegistryTest.java | 25 ++ .../runtime/SecretSourceValidatorTest.java | 63 +++- src/application-core/CLAUDE.md | 22 +- src/application-core/README.md | 9 +- .../idempotency/IdempotencyClaimOutcome.java | 12 + ...IdempotencyCompletionUnknownException.java | 17 +- .../idempotency/IdempotencyExecutor.java | 3 + .../outbox/OutboxAggregateSequencePort.java | 12 +- .../application/outbox/OutboxStorePort.java | 7 +- .../application/security/SystemCaller.java | 34 +++ .../application/security/SystemInvoked.java | 27 +- .../testkit/IdempotencyStoreContract.java | 145 ++++++++- .../testkit/DirectTransactionPort.java | 62 ++-- .../testkit/TransactionPortContract.java | 54 ++++ .../InMemoryQuotaLedgerContractTest.java | 6 + .../testkit/FakeFileQuotaService.java | 29 ++ .../testkit/FileQuotaLedgerContract.java | 41 ++- src/build-logic/README.md | 2 +- .../config/ConfigContractPlugin.java | 6 +- .../config/EnvContractVerifier.java | 90 +++++- .../buildlogic/config/VerifyEnvKeysTask.java | 3 +- .../SpringConfigConventionPlugin.java | 31 ++ .../strictlane/StrictLaneStagesExtension.java | 14 +- .../strictlane/StrictLaneStagesPlugin.java | 60 ++-- .../buildlogic/strictlane/TaskSelectors.java | 214 +++++++++++++ .../strictlane/WorkflowGradleInvocations.java | 287 ++++++++++++++---- .../config/EnvContractVerifierTest.java | 52 ++++ .../StrictLaneStagesFunctionalTest.java | 142 ++++++++- .../strictlane/TaskSelectorsTest.java | 71 +++++ .../WorkflowGradleInvocationsTest.java | 151 ++++++++- src/composition/idempotency/README.md | 10 +- .../idempotency/IdempotencySettings.java | 17 +- .../config/module-defaults/idempotency.yml | 3 + .../idempotency/IdempotencySettingsTest.java | 43 +++ src/composition/outbox-relay/README.md | 4 +- src/composition/outbox-relay/build.gradle | 18 ++ .../OutboxLongRetryRunbookContractTest.java | 61 ++++ src/composition/service-runtime/README.md | 4 +- .../PermissionEnforcementGuard.java | 20 +- .../PermissionEnforcementGuardTest.java | 40 ++- src/config/spotbugs/exclude.xml | 8 +- .../grpc/api/ledger/GrpcClaimToken.java | 7 +- .../grpc/api/ledger/GrpcOperationLedger.java | 18 +- .../ledger/GrpcOperationLedgerContract.java | 54 ++++ .../api/ledger/InMemoryOperationLedger.java | 8 +- ...rpcOperationLedgerMigrationLockWaitIT.java | 155 ++++++++++ .../ledger/GrpcOperationLedgerMigrations.java | 34 ++- .../JpaGrpcOperationLedgerContractIT.java | 5 +- .../ledger/GrpcOperationLedgerRepository.java | 21 +- .../grpc/ledger/JpaGrpcOperationLedger.java | 5 +- ...ex_grpc_operation_ledger_finished_rows.sql | 13 + ...dd_grpc_operation_ledger_claim_attempt.sql | 13 + .../GrpcOperationLedgerRepositoryTest.java | 30 +- .../KafkaDeadLetterQuarantineTest.java | 82 +++++ src/optional-platforms/build.gradle | 4 +- src/sample-portfolio/README.md | 4 +- .../SampleDemoAccountsProductionGuard.java | 8 +- .../src/main/resources/application.yml | 12 +- .../V8_1__poster_title_unique.sql | 12 + .../SampleDemoProfilesStartupSystemTest.java | 65 ++++ .../systemTest/resources/application-test.yml | 4 +- .../SampleMigrationLockWaitTest.java | 63 ++++ .../SampleDemoSecurityConfigTest.java | 5 +- .../shared/error/OperationalError.java | 2 +- 168 files changed, 5511 insertions(+), 602 deletions(-) create mode 100644 src/adapter/inbound/messaging/src/test/java/dev/caskeleton/adapter/inbound/messaging/SystemInvokedConsumePathTest.java create mode 100644 src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/error/EnvelopeErrorControllerTest.java create mode 100644 src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/api/error/HttpBodyReadException.java create mode 100644 src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/jdk/JdkStreamingRestrictedHeadersTest.java create mode 100644 src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/jdk/ReadIdleInputStreamTest.java create mode 100644 src/adapter/outbound/objectstorage/src/main/resources/META-INF/additional-spring-configuration-metadata.json create mode 100644 src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/readiness/PostgreSqlMigrationLockWaitIntegrationTest.java create mode 100644 src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/MigrationLockWaitTest.java create mode 100644 src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/MigrationTrees.java create mode 100644 src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/entity/MutualForeignKeyInvoiceEntity.java create mode 100644 src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/entity/MutualForeignKeyOrderEntity.java create mode 100644 src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/migration/LockWaitMigrationInspectorTest.java create mode 100644 src/adapter/outbound/persistence-jpa/src/testFixtures/java/dev/caskeleton/adapter/outbound/persistence/testkit/migration/LockWaitMigrationInspector.java create mode 100644 src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/allowed/adapter/inbound/messaging/SystemCallerHandlerFixture.java create mode 100644 src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/allowed/adapter/inbound/messaging/package-info.java create mode 100644 src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/messaging/SystemCallerRestControllerFixture.java create mode 100644 src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/messaging/UnmarkedSystemInvokedHandlerFixture.java create mode 100644 src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/web/controller/SystemCallerControllerFixture.java create mode 100644 src/application-core/src/main/java/dev/caskeleton/application/security/SystemCaller.java create mode 100644 src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/TaskSelectors.java create mode 100644 src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/TaskSelectorsTest.java create mode 100644 src/composition/outbox-relay/src/test/java/dev/caskeleton/composition/outbox/OutboxLongRetryRunbookContractTest.java create mode 100644 src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerMigrationLockWaitIT.java create mode 100644 src/messaging/messaging-spring-boot-starter-kafka/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantineTest.java create mode 100644 src/sample-portfolio/src/systemTest/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoProfilesStartupSystemTest.java create mode 100644 src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/bootstrap/persistence/SampleMigrationLockWaitTest.java diff --git a/.github/workflows/object-storage-qualification.yml b/.github/workflows/object-storage-qualification.yml index 8a1a43d0..da628638 100644 --- a/.github/workflows/object-storage-qualification.yml +++ b/.github/workflows/object-storage-qualification.yml @@ -1,7 +1,8 @@ name: object-storage-qualification # Only pull requests that touch object storage start the MinIO container (review BUILD-계획-B3). The -# MinIO lanes left the object storage leaf's `check`; they run in its `mainCheck` on main and here. +# MinIO lanes left the object storage leaf's `check`; they run in the root `integrationCheck` (main push +# and nightly, integration-main.yml) and here. on: pull_request: paths: diff --git a/AGENTS.md b/AGENTS.md index 0b68af7d..bf95ad38 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -82,8 +82,9 @@ root `CLAUDE.md`는 이 목록의 동기화된 요약이다. 두 문서가 어 `releaseCheck`, optional-platforms 빌드: `-p optional-platforms`로 부르는 `ci`·`integrationCheck`· `performanceCheck`·`releaseCheck`) 중 하나에 들어 있는지, 그리고 workflow의 접힌(`>`) run 블록이 빈 줄 때문에 옵션을 별개 명령으로 떼어 내지 않는지 검사한다. 두 빌드가 각자 검사를 돌린다. `-x`로 뺀 - lane, `continue-on-error` 아래 명령, 상태 함수(`always()`·`success()`·`!cancelled()`)가 아닌 `if:` - 아래 명령은 실행으로 세지 않는다. + lane(Gradle처럼 축약 이름도 맞춰 본다), `continue-on-error` 아래 명령, 상태 함수(`always()`·`success()`· + `!cancelled()`)가 아닌 `if:` 아래 명령, `workflow_dispatch`로만 시작하는 workflow의 명령은 실행으로 세지 + 않는다. 명령이 어느 빌드를 실행하는지는 `working-directory`와 `-p`를 합친 디렉터리로 정한다. 작업 파일의 소유 leaf는 registry의 `source_path`로 판단하고 가장 가까운 `src/**/CLAUDE.md`를 함께 읽는다. focused test는 registry의 `gradle_path`에서 diff --git a/CLAUDE.md b/CLAUDE.md index facf5c12..fb3b1eca 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -40,8 +40,10 @@ If this summary drifts from `AGENTS.md`, `AGENTS.md` wins and this summary must `releaseCheck` and the named repository-wide gates). `verifyStrictLaneStages --dry-run` (pull request; the main build and, as `-p optional-platforms`, the optional build each run their own) fails when a non-`explicitOnly` strict lane is in no stage a workflow executes — a lane the - command removes with `-x` and a command under `continue-on-error` or a non-status `if:` do not - count — or a folded workflow `run:` block splits an option off its command. + command removes with `-x` (abbreviations matched as Gradle does), a command under + `continue-on-error` or a non-status `if:`, and a `workflow_dispatch`-only workflow do not count; + the build a command runs is its `working-directory` joined with `-p` — or a folded workflow + `run:` block splits an option off its command. Commit policy is `human-only`: agents do not stage, commit, amend, or push implementation changes. diff --git a/docs/adr/ADR-GRPC-004-retry-ownership-and-durable-idempotency.md b/docs/adr/ADR-GRPC-004-retry-ownership-and-durable-idempotency.md index 7307c654..78984639 100644 --- a/docs/adr/ADR-GRPC-004-retry-ownership-and-durable-idempotency.md +++ b/docs/adr/ADR-GRPC-004-retry-ownership-and-durable-idempotency.md @@ -83,6 +83,17 @@ load-check-save let two commits that both read `IN_PROGRESS` both write; now exa A claim whose conflicting row is purged between insert and read retries the insert once instead of failing the call. +`release(identity, claimToken)` is fenced the same way (re-review 2026-11-03 round 2, RR-01-15): it is a +conditional `DELETE … WHERE state = 'IN_PROGRESS' AND claim_attempt = :attempt` and returns whether it +deleted. It used to delete by key alone, so any caller could erase the claim of the attempt that held +it, or a finished record a retry replays. The counter lives in the row, so a deleted row's key starts +again at attempt 1: the token separates the attempts of one row, not attempts across a delete. Both +deletions are limited accordingly — release only for a claim whose owner is known to be gone, purge +only for rows finished longer ago than any retry. Rejected: dropping `release` from the port +(`markFailed` then `reclaimFailed` keeps the counter, but a deployment that must free a key for a +different request has no other path), and a tombstone row that keeps the counter (a second state to +purge, for a window the two limits already close). + Chosen over the claim moment as the token (`WHERE claimed_at = :claimedAt`), which needs no column: PostgreSQL stores microseconds while `Instant` carries nanoseconds, so the token a caller holds and the one stored can differ by rounding, and two processes' clocks can give two attempts the same diff --git a/docs/capability-status.md b/docs/capability-status.md index 7b701ff0..b70e0120 100644 --- a/docs/capability-status.md +++ b/docs/capability-status.md @@ -39,7 +39,7 @@ lanes and CI; it is not a deployment artifact. | GraphQL transport | `backend.graphql.enabled` / `APP_GRAPHQL_ENABLED`, `true` | yes | yes (one line: `runtimeOnly project(':adapter:inbound:graphql')`) | leaf-level | On when present; `false` also excludes Spring GraphQL's own auto-configuration, so no `/graphql` (`GraphQlCapabilityActivationTest`). `graphqlTransportQualificationTest` runs the HTTP boundary. The deployment-mode/runtime-profile rule is the module's own `GraphQlPlatformStartupValidator`. The default jar does not ship GraphQL, so a compose lane that exercises it runs the qualification jar. | | Object storage | `app.object-storage.enabled` / `APP_OBJECTSTORAGE_ENABLED`, `true` | yes | yes (one line: `runtimeOnly project(':adapter:outbound:objectstorage')`) | MinIO only | Own `ObjectStorageAutoConfiguration` (was bound through bootstrap's properties scan); on when present, `false` leaves only the explicit legacy seam (`ObjectStorageAutoConfigurationTest`). `app-bootstrap:integrationTest` assembles it against digest-pinned MinIO. | | gRPC platform | — (build-level) | yes | **no** | yes | Absent from the main build entirely: `optional-platforms` is a separate Gradle build, so the switch is the `include`, not an environment variable. `-p optional-platforms ci` runs on every PR and `integrationCheck` certifies the operation ledger against real PostgreSQL. | -| gRPC transport (`adapter:inbound:grpc`) | `ca-skeleton.grpc.server.enabled` (no `APP_*` row), `true` | yes | yes (one line: `runtimeOnly project(':adapter:inbound:grpc')`) | leaf-level | Own `AutoConfiguration.imports` (`GrpcServerConfig`); on when present, and the loopback plaintext listener still needs `ca-skeleton.grpc.server.allow-insecure-local=true` (`GrpcSafeActivationTest`, `grpcTransportQualificationTest`). | +| gRPC transport (`adapter:inbound:grpc`) | `app.grpc.server.enabled` / `APP_GRPC_SERVER_ENABLED`, `true` | yes | yes (one line: `runtimeOnly project(':adapter:inbound:grpc')`) | leaf-level | Own `AutoConfiguration.imports` (`GrpcServerConfig`); on when present, and the loopback plaintext listener still needs `app.grpc.server.allow-insecure-local=true` (`GrpcSafeActivationTest`, `grpcTransportQualificationTest`). | | WebSocket transport | `app.websocket.enabled` / `APP_WEBSOCKET_ENABLED`, `true` | yes | yes (one line: `runtimeOnly project(':adapter:inbound:websocket')`) | — | Own `AutoConfiguration.imports` (`WebSocketConfig`), on in a servlet app when present (`WebSocketAutoConfigurationTest`); `ConditionalTransportBuildContractTest` asserts it has no default production edge. | | Web Advanced library (`adapter:inbound:web-advanced`) | — (build-level; three `backend.web.advanced.*.enabled` switches inside) | yes | **no** | — | No composition root depends on it; `app-bootstrap` takes it `testRuntimeOnly` for the ArchUnit import. No real-provider lane: `webAdvancedTest` (the `web-advanced` tag, run by `release.yml` → `web-stable-release-gate`) is unit and context-runner level. The Nginx lane went with the proxy library on 2026-09-25. | diff --git a/docs/fileserver/operations.md b/docs/fileserver/operations.md index 2c24969a..cc7a14b0 100644 --- a/docs/fileserver/operations.md +++ b/docs/fileserver/operations.md @@ -126,3 +126,72 @@ curl -s -X POST "$ADMIN/internal/fileserver/uploads:cleanup?maxItems=500&maxByte If the deferrals are all `ACTIVE_WRITER_LEASE`, the backlog resolves itself as those uploads expire. Never delete staging content to clear a backlog: an upload that is mid-flight will corrupt. + +## Namespace quota drift + +**Signal** — a namespace's usage (`fileserver.quota` rejections, or the drift query below) disagrees +with the bytes its files actually hold. The known cause is history: before the 2026-11 maturity +review, `move` relocated a file without moving its quota. The bytes of every file moved before that +fix are still booked on the namespace it left, and deleting such a file reclaims from the namespace +it now lives in, where the reclaim stops at zero and leaves the old booking behind. A derived +service that moved files across namespaces before taking the fix has this drift; one that never did +should see none. + +What the ledger must equal: quota is committed when a file is published and reclaimed when cleanup +deletes its content, so a namespace's committed bytes are the sizes of its published files that are +not yet `DELETED`. The query compares the two and changes nothing. + +```sql +WITH booked AS ( + SELECT scope_value AS namespace, sum(committed_bytes) AS booked_bytes + FROM fs_quota_reservation + WHERE scope_type = 'namespace' AND status = 'COMMITTED' + GROUP BY scope_value), +held AS ( + SELECT namespace, sum(actual_size) AS held_bytes + FROM fs_file + WHERE published_at IS NOT NULL AND state <> 'DELETED' + GROUP BY namespace) +SELECT coalesce(b.namespace, h.namespace) AS namespace, + coalesce(b.booked_bytes, 0) AS booked_bytes, + coalesce(h.held_bytes, 0) AS held_bytes, + coalesce(b.booked_bytes, 0) - coalesce(h.held_bytes, 0) AS drift_bytes + FROM booked b + FULL JOIN held h ON h.namespace = b.namespace + WHERE coalesce(b.booked_bytes, 0) <> coalesce(h.held_bytes, 0); +``` + +A positive `drift_bytes` is booked capacity no file holds (the namespace a file left); a negative one +is held content nothing booked (the namespace it arrived in). While uploads, copies, moves or +cleanup run, a row can show a drift that only means work in flight, so judge drift with the +fileserver's writes stopped. + +To correct a namespace, stop every fileserver write to it first — uploads, copy, move, delete, and +the cleanup worker (`APP_FILESERVER_PLATFORM_CLEANUP_ENABLED=false`) — because the statement below +recomputes from `fs_file` and a publish or reclaim that lands between its two steps would be lost. +It zeroes the namespace's committed rows and books the recomputed total as one row; the per-row +split has no meaning once reclaims have drawn rows down, and enforcement only ever reads the sum. +`RESERVED` rows (uploads in progress) are left alone. + +```sql +BEGIN; +SET LOCAL lock_timeout = '5s'; +UPDATE fs_quota_reservation + SET committed_bytes = 0, version = version + 1, updated_at = now() + WHERE scope_type = 'namespace' AND scope_value = :'namespace' + AND status = 'COMMITTED' AND committed_bytes > 0; +INSERT INTO fs_quota_reservation ( + reservation_id, scope_type, scope_value, reserved_bytes, committed_bytes, + expires_at, status, version, created_at, updated_at) +SELECT gen_random_uuid(), 'namespace', :'namespace', held, held, + now(), 'COMMITTED', 0, now(), now() + FROM (SELECT coalesce(sum(actual_size), 0) AS held + FROM fs_file + WHERE namespace = :'namespace' + AND published_at IS NOT NULL AND state <> 'DELETED') AS recomputed + WHERE held > 0; +COMMIT; +``` + +Run it with `psql -v namespace=`, then run the drift query again: the namespace must be gone +from its result before writes resume. diff --git a/docs/httpclient/configuration-reference.md b/docs/httpclient/configuration-reference.md index 206d81f7..7172e5c2 100644 --- a/docs/httpclient/configuration-reference.md +++ b/docs/httpclient/configuration-reference.md @@ -106,9 +106,12 @@ instead, so a long download is not cut at an arbitrary total (design §15.3). Th transports. The JDK transport keeps its exchange-wide timeout off a streamed body and bounds each read with its own `read-idle` timer, because the JDK client has no socket read timeout; before review 2026-11-03 RR-04-03 every JDK download longer than `response-header` was cut mid-body. A body -failure after hand-over is thrown as `HttpResponseTimeoutException` (a gap longer than `read-idle`) -or `HttpResponseTruncatedException` (the connection ended early), stage `RESPONSE_BODY`, evidence -`PARTIAL_RESPONSE`. +failure after hand-over is thrown from `read` as `HttpBodyReadException` — an `IOException`, as +`InputStream` promises, so `catch (IOException)` still sees it — whose `failure()` is +`HttpResponseTimeoutException` (a wait longer than `read-idle`) or `HttpResponseTruncatedException` +(the connection ended early), stage `RESPONSE_BODY`, evidence `PARTIAL_RESPONSE` (re-review +2026-11-04 RR-03-14). On the JDK transport `read-idle` bounds each read's wait, `skip` included, and +time the caller spends between reads does not count (RR-03-12). ### Removed settings @@ -140,6 +143,15 @@ RR-04-01): | The caller's own deadline (`HttpOperation.deadline()`, when earlier than `total-call`) | Not recorded; the permission is given back | The caller ran out of time. Counting it opened the circuit on a healthy upstream whenever callers passed short budgets | | The profile's `timeout.total-call` | Recorded as a failure | `total-call` is the operator's statement of how long a healthy upstream may take; a slow-drip upstream that never trips a per-gap timeout is seen only through it | +The first row has a blind spot (re-review 2026-11-04 RR-03-16). The attempt is let off whenever the +caller's deadline was the earlier one and had passed when the attempt failed — the code does not ask +whether the upstream was still answering within the profile's limits. If every caller of a profile +passes a budget shorter than `connect` or `response-header`, an upstream that hangs (drops packets, +never answers) ends every attempt at the caller's deadline first, so no attempt is recorded and the +circuit never opens; every call waits out its own budget. A failure that comes before the caller's +deadline — a refused connection — is still counted. For such a profile, set `connect` and +`response-header` shorter than the shortest caller budget. + | Property | Default | Meaning | |---|---|---| | `failure-rate-threshold` | `50` | Percent of failed calls in the window that opens the circuit, 1–100 | diff --git a/docs/httpclient/operations.md b/docs/httpclient/operations.md index b4bfc59d..bf95cfae 100644 --- a/docs/httpclient/operations.md +++ b/docs/httpclient/operations.md @@ -111,6 +111,11 @@ evidence, replayability, stage, retryability, attempt, elapsed, remaining deadli | `HttpUnknownTransportException` | the engine failed in a way no category above describes; the metadata evidence, not the type, says whether it was sent | never | | `HttpAmbiguousExecutionException` | a non-idempotent request was sent and the outcome is unknown | never — reconcile instead | +A streamed body that fails after hand-over is the one place these arrive wrapped: `read` throws +`HttpBodyReadException`, an `IOException` as `InputStream` promises, and its `failure()` is the +`HttpResponseTimeoutException` or `HttpResponseTruncatedException` above (re-review 2026-11-04 +RR-03-14). + ## Traces ```text diff --git a/docs/jpa/entity-mapping-guide.md b/docs/jpa/entity-mapping-guide.md index 4d3aa55e..8c3b39d7 100644 --- a/docs/jpa/entity-mapping-guide.md +++ b/docs/jpa/entity-mapping-guide.md @@ -85,8 +85,15 @@ reason those are ID references. bidirectional pair, on `@Entity`, `@MappedSuperclass` and `@Embeddable` types: an association that declares `mappedBy` (only that inverse half is reported), and two entities whose associations point at each other without `mappedBy` (both halves are reported — the `@JoinColumn` form above, review -RR-03-04). A class that links to itself is a pair only when a second member points back the other -way (`@ManyToOne parent` with `@OneToMany children`). `JpaProductionArchitectureTest` (app-bootstrap +RR-03-04). That last check also refuses two entities that each hold their own foreign key to the +other (`Order.@OneToOne latestInvoice` with `Invoice.@ManyToOne order`). It is not one link mapped +twice, and the message says so; it is refused on purpose, because the two tables then reference each +other: neither row can be inserted or deleted first, one key has to be nullable and written by a +second `UPDATE`, and the entity graph is a cycle. Keep the association on one side and give the other +an ID or a query (re-review round 2, RR-02-15). A class that links to itself is a pair only when a +second member points back the other way (`@ManyToOne parent` with `@OneToMany children`); two +`@ManyToOne` links of a class to itself (a manager and a mentor) pass, because both keys point into +the one table and there are no two tables to order. `JpaProductionArchitectureTest` (app-bootstrap `architectureTest`) runs it over the production graph and `SampleArchitectureTest` over the sample. The failure message names the fields and these alternatives. diff --git a/docs/jpa/migration-guide.md b/docs/jpa/migration-guide.md index 550e4b03..d79e63c8 100644 --- a/docs/jpa/migration-guide.md +++ b/docs/jpa/migration-guide.md @@ -32,6 +32,34 @@ reason, and an approval (design §8.4). Only Flyway's structured error codes reach the exception. Its messages embed the script path and part of the failing statement. +## Bound the lock wait + +A migration that locks a table already in use starts with `SET LOCAL lock_timeout = '5s';`, before +the first statement that takes the lock. `ALTER TABLE` takes the strongest lock there is, which +conflicts even with a plain read; a non-concurrent `CREATE INDEX` takes one that stops every write; +`UPDATE` and `DELETE` wait on the rows other transactions hold. Unbounded, the migration waits for +the oldest open transaction that touched the table — an operator's forgotten `BEGIN` in psql has no +deadline — and every read and write that arrives meanwhile queues behind the migration, holding a +pool connection while it waits. With the bound, the migration fails after five seconds, Flyway +rolls it back whole, the database stays on the version before it and the new release does not +start: a failed deploy instead of an outage. Deploy again when the transaction has ended. + +`LOCAL`, because a plain `SET` outlives the migration on the pooled connection Flyway borrowed and +changes every statement that later runs on it. A statement on a table the same migration created a +moment earlier waits for nothing and needs no bound. + +Enforced, not remembered: `LockWaitMigrationInspector` (persistence-jpa testkit) reads a script +and names every statement that locks an existing table before the bound. `MigrationLockWaitTest` +runs it over every tree this leaf's database runs, `SampleMigrationLockWaitTest` over the sample's +stream; a service stream copies the latter with its own directory. The migrations released before +the rule wait unbounded and stay that way (they are applied, and applied files are never edited); +the test lists them and refuses any new one. `PostgreSqlMigrationLockWaitIntegrationTest` +(`postgresqlMigrationLockWaitIntegrationTest`) and the gRPC ledger's +`GrpcOperationLedgerMigrationLockWaitIT` show on real PostgreSQL that a bounded migration gives up +within the bound while another transaction holds the table, and leaves the version below it. The +rule came from V21 having the bound and V22, in the same commit, not having it (re-review +2026-11-03 round 2, RR-02-09). + ## Concurrent index builds `CREATE INDEX CONCURRENTLY` cannot run inside a transaction block, and Flyway wraps migrations in diff --git a/docs/manual/04-persistence.md b/docs/manual/04-persistence.md index 4d5faddd..ec7e99dd 100644 --- a/docs/manual/04-persistence.md +++ b/docs/manual/04-persistence.md @@ -128,6 +128,12 @@ capability 스트림 db/migration/jpa/ 각자 histor (리뷰 README 13.2, 09-02). - 기존 데이터가 새 제약을 어기면 마이그레이션이 이유를 남기고 멈추게 쓴다(`V8_1`의 `DO $$` 블록). 조용히 지우지 않는다. +- 운영 중인 테이블을 잠그는 문장(`ALTER TABLE`, `CONCURRENTLY` 없는 `CREATE INDEX`, `UPDATE`·`DELETE`)이 있으면 + 그 앞에 `SET LOCAL lock_timeout = '5s';`를 둔다(`V8_1`, `V22`). 상한이 없으면 그 테이블을 건드린 열린 + 트랜잭션 하나(`ALTER TABLE`이면 읽기만 한 것도) 뒤에서 마이그레이션이 끝없이 기다리고, 그동안 뒤에 온 그 + 테이블의 쓰기(`ALTER TABLE`이면 읽기까지)가 모두 마이그레이션 뒤에 줄을 선다. 상한이 있으면 5초 뒤 실패·롤백되고 새 버전이 뜨지 않을 뿐이다. 자기 스트림에도 + `SampleMigrationLockWaitTest`처럼 `LockWaitMigrationInspector`(persistence-jpa testkit) 검사를 둔다 + (`docs/jpa/migration-guide.md` "Bound the lock wait"). ## 지금 이해하면 되는 것 diff --git a/docs/manual/05-idempotency.md b/docs/manual/05-idempotency.md index 1883f5a7..0effa752 100644 --- a/docs/manual/05-idempotency.md +++ b/docs/manual/05-idempotency.md @@ -54,6 +54,11 @@ DB 커밋 중 연결 끊김 ───────────────── | 500 `IDEMPOTENCY_RECOVERY_REQUIRED` | 이전 시도 효과를 증명 못함 | 재전송·재실행 모두 위험. 업무 데이터로 확인 | | 500 `DB_COMPLETION_UNKNOWN`, `retryable=false` | DB 커밋 중 연결 끊김. 커밋됐을 수도 있음 | 키 없이 재시도 금지. KEYED면 같은 키로 재전송하면 서버가 판정 | +같은 키 재전송은 작업을 다시 돌리지 않는다. 첫 시도가 실행을 시작한 뒤 processing lease(기본 30s)가 +지나면 두 저장소 모두 그 기록을 넘겨주지 않고 500 `IDEMPOTENCY_RECOVERY_REQUIRED`로 답한다. 넘겨받는 것은 +실행 전 claim과 재시도 가능한 실패뿐이다. 기록은 lease + `replay-ttl` 동안 남고, `replay-ttl`이 lease보다 +짧으면 기동이 실패한다(`IdempotencyStoreContract`, 재리뷰 2026-11-04 RR-01-11·RR-03-11). + 409를 `retryable=true`로 바꾼 이유(리뷰 02-01 결정): 같은 키 재전송은 서버가 안전하게 판정하므로 다시 보내는 것이 맞고, 새 키로 보내면 서버가 막지 못한다. 짝 프론트 분류기도 이 표를 따른다. diff --git a/docs/manual/07-authorization.md b/docs/manual/07-authorization.md index 4aef0641..a6b8443a 100644 --- a/docs/manual/07-authorization.md +++ b/docs/manual/07-authorization.md @@ -64,18 +64,34 @@ JWT의 `realm_access.roles`와 **자기 client**(`app.security.audience`)의 `re ## 시스템이 부르는 쓰기: `@SystemInvoked` -스케줄러·relay처럼 사람이 아니라 시스템이 부르는 쓰기 유스케이스는 `@RequiresPermission` 대신 +스케줄러·relay·메시지 소비자처럼 사람이 아니라 시스템이 부르는 쓰기 유스케이스는 `@RequiresPermission` 대신 `@SystemInvoked(reason = "...")`를 단다(`src/application-core/src/main/java/dev/caskeleton/application/security/SystemInvoked.java`). -스케줄러 스레드에는 인증이 없어 권한 선언은 장식이 되기 때문이다. 이 표시가 붙은 유스케이스는 **권한 -검사가 없는 쓰기**이므로 전송 계층이 닿으면 안 된다. 그래서 두 가지가 막는다. +스케줄러·소비 스레드에는 인증(`Authentication`)이 없어서 권한 선언은 매번 거부되거나 아예 검사되지 않는 +장식이 되기 때문이다. 이 표시가 붙은 유스케이스는 **권한 검사가 없는 쓰기**다. 그래서 사용자 요청이 +닿으면 안 된다. -| 막는 것 | 어디 | -| --- | --- | -| bean으로 등록하면 기동 실패. 조립 모듈이 `new`로 만들어 스케줄러에만 준다(예: `composition:outbox-relay`) | `src/composition/service-runtime/src/main/java/dev/caskeleton/composition/runtime/authorization/PermissionEnforcementGuard.java` | -| inbound adapter가 그 클래스를 직접 쓰면 빌드 실패 | `INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES`(`ApplicationBoundaryArchitectureTest`) | +```text +사용자 요청 ─▶ web·GraphQL·gRPC·WebSocket ──✗──▶ @SystemInvoked 유스케이스 (빌드 실패) +시스템 ─▶ 스케줄러(조립 모듈이 new로 넘김) ─────▶ @SystemInvoked 유스케이스 +시스템 ─▶ 메시지 handler(@SystemCaller, 직접 new) ─▶ @SystemInvoked 유스케이스 +어디서든 ─▶ @SystemInvoked 유스케이스를 bean으로 등록 ──✗ (기동 실패) +``` + +- 사용자 전송: 사용자가 보낸 요청이 들어오는 inbound adapter다(web·GraphQL·gRPC·WebSocket). +- 시스템 호출자: broker 전달이나 시간(tick)이 깨우는 코드다. 사용자 요청은 여기로 들어오지 않는다. + +| 규칙 | 무엇을 막나 | 어디 | +| --- | --- | --- | +| bean 금지 | `@SystemInvoked` 유스케이스가 bean이면 기동 실패. 호출자가 `new`로 만들어 혼자 들고 있는 것만 허용한다 | `src/composition/service-runtime/src/main/java/dev/caskeleton/composition/runtime/authorization/PermissionEnforcementGuard.java` | +| 시스템 호출자만 | inbound adapter가 그 클래스에 의존하면 빌드 실패. `@SystemCaller(reason = "...")`를 단 클래스만 예외 | `ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES`(`ApplicationBoundaryArchitectureTest`) | +| 표시 검사 | `@SystemCaller`는 `..adapter.inbound.messaging..`·`..adapter.inbound.scheduler..`에만 있고, 사용자 전송 타입(controller·GraphQL·STOMP mapping·gRPC·servlet)에 의존하지 않는다 | `SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS`(같은 파일) | + +규칙은 "누가 닿으면 안 되나"가 아니라 "누가 닿아도 되나"를 적는다. 그래서 나중에 새 전송이 생겨도 따로 +고치지 않아도 막힌다. 메시지 소비자의 작업 순서는 09장 「소비」 4단계다. D4 실패(`MUTATING_USE_CASES_DECLARE_REQUIRED_PERMISSION`)를 끄려고 이 표시를 쓰지 않는다. 사용자가 부르는 -쓰기라면 권한을 선언하고 역할에 매핑한다. +쓰기라면 권한을 선언하고 역할에 매핑한다. 같은 쓰기를 사용자도 시스템도 부르면 유스케이스를 둘로 나눈다 +(사용자용 `@RequiresPermission`, 시스템용 `@SystemInvoked`). 업무 규칙은 domain에 두고 둘이 같이 쓴다. ## web 없는 조립 @@ -86,6 +102,7 @@ web이 빠진 조립(gRPC 전용·배치 전용)에는 `@RequiresPermission`을 ## 지금 이해하면 되는 것 - 쓰기 = `@RequiresPermission` + non-final + bean + 역할 매핑. -- 시스템만 부르는 쓰기 = `@SystemInvoked` + bean 아님 + 전송에서 안 닿음. +- 시스템만 부르는 쓰기 = `@SystemInvoked` + bean 아님 + 사용자 전송에서 안 닿음. 부르는 쪽은 스케줄러 + 조립이나 `@SystemCaller` 메시지 handler뿐이다. - 객체 판정은 유스케이스 안, 남의 것은 404. - 역할은 자기 client 것만 읽는다. diff --git a/docs/manual/09-outbox-events.md b/docs/manual/09-outbox-events.md index 753781b4..61b75b94 100644 --- a/docs/manual/09-outbox-events.md +++ b/docs/manual/09-outbox-events.md @@ -73,7 +73,9 @@ draft는 `IntegrationEventDrafts.draft(contract, eventId, aggregate, order, occu 시각(벽시계)은 쓰지 않는다 — 서버 시계가 어긋나면 순서가 뒤집힌다(리뷰 06-07). 벽시계 번호를 쓰던 서비스가 포트로 옮기면 V21 floor 덕분에 새 번호는 옛 번호 뒤에서 시작한다. 그래서 첫 번호가 1이 아니라 약 1.7×10¹⁵일 수 있다. 소비자는 번호를 "마지막으로 본 값보다 큰가"로만 비교하고 연속성을 - 가정하지 않는다. + 가정하지 않는다. 비교는 aggregate identity(tenant scope 포함) 단위다. tenant scope를 바꾸는 배포는 옛 + 버전 인스턴스를 모두 멈춘 뒤 새 버전을 띄운다 — 섞여 돌면 새 scope의 번호가 옛 버전이 방금 쓴 번호보다 + 작을 수 있다(runbook `outbox-dead-letter.md` §8, 재리뷰 2차 RR-02-10). ## relay와 실패 @@ -112,17 +114,47 @@ Kafka ─▶ platform consumer ─▶ (INBOX_TRANSACTIONAL이면) InboxMessageHa 적용한다. 3. destination을 선언한다: `guarantees.external-side-effect: INBOX_TRANSACTIONAL`, `consumer.group`, `retry.mode`·`retry.max-attempts`, `dlq.destination`. 예: `src/adapter/inbound/messaging/README.md`. -4. handler(`MessageHandler`)를 `adapter:inbound:messaging`에 둔다. delivery 하나를 use case 호출 - 하나로 바꾸고 업무 규칙은 use case에 둔다. use case는 평소처럼 `tx.inWrite`로 쓴다. 바깥 inbox - 트랜잭션에 합류한다. +4. 쓰기 use case와 handler를 만든다. 소비 스레드에는 인증이 없어서 `@RequiresPermission` use case는 + 매번 거부된다. 그래서 handler가 부르는 쓰기 use case는 `@SystemInvoked`이고, handler는 그 use case를 + **직접 `new`로 만들어 혼자 들고 있는** `@SystemCaller`다(07장 「시스템이 부르는 쓰기」). 이 방법 하나만 + 기동 가드와 빌드 규칙을 모두 통과한다. + - use case: `@SystemInvoked(reason = ...)` + `@UseCaseCapability(WRITE, …, WRITE_REPOSITORY)`. 평소처럼 + `tx.inWrite`로 쓰고, 바깥 inbox 트랜잭션에 합류한다. 업무 규칙은 여기에 둔다. + - handler(`MessageHandler`): `adapter:inbound:messaging`(또는 서비스의 `..adapter.inbound.messaging..` + 패키지)에 두고 `@SystemCaller(reason = ...)`를 단다. 생성자가 port를 받아 use case를 만든다. + delivery 하나를 use case 호출 하나로 바꾼다. + - bean은 handler 하나다. use case를 bean으로 등록하면 `PermissionEnforcementGuard`가 기동을 막는다. + 표시 없이 use case에 의존하면 `ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES`가 빌드를 막는다. + + ```java + @SystemCaller(reason = "order-events 소비자: platform consumer 스레드만 부른다") + public final class OrderCreatedHandler implements MessageHandler { + private final ProjectOrderUseCase useCase; // @SystemInvoked, bean 아님 + + public OrderCreatedHandler(TransactionPort tx, OrderProjectionPort projections) { + this.useCase = new ProjectOrderUseCase(tx, projections); + } + // destination(), handle(delivery) → useCase.project(...) + } + + @Bean // 이 leaf의 설정 클래스 + OrderCreatedHandler orderCreatedHandler(TransactionPort tx, OrderProjectionPort projections) { + return new OrderCreatedHandler(tx, projections); + } + ``` + + 같은 쓰기를 사용자 요청(HTTP 등)도 부르면 use case를 둘로 나눈다. 사용자용은 `@RequiresPermission`, + 소비용은 `@SystemInvoked`이고 업무 규칙은 domain에 둔다. payload는 다른 서비스가 보낸 입력이므로 use + case가 검증한다 — 권한 검사가 없다는 것이지 검증이 없다는 것이 아니다. 5. payload 타입을 정한다. 템플릿 relay가 보낸 메시지의 payload는 canonical envelope JSON 전체다 (`eventId`, `payload` 등). handler의 record는 그 모양이다. 등록 코드는 쓰지 않는다: handler의 `destination()`(계약 id, record class)과 `schemaVersions()`(읽는 payloadVersion, 기본 v1)가 decode 목록이 된다. 발행 쪽이 v2를 보내기 시작하면 handler가 `schemaVersions()`에 v2를 더한다 — 선언하지 않은 버전은 `SCHEMA_VERSION_NOT_REGISTERED`로 DLQ에 간다. -6. 테스트 본보기: `InboxRedeliveryIntegrationTest`(lane `messagingInboxRedeliveryIntegrationTest`, 실제 - Kafka + PostgreSQL). offset을 되감아 다시 전달해도, 처리 중 예외 뒤 재전달돼도 효과와 inbox 행이 - 하나인지 본다. +6. 테스트 본보기: `SystemInvokedConsumePathTest`(단위, 4단계 모양이 권한 가드 아래에서 기동하고 delivery가 + use case에 한 번 닿는지, use case를 bean으로 두면 기동이 막히는지)와 `InboxRedeliveryIntegrationTest` + (lane `messagingInboxRedeliveryIntegrationTest`, 실제 Kafka + PostgreSQL, 같은 handler 모양). offset을 + 되감아 다시 전달해도, 처리 중 예외 뒤 재전달돼도 효과와 inbox 행이 하나인지 본다. ### 판단 지점 @@ -162,3 +194,4 @@ Kafka ─▶ platform consumer ─▶ (INBOX_TRANSACTIONAL이면) InboxMessageHa - 계약(id·버전·schema·해시)을 먼저 정하고 계약 bean을 등록한다. - 전달은 적어도 한 번. 소비자는 inbox(`INBOX_TRANSACTIONAL`)로 예약과 업무 쓰기를 한 트랜잭션에 묶는다. - 실패한 처리는 예약까지 롤백되어 재전달에서 다시 처리된다. inbox 보존 기간은 재전달 창보다 길게. +- 소비 handler = `@SystemCaller` + `@SystemInvoked` use case를 직접 `new`. use case는 bean이 아니다. diff --git a/docs/messaging/outbox-inbox.md b/docs/messaging/outbox-inbox.md index 08f18189..1567cc93 100644 --- a/docs/messaging/outbox-inbox.md +++ b/docs/messaging/outbox-inbox.md @@ -154,7 +154,9 @@ consume 계약은 `MessageHandler` 하나다. handler를 bean으로 등록하 실제 왕복·중복 제거·DLQ는 `MessagingKafkaConsumeQualificationTest`(Kafka + PostgreSQL Testcontainers)가 확인한다. 애플리케이션 경로(앱 outbox relay가 발행 → `adapter:inbound:messaging`이 조립한 inbox와 JPA 트랜잭션 → handler)의 중복 전달·실패 재처리는 `InboxRedeliveryIntegrationTest`(lane -`messagingInboxRedeliveryIntegrationTest`)가 확인한다. +`messagingInboxRedeliveryIntegrationTest`)가 확인한다. 애플리케이션 handler가 쓰기 use case를 부르는 방법 +(`@SystemCaller` handler가 `@SystemInvoked` use case를 직접 만든다)은 `docs/manual/09-outbox-events.md` +「소비」 4단계다. ### 복합 키 diff --git a/docs/registries/env-keys.yaml b/docs/registries/env-keys.yaml index 61e9ea02..85391be2 100644 --- a/docs/registries/env-keys.yaml +++ b/docs/registries/env-keys.yaml @@ -1601,7 +1601,9 @@ env_keys: - name: APP_IDEMPOTENCY_TTL # app.capabilities.idempotency.replay-ttl: how long a completed response stays - # replayable. One value for every provider; IdempotencySettings refuses more than 72h at startup. + # replayable. One value for every provider; IdempotencySettings refuses more than 72h, and less + # than APP_IDEMPOTENCY_PROCESSING_LEASE, at startup (re-review 2026-11-04 RR-03-11). Each record + # lives for the processing lease plus this value in both stores. type: duration default: 24h allowed_values: null @@ -1609,7 +1611,7 @@ env_keys: required: false reload_policy: restart-only owner_branch: feature-rate-limit-idempotency-contract - validation: spring_duration_shorthand_le_72h + validation: spring_duration_shorthand_le_72h_ge_processing_lease compatibility_impact: behavior-change required_test: idempotency-contract:ttl-applied @@ -1663,6 +1665,10 @@ env_keys: required_test: redis-idempotency-contract:key-namespace - name: APP_IDEMPOTENCY_PROCESSING_LEASE + # app.capabilities.idempotency.processing-lease: how long one attempt holds its claim. Longer + # than the in-flight Retry-After (2s), at most 1h, and not longer than APP_IDEMPOTENCY_TTL. A + # claim whose lease runs out is taken over only if it never started; an execution whose lease + # ran out is answered IDEMPOTENCY_RECOVERY_REQUIRED and never run again. type: duration default: 30s allowed_values: null diff --git a/docs/registries/metrics.yaml b/docs/registries/metrics.yaml index 043fccf4..a3ba89d8 100644 --- a/docs/registries/metrics.yaml +++ b/docs/registries/metrics.yaml @@ -390,7 +390,10 @@ metrics: allowed_values: [PENDING, IN_FLIGHT, PUBLISHED, FAILED, DEAD, PARKED] percentiles: null histogram_buckets: null + # DEAD > 0: the claim moves a row it will not hand out to DEAD without a relay result, so + # outbox.publisher.published.total{outcome=DEAD} misses it (re-review round 2, RR-02-12). alert_severity_thresholds: + p1: "DEAD > 0 (runbook://outbox/dead-letter)" p2: "PENDING size growing for 10m" owner_branch: feature-domain-event-outbox-contract log_field_mapping: [status] @@ -550,8 +553,11 @@ metrics: - name: outcome cardinality_limit: 2 allowed_values: [admitted, refused] - percentiles: [0.5, 0.9, 0.95, 0.99] - histogram_buckets: slo_driven + # As RedisLaneMetrics registers it: a percentile histogram (publishPercentileHistogram, expected + # range 1ms..10s) and no client-side percentiles and no SLO boundaries. The p99 below is a + # histogram_quantile over the buckets (re-review 2026-11-04 RR-03-17). + percentiles: null + histogram_buckets: percentile_histogram alert_severity_thresholds: p2: "SCRIPT admitted p99 > 100ms for 10m (the correctness lane is queueing)" p3: "CACHE_SCRIPT admitted p99 > 100ms for 30m" diff --git a/docs/registries/secrets-classification.yaml b/docs/registries/secrets-classification.yaml index c10c2c2a..e6b4e67f 100644 --- a/docs/registries/secrets-classification.yaml +++ b/docs/registries/secrets-classification.yaml @@ -10,9 +10,11 @@ # (feature-secrets-config-source-contract 2026-05-22: "dev/local sentinel value prefix = __LOCAL_DEV_"). # - prod profile에서 `__LOCAL_DEV_` prefix 발견 시 startup fail # (feature-secrets-config-source-contract 2026-05-22). -# - prod profile에서 `fixture-` prefix 값(이 저장소에 적힌 lane fixture 비밀번호 — infra/redis-sdk/acl, -# compose lane의 REDIS_LANE_*)도 같은 이유로 startup fail. 어떤 변수 이름이든 검사한다 -# (review 2026-11-03 RR-07-04). +# - prod profile에서 비밀값이 `fixture-`로 시작하면(이 저장소에 적힌 lane fixture 비밀번호 — +# infra/redis-sdk/acl, compose lane의 REDIS_LANE_*) 같은 이유로 startup fail (review 2026-11-03 +# RR-07-04). 비밀값만 본다: 이 파일의 `secret` 행(SecretSourceValidator.CLASSIFIED_SECRETS, +# SecretsClassificationRegistryTest가 1:1로 묶음), 모듈이 기여한 비밀, `secret://…/` 참조가 +# 가리키는 변수. HOSTNAME·앱 이름 같은 비밀이 아닌 값은 보지 않는다 (review 2026-11-04 RR2-04-01). # - Masking 기본 = `full_except_last_4` (feature-secrets-config-source-contract 2026-05-22: # "full mask except last 4 chars for non-secret tokens"). 진짜 secret(password/private key)은 `full`. # - Naming suffix는 보조 신호 (feature-secrets-config-source-contract: "_TOKEN, _KEY, _PASSWORD"). diff --git a/docs/reviews/2026-11-03-rereview/README.md b/docs/reviews/2026-11-03-rereview/README.md index 5fbe2cb2..bf9aba83 100644 --- a/docs/reviews/2026-11-03-rereview/README.md +++ b/docs/reviews/2026-11-03-rereview/README.md @@ -143,3 +143,44 @@ Important 7건: | RR-02-09 V22 잠금 | V21과 같은 `SET LOCAL lock_timeout`과 같은 모양의 lane 테스트 | 같은 커밋 안의 두 마이그레이션이 다른 규칙을 따르면 안 된다 | 멈추는 기준: 3차 수정 뒤 그 diff만 보는 확인 리뷰에서 새 Important가 0이면 멈춘다. + +## 10. 3차 수정 결과 (2026-11-04) + +9.1절 결정대로 2차 재리뷰 발견 29건(Important 7, Minor 22)을 영역 묶음 4개가 `ecc74b7d`에서 `test-driven-development` 스킬 절차로 RED→GREEN 했다. 오케스트레이터가 합쳤다. 하위 에이전트 없음. + +| 묶음 | 발견 | 상태 | 검증 (에이전트) | 처리 내용 | 결정 | +|---|---|---|---|---|---| +| r5-a 멱등성·outbound | RR-01-11, RR-03-11~18 | 완료 | leaf check 9, Redis 컨테이너 계약, PG 멱등성 계약 19/19, `postgresqlIdempotencyIntegrationTest` 8/8, architectureCheck, verifyEnvKeys, documentContractTest, topology standalone·cluster | RR-01-11 Redis CLAIM이 lease 지난 EXECUTING을 ABANDONED로 바꾸고 `RECOVERY_REQUIRED`로 답한다(PG와 같음, RED 실 Redis), 공유 계약에 시간 케이스 5개. RR-03-11 두 store의 기록 수명을 lease + replay TTL로(claim·takeover·renew), `replay-ttl < processing-lease`면 기동 실패(RED 실 Redis·PG). RR-03-12 skip을 read 단위로(RED). RR-03-13 read마다 걸던 타이머를 스트림당 watchdog 하나로(read당 2.6–3.1µs → 25–32ns). RR-03-14 본문 실패를 `IOException` 하위 `HttpBodyReadException`으로(RED). RR-03-15 시스템 속성 목록을 Spring과 비교(RED). RR-03-16 breaker 맹점 문서. RR-03-17 listener 첫 실패 WARN(RED). RR-03-18 문서 | lease가 지난 실행은 abandon해서, 늦게 끝난 첫 시도의 결과도 기록하지 않는다(9.1절 "다시 돌리지 않는다"와 같은 방향, PG 동작 유지). cluster topology 첫 실행의 캐시 테스트 2건 실패는 재실행·기준 커밋에서 재현 안 됨 | +| r5-b core·inbound | RR-01-12~17 | 완료 | application-core·fileserver·web·graphql·grpc·inbound messaging·service-runtime check, app-bootstrap architectureTest, architectureCheck, verifyDependencyLocks, inbox Kafka+PG, gRPC ledger PG 44, jpaPlatformContract | RR-01-12 새 표시 `@SystemCaller(reason)`: 메시지 handler·scheduler가 `@SystemInvoked` use case를 생성자에서 `new`로 만든다. ArchUnit `ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES`·`SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS`, 매뉴얼 09장 경로를 `SystemInvokedConsumePathTest`로 고정. RR-01-13 `/error` status 접기를 문서·`EnvelopeErrorControllerTest`로. RR-01-14 `GraphQlClientPolicy`의 page 규칙 복사본 삭제. RR-01-15 ledger `release(identity, claimToken)`(RED 2). RR-01-16 `DirectTransactionPort`가 스레드별 트랜잭션(RED). RR-01-17 쿼터 롤백 계약 케이스(RED) | scheduler 패키지도 `@SystemCaller` 허용, `/error`는 코드 표에 없는 status를 class 코드로 접는다, release도 claim token으로 막는다 | +| r5-c persistence·messaging | RR-02-09~16 | 완료 | persistence-jpa(587)·persistence-notification·starter-kafka·outbox-relay·application-core·sample check, 실 PG lane 11개, `-p optional-platforms integrationCheck`, architectureCheck, documentContractTest, verifyStrictLaneStages | RR-02-09 V20·V22·알림 V11~V13·sample V8_1·grpc V002·V003에 `SET LOCAL lock_timeout = '5s'`, 텍스트 검사기 `LockWaitMigrationInspector`(testkit)와 실 PG lane `postgresqlMigrationLockWaitIntegrationTest`(RED 9). RR-02-10 scope를 바꾸는 배포는 rolling 금지 문서. RR-02-11 모르는 상태 값은 집계에서 빼고 WARN 1회(RED). RR-02-12 claim이 DEAD로 옮길 때 `last_failure_code`를 쓰고 `outbox.pending.size{status=DEAD} > 0` p1 경보(RED 단위·실 PG). RR-02-13 Javadoc. RR-02-14 runbook 숫자를 상수와 비교하는 document-contract lane. RR-02-15 메시지·가이드·fixture. RR-02-16 header 테스트, 업그레이드 안내 B-39 | 브리프에 없던 V20·V8_1도 같은 근거(아직 어디에도 미적용)로 수정. 규칙 이전에 배포된 마이그레이션 24개는 고칠 수 없어 동결 목록(줄기만 함). DEAD 원인 코드는 claim의 row lock 아래 flush | +| r5-d bootstrap·build·sample | RR2-04-01~06 | 완료 | support·app-bootstrap·sample·objectstorage check, app-bootstrap systemTest·documentContractTest, sample systemTest·webSystemTest, `-p build-logic check`, 두 빌드 verifyStrictLaneStages, verifyEnvKeys, compose 18 lane, `docker build`와 실제 기동(프로필 없음 exit 71, `demo` 8초 ready) | RR2-04-01 prod `fixture-` 검사는 비밀값만(레지스트리 `secret` 행, 모듈 기여 비밀, `secret://…/` 대상)(RED). RR2-04-02 sample `application.yml`이 `local`·`test`·`demo`에서 object storage를 끔(RED 9). RR2-04-03 rule·비밀 목록 bean에 주입 멤버·`@ConfigurationProperties`가 있으면 모든 프로필에서 기동 실패(RED 4). RR2-04-04 축약 `-x`, `workflow_dispatch` 전용 workflow, `working-directory`+`-p`, `uses:`의 `with: run:`(RED 7). RR2-04-05 lane json의 `APP_` 밖 키 거부, `property:` 행은 설정 metadata에 있을 때만 읽힘(RED 2). RR2-04-06 문서 | 리뷰어 제안 "static @Bean 선언 강제" 대신 bean이 실제로 필요로 하는 것을 본다(선언 강제는 문제없는 등록을 막고 `@Value` 필드는 통과시킨다). `workflowProjectDirectory` 설정을 없애고 저장소 구조로 정한다 — `working-directory` 없이 `./gradlew`를 부르는 workflow 명령은 실행으로 세지 않는다(B-36). `ca.spring-config`의 `compileJava`가 `processResources` 뒤에 돈다 | + +### 10.1 합칠 때 오케스트레이터가 한 것 + +- 업그레이드 안내 `docs/upgrade/2026-11-maturity-review.md`를 세 방향(r5-b·r5-c·r5-d)에서 손으로 합쳤다. B-14·B-15는 r5-d, B-16은 r5-b, B-39는 r5-c. +- RR-01-17의 남은 절반(이 수정 전에 move된 파일의 장부)을 직접 마무리했다. `docs/fileserver/operations.md` 「Namespace quota drift」에 차이를 찾는 쿼리와 namespace 하나를 다시 계산하는 SQL을 적었다. 실 PostgreSQL 16에 V1 스키마와 move 전 상태(출발 +100, 도착 −100, 삭제된 파일, 진행 중 예약)를 넣고 돌려, 차이 2행 → 0행, `RESERVED` 행은 그대로인 것을 확인했다. 업그레이드 안내 4절의 "재계산 SQL은 아직 없다"를 이 절로 바꿨다. +- 프론트 인계 문서에 이번 계약 변화(같은 키 재전송은 작업을 다시 돌리지 않음, `replay-ttl` 24h, `/error` status 접기)를 더했다. + +### 10.2 게이트 (2026-11-04, 합친 메인 작업 트리, 공유 잠금) + +첫 실행에서 모두 통과했다. + +| 게이트 | 결과 | 시간 | 실행 직전 부하(1분) | +|---|---|---|---| +| `ci` | rc=0 | 6분 22초 | 7.75 | +| `verifyStrictLaneStages --dry-run` (주 빌드 / `-p optional-platforms`) | rc=0 / rc=0 | 10초 / 3초 | 16.51 / 15.10 | +| `mainCheck` | rc=0 | 3분 6초 | 14.37 | +| `:integrationCheck` | rc=0 | 14분 22초 | 15.03 | +| `-p optional-platforms ci integrationCheck performanceCheck` | rc=0 | 1분 3초 | 1.88 | +| `:performanceCheck` | rc=0 | 22초 | 9.69 | +| `:releaseCheck -PgitRevision=` | rc=0 | 3초 | 7.63 | +| `scripts/verify-compose-profile-contracts.sh` | rc=0, 18 lane | 4초 | 7.82 | + +`integrationCheck`는 입력이 없는 lane 4개(`redisTopologyTest` 3개, `objectStorageAwsQualificationTest`)를 이름을 대며 뺐다(설계대로). 이 절과 `docs/fileserver/operations.md`·업그레이드 안내 문구는 `ci`가 도는 동안 고쳤다. 그 뒤의 `mainCheck`(document-contract lane 포함)는 고친 문서로 돌았다. + +### 10.3 남은 것 + +- `spotbugsTestFixtures`: 게이트 밖이고 `ecc74b7d`에서도 실패한다. r5-c의 새 검사기(`LockWaitMigrationInspector`)가 정규식 ReDoS 경고 8건을 더했다(입력은 저장소 마이그레이션 파일뿐). +- 실제 CI runner 실행: Gitea repo의 Actions가 꺼져 있어(`has_actions: false`) workflow가 돈 적이 없다. 켜는 것은 소유자 일이다. +- 소유자 확인이 필요한 기본값: 10절 "결정" 칸 — 특히 lease 뒤 늦게 끝난 첫 시도의 결과를 버리는 것, workflow 명령에 `working-directory`가 필요해진 것. +- 다음: 이 수정 커밋만 보는 확인 리뷰. 새 Important가 0이면 멈춘다(9.1절). diff --git a/docs/runbooks/migration-failed.md b/docs/runbooks/migration-failed.md index 540c6ef2..4a1c5c32 100644 --- a/docs/runbooks/migration-failed.md +++ b/docs/runbooks/migration-failed.md @@ -4,7 +4,7 @@ category: INTERNAL error_codes: [MIGRATION_FAILED] severity: P1 owner: oncall -last_updated: 2026-07-29 +last_updated: 2026-11-04 status: active --- @@ -40,7 +40,13 @@ status: active 4. Check whether any owned relation was created without a successful history entry. Compare only against the owned tables in the reviewed migration; do not drop unrelated relations. 5. Classify the failure: - - lock/statement timeout: remove the blocker or reduce rollout concurrency, then rerun; + - lock/statement timeout: remove the blocker or reduce rollout concurrency, then rerun. A + migration that locks a table already in use gives up after its own `SET LOCAL lock_timeout` + (5 s; SQLState `55P03` `lock_not_available`) and rolls back whole, so the history ends one + version below it. The blocker is a transaction that touched the table and is still open — + often an idle-in-transaction session: `SELECT pid, state, xact_start, query FROM + pg_stat_activity WHERE xact_start < now() - interval '5 seconds' ORDER BY xact_start;`. End + it, then rerun (`docs/jpa/migration-guide.md` "Bound the lock wait"); - SQL/data precondition: create a new forward migration that makes the precondition explicit; a duplicate outbox order key stopping V21 (`ux_outbox_event_aggregate_order`) is the known case — clean it up with `outbox-dead-letter.md` §8 and rerun, the migration stays as it is; diff --git a/docs/runbooks/outbox-dead-letter.md b/docs/runbooks/outbox-dead-letter.md index f0d212bd..1ccd6d13 100644 --- a/docs/runbooks/outbox-dead-letter.md +++ b/docs/runbooks/outbox-dead-letter.md @@ -4,7 +4,7 @@ category: INTERNAL error_codes: [OUTBOX_DEAD_LETTER] severity: P1 owner: oncall -last_updated: 2026-11-03 +last_updated: 2026-11-04 status: stub --- @@ -16,7 +16,10 @@ status: stub - alert name: `outbox_dead_letter` - alert payload 필수 field: `error.code=OUTBOX_DEAD_LETTER`, `event_type`, `event_id`, `correlation_id`, `runbook_link` -- 임계: `outbox.publisher.published.total{outcome=DEAD}` > 0 (DEAD 전이는 자동 회복이 없으므로 단건도 점검 대상) +- 임계: `outbox.publisher.published.total{outcome=DEAD}` > 0 또는 `outbox.pending.size{status=DEAD}` > 0 (DEAD 전이는 + 자동 회복이 없으므로 단건도 점검 대상). 두 번째 조건이 필요한 이유: claim이 행을 DEAD로 옮기는 경우 + (`ATTEMPT_COUNTERS_INCONSISTENT`, `UNREADABLE_ROW`)는 relay 결과를 거치지 않아 첫 번째 카운터가 오르지 않는다 + (재리뷰 2026-11-03 2차 RR-02-12) ## 2. First Response (5분 이내) @@ -26,7 +29,8 @@ status: stub `last_failure_code`에 남는다. ERROR가 없고 WARN `outbox claim moved a row ... to DEAD`만 있으면 relay가 아니라 claim이 행을 DEAD로 옮긴 것이다 — `error.reason`이 `ATTEMPT_COUNTERS_INCONSISTENT`면 `deferred_attempt_count`를 되돌리지 않은 수동 재발행이 원인이니 §4의 SQL로 다시 재발행한다. `UNREADABLE_ROW`면 행 자체가 event로 읽히지 - 않는다(canonical 봉투 없음, 저장된 해시와 바이트 불일치) + 않는다(canonical 봉투 없음, 저장된 해시와 바이트 불일치). 이 경우에도 같은 코드가 행의 `last_failure_code`에 + 남는다 — 그 전 발행 실패의 코드는 덮어쓴다 2. DB에서 DEAD row 확인: `SELECT * FROM outbox_event WHERE status = 'DEAD' ORDER BY occurred_at;` 3. **차단 영향 파악 (중요)**: strict per-aggregate FIFO 정책상 DEAD row는 같은 aggregate(`tenant_scope`, `aggregate_type`, `aggregate_id`)의 후행 이벤트를 계속 차단함. 순서는 `(aggregate_sequence, event_index)`로 비교한다(claim SQL `PostgreSqlOutboxClaimRepository`와 같은 키) — `SELECT count(*) FROM outbox_event b WHERE b.status <> 'PUBLISHED' AND EXISTS (SELECT 1 FROM outbox_event d WHERE d.status='DEAD' AND d.tenant_scope=b.tenant_scope AND d.aggregate_type=b.aggregate_type AND d.aggregate_id=b.aggregate_id AND (d.aggregate_sequence < b.aggregate_sequence OR (d.aggregate_sequence = b.aggregate_sequence AND d.event_index < b.event_index)));` @@ -121,10 +125,24 @@ V21이 하는 일은 세 가지 더 있다. 5초 안에 못 잡으면 V21이 실패하고 V20에 남는다(`MIGRATION_FAILED`). 덜 바쁜 시각에 다시 배포하거나 Step 0으로 인덱스를 먼저 만든다. +V20(`deferred_attempt_count`)과 V22(`last_failure_code`)도 같은 상한을 둔다(재리뷰 2026-11-03 2차 RR-02-09). +둘 다 `ALTER TABLE outbox_event`라 가장 강한 잠금(ACCESS EXCLUSIVE)을 건다. 이 잠금은 읽기와도 충돌하므로, +`outbox_event`를 읽기만 한 트랜잭션 하나(운영자가 `BEGIN` 안에서 조회하고 둔 psql 세션 포함)가 열려 있어도 +기다린다. 5초 안에 못 잡으면 그 마이그레이션이 실패·롤백되고 DB는 한 버전 아래(V19 또는 V21)에 남으며 새 버전은 +기동하지 않는다(`MIGRATION_FAILED`, exit 70). 열린 트랜잭션을 찾아 끝낸 뒤 다시 배포한다 — +`SELECT pid, state, xact_start, query FROM pg_stat_activity WHERE xact_start < now() - interval '5 seconds' ORDER BY xact_start;` + rolling 배포 중 아직 옛 버전인 인스턴스가 벽시계 번호로 쓴 행은 floor보다 크다. 새 버전의 counter는 그 aggregate의 가장 큰 행 번호보다 작아지지 않으므로 그 행 뒤에 번호를 준다. 서로 커밋되지 않은 채 겹친 옛·새 쓰기 사이의 순서는 보장하지 않는다 — 옛 벽시계 방식도 보장하지 않던 것이다. +이 "그 행 뒤에"는 **같은 aggregate identity**의 행에만 성립한다. tenant scope를 바꾸는 배포에서는 옛 버전이 +옛 scope로, 새 버전이 새 scope로 쓴다. 새 버전의 counter는 옛 scope 행을 보지 못하므로 floor + 1부터 시작하고, +rolling 중 옛 버전이 방금 쓴 번호(floor보다 초당 10⁶씩 큰 벽시계 번호)보다 **작은** 번호를 준다(재리뷰 +2026-11-03 2차 RR-02-10, 실 PG 확인). scope를 무시하고 aggregate id로 "마지막 번호보다 작으면 버린다"를 구현한 +소비자는 그 사이 새 이벤트를 버린다. 그래서 **tenant scope를 바꾸는 배포는 rolling으로 하지 않는다**: 옛 버전 +인스턴스를 모두 멈춘 뒤(옛·새 버전이 함께 쓰는 구간 없이) 새 버전을 띄운다. + tenant scope를 바꾼 서비스(샘플은 `portfolio` → 단일 scope)는 배포 전에 옛 scope의 미발행 행이 없는지 확인한다. 옛 scope 행과 새 scope 행은 서로 다른 aggregate로 보여 claim gate가 서로를 막지 않고, partition key도 달라 다른 partition으로 간다. @@ -231,7 +249,8 @@ Step 1을 다시 돌려 0행이면 배포한다. 정리와 배포 사이에 옛 확인 쿼리, 정리 SQL, Step 0의 인덱스 SQL은 이 문서에서 그대로 읽어 실제 PostgreSQL에서 실행하는 테스트가 있다(`PostgreSqlOutboxOrderKeyRunbookIntegrationTest`, lane `postgresqlOutboxOrderKeyRunbookIntegrationTest`). 같은 lane이 V21의 counter·floor, 잠금 대기 실패, -INVALID 인덱스 거부도 본다. 이 절의 `runbook-sql` 블록을 고치면 그 lane을 다시 돌린다. +INVALID 인덱스 거부도 본다. 이 절의 `runbook-sql` 블록을 고치면 그 lane을 다시 돌린다. V20·V22의 잠금 대기 +실패는 `PostgreSqlMigrationLockWaitIntegrationTest`(lane `postgresqlMigrationLockWaitIntegrationTest`)가 본다. --- diff --git a/docs/runbooks/outbox-publish-failed.md b/docs/runbooks/outbox-publish-failed.md index 41e9dd6b..b676e14e 100644 --- a/docs/runbooks/outbox-publish-failed.md +++ b/docs/runbooks/outbox-publish-failed.md @@ -4,7 +4,7 @@ category: TRANSIENT_DEPENDENCY error_codes: [OUTBOX_PUBLISH_FAILED] severity: P2 owner: oncall -last_updated: 2026-11-03 +last_updated: 2026-11-04 status: stub --- @@ -86,7 +86,8 @@ status: stub **Step 1 — 찾기 (읽기 전용).** 미룬 claim(`deferred_attempt_count`)이 10번 이상인 FAILED 행이다. backoff 상한이 5분이라 10번은 적어도 약 35분 동안 계속 실패했다는 뜻이다. `outbox.retry.long.size`·`outbox.retry.long.age`가 -같은 기준(`OutboxMetrics.LONG_RETRY_DEFERRED_CLAIMS`)으로 센다. +같은 기준(`OutboxMetrics.LONG_RETRY_DEFERRED_CLAIMS`)으로 센다. 아래 SQL의 숫자를 바꾸면 그 상수도 같이 바꾼다 — +`OutboxLongRetryRunbookContractTest`(lane `:composition:outbox-relay:outboxRelayDocumentContractTest`)가 둘을 비교한다. ```sql @@ -114,6 +115,12 @@ UPDATE outbox_event SET status = 'PARKED' WHERE event_id = '' AND status = ' `UPDATE 0`이면 그 사이 relay가 행을 claim해 IN_FLIGHT다. `next_attempt_at`이 지난 뒤 다시 실행한다. 보류된 행 수는 `outbox.pending.size{status=PARKED}`로 보인다. +**롤백 전에 보류를 푼다.** `PARKED`는 이 버전(`bbe944bd` 이후)에 생긴 상태다. 그보다 옛 버전은 이 이름을 몰라 +상태별 집계 전체가 실패하고, `outbox.pending.size` 게이지가 PENDING까지 통째로 꺼진다 — `PENDING size growing` +경보의 입력이 사라진다(재리뷰 2026-11-03 2차 RR-02-11). 옛 버전으로 롤백해야 하면 그 전에 보류한 행을 모두 +Step 3으로 재개하거나 버린다. 이 버전부터는 모르는 상태가 있어도 그 상태만 빼고 세고 WARN 한 번으로 이름을 +말한다(`outbox_event holds rows in a status this version does not know`). + **Step 3 — 재개(resume) 또는 버리기(skip).** 원인을 고쳤으면 재개한다. 행은 FAILED로 돌아가 바로 claim 대상이 된다. 시도 횟수는 그대로 둔다(미룬 claim은 예산을 쓰지 않으므로 DEAD로 바로 가지 않는다). diff --git a/docs/runbooks/redis-capability-incident.md b/docs/runbooks/redis-capability-incident.md index a40d93b4..481bb7fd 100644 --- a/docs/runbooks/redis-capability-incident.md +++ b/docs/runbooks/redis-capability-incident.md @@ -47,7 +47,9 @@ Redis liveness 실패만으로 pod를 재시작하지 않는다. 재시작 폭 로그와 health detail에서 확인한다. timeout이나 연결 끊김은 미실행 증거가 아니다. - readiness는 최신 health detail의 state/reason/requirement로 본다. optional cache의 degraded serving과 required coordination/session의 fail-closed 결정을 같은 availability 의미로 합치지 않는다. -- reconnect, cache source-load, session repository error 지표의 변화를 함께 본다. +- 재연결은 `redis.lane.connections.opened.total`과 `discarded.total`을 함께 본다(Cluster `TRANSACTION` + lane은 borrow마다 열어 opened가 꾸준히 오른다 — `cache-redis/README.md`). cache source-load·session + repository 오류 지표는 내보내는 코드가 없다(설계만 있음). - Redis server 측에서는 memory/eviction, rejected clients, replication link/lag, persistence error, Cluster coverage를 operator dashboard에서 확인한다. - `NOSCRIPT`, result-schema mismatch, ACL denial, TLS/auth failure, OOM, timeout을 서로 다른 @@ -57,21 +59,21 @@ Redis liveness 실패만으로 pod를 재시작하지 않는다. 재시작 폭 - `redis.lane.*` meter의 tag는 닫힌 lane 이름뿐이다. key, subject, session id, token, endpoint, exception text, script/SHA, value 같은 identity/wire material을 metric이나 ticket에 복사하지 않는다. -- semantic operation 계측은 logical provider가 실제로 반환한 hit/miss/denied/conflict/ - unavailable/indeterminate 결과를 기록한다. cache의 `stale`/`skipped`, session의 - `tombstoned`/`absolute_expired`도 정상 hit/miss와 분리한다. meter registry, classifier, - monotonic ticker 장애는 command 결과나 원래 exception instance를 바꾸지 않는다. +- semantic operation 단위 계측(hit/miss/denied/conflict/unavailable/indeterminate 결과, cache의 + `stale`/`skipped`, session의 `tombstoned`/`absolute_expired`)은 **설계만 있고 내보내지 않는다** + (`redis.capability.*`, 위 Detection 절). operation 결과는 로그와 health detail로 본다. `redis.lane.*` + 계측(listener)이 실패해도 borrow 결과나 원래 exception은 바뀌지 않는다. 처음 실패는 WARN 로그 한 줄로 + 남고(`RedisRuntimeOwner`), 그 뒤는 DEBUG다 — 지표가 통째로 비면 그 WARN을 찾는다. - route 응답이 설정된 byte/collection bound를 넘으면 동일 logical operation을 `unavailable`로 종료한다. GET/read-only 응답은 `not_applied`, mutation VALUE/MULTI 응답은 - 서버 실행 여부를 되돌릴 수 없으므로 `indeterminate`다. 앞선 `success` 표본과 이 실패를 두 - operation으로 합산하지 않는다. + 서버 실행 여부를 되돌릴 수 없으므로 `indeterminate`다. - Spring 종료의 dependency order는 invalidation subscription 같은 capability dependent를 먼저 닫고, capability bean을 닫은 다음 canonical registry가 router admission을 닫아 in-flight를 bounded drain하고 마지막에 runtime을 닫는 순서다. 종료 중 새 command를 허용하거나 drain timeout 뒤 무기한 기다리지 않는다. - 현재 composition에는 active Redis scheduler나 dormant credential-rotation coordinator가 없다. 존재하지 않는 lifecycle coordinator를 복구 절차에서 찾거나 수동 호출하지 않는다. -- 이 meter와 단일-process lifecycle test는 Sentinel/Cluster failover, TLS/ACL 배포 적합성, +- `redis.lane.*` meter와 단일-process lifecycle test는 Sentinel/Cluster failover, TLS/ACL 배포 적합성, k3s multi-node, L1/L2 분산 일관성, distributed session 동작의 qualification 증거가 아니다. 해당 label은 별도 topology/conformance lane의 실제 증거가 있어야 한다. diff --git a/docs/upgrade/2026-11-maturity-review.md b/docs/upgrade/2026-11-maturity-review.md index 6a96a80a..995b634a 100644 --- a/docs/upgrade/2026-11-maturity-review.md +++ b/docs/upgrade/2026-11-maturity-review.md @@ -21,13 +21,13 @@ |---|---|---|---| | 1 | JWT 모드면 `APP_SECURITY_JWT_AUDIENCE`에 이 서비스의 client id를 쉼표로 넣는다 | 기동 실패 `APP_SECURITY_JWT_AUDIENCE (app.security.audience) is required` | JWT 모드(기본값) 전부 | | 2 | dev·prod 환경 변수의 `APP_FLYWAY_ENABLED` 값을 확인한다 | `false`가 남아 있으면 이제 정말로 마이그레이션이 안 돈다 | persistence-jpa 선택 | -| 3 | prod 설정 어디에도 `fixture-`로 시작하는 값이 없게 한다 | prod 기동 실패(exit 78) | prod 배포 전부 | -| 4 | 직접 만든 `StartupRule`·`RequiredProductionSecrets` bean을 `static @Bean`, 의존성 없이 바꾼다 | 기동 실패 `startup rule '<이름>' created [...]`(exit 78) | 자기 rule이 있는 서비스 | +| 3 | prod 비밀값(레지스트리 `secret` 행, 모듈이 기여한 비밀, `secret://…/`이 가리키는 변수)이 `fixture-`로 시작하지 않게 한다 | prod 기동 실패(exit 78) | prod 배포 전부 | +| 4 | 직접 만든 `StartupRule`·`RequiredProductionSecrets` bean을 `static @Bean`, 의존성 없이 바꾸고, 그 클래스에서 `@Value`·`@Autowired` 멤버와 `@ConfigurationProperties`를 뺀다 | 기동 실패 `startup rule '<이름>' created [...]` 또는 `'<이름>' (...) is a StartupRule or RequiredProductionSecrets ... but it needs one`(exit 78) | 자기 rule이 있는 서비스 | | 5 | 직접 구현한 outbox·발행·알림·파일 서버 포트를 새 시그니처로 고친다(3.1절) | 컴파일 오류 | 해당 포트를 구현·호출하는 서비스 | | 6 | `dev.caskeleton.application.outbound.*` import를 `application.secret.*`·`application.objectstorage.content.CallBudget`으로 바꾼다 | 컴파일 오류 | 해당 타입을 쓰는 서비스 | | 7 | 이벤트 id를 소문자 UUIDv7로 만들고 `IntegrationEventDraft`는 `IntegrationEventDrafts.draft(...)`로만 만든다 | 쓰기 요청이 `IllegalArgumentException`으로 실패, architectureTest 실패 | 통합 이벤트를 내는 서비스 | | 8 | V21 배포 전에 `docs/runbooks/outbox-dead-letter.md` §8 Step 1 쿼리를 운영 DB에 돌린다 | 중복 순서 키가 있으면 기동 실패(exit 70, V20에 남음) | persistence-jpa + outbox | -| 9 | 이 브랜치의 이전 V21을 적용한 개발 DB를 다시 만든다(4.2절) | Flyway checksum 불일치로 기동 실패(exit 70) | 이 브랜치를 미리 받아 돌린 개발 DB | +| 9 | 이 브랜치의 이전 V21(또는 3차 수정 전 V20·V22·V11~V13·V8_1·V002·V003)을 적용한 개발 DB를 다시 만든다(4.2절) | Flyway checksum 불일치로 기동 실패(exit 70) | 이 브랜치를 미리 받아 돌린 개발 DB | | 10 | 자기 SecurityFilterChain이 있으면 `DispatcherType.ERROR`를 허용한다 | 핸들러 밖 오류가 401로 바뀌어 나간다 | 자기 보안 체인을 둔 서비스 | | 11 | 짝 프론트(또는 다른 클라이언트)에 7절 계약을 맞추게 한다 | 409를 재시도 불가로 읽거나, `null` 키를 기대하다 깨진다 | HTTP 클라이언트가 있는 서비스 | | 12 | 자기 Flyway 마이그레이션 번호가 템플릿의 새 번호와 겹치지 않는지 본다(4절) | 같은 버전 두 개로 Flyway 실패 | 같은 stream에 자기 마이그레이션을 둔 서비스 | @@ -56,8 +56,9 @@ | B-07 | 패키지 이동: `dev.caskeleton.application.outbound.SecretSourcePort`·`SecretReference` → `dev.caskeleton.application.secret.*`. `dev.caskeleton.application.outbound.CallBudget` → `dev.caskeleton.application.objectstorage.content.CallBudget`(leaf `application-objectstorage`). `application.outbound` 패키지 삭제 | `package dev.caskeleton.application.outbound does not exist` | import를 바꾼다. `CallBudget`을 쓰는 leaf가 `application-objectstorage`에 의존해도 되는지 `modules.json` 간선을 확인 | `5786126c` · `src/application-core/.../secret/`, `src/application-objectstorage/.../content/CallBudget.java` | | B-08 | `shared-contract`에서 `dev.caskeleton.shared.operation.Operation`·`OperationStatus` 삭제. sample-portfolio(`...sample.portfolio.adapter.inbound.web.operation`)로 옮겨졌다 | import 실패 | 쓰고 있었다면 자기 inbound adapter로 복사한다 | `1400dc15` · `src/shared-contract/src/main/java/dev/caskeleton/shared/operation/`(삭제) | | B-09 | `SecuritySettings`(adapter:inbound:web): `audience`가 `String` → `List`. `@ConfigurationProperties` 제거, `SecuritySettings.bind(Environment)`로 strict 바인딩. 생성자 `(String, String, List)` → `(String, List, List)`. `bootstrap.security.AuthenticationModeSettings` 삭제 | `audience()`를 문자열로 쓰는 코드, 직접 생성하는 테스트 컴파일 오류 | `List`으로 바꾼다. 인증 모드는 `SecuritySettings.authMode()`에서 읽는다 | `34565130`(List), `5786126c`(strict·삭제) · `src/adapter/inbound/web/.../settings/SecuritySettings.java` | -| B-10 | adapter 공개 타입 삭제·이동: graphql `execution.GraphQlTimeoutPolicy`·`dataloader.GraphQlDataLoaderRequestRegistry` 삭제(`runtime.GraphQlDeadlineInstrumentation`·`GraphQlPerParentFetchInstrumentation` 추가). jpa `experimental.tenant.TenantId` 삭제 → `dev.caskeleton.application.tenant.TenantId`. persistence-notification의 엔티티는 `...platform.entity`, 저장소는 `...platform.repository`로 이동, `inbox` 하위 클래스는 `platform`으로 올라옴, `PreferenceJpaRepository`·`SuppressionJpaRepository` 삭제, `JpaNotificationSideEffect` → application-notification `dispatch.SuppressionSideEffect` | import 실패. leaf `check`의 API surface 비교가 스냅샷과 달라 실패(자기 공개 타입을 더했다면 merge 충돌) | import를 바꾼다. 스냅샷 파일은 upstream 것을 받고 자기 추가분을 다시 승인한다 | `1400dc15`·`5786126c` · `docs/architecture/graphql-api-surface.txt`, `docs/architecture/jpa-api-surface.txt`(166→165), `docs/notification/api-surface-snapshot.txt`(362→364) | +| B-10 | adapter 공개 타입 삭제·이동: graphql `execution.GraphQlTimeoutPolicy`·`dataloader.GraphQlDataLoaderRequestRegistry` 삭제(`runtime.GraphQlDeadlineInstrumentation`·`GraphQlPerParentFetchInstrumentation` 추가). jpa `experimental.tenant.TenantId` 삭제 → `dev.caskeleton.application.tenant.TenantId`. persistence-notification의 엔티티는 `...platform.entity`, 저장소는 `...platform.repository`로 이동, `inbox` 하위 클래스는 `platform`으로 올라옴, `PreferenceJpaRepository`·`SuppressionJpaRepository` 삭제, `JpaNotificationSideEffect` → application-notification `dispatch.SuppressionSideEffect`. graphql `GraphQlClientPolicy.effectivePageSize(Integer, Integer)` 삭제 → `pagination.GraphQlConnectionPolicy.effectivePageSize`(상한 검사 포함, 재리뷰 RR-01-14). grpc(optional-platforms) `GrpcOperationLedger.release(identity)` → `boolean release(identity, GrpcClaimToken)`: 진행 중이고 그 token이 가진 claim만 지운다(RR-01-15) | import 실패, 호출부·직접 구현한 ledger 컴파일 오류. leaf `check`의 API surface 비교가 스냅샷과 달라 실패(자기 공개 타입을 더했다면 merge 충돌) | import를 바꾼다. 스냅샷 파일은 upstream 것을 받고 자기 추가분을 다시 승인한다 | `1400dc15`·`5786126c` · `docs/architecture/graphql-api-surface.txt`, `docs/architecture/jpa-api-surface.txt`(166→165), `docs/notification/api-surface-snapshot.txt`(362→364) | | B-11 | 조립·지원 타입: `bootstrap.autoconfigure.operation.DurableOperationConfig` → `composition:durable-operation`의 `composition.operation.DurableOperationAutoConfiguration`. `bootstrap.settings.LoggingSettings` 삭제. `composition.outbox.OutboxLeaderElectionToken` 삭제. `TracingSamplingEnvironmentPostProcessor` → `TracingEnvironmentPostProcessor`, `StartupOrder.TRACING_SAMPLING` → `TRACING_PROPERTIES`. `StartupRuleRunner(Environment, List)` → `StartupRuleRunner(Environment)`. `SecretSourceValidator(ConfigurableEnvironment, SecretSourcePort, List)` → `SecretSourceValidator(Supplier>)` | 이 타입을 참조하는 코드·테스트 컴파일 오류 | 새 이름으로 바꾼다 | `1400dc15`, `34565130`, `bbe944bd` · `src/composition/durable-operation/`, `src/adapter/outbound/support/.../startup/` | +| B-39 | vendored messaging SPI `KafkaQuarantinePublisher`(`messaging-kafka`): `quarantine(ConsumerRecord, Throwable)` → `quarantine(ConsumerRecord, FailureCategory category, Throwable reason)`. 호출자가 격리 이유(읽을 수 없는 레코드 `DESERIALIZATION`, 시도 예산 소진은 실패의 category 또는 `POISON_MESSAGE`)를 넘기고, 구현은 그것을 `msg.failure-category` header로 쓴다 | 이 SPI를 직접 구현한 서비스는 `does not override abstract method` 컴파일 오류. 람다로 만든 것도 인자 수 오류 | 세 인자 메서드로 고친다. category를 header나 저장소에 그대로 남겨야 redrive가 두 경우를 가른다(템플릿 `KafkaDeadLetterQuarantine`, 테스트 `KafkaDeadLetterQuarantineTest`) | `bbe944bd`(리뷰 06-14), 3차 수정에서 목록에 추가(재리뷰 2차 RR-02-16) · `src/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java` | ### 3.2 설정·기동 — 기동이 실패하거나 런타임 예외가 난다 @@ -65,9 +66,9 @@ |---|---|---|---|---| | B-12 | JWT 모드에서 `APP_SECURITY_JWT_AUDIENCE` 필수(빈 값 거부, `${...}`가 안 풀린 값도 없는 값으로 봄). 토큰 `aud`가 목록 중 하나를 담아야 통과. `resource_access` role은 audience에 적은 client 것만 읽는다(realm role은 그대로) | 기동 실패 `APP_SECURITY_JWT_AUDIENCE (app.security.audience) is required`(exit 코드 확인 못 함). `.env.example`이 `APP_SECURITY_JWT_AUDIENCE=`(빈 값)를 싣고 있다. 다른 client의 role에 기대던 권한이 사라짐 | client id를 쉼표로 넣는다. 다른 client role로 권한을 받던 사용자가 있는지 IdP에서 확인 | `34565130`(필수·role), `1400dc15`(안 풀린 `${...}`) · `src/adapter/inbound/web/.../auth/JwtDecoderConfig.java`, `.../JwtToAuthenticatedPrincipalConverter.java`, `.../settings/SecuritySettings.java` | | B-13 | `app.security.*` strict 바인딩: 접두어 아래 오타 키, 또는 설정·placeholder 어디에도 쓰이지 않는 `APP_SECURITY_*` 환경 변수가 기동을 막는다 | 기동 실패(메시지는 `StrictSettingsBinder`가 키 이름을 담아 냄, 정확한 문구 확인 못 함) | 쓰지 않는 `APP_SECURITY_*` 변수를 지운다 | `5786126c` · `SecuritySettings.bind`, `src/adapter/outbound/support/.../StrictSettingsBinder.java` | -| B-14 | prod 프로필에서 `fixture-`로 시작하는 설정 값(변수 이름 무관)을 거부. Redis ACL fixture 파일 이름 변경: `infra/redis-sdk/acl/all-accounts.acl` → `lane-fixture-accounts.acl` | prod 기동 실패, 요약 `prod secret/config-source contract is invalid`(exit 78). 옛 ACL 경로를 mount하던 compose는 파일 없음 | prod 값을 진짜 비밀로 바꾼다. compose의 ACL 경로를 새 이름으로 | `bbe944bd` · `src/app-bootstrap/.../runtime/SecretSourceValidator.java`, `docs/registries/secrets-classification.yaml` | -| B-15 | 기동 rule이 모든 singleton보다 먼저 돈다(`StartupRuleRunner`가 `BeanFactoryPostProcessor`). `StartupRule`과 `RequiredProductionSecrets` bean은 `static @Bean`이고 `Environment`, bean factory, static 무인자 `RequiredProductionSecrets`의 `ObjectProvider`만 받아야 한다 | non-static `@Bean`이거나 다른 bean을 받으면 기동 실패 `startup rule '<이름>' created [] while creating it ... Declare the rule with a static @Bean method ...`(exit 78) | rule 선언을 `static`으로, 인자를 위 셋으로 줄인다. 다른 bean이 필요한 검사는 rule이 아니라 일반 bean으로 둔다 | `34565130`, `bbe944bd` · `src/adapter/outbound/support/.../startup/StartupRule.java`, `StartupRuleRunner.java` | -| B-16 | 권한 가드(새 `PermissionEnforcementGuard`): `@SystemInvoked` 유스케이스가 bean이면 기동 실패. `@RequiresPermission`을 단 bean에 적용되는 advisor가 없어도 기동 실패. ArchUnit `SYSTEM_INVOKED_USE_CASES_DECLARE_NO_PERMISSION`, `INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES` 추가 | 기동 실패 `@SystemInvoked use cases are registered as beans ...` 또는 `beans declare @RequiresPermission but no advisor enforces it ...`(exit 72). inbound adapter가 그 유스케이스를 부르면 architectureTest 실패 | 시스템만 부르는 쓰기 유스케이스는 composition에서 `new`로 만들어 호출자(scheduler 등)에 넘긴다. `@RequiresPermission`을 쓰면 web adapter(`MethodSecurityConfig`)가 조립돼 있어야 한다 | `1400dc15`, `bbe944bd` · `src/composition/service-runtime/.../authorization/PermissionEnforcementGuard.java`, `src/application-core/.../security/SystemInvoked.java` | +| B-14 | prod 프로필에서 `fixture-`로 시작하는 비밀값을 거부(레지스트리 `secret` 행, 모듈이 기여한 비밀, `secret://…/` 참조 대상. `HOSTNAME`·앱 이름 같은 비밀 아닌 값은 보지 않는다 — RR2-04-01). Redis ACL fixture 파일 이름 변경: `infra/redis-sdk/acl/all-accounts.acl` → `lane-fixture-accounts.acl` | prod 기동 실패, 요약 `prod secret/config-source contract is invalid`(exit 78). 옛 ACL 경로를 mount하던 compose는 파일 없음 | prod 값을 진짜 비밀로 바꾼다. compose의 ACL 경로를 새 이름으로 | `bbe944bd` · `src/app-bootstrap/.../runtime/SecretSourceValidator.java`, `docs/registries/secrets-classification.yaml` | +| B-15 | 기동 rule이 모든 singleton보다 먼저 돈다(`StartupRuleRunner`가 `BeanFactoryPostProcessor`). `StartupRule`과 `RequiredProductionSecrets` bean은 `static @Bean`이고 `Environment`, bean factory, static 무인자 `RequiredProductionSecrets`의 `ObjectProvider`만 받아야 한다. 두 타입의 클래스는 `@Value`·`@Autowired`·`@Inject`·`@Resource` 멤버, `@PostConstruct`·`@PreDestroy`, `@ConfigurationProperties`를 가질 수 없다(post-processor보다 먼저 만들어져 비어 있게 된다, RR2-04-03) | non-static `@Bean`이거나 다른 bean을 받으면 기동 실패 `startup rule '<이름>' created [] while creating it ... Declare the rule with a static @Bean method ...`, 위 멤버가 있으면 `'<이름>' (...) is a StartupRule or RequiredProductionSecrets ... but it needs one: [...]`(둘 다 exit 78, 후자는 모든 프로필) | rule 선언을 `static`으로, 인자를 위 셋으로 줄인다. 값은 `violations(Environment)`가 받는 `Environment`에서 읽는다. 다른 bean이 필요한 검사는 rule이 아니라 일반 bean으로 둔다 | `34565130`, `bbe944bd` · `src/adapter/outbound/support/.../startup/StartupRule.java`, `StartupRuleRunner.java` | +| B-16 | 권한 가드(새 `PermissionEnforcementGuard`): `@SystemInvoked` 유스케이스가 bean이면 기동 실패. `@RequiresPermission`을 단 bean에 적용되는 advisor가 없어도 기동 실패. ArchUnit `SYSTEM_INVOKED_USE_CASES_DECLARE_NO_PERMISSION`, `ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES`, `SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS` 추가, 새 표시 `@SystemCaller` | 기동 실패 `@SystemInvoked use cases are registered as beans ...` 또는 `beans declare @RequiresPermission but no advisor enforces it ...`(exit 72). `@SystemCaller`가 아닌 inbound adapter가 그 유스케이스를 부르면 architectureTest 실패 | 시스템만 부르는 쓰기 유스케이스는 composition에서 `new`로 만들어 호출자(scheduler 등)에 넘기거나, 메시지 handler면 handler에 `@SystemCaller`를 달고 생성자에서 `new`로 만든다(매뉴얼 09장 「소비」 4단계). `@RequiresPermission`을 쓰면 web adapter(`MethodSecurityConfig`)가 조립돼 있어야 한다 | `1400dc15`, `bbe944bd` · `src/composition/service-runtime/.../authorization/PermissionEnforcementGuard.java`, `src/application-core/.../security/SystemInvoked.java`, `SystemCaller.java` | | B-17 | `IntegrationEventDraft`: `eventId`·`causationId`는 소문자 UUIDv7만. ArchUnit `INTEGRATION_EVENT_DRAFTS_ARE_BUILT_BY_THE_SHARED_HELPER`: `new IntegrationEventDraft<>(...)` 금지, `IntegrationEventDrafts.draft(...)`만 | 이벤트를 만드는 쓰기 요청이 `IllegalArgumentException: eventId must be a canonical lower-case UUIDv7...`로 실패. architectureTest 실패 | event id를 UUIDv7로 만든다(예: sample `UuidOutboxEventIdFactory`가 `UuidCreator.getTimeOrderedEpochPlus1()`). draft는 헬퍼로 만든다. tenant가 없는 이벤트는 `IntegrationEventDrafts.SINGLE_TENANT_SCOPE`(`global`) | `1400dc15`, `bbe944bd` · `src/application-core/.../messaging/event/IntegrationEventDraft.java`, `IntegrationEventDrafts.java` | | B-18 | `MessageHandler.schemaVersions()`(vendored `messaging-core-api`, default `{v1}`): 소비자 decode 목록을 handler 선언으로 만든다. 같은 (메시지 타입, 버전)에 다른 payload class를 선언하면 실패 | 컴파일은 된다. v1이 아닌 메시지는 `SCHEMA_VERSION_NOT_REGISTERED`로 거부. 발행 등록(`MessageContracts`)과 class가 다르면 기동 실패 `MESSAGING_CONTRACT_CONFLICT` | v2 이상을 읽는 handler는 `schemaVersions()`를 override한다. 소비 타입을 `MessageContracts`에 따로 등록하던 코드는 지운다 | `5786126c` · `src/messaging/messaging-core-api/.../delivery/MessageHandler.java`, `messaging-spring-boot-starter/.../MessageContracts.java` | | B-19 | `APP_FLYWAY_ENABLED`가 모든 프로필에서 읽힌다. dev·prod 프로필 파일의 `spring.flyway.enabled: true`·`app.persistence.vendor` 고정값을 지우고 persistence-jpa `config/module-defaults`가 `${APP_FLYWAY_ENABLED:true}`를 싣는다 | dev·prod 환경에 `APP_FLYWAY_ENABLED=false`가 있으면 전에는 무시됐지만 이제 기동 때 마이그레이션이 안 돈다 | dev·prod 환경 값을 확인한다. 자기 `application-dev.yml`·`application-prod.yml`에 같은 고정값을 다시 넣지 않는다 | `5786126c` · `src/app-bootstrap/src/main/resources/application-{dev,prod}.yml`, `src/adapter/outbound/persistence-jpa/.../module-defaults/persistence-jpa.yml` | @@ -75,10 +76,10 @@ | B-21 | 트랜잭션 제한: `TransactionPort` 쓰기·읽기 트랜잭션 30초(`app.persistence-jpa.transaction.write-timeout`·`read-timeout`, env 키 아님) | 30초를 넘는 트랜잭션은 문장이 취소되고 `DB_QUERY_CANCELED`로 실패 | 긴 일괄 작업은 쪼갠다. 꼭 필요하면 프로필 yml에서 값을 올린다 | `1400dc15` · `persistence-jpa.yml` | | B-22 | 격리 수준을 Hikari에서 고정: `spring.datasource.hikari.transaction-isolation: TRANSACTION_READ_COMMITTED`. 다른 값이면 거부. `TransactionPort`는 트랜잭션마다 격리 수준을 정하지 않는다 | 다른 격리 수준을 설정했다면 기동 실패(`HikariPoolConstraintValidator`) | 값을 지운다. 더 높은 격리가 필요한 곳은 따로 설계한다 | `1400dc15` · `src/adapter/outbound/persistence-jpa/.../startup/HikariPoolConstraintValidator.java` | | B-23 | H2 드라이버가 운영 jar에서 빠짐(persistence-jpa `runtimeOnly` → `testRuntimeOnly`). `bootRunH2` 작업만 따로 넣는다 | `APP_PERSISTENCE_VENDOR=h2`로 `bootRunH2` 밖에서 띄우면 드라이버가 없다(정확한 오류 확인 못 함) | 로컬은 Compose PostgreSQL 또는 `bootRunH2`를 쓴다 | `1400dc15` · `src/adapter/outbound/persistence-jpa/build.gradle` | -| B-24 | `APP_IDEMPOTENCY_PROCESSING_LEASE`는 2초(`IdempotencyExecutor.IN_FLIGHT_RETRY_AFTER`)보다 길어야 한다 | 2초 이하면 기동 실패 | 값을 2초보다 길게(기본 30s) | `5786126c`, `bbe944bd` · `src/composition/idempotency/.../IdempotencySettings.java` | +| B-24 | `APP_IDEMPOTENCY_PROCESSING_LEASE`는 2초(`IdempotencyExecutor.IN_FLIGHT_RETRY_AFTER`)보다 길어야 한다. `APP_IDEMPOTENCY_TTL`(replay TTL)은 그 lease보다 짧으면 안 된다(재리뷰 RR-03-11) | 2초 이하면, 또는 replay TTL < lease면 기동 실패 | lease를 2초보다 길게(기본 30s), replay TTL을 lease 이상으로(기본 24h) | `5786126c`, `bbe944bd`, 3차 수정 · `src/composition/idempotency/.../IdempotencySettings.java` | | B-25 | 알림 콜백 본문 상한 65508 → 65251(바인딩에서 상한 검사) | `.env`에 옛 예제 값 `APP_NOTIFICATION_PLATFORM_CALLBACK_MAX_BODY_BYTES=65508`이 있으면 기동 실패 `max-body-bytes must be 1..65251` | 값을 지우거나 65251 이하로 | `5786126c` · `src/adapter/outbound/notification/.../autoconfigure/NotificationPlatformSettings.java` | | B-26 | `config/module-defaults/`에 같은 이름 파일이 둘이거나, 두 파일이 같은 키를 실으면 기동 실패 | 기동 실패 `two capability modules ship config/module-defaults/` 또는 `... both ship `(exit 78) | 자기 leaf의 module-defaults 파일 이름·키가 템플릿 모듈과 겹치지 않게 한다 | `1400dc15` · `src/adapter/outbound/support/.../startup/ModuleDefaultsEnvironmentPostProcessor.java` | -| B-27 | (sample-portfolio를 남겨 둔 경우) 데모 계정은 활성 프로필이 모두 `local`·`test`·`demo` 중 하나일 때만. 프로필이 없어도 거부 | 기동 실패 `the sample's demo accounts authenticate only under the local, test or demo profile` | `SPRING_PROFILES_ACTIVE=local` 등으로 띄우거나 sample 보안 설정을 지운다 | `5786126c`, `bbe944bd` · `src/sample-portfolio/.../security/SampleDemoAccountsProductionGuard.java` | +| B-27 | (sample-portfolio를 남겨 둔 경우) 데모 계정은 활성 프로필이 모두 `local`·`test`·`demo` 중 하나일 때만. 프로필이 없어도 거부 | 기동 실패 `the sample's demo accounts authenticate only under the local, test or demo profile` | `SPRING_PROFILES_ACTIVE=demo`(이미지·jar), `local`(bootRun)로 띄우거나 sample 보안 설정을 지운다. 세 프로필 모두 sample `application.yml`이 object storage를 끈다(RR2-04-02) | `5786126c`, `bbe944bd` · `src/sample-portfolio/.../security/SampleDemoAccountsProductionGuard.java` | ### 3.3 HTTP wire — 응답이 달라진다 @@ -88,7 +89,7 @@ |---|---|---|---|---| | B-28 | 처리 중 409 `IDEMPOTENT_IN_FLIGHT`: `retryable` false→true, `Retry-After: 2` 추가. gRPC trailer `error-retryable`과 GraphQL extension `retryable`도 true | 409를 "재시도 금지"로 처리하던 클라이언트가 틀린다 | **같은 `Idempotency-Key`로** `Retry-After` 뒤 재전송하게 한다. 새 키 금지 | `34565130` · `src/shared-contract/.../error/OperationalError.java`, `docs/registries/error-codes.yaml` | | B-29 | Envelope의 `null` 필드를 키째 생략: 성공은 `error` 키 없음, 실패는 `data` 키 없음, `meta.page`·`meta.correlationId`·`error.details`도 값이 없으면 키 없음. 빈 목록은 그대로 | `"error": null`·`"data": null`을 기대하던 클라이언트·계약 테스트가 깨진다 | `success`로 가르고 키 존재로 판단하지 않는다 | `34565130` · `src/adapter/inbound/web/.../envelope/EnvelopeWireFormatConfig.java` | -| B-30 | 새 코드 `IDEMPOTENCY_COMPLETION_UNKNOWN`(503, `TRANSIENT_DEPENDENCY`, retryable=true): 작업 실행 **뒤** complete 단계에서 멱등성 저장소가 응답하지 않음. 전에는 `IDEMPOTENCY_UNAVAILABLE` | 503을 "적용 안 됨"으로 읽던 클라이언트가 새 키로 다시 보내 중복이 생길 수 있다 | `error.code`로 가른다. 결과 불명이므로 같은 키로만 재전송 | `bbe944bd` · `src/application-core/.../idempotency/IdempotencyCompletionUnknownException.java`, `IdempotencyExecutor.java` | +| B-30 | 새 코드 `IDEMPOTENCY_COMPLETION_UNKNOWN`(503, `TRANSIENT_DEPENDENCY`, retryable=true): 작업 실행 **뒤** complete 단계에서 멱등성 저장소가 응답하지 않음. 전에는 `IDEMPOTENCY_UNAVAILABLE` | 503을 "적용 안 됨"으로 읽던 클라이언트가 새 키로 다시 보내 중복이 생길 수 있다 | `error.code`로 가른다. 결과 불명이므로 같은 키로만 재전송. 실행 중 lease가 지난 기록은 두 저장소 모두 500 `IDEMPOTENCY_RECOVERY_REQUIRED`로 답한다 — Redis 저장소는 전에 넘겨받아 작업을 다시 돌렸다(재리뷰 RR-01-11) | `bbe944bd`, 3차 수정 · `src/application-core/.../idempotency/IdempotencyCompletionUnknownException.java`, `IdempotencyExecutor.java`, `src/adapter/outbound/cache-redis/.../idempotency/IdempotencyScripts.java` | | B-31 | 핸들러 밖 오류(`/error` 디스패치)를 `EnvelopeErrorController`가 실패 Envelope로 답한다. 전에는 Boot `/error` 맵이 `success:true`로 감싸져 요청 경로까지 실려 나갔다. 기본 JWT 체인에서 익명 400이 401로 바뀌지 않는다. 4xx·5xx 본문은 `success:true`로 감싸지 않는다. 보안 체인이 `DispatcherType.ERROR`를 허용한다 | 오류 본문 모양이 바뀐다. 자기 SecurityFilterChain이 ERROR 디스패치를 막으면 400이 401로 나간다 | 자기 체인에 `auth.dispatcherTypeMatchers(DispatcherType.ERROR).permitAll()`을 넣는다(템플릿 `SecurityConfig`와 같게) | `bbe944bd` · `src/adapter/inbound/web/.../error/EnvelopeErrorController.java`, `.../auth/SecurityConfig.java` | | B-32 | 새·바뀐 오류 코드: 비동기 요청 timeout 500 `INTERNAL_ERROR` → 503 `REQUEST_TIMEOUT`(retryable). `IDEMPOTENCY_KEY_INVALID` 400(`details.reason`=`MISSING`·`MALFORMED`, `IdempotencyKeyHeader.parse`를 쓰는 엔드포인트, sample은 `BAD_PARAMETER`에서 바뀜). web-advanced 예외 10종이 모두 500 `INTERNAL_ERROR`였다가 각자 코드(`ADMISSION_REFUSED` 503 등) | 코드로 분기하던 클라이언트가 새 코드를 모른다 | 클라이언트 분기표에 더한다. web-advanced 표는 `src/adapter/inbound/web-advanced/README.md` 「오류 응답」 | `1400dc15` · `src/adapter/inbound/web/.../error/WebTransportError.java`, `HttpStatusErrorCodes.java`, `src/adapter/inbound/web-advanced/.../error/WebAdvancedError.java` | | B-33 | DB 실패 분류: 낙관적 잠금 충돌 500 → 409 `DB_OPTIMISTIC_CONFLICT`. 커밋 단계 연결 끊김(08xxx, 40003 등)을 재시도 가능으로 답하던 것 → 500 `DB_COMPLETION_UNKNOWN`(retryable=false) | 응답 status·code가 바뀐다 | 409는 다시 읽고 재시도, `DB_COMPLETION_UNKNOWN`은 업무 데이터로 결과를 확인 | `34565130`, `1400dc15` · `src/adapter/outbound/persistence-jpa/.../failure/PersistenceExceptionTranslator.java` | @@ -99,7 +100,7 @@ |---|---|---|---|---| | B-34 | 템플릿 패키지 규칙(ArchUnit): `FEATURE_PACKAGES_DO_NOT_REUSE_TEMPLATE_PACKAGE_NAMES`, `TEMPLATE_PACKAGES_DO_NOT_DEPEND_ON_FEATURE_PACKAGES`. 템플릿 이름: application `usecase command query capability transaction security tenant observability secret messaging outbox idempotency operation lease lock cache realtime notification fileserver objectstorage`, domain `identifier stereotype` | 기능 패키지 이름이 이 목록과 겹치거나(예: `domain.lock`), 업무 코드를 템플릿 패키지(예: `application.usecase.*`)에 두고 기능 패키지를 import하면 `:app-bootstrap:architectureTest` 실패 | 기능 패키지 이름을 바꾸고, 업무 코드를 `dev.caskeleton.application.`로 옮긴다. 규칙 정본은 `src/application-core/CLAUDE.md` "Package ownership" | `5786126c` · `src/app-bootstrap/src/architectureTest/.../TemplatePackageOwnership.java` | | B-35 | JPA 규칙: 양방향 연관은 한쪽에서만 매핑(`mappedBy`), notification·fileserver 저장소 엔티티는 그 leaf 안에서만, `@Version` 엔티티의 bulk update는 version을 올려야 함 | 위반 엔티티·쿼리가 있으면 architectureTest 실패 | 매핑·쿼리를 고친다 | `34565130`, `1400dc15`, `bbe944bd` · `src/app-bootstrap/src/architectureTest/.../JpaProductionArchitectureTest.java` | -| B-36 | 빌드 검사 강화: `modules.json`에 없는 `build.gradle`이 있으면 `verifyCleanArchitectureDependencies` 실패. `verifyStrictLaneStages --dry-run`은 모든 strict lane이 workflow가 실행하는 단계(`:ci` `:sample-portfolio:check` `:mainCheck` `:integrationCheck` `:performanceCheck` `:releaseCheck`)에 속해야 하고 `.github/workflows`를 읽는다. SpotBugs 신뢰도 HIGH → MEDIUM | 등록 안 한 leaf, 단계에 안 걸린 자기 lane, 자기 코드의 MEDIUM 발견이 `ci`·`qualityCheck`를 막는다 | leaf를 등록하거나 지운다. 자기 lane은 단계에 건다. SpotBugs 발견은 고치거나 이유를 단 제외로 | `1400dc15`, `34565130`, `bbe944bd`, `5786126c` · `src/build-logic/.../ModuleRegistry.java`, `.../strictlane/StrictLaneStagesPlugin.java`, `.../quality/QualityConventionsPlugin.java`, `src/build.gradle` | +| B-36 | 빌드 검사 강화: `modules.json`에 없는 `build.gradle`이 있으면 `verifyCleanArchitectureDependencies` 실패. `verifyStrictLaneStages --dry-run`은 모든 strict lane이 workflow가 실행하는 단계(`:ci` `:sample-portfolio:check` `:mainCheck` `:integrationCheck` `:performanceCheck` `:releaseCheck`)에 속해야 하고 `.github/workflows`를 읽는다. 명령의 빌드는 `working-directory`(단계·job·workflow `defaults`)와 `-p`를 합친 디렉터리로 정하고, `workflow_dispatch`로만 시작하는 workflow의 명령은 세지 않으며, 축약한 `-x`도 Gradle처럼 맞춰 본다(RR2-04-04). `verifyEnvKeys`: lane 계약 json의 `APP_` 밖 키는 `SPRING_`·`MANAGEMENT_`·`SERVER_`이거나 `secret://…/`가 가리키는 변수여야 하고, 레지스트리 `property:` 행은 그 property가 설정 metadata에 있어야 읽히는 것으로 친다(`@ConditionalOnProperty`로만 읽는 스위치는 `META-INF/additional-spring-configuration-metadata.json`에 선언, RR2-04-05). SpotBugs 신뢰도 HIGH → MEDIUM | 등록 안 한 leaf, 단계에 안 걸린 자기 lane, 자기 코드의 MEDIUM 발견이 `ci`·`qualityCheck`를 막는다. `working-directory: src` 없이 `./gradlew`를 부르는 자기 workflow 단계는 실행으로 세지 않는다 | leaf를 등록하거나 지운다. 자기 lane은 단계에 건다. 단계를 부르는 workflow 명령에 `working-directory`를 둔다. SpotBugs 발견은 고치거나 이유를 단 제외로 | `1400dc15`, `34565130`, `bbe944bd`, `5786126c` · `src/build-logic/.../ModuleRegistry.java`, `.../strictlane/StrictLaneStagesPlugin.java`, `.../quality/QualityConventionsPlugin.java`, `src/build.gradle` | | B-37 | app-bootstrap 테스트 lane 분리: 저장소 파일을 읽는 테스트는 태그 `document-contract`(main 단계) 또는 `repository-contract`(`check`)여야 하고, 일반 `test` lane에는 `ca.repository.root`가 없다 | app-bootstrap에 둔 자기 테스트가 저장소 파일을 읽으면 `test`에서 실패 | 태그를 단다 | `1400dc15` · `src/app-bootstrap/build.gradle` | ### 3.5 모듈 registry·새 leaf @@ -114,26 +115,37 @@ | stream (history 표) | 버전 | 하는 일 | 기존 데이터 영향 | 실패 조건 | 근거 | |---|---|---|---|---|---| -| postgresql (`flyway_schema_history`, persistence-jpa 선택 시) | V20 | `outbox_event.deferred_attempt_count INTEGER NOT NULL DEFAULT 0` + CHECK ≥ 0 | 기존 행 0. 지난 claim은 전부 시도 예산에 센다 | 확인된 것 없음 | `1400dc15` | +| postgresql (`flyway_schema_history`, persistence-jpa 선택 시) | V20 | `outbox_event.deferred_attempt_count INTEGER NOT NULL DEFAULT 0` + CHECK ≥ 0 | 기존 행 0. 지난 claim은 전부 시도 예산에 센다 | `outbox_event`를 건드린 트랜잭션이 열려 있어 5초 안에 잠금을 못 잡으면 실패·롤백, V19에 남음(exit 70) | `1400dc15`, 3차 수정(RR-02-09) | | 〃 | V21 | `outbox_aggregate_sequence`(aggregate별 counter), `outbox_aggregate_sequence_floor`(V21 적용 시각 마이크로초 한 행), 남은 행 최대 번호로 counter 미리 생성, 유일 인덱스 `ux_outbox_event_aggregate_order (tenant_scope, aggregate_type, aggregate_id, aggregate_sequence, event_index)` | 새로 생기는 aggregate의 첫 번호가 floor+1(약 1.7×10¹⁵)부터. 순서 열이 NULL인 옛 행은 비교 안 함 | ① 같은 순서 키 행이 이미 둘 이상 → 실패, V21 전체 롤백, V20에 남음(exit 70). ② `SET LOCAL lock_timeout = '5s'` 안에 `outbox_event` 잠금을 못 잡으면 실패·롤백. ③ 같은 이름 인덱스가 유효한 유일 인덱스가 아니면(실패한 CONCURRENTLY) `RAISE EXCEPTION` | `1400dc15`, `bbe944bd` · `docs/runbooks/outbox-dead-letter.md` §8 | -| 〃 | V22 | `outbox_event.last_failure_code VARCHAR(120)` NULL, 기본값 없음. 새 상태 `PARKED`는 DDL 없음 | 행 재작성 없음. 기존 행은 다음 실패 때까지 NULL | 확인된 것 없음 | `bbe944bd` | -| notification (`flyway_jpa_notification_history`, notification-platform 선택 시) | V11 | `notification_reconciliation_job.claimed_until timestamptz`, `claim_fence bigint NOT NULL DEFAULT 0` + CHECK | 기존 job은 fence 0, lease 없음 | 확인된 것 없음 | `34565130` | -| 〃 | V12 | `notification_suppression`의 `(tenant_id, normalized_target_hmac, reason)` 중복 행을 **삭제**(만료 없는 것 > 늦은 만료 > 이른 `effective_at` > 이른 `created_at` > 작은 id 순으로 하나 남김) 후 UNIQUE `uk_notification_suppression_target` | 지운 행의 `SuppressionId`는 더 이상 조회되지 않는다. 되돌리려면 백업 필요 | 확인된 것 없음(잠금 대기 제한 없음) | `1400dc15` | -| 〃 | V13 | `notification_request.acceptance varchar(40) NOT NULL DEFAULT 'ACCEPTED'` + CHECK(`ACCEPTED`, `DROPPED_AS_DUPLICATE`) | 기존 요청은 모두 `ACCEPTED` | 확인된 것 없음 | `5786126c` | -| sample (`flyway_sample_schema_history`, sample-portfolio를 남긴 경우) | V8_1 | 유일 인덱스 `uq_poster_title_ci ON poster (upper(title))` | 없음 | 대소문자만 다른 제목이 있으면 `poster has titles that differ only by case or repeat: ...`로 실패 | `34565130` | -| grpc-ledger (optional-platforms, history 표 확인 못 함) | V002 | 부분 인덱스 `ix_grpc_operation_ledger_finished_completed_at` (일반 `CREATE INDEX`) | 없음 | 큰 ledger면 만드는 동안 쓰기가 멈춘다. 파일 주석은 손으로 `CONCURRENTLY`를 먼저 돌리라고 한다 | `1400dc15` | -| 〃 | V003 | `grpc_operation_ledger.claim_attempt BIGINT NOT NULL DEFAULT 1` + CHECK > 0 | 기존 행은 첫 시도(1) | 확인된 것 없음 | `bbe944bd` | +| 〃 | V22 | `outbox_event.last_failure_code VARCHAR(120)` NULL, 기본값 없음. 새 상태 `PARKED`는 DDL 없음 | 행 재작성 없음. 기존 행은 다음 실패 때까지 NULL | `outbox_event`를 건드린(읽기만 한 것도) 트랜잭션이 열려 있어 5초(`SET LOCAL lock_timeout`) 안에 잠금을 못 잡으면 실패·롤백, V21에 남음(exit 70) | `bbe944bd`, 3차 수정(RR-02-09) | +| notification (`flyway_jpa_notification_history`, notification-platform 선택 시) | V11 | `notification_reconciliation_job.claimed_until timestamptz`, `claim_fence bigint NOT NULL DEFAULT 0` + CHECK | 기존 job은 fence 0, lease 없음 | 그 표를 건드린 트랜잭션이 열려 있어 5초 안에 잠금을 못 잡으면 실패·롤백, V10에 남음(exit 70) | `34565130`, 3차 수정(RR-02-09) | +| 〃 | V12 | `notification_suppression`의 `(tenant_id, normalized_target_hmac, reason)` 중복 행을 **삭제**(만료 없는 것 > 늦은 만료 > 이른 `effective_at` > 이른 `created_at` > 작은 id 순으로 하나 남김) 후 UNIQUE `uk_notification_suppression_target` | 지운 행의 `SuppressionId`는 더 이상 조회되지 않는다. 되돌리려면 백업 필요 | 그 표를 건드린 트랜잭션이 열려 있어 5초 안에 잠금(행 잠금 포함)을 못 잡으면 실패·롤백(삭제도 롤백), V11에 남음(exit 70) | `1400dc15`, 3차 수정(RR-02-09) | +| 〃 | V13 | `notification_request.acceptance varchar(40) NOT NULL DEFAULT 'ACCEPTED'` + CHECK(`ACCEPTED`, `DROPPED_AS_DUPLICATE`) | 기존 요청은 모두 `ACCEPTED` | 그 표를 건드린 트랜잭션이 열려 있어 5초 안에 잠금을 못 잡으면 실패·롤백, V12에 남음(exit 70) | `5786126c`, 3차 수정(RR-02-09) | +| sample (`flyway_sample_schema_history`, sample-portfolio를 남긴 경우) | V8_1 | 유일 인덱스 `uq_poster_title_ci ON poster (upper(title))` | 없음 | 대소문자만 다른 제목이 있으면 `poster has titles that differ only by case or repeat: ...`로 실패. `poster`에 쓴 트랜잭션이 열려 있어 5초 안에 잠금을 못 잡으면 실패·롤백, V8에 남음 | `34565130`, 3차 수정(RR-02-09) | +| grpc-ledger (optional-platforms, history 표 확인 못 함) | V002 | 부분 인덱스 `ix_grpc_operation_ledger_finished_completed_at` (일반 `CREATE INDEX`) | 없음 | 큰 ledger면 만드는 동안 쓰기가 멈춘다. 파일 주석은 손으로 `CONCURRENTLY`를 먼저 돌리라고 한다. ledger에 쓴 트랜잭션이 열려 있어 5초 안에 잠금을 못 잡으면 실패·롤백, V001에 남음 | `1400dc15`, 3차 수정(RR-02-09) | +| 〃 | V003 | `grpc_operation_ledger.claim_attempt BIGINT NOT NULL DEFAULT 1` + CHECK > 0 | 기존 행은 첫 시도(1) | ledger를 건드린 트랜잭션이 열려 있어 5초 안에 잠금을 못 잡으면 실패·롤백, V002에 남음 | `bbe944bd`, 3차 수정(RR-02-09) | - **새로 연결된 stream**: `adapter:inbound:messaging`을 `runtimeOnly`로 넣으면 platform의 `V1101__messaging_inbox.sql`을 `MessagingInboxSchemaStream`이 `flyway_messaging_inbox_history`에 적용한다(baseline 1100). 넣지 않으면 영향 없음 (`src/adapter/inbound/messaging/README.md`). +- **잠금 대기 상한(3차 수정, RR-02-09)**: 위 표에서 이미 쓰이는 테이블을 잠그는 마이그레이션은 모두 그 잠금 앞에 + `SET LOCAL lock_timeout = '5s'`를 둔다. 실패하면 SQLState `55P03`이고, 열린 트랜잭션을 끝낸 뒤 다시 배포한다 + (`docs/runbooks/migration-failed.md` "Classify the failure"). 자기 스트림에도 같은 규칙과 검사 + (`LockWaitMigrationInspector`, 예: `SampleMigrationLockWaitTest`)를 둔다(`docs/jpa/migration-guide.md` "Bound the lock wait"). - **번호 충돌**: 템플릿이 이제 postgresql V20~V22, notification V11~V13, sample V8_1, grpc V002·V003을 쓴다. 파생 서비스가 같은 위치에 같은 번호를 두었다면 Flyway가 같은 버전 두 개로 실패한다. 자기 마이그레이션은 템플릿과 떨어진 번호대로 옮긴다 (이미 적용된 파일 이름을 바꾸면 checksum·history가 어긋나므로 운영 DB에 적용된 것이면 DB 담당과 정한다). - **migration-on-startup=false로 운영하는 서비스**: `infra/postgres/apply-capability-streams.sh`가 바뀌었다(stream을 `/flyway/streams/`에서 먼저 찾음). 운영자가 직접 적용한다. -### 4.2 이 브랜치의 이전 V21을 적용한 DB +### 4.2 이 브랜치의 이전 V21(과 3차 수정 전 V20·V22·V11~V13·V8_1·V002·V003)을 적용한 DB + +3차 수정(RR-02-09)은 이 브랜치가 더한 마이그레이션 중 이미 쓰이는 테이블을 잠그는 여덟 개(postgresql V20·V22, +notification V11·V12·V13, sample V8_1, grpc-ledger V002·V003)에 `SET LOCAL lock_timeout`을 넣으려고 파일을 직접 +고쳤다. V21과 같은 전제(어느 배포에도 적용되지 않음)다. 그 이전 내용을 적용한 개발 DB도 아래 V21과 같이 checksum +불일치로 기동이 실패하므로 다시 만든다. 새 checksum은 `applied-migrations.sha256`에 있다(sample·grpc-ledger는 고정 목록 +없음). + V21은 `bbe944bd`에서 V22를 따로 두지 않고 직접 고쳤다(floor 표와 counter 미리 만들기를 넣음). `1400dc15`·`5786126c` 시점의 V21(sha256 `f830cda5…`)을 적용한 DB는 새 V21(sha256 `21744ea1…`, `src/adapter/outbound/persistence-jpa/src/test/resources/db/applied-migrations.sha256`)과 @@ -182,7 +194,7 @@ V21 실패의 분류는 `docs/runbooks/migration-failed.md` "Classify the failur | 영역 | 바뀐 것 | 영향 | 근거 | |---|---|---|---| -| outbox relay | 일시 거부(`REJECTED_RETRYABLE`)·무응답(`AMBIGUOUS`)은 시도 예산을 쓰지 않고 최대 5분 간격(`OutboxBackoffPolicy.MAX_DELAY`)으로 끝없이 재시도. 실패 원인 코드를 `last_failure_code`에 남김. 수동 보류 상태 `PARKED`. 지표 `outbox.retry.long.size`·`outbox.retry.long.age` | 진짜 인증 오류도 DEAD가 아니라 계속 재시도한다. 경보는 새 지표로 잡는다. 처분 SQL은 `docs/runbooks/outbox-dead-letter.md`·`outbox-publish-failed.md` | `34565130`, `1400dc15`, `bbe944bd` | +| outbox relay | 일시 거부(`REJECTED_RETRYABLE`)·무응답(`AMBIGUOUS`)은 시도 예산을 쓰지 않고 최대 5분 간격(`OutboxBackoffPolicy.MAX_DELAY`)으로 끝없이 재시도. 실패 원인 코드를 `last_failure_code`에 남김. 수동 보류 상태 `PARKED`. 지표 `outbox.retry.long.size`·`outbox.retry.long.age`. claim이 DEAD로 옮긴 행(`ATTEMPT_COUNTERS_INCONSISTENT`·`UNREADABLE_ROW`)도 그 이유를 `last_failure_code`에 쓰고, 경보에 `outbox.pending.size{status=DEAD}` > 0을 더함(RR-02-12). 이 버전이 모르는 상태 값은 `outbox.pending.size`에서 빼고 WARN 한 번(RR-02-11) | 진짜 인증 오류도 DEAD가 아니라 계속 재시도한다. 경보는 새 지표로 잡는다. 처분 SQL은 `docs/runbooks/outbox-dead-letter.md`·`outbox-publish-failed.md` | `34565130`, `1400dc15`, `bbe944bd` | | outbox append | 읽기 전용 트랜잭션 안의 `append`도 거부(`CallerTransaction.requireReadWrite`). 전에는 트랜잭션 밖만 거부했고 읽기 전용에서는 이벤트가 조용히 사라졌다 | `inRead` 안에서 이벤트를 쓰던 코드가 이제 예외로 드러난다 | `5786126c` · `JpaOutboxStore.java` | | 순서 번호 | aggregate 순서 번호를 DB counter가 매긴다(`OutboxAggregateSequencePort.nextSequence`). 한 aggregate에 이벤트가 하나면 `AggregateOrder.first()`. 같은 순서 키 두 번째 append는 유일 인덱스가 거부(정확한 오류 코드 확인 못 함) | 소비자가 보는 번호가 새 aggregate에서 floor+1부터 시작해 크게 뛴다. 벽시계로 번호를 만들던 코드는 바꾼다 | `1400dc15`, `bbe944bd` | | durable operation | operation id가 UUIDv7(`UuidV7DurableOperationIdFactory`). 제출 이벤트의 event id·aggregate id가 UUIDv7이고 tenant scope는 `global` | 전에는 이 이벤트가 messaging bridge에서 전부 DEAD가 됐다(커밋 `1400dc15` 본문). 소비자가 보는 id 모양이 바뀐다 | `1400dc15`, `bbe944bd` · `SubmitDurableOperationUseCase.java` | @@ -192,7 +204,7 @@ V21 실패의 분류는 `docs/runbooks/migration-failed.md` "Classify the failur | GraphQL | 크기를 선언하지 않은 list를 원소 20개로 계산, `first`·`last` 둘 다면 큰 쪽. per-parent 경고 임계 10(`APP_GRAPHQL_PER_PARENT_FETCH_WARNING_THRESHOLD`). deadline instrumentation이 `REQUEST_TIMEOUT` | 전에 통과하던 깊은 목록 쿼리가 복잡도로 거부될 수 있다. 필요한 필드에 `estimatedListSize`를 선언 | `34565130`, `1400dc15`, `bbe944bd` | | WebSocket | 노드당 STOMP 세션 1000개(`APP_WEBSOCKET_MAX_CONNECTIONS`), 초과 시 연결 후 1013으로 닫음. 메시지 64KB, 송신 버퍼 512KB, 송신 시간 10s, 첫 메시지 60s를 명시 | 동시 연결이 많은 서비스는 상한을 올린다 | `1400dc15` · `env-keys.yaml` | | gRPC 서버(inbound adapter) | 수신 메시지 1 MiB(io.grpc 기본 4 MiB에서 줄어듦), 메타데이터 8 KiB, 연결당 동시 호출 100, 실행 스레드 32·대기열 256, 기본 deadline 10s, keepalive 30s/10s, 최대 연결 수명 30분, drain 2s | 1 MiB를 넘는 메시지는 거부된다 | `1400dc15` · `src/adapter/inbound/grpc/.../GrpcServerSettings.java` | -| 파일 서버 | 정리 worker 기본 켜짐(`APP_FILESERVER_PLATFORM_CLEANUP_ENABLED` false→true, 끄면 기동 경고). copy·move 대상은 기본 namespace만(`APP_FILESERVER_PLATFORM_TARGET_NAMESPACES`, 그 밖은 `ACCESS_DENIED`). namespace 쿼터 상한(`APP_FILESERVER_PLATFORM_QUOTA_NAMESPACE_LIMIT`, 비우면 무제한, 넘으면 413 `QUOTA_EXCEEDED`). move가 쿼터를 옮긴다 | 다른 namespace로 copy·move하던 서비스는 허용 목록을 설정해야 한다. 정리 worker가 물리 객체를 지운다 | `5786126c`, `bbe944bd` · `CopyFileApplicationUseCase.java`, `env-keys.yaml` | +| 파일 서버 | 정리 worker 기본 켜짐(`APP_FILESERVER_PLATFORM_CLEANUP_ENABLED` false→true, 끄면 기동 경고). copy·move 대상은 기본 namespace만(`APP_FILESERVER_PLATFORM_TARGET_NAMESPACES`, 그 밖은 `ACCESS_DENIED`). namespace 쿼터 상한(`APP_FILESERVER_PLATFORM_QUOTA_NAMESPACE_LIMIT`, 비우면 무제한, 넘으면 413 `QUOTA_EXCEEDED`). move가 쿼터를 옮긴다 | 다른 namespace로 copy·move하던 서비스는 허용 목록을 설정해야 한다. 정리 worker가 물리 객체를 지운다. 이 수정 전에 다른 namespace로 move된 파일의 바이트는 여전히 출발 namespace 사용량에 잡혀 있고 도착 namespace에는 없다 — 그 파일을 지우면 도착 namespace에서 회수되어(0에서 멈춘다) 출발 쪽은 그대로 남는다. 상한을 켜기 전에 `docs/fileserver/operations.md` 「Namespace quota drift」의 쿼리로 namespace별 차이를 보고, 있으면 같은 절의 재계산 SQL로 고친다(쓰기를 멈춘 뒤, 재리뷰 RR-01-17) | `5786126c`, `bbe944bd` · `CopyFileApplicationUseCase.java`, `env-keys.yaml` | | 알림 | SES 콜백 replay 허용 차이 5m→1h. 중복 제거 창에서 버린 제출을 `CANCELED`·`DROPPED_AS_DUPLICATE`로 저장해 같은 멱등성 키에 같은 영수증. reconciliation job은 DB lease로 소유. INGEST_ONLY도 accept 때 route 이름 기록 | 콜백·영수증 결과가 바뀐다 | `34565130`, `5786126c` | | Redis | 캐시 조건부 쓰기가 자기 SCRIPT lane(`APP_CACHE_REDIS_MAXIMUM_SCRIPT_CONNECTIONS`, 기본 4)을 쓴다. registry에서 `redis.capability.*` 지표 6개를 빼고 `redis.lane.*` 6개를 더했다 | Redis 연결이 캐시 lane만큼 늘어난다. 옛 지표 이름으로 만든 대시보드·경보는 바꾼다 | `bbe944bd` · `docs/registries/metrics.yaml` | | 설정 바인딩 | strict 설정에서 선택 숫자·크기·기간 키의 빈 값은 "미설정". 목록 원소 아래 오타 키도 거부(`StrictSettingsBinder`) | `.env.example`의 빈 값으로 기동 가능. 목록 원소 아래 오타는 이제 기동 실패 | `5786126c` · `StrictSettingsBinder.java` | diff --git a/docs/web/http-contract.md b/docs/web/http-contract.md index 3e8c1f9b..de18900b 100644 --- a/docs/web/http-contract.md +++ b/docs/web/http-contract.md @@ -56,9 +56,16 @@ JSON 응답은 모두 `Envelope` 하나의 모양이다(소유자 결정 2). ** 컨트롤러에 닿기 전에 거부된 요청(보안 방화벽이 막은 경로, 필터의 `sendError`, 필터가 던진 예외)도 같은 실패 Envelope다. 컨테이너가 `/error`로 넘긴 요청을 `EnvelopeErrorController`가 status에 맞는 코드로 답한다. -본문에 요청 경로나 시각을 싣지 않는다. 원래 status가 그대로 나가고, 기본 JWT 체인에서도 익명 400이 401로 -바뀌지 않는다(재리뷰 RR-02-02). 다만 요청 id 필터를 지나기 전에 끊긴 응답은 `meta`의 id 키가 빠질 수 있다. -클라이언트는 `meta`의 모르는 키를 무시한다. +본문에 요청 경로나 시각을 싣지 않는다. 기본 JWT 체인에서도 익명 400이 401로 바뀌지 않는다(재리뷰 RR-02-02). +다만 요청 id 필터를 지나기 전에 끊긴 응답은 `meta`의 id 키가 빠질 수 있다. 클라이언트는 `meta`의 모르는 키를 +무시한다. + +status는 코드 표(`HttpStatusErrorCodes`)에 있는 것만 그대로 나간다: 401·403·404·405·406·412·413·414·415·429·503. +표에 없는 status는 그 class의 코드로 접힌다 — 4xx(408·409·410·422 등)는 `BAD_PARAMETER` 400, 5xx(501·502·504 +등)는 `INTERNAL_ERROR` 500(재리뷰 RR-01-13, `EnvelopeErrorControllerTest`). `error.code`와 status가 registry +(`docs/registries/error-codes.yaml`)와 어긋나지 않게 하려는 것이다. 클라이언트는 status가 아니라 `error.code`로 +가르기 때문이다. 필터가 다른 status를 내야 하면 `sendError` 대신 그 status의 코드로 `EnvelopeResponseWriter`를 +쓰거나, 표에 행을 더한다. 4xx·5xx status로 나가는 본문은 `success:true`로 감싸지 않는다. 실패 Envelope가 아닌 본문을 오류 status로 돌려주는 핸들러는 그 본문이 그대로 나간다. @@ -105,13 +112,19 @@ JSON 응답은 모두 `Envelope` 하나의 모양이다(소유자 결정 2). ** | 작업 실행 뒤 결과 기록(complete 단계) 중 저장소 응답 없음 — **결과 불명** | 503 | `IDEMPOTENCY_COMPLETION_UNKNOWN` | `true` | 없음 | "적용되지 않았다"고 알리지 않는다. **같은 키로** 나중에 다시 보낸다. 새 키 금지 | | 이전 시도의 효과를 증명할 수 없음 | 500 | `IDEMPOTENCY_RECOVERY_REQUIRED` | `false` | 없음 | 재전송도 재실행도 안전하지 않다. 업무 데이터로 결과를 확인한다 | -- **결과 불명이 무엇을 뜻하나.** 작업은 이미 돌았다. 그 작업의 트랜잭션 안 DB 변경은 예외와 함께 - 롤백되지만, 트랜잭션 밖 효과(다른 시스템 호출, 메시지)는 일어났을 수 있다. 그래서 실행 전 거부와 같은 +- **결과 불명이 무엇을 뜻하나.** 작업은 이미 돌았다. 유스케이스가 executor를 쓰기 트랜잭션 안에서 + 불렀다면 그 트랜잭션의 DB 변경은 예외와 함께 롤백된다. 작업이 자기 트랜잭션을 따로 커밋했다면 그 변경은 + 남는다. 트랜잭션 밖 효과(다른 시스템 호출, 메시지)는 일어났을 수 있다. 그래서 실행 전 거부와 같은 503이어도 code가 다르다. 클라이언트는 status가 아니라 `error.code`로 가른다. -- **같은 키로 다시 보내면 서버가 판정한다.** 결과가 실제로는 기록됐으면 재생(첫 요청의 성공 응답), 첫 시도의 - processing lease가 아직 남았으면 409 `IDEMPOTENT_IN_FLIGHT`, lease가 지났으면 500 - `IDEMPOTENCY_RECOVERY_REQUIRED`다. 새 키로 보내면 서버는 다른 요청으로 보고 막지 못한다 - (재리뷰 2026-11-03 README 4절 결정, RR-02-03). +- **같은 키로 다시 보내면 서버가 판정하고, 작업을 다시 돌리지 않는다.** 결과가 실제로는 기록됐으면 + 재생(첫 요청의 성공 응답), 첫 시도의 processing lease(`processing-lease`, 기본 30s)가 아직 남았으면 409 + `IDEMPOTENT_IN_FLIGHT`, lease가 지났으면 500 `IDEMPOTENCY_RECOVERY_REQUIRED`다. lease가 지난 실행 기록을 다음 + 요청이 만나면 두 저장소(PostgreSQL·Redis) 모두 넘겨주지 않고 포기(abandoned)로 표시한다 — 그 뒤 첫 시도가 + 늦게 끝나도 결과는 기록되지 않고 그 요청도 500을 받는다. 새 키로 보내면 서버는 다른 요청으로 보고 막지 못한다 + (재리뷰 2026-11-03 README 4절 결정 RR-02-03, 2026-11-04 9.1절 결정 RR-01-11). +- **서버가 기억하는 기간.** 완료된 응답은 완료부터 `replay-ttl`(기본 24h), 끝나지 않은 시도의 기록은 + claim부터 processing lease + `replay-ttl` 동안 남는다. 그 뒤 같은 키는 새 요청으로 실행된다. + `replay-ttl`이 `processing-lease`보다 짧으면 기동이 실패한다(재리뷰 2026-11-04 RR-03-11). ## 3. 헤더 diff --git a/infra/redis-sdk/acl/README.md b/infra/redis-sdk/acl/README.md index 0117e9b1..6c20d498 100644 --- a/infra/redis-sdk/acl/README.md +++ b/infra/redis-sdk/acl/README.md @@ -8,9 +8,12 @@ a real Redis, that line alone gives anyone who can reach the server and read thi control of it (`CONFIG`, `MODULE`, every key). A deployment writes its own ACL file of the same shape: the application accounts it actually uses, passwords from its secret store, and none of the lane-only accounts (review 2026-11-03 RR-07-04). The application enforces its half of that: under -the `prod` runtime profile it refuses to start while any setting or environment variable holds a -value beginning with `fixture-` (`SecretSourceValidator`), so a deployment that copied a lane's -`REDIS_LANE_*` values stops before it connects. +the `prod` runtime profile it refuses to start while a secret holds a value beginning with +`fixture-` (`SecretSourceValidator`): a key the secrets registry classifies `secret`, or the +variable a `secret://…/` credential reference names. A deployment that copied a lane's +`APP_REDIS_AUTHENTICATION_*_REFERENCE` and `REDIS_LANE_*` values stops before it connects. Values +that are not secrets — a pod's `HOSTNAME`, an application name — are not read, because `fixture-` +is an ordinary word as well. One file per `CommandAccess` level. Each account is deliberately narrower than the SDK's own rules: the account is the last enforcement boundary and a permit issued inside the process never widens it, diff --git a/src/.env.example b/src/.env.example index 200bb942..e1b49131 100644 --- a/src/.env.example +++ b/src/.env.example @@ -121,6 +121,7 @@ APP_RATE_LIMIT_REFILL_TOKENS=100 APP_RATE_LIMIT_REFILL_PERIOD=1s APP_RATE_LIMIT_MAXIMUM_COST=10 APP_RATE_LIMIT_CLEANUP_GRACE=5s +# Replay TTL: at most 72h and not shorter than APP_IDEMPOTENCY_PROCESSING_LEASE (startup fails). APP_IDEMPOTENCY_TTL=24h # disabled | redis | postgresql. postgresql refuses a non-PostgreSQL data source, so bootRunH2 # runs with disabled or redis. @@ -294,7 +295,8 @@ APP_GRAPHQL_PER_PARENT_FETCH_WARNING_THRESHOLD=10 # Optional build-composition: these bind only when :adapter:inbound:websocket is present. # The default bootJar does not contain that leaf. APP_WEBSOCKET_ENABLED=true -# Explicit offs for two more capabilities that are on when present (the Compose lanes set them). +# Switches for two more capabilities that are on when present: `true` here is their default, and +# `false` is the explicit off the Compose lanes set. APP_OBJECTSTORAGE_ENABLED=true APP_GRPC_SERVER_ENABLED=true APP_WEBSOCKET_ENDPOINT=/ws diff --git a/src/Dockerfile.sample b/src/Dockerfile.sample index 7c04fa00..064714f5 100644 --- a/src/Dockerfile.sample +++ b/src/Dockerfile.sample @@ -5,14 +5,20 @@ # Builds and runs the REFERENCE app (SamplePortfolioApplication), not the # production composition root (CaSkeletonApplication) that src/Dockerfile builds. # The sample is the demo-friendly entrypoint: its application.yml self-provides -# defaults for every env placeholder, so the only external dependency it needs -# to boot is a reachable PostgreSQL (datasource + Flyway sample migrations). +# defaults for every env placeholder, so what it needs to boot is a demo profile +# and a reachable PostgreSQL (datasource + Flyway sample migrations). +# +# The profile is not baked in. The demo accounts (a published admin password) start only under an +# explicitly active `demo`, `local` or `test` profile (SampleDemoAccountsProductionGuard), and an +# image that defaulted to one would carry them into any deployment that ran it as is. Without +# SPRING_PROFILES_ACTIVE the container stops with exit code 71 and says so. # # Build (no build-args required — this is a disposable demo, not a release artifact): # docker build -f src/Dockerfile.sample src/ -t ca-sample:local # # Run (point APP_DATASOURCE_URL at a reachable Postgres; localhost default shown): # docker run --rm -p 8080:8080 -p 9001:9001 \ +# -e SPRING_PROFILES_ACTIVE=demo \ # -e APP_DATASOURCE_URL=jdbc:postgresql://host.docker.internal:5432/ca_skeleton \ # ca-sample:local # diff --git a/src/README.md b/src/README.md index 5e77ea3c..a1ffcf78 100644 --- a/src/README.md +++ b/src/README.md @@ -236,7 +236,8 @@ drift하기 시작하면 그때 자동화합니다. 진짜 항목을 상대로, `postgresql` 은 PostgreSQL data source 가 아니면 기동을 거부하므로 `bootRunH2` 에서는 `disabled` 나 `redis` 를 씁니다. - **`APP_IDEMPOTENCY_TTL`** — 완료 응답을 재생할 수 있는 기간(`replay-ttl`). duration(예: `24h`). - 최대 72h 이고 넘으면 기동을 거부합니다. (D6) + 최대 72h 이고 넘으면 기동을 거부합니다. (D6) `APP_IDEMPOTENCY_PROCESSING_LEASE`(기본 30s)보다 + 짧아도 기동을 거부합니다. ### Async executor diff --git a/src/adapter/inbound/graphql/src/main/java/dev/caskeleton/adapter/inbound/graphql/policy/GraphQlClientPolicy.java b/src/adapter/inbound/graphql/src/main/java/dev/caskeleton/adapter/inbound/graphql/policy/GraphQlClientPolicy.java index 71a9c11b..d00ea632 100644 --- a/src/adapter/inbound/graphql/src/main/java/dev/caskeleton/adapter/inbound/graphql/policy/GraphQlClientPolicy.java +++ b/src/adapter/inbound/graphql/src/main/java/dev/caskeleton/adapter/inbound/graphql/policy/GraphQlClientPolicy.java @@ -23,7 +23,10 @@ import java.time.Duration; * @param maxFragments most accepted fragment definitions * @param maxInputListElements most accepted elements in one input list * @param defaultPageSize page size used for cost when a connection omits {@code first}/{@code last} - * @param maxPageSize largest accepted page size + * @param maxPageSize largest accepted page size. This record only holds the two numbers; the page a + * resolver serves is decided by {@code GraphQlConnectionPolicy.effectivePageSize} and the page + * a document is charged for by the complexity scorer — one rule each, not a third copy here + * (review 2026-11-03 RR-01-14) * @param maxComplexity largest accepted pre-execution complexity score * @param maxExecutionTime request execution budget * @param introspectionAllowed whether this profile may run introspection @@ -92,20 +95,6 @@ public record GraphQlClientPolicy( false); } - /** - * The effective page size for a connection request. - * - *

An omitted {@code first}/{@code last} must cost the default page size rather than one, or a - * client could bypass the cost budget simply by leaving the argument out. - */ - public int effectivePageSize(Integer first, Integer last) { - int requested = first != null ? first : last != null ? last : defaultPageSize; - if (requested < 1) { - throw new IllegalArgumentException("page size must be positive"); - } - return requested; - } - private static void requirePositive(long value, String name) { if (value < 1) { throw new IllegalArgumentException(name + " must be positive"); diff --git a/src/adapter/inbound/graphql/src/test/java/dev/caskeleton/adapter/inbound/graphql/pagination/GraphQlConnectionAssemblerTest.java b/src/adapter/inbound/graphql/src/test/java/dev/caskeleton/adapter/inbound/graphql/pagination/GraphQlConnectionAssemblerTest.java index 09b63d6f..a3217aa2 100644 --- a/src/adapter/inbound/graphql/src/test/java/dev/caskeleton/adapter/inbound/graphql/pagination/GraphQlConnectionAssemblerTest.java +++ b/src/adapter/inbound/graphql/src/test/java/dev/caskeleton/adapter/inbound/graphql/pagination/GraphQlConnectionAssemblerTest.java @@ -91,6 +91,18 @@ class GraphQlConnectionAssemblerTest { .hasMessageContaining("exceeds the maximum"); } + /** The {@code last} case of the page-size rule that used to be copied in the client policy. */ + @Test + void aBackwardPageIsSizedByItsLastAndAnOmittedOneByTheDefault() { + var policy = GraphQlConnectionPolicy.from(clientPolicy()); + + assertThat( + List.of( + policy.effectivePageSize(GraphQlConnectionRequest.backward(30, null)), + policy.effectivePageSize(GraphQlConnectionRequest.backward(null, null)))) + .containsExactly(30, 20); + } + @Test void totalCountIsOptInRatherThanAlwaysComputed() { var connection = diff --git a/src/adapter/inbound/graphql/src/test/java/dev/caskeleton/adapter/inbound/graphql/policy/GraphQlClientPolicyTest.java b/src/adapter/inbound/graphql/src/test/java/dev/caskeleton/adapter/inbound/graphql/policy/GraphQlClientPolicyTest.java index cb3dbed2..785d02e4 100644 --- a/src/adapter/inbound/graphql/src/test/java/dev/caskeleton/adapter/inbound/graphql/policy/GraphQlClientPolicyTest.java +++ b/src/adapter/inbound/graphql/src/test/java/dev/caskeleton/adapter/inbound/graphql/policy/GraphQlClientPolicyTest.java @@ -38,15 +38,6 @@ class GraphQlClientPolicyTest { .isInstanceOf(IllegalArgumentException.class); } - @Test - void omittedPageSizeCostsTheDefaultRatherThanOne() { - GraphQlClientPolicy policy = policy(65536, 20, 100, Duration.ofSeconds(5)); - - assertThat(policy.effectivePageSize(null, null)).isEqualTo(20); - assertThat(policy.effectivePageSize(50, null)).isEqualTo(50); - assertThat(policy.effectivePageSize(null, 30)).isEqualTo(30); - } - @Test void productionProfileCanRequireNamedOperations() { GraphQlClientPolicy publicProfile = diff --git a/src/adapter/inbound/messaging/CLAUDE.md b/src/adapter/inbound/messaging/CLAUDE.md index 44f14572..a13b56d8 100644 --- a/src/adapter/inbound/messaging/CLAUDE.md +++ b/src/adapter/inbound/messaging/CLAUDE.md @@ -10,6 +10,9 @@ else: the platform decodes what its `destination()` and `schemaVersions()` declare. - Boundaries: no business rule in a handler (HARD-STOP 4); no persistence entity or repository; no broker SDK type. Allowed edges come only from the registry entry. +- A handler that calls a write use case is `@SystemCaller(reason = ...)` and constructs the + `@SystemInvoked` use case itself; the use case is never a bean (manual 09 「소비」 step 4, + `ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES`, `SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS`). - Tests: focused auto-configuration and inbox behavior tests; `./gradlew :adapter:inbound:messaging:check`. The real Kafka + PostgreSQL redelivery lane is `messagingInboxRedeliveryIntegrationTest` (`:integrationCheck`). diff --git a/src/adapter/inbound/messaging/README.md b/src/adapter/inbound/messaging/README.md index 042b5d84..3443f2a8 100644 --- a/src/adapter/inbound/messaging/README.md +++ b/src/adapter/inbound/messaging/README.md @@ -31,6 +31,10 @@ Kafka ─▶ platform consumer (starter-kafka) 만든다. platform은 애플리케이션의 datasource·트랜잭션을 스스로 고르지 않는다는 기존 설계를 그대로 둔다. - 애플리케이션의 handler(`MessageHandler` 구현)는 이 leaf에 둔다. 컨트롤러가 HTTP 요청을 use case 호출로 바꾸듯 delivery를 use case 호출 하나로 바꾸는 inbound adapter다. 업무 규칙은 use case에 둔다. +- 쓰기 use case를 부르는 handler는 `@SystemCaller(reason = ...)`이고, `@SystemInvoked` use case를 생성자에서 + `new`로 만들어 혼자 든다. use case는 bean이 아니다. 소비 스레드에는 인증이 없어 `@RequiresPermission` + use case는 매번 거부되기 때문이다. 작업 순서는 `docs/manual/09-outbox-events.md` 「소비」 4단계, 규칙은 + 07장 「시스템이 부르는 쓰기」(리뷰 2026-11-03 RR-01-12, 결정 9.1). ## 추가할 한 줄 @@ -85,7 +89,8 @@ app: destination: order-events-dlq ``` -handler 예시는 `MessagingInboxAutoConfigurationTest#exampleHandler`다. +handler 예시: 쓰기 use case를 부르는 모양은 `SystemInvokedConsumePathTest`의 `OrderCreatedHandler`, 실제 Kafka + +PostgreSQL에서 같은 모양은 `InboxRedeliveryIntegrationTest`의 `ProjectionHandler`다. ## 고른 방법과 버린 대안 @@ -96,6 +101,19 @@ handler 예시는 `MessagingInboxAutoConfigurationTest#exampleHandler`다. (HARD-STOP 4). - 버림: `adapter:outbound:messaging`(발행 bridge)에 소비까지 넣기 — 발행 bridge는 `messaging-core-api`만 보는 anti-corruption 경계이고, 소비는 inbound 방향이다. +- 고름(RR-01-12): handler가 `@SystemCaller`를 달고 `@SystemInvoked` use case를 직접 `new`로 만든다. 사용자 + 요청이 들어오는 전송(web·GraphQL·gRPC·WebSocket)만 그 use case에 닿지 못하고, 표시는 ArchUnit이 검사한다 + (`SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS`: messaging·scheduler inbound 패키지에만, 사용자 전송 + 타입에 의존하지 않음). 기동 가드는 바꾸지 않는다: `@SystemInvoked` bean은 예외 없이 실패다. + - 버림: use case를 bean으로 두고 가드에 "handler만 주입받으면 허용" 예외 — 누가 주입받는지는 기동 때 + 확정되지 않는다(`ObjectProvider`, `getBean`, 나중에 추가된 bean). 예외 조건이 곧 우회 경로가 된다. + - 버림: handler는 `CommandUseCase` 인터페이스만 받고 조립 모듈이 `new`로 넣기 — 소비자마다 composition + 모듈이 필요하거나 app-bootstrap에 업무 조립이 들어간다(위 HARD-STOP 4). 조립 클래스도 결국 그 use case + 클래스에 의존한다. + - 버림: ArchUnit 대상에서 messaging 패키지를 통째로 빼기 — 클래스에 표시가 없어 의도가 안 보이고, 그 + 패키지의 아무 클래스(HTTP endpoint 포함)나 닿는다. + - 버림: 사용자 전송 패키지만 나열해 금지 — 나중에 새 전송이 생기면 조용히 열린다. 규칙은 "닿아도 되는 + 쪽"을 적는다. - 버림: application-core에 소비 포트를 새로 만들어 platform `MessageHandler`로 감싸는 범용 bridge — payload 타입 매핑과 결과 매핑 규칙을 먼저 정해야 해서 이번 범위보다 크다. 지금은 handler가 직접 use case를 부른다. @@ -104,12 +122,17 @@ handler 예시는 `MessagingInboxAutoConfigurationTest#exampleHandler`다. - 단위: `MessagingInboxAutoConfigurationTest` — 조립 조건, inbox 마이그레이션 stream 조립 스위치, runner의 커밋·롤백, in-memory inbox로 중복 1회. decode 목록은 starter `MessagingConsumeConfigurationTest`. +- 단위: `SystemInvokedConsumePathTest` — 이 leaf의 inbox와 `composition:service-runtime`의 권한 가드를 함께 + 띄워, `@SystemCaller` handler가 만든 `@SystemInvoked` use case로 기동하고 같은 delivery 두 번에 쓰기가 + 한 번인지, use case를 bean으로 두면 기동이 실패하는지 본다. - 실제 Kafka + PostgreSQL: `InboxRedeliveryIntegrationTest`(lane `messagingInboxRedeliveryIntegrationTest`). Kafka capability, outbox relay, persistence-jpa, 이 leaf를 auto-configuration으로 띄운다. 앱 relay가 PostgreSQL outbox 행을 Kafka로 발행하고, 소비자가 inbox 예약과 업무 쓰기를 한 JPA 트랜잭션에서 한다. - offset을 0으로 되감고 다시 띄워 같은 레코드를 다시 읽혀도 handler 호출 1회, 업무 행 1개, inbox 행 1개. - handler가 업무 쓰기 뒤 예외를 던지면 예약과 쓰기가 함께 롤백되고, platform 재시도에서 다시 처리돼 handler 호출 2회, 업무 행 1개, inbox 행 1개. + - handler는 4단계 모양(`@SystemCaller` + 직접 만든 `@SystemInvoked` use case)이고, `composition:service-runtime`이 + classpath에 있어 서비스와 같은 권한 가드 아래에서 기동한다. - 소비 leaf가 기동하면 inbox 마이그레이션만 `flyway_messaging_inbox_history`에 적용되고 platform outbox 표는 생기지 않는다. - 테스트는 payload 타입을 등록하지 않고 inbox 표도 만들지 않는다 — 서비스가 받는 그대로다. diff --git a/src/adapter/inbound/messaging/build.gradle b/src/adapter/inbound/messaging/build.gradle index c3de3ac8..11ed657a 100644 --- a/src/adapter/inbound/messaging/build.gradle +++ b/src/adapter/inbound/messaging/build.gradle @@ -46,6 +46,12 @@ dependencies { implementation 'org.springframework:spring-tx' testImplementation 'org.springframework.boot:spring-boot-starter-test' + // The consume path a service runs (review 2026-11-03 RR-01-12): the permission guard every + // service takes from composition:service-runtime, the typed startup failure it raises, and the + // transaction port's fake. Test edges are outside the registry gate. + testImplementation project(':composition:service-runtime') + testImplementation project(':adapter:outbound:support') + testImplementation testFixtures(project(':application-core')) } testing { @@ -73,7 +79,6 @@ dependencies { integrationTestImplementation project(':adapter:outbound:persistence-jpa') integrationTestImplementation project(':application-core') integrationTestImplementation testFixtures(project(':application-core')) - integrationTestImplementation project(':shared-contract') integrationTestImplementation project(':messaging:messaging-core-api') integrationTestImplementation project(':messaging:messaging-spring-boot-starter') integrationTestImplementation 'org.springframework:spring-jdbc' diff --git a/src/adapter/inbound/messaging/gradle.lockfile b/src/adapter/inbound/messaging/gradle.lockfile index 30469b12..fb62ebfa 100644 --- a/src/adapter/inbound/messaging/gradle.lockfile +++ b/src/adapter/inbound/messaging/gradle.lockfile @@ -1,6 +1,7 @@ # This is a Gradle generated file for dependency locking. # Manual edits can break the build and are not advised. # This file is expected to be part of source control. +aopalliance:aopalliance:1.0=integrationTestRuntimeClasspath,testRuntimeClasspath at.yawk.lz4:lz4-java:1.10.1=integrationTestRuntimeClasspath biz.aQute.bnd:biz.aQute.bnd.annotation:7.1.0=integrationTestCompileClasspath,testCompileClasspath ch.qos.logback:logback-classic:1.5.38=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath @@ -14,6 +15,7 @@ com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,integrationTest com.github.docker-java:docker-java-api:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath com.github.docker-java:docker-java-transport-zerodep:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath com.github.docker-java:docker-java-transport:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath +com.github.f4b6a3:uuid-creator:6.1.1=integrationTestRuntimeClasspath,testRuntimeClasspath com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor com.github.luben:zstd-jni:1.5.6-10=integrationTestRuntimeClasspath com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs @@ -53,9 +55,28 @@ commons-logging:commons-logging:1.3.6=compileClasspath,integrationTestCompileCla info.picocli:picocli:4.7.7=checkstyle io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor +io.micrometer:context-propagation:1.2.1=integrationTestRuntimeClasspath,testRuntimeClasspath io.micrometer:micrometer-commons:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath +io.micrometer:micrometer-core:1.16.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testRuntimeClasspath +io.micrometer:micrometer-jakarta9:1.16.7=integrationTestRuntimeClasspath,testRuntimeClasspath io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath +io.micrometer:micrometer-tracing-bridge-otel:1.6.7=integrationTestRuntimeClasspath,testRuntimeClasspath +io.micrometer:micrometer-tracing:1.6.7=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry.semconv:opentelemetry-semconv:1.37.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-api:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-common:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-context:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-exporter-common:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-exporter-otlp-common:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-exporter-otlp:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-exporter-sender-jdk:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-extension-trace-propagators:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-sdk-common:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-sdk-extension-autoconfigure-spi:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-sdk-logs:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-sdk-metrics:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-sdk-trace:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath +io.opentelemetry:opentelemetry-sdk:1.55.0=integrationTestRuntimeClasspath,testRuntimeClasspath io.projectreactor:reactor-core:3.8.7=integrationTestRuntimeClasspath jakarta.activation:jakarta.activation-api:2.1.4=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath jakarta.annotation:jakarta.annotation-api:3.0.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath @@ -68,7 +89,7 @@ jaxen:jaxen:2.0.6=spotbugs net.bytebuddy:byte-buddy-agent:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath net.java.dev.jna:jna:5.18.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -net.logstash.logback:logstash-logback-encoder:8.0=integrationTestRuntimeClasspath +net.logstash.logback:logstash-logback-encoder:8.0=integrationTestRuntimeClasspath,testRuntimeClasspath net.minidev:accessors-smart:2.6.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath net.minidev:json-smart:2.6.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs @@ -109,7 +130,7 @@ org.glassfish.jaxb:jaxb-core:4.0.9=integrationTestRuntimeClasspath org.glassfish.jaxb:jaxb-runtime:4.0.9=integrationTestRuntimeClasspath org.glassfish.jaxb:txw2:4.0.9=integrationTestRuntimeClasspath org.hamcrest:hamcrest:3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.hdrhistogram:HdrHistogram:2.2.2=integrationTestRuntimeClasspath +org.hdrhistogram:HdrHistogram:2.2.2=integrationTestRuntimeClasspath,testRuntimeClasspath org.hibernate.models:hibernate-models:1.0.1=integrationTestRuntimeClasspath org.hibernate.orm:hibernate-core:7.2.24.Final=integrationTestRuntimeClasspath org.javassist:javassist:3.28.0-GA=checkstyle @@ -125,7 +146,7 @@ org.junit.platform:junit-platform-engine:6.0.3=integrationTestRuntimeClasspath,t org.junit.platform:junit-platform-launcher:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=integrationTestRuntimeClasspath +org.latencyutils:LatencyUtils:2.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath org.mockito:mockito-core:5.20.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,mockitoAgent,testCompileClasspath,testRuntimeClasspath org.mockito:mockito-junit-jupiter:5.20.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.objenesis:objenesis:3.3=integrationTestRuntimeClasspath,testRuntimeClasspath @@ -150,21 +171,30 @@ org.slf4j:jul-to-slf4j:2.0.18=integrationTestCompileClasspath,integrationTestRun org.slf4j:slf4j-api:2.0.18=integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs,spotbugsSlf4j,testCompileClasspath,testRuntimeClasspath org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j org.slf4j:slf4j-simple:2.0.18=checkstyle -org.springframework.boot:spring-boot-actuator:4.0.8=integrationTestRuntimeClasspath +org.springframework.boot:spring-boot-actuator-autoconfigure:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath +org.springframework.boot:spring-boot-actuator:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-autoconfigure:4.0.8=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-configuration-processor:4.0.8=annotationProcessor org.springframework.boot:spring-boot-data-commons:4.0.8=integrationTestRuntimeClasspath org.springframework.boot:spring-boot-data-jpa:4.0.8=integrationTestRuntimeClasspath org.springframework.boot:spring-boot-flyway:4.0.8=integrationTestRuntimeClasspath +org.springframework.boot:spring-boot-health:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-hibernate:4.0.8=integrationTestRuntimeClasspath org.springframework.boot:spring-boot-http-converter:4.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-jackson:4.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-jdbc:4.0.8=integrationTestRuntimeClasspath org.springframework.boot:spring-boot-jpa:4.0.8=integrationTestRuntimeClasspath +org.springframework.boot:spring-boot-micrometer-metrics:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath +org.springframework.boot:spring-boot-micrometer-observation:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath +org.springframework.boot:spring-boot-micrometer-tracing-opentelemetry:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath +org.springframework.boot:spring-boot-micrometer-tracing:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath +org.springframework.boot:spring-boot-opentelemetry:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-persistence:4.0.8=integrationTestRuntimeClasspath org.springframework.boot:spring-boot-resttestclient:4.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath +org.springframework.boot:spring-boot-security:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-servlet:4.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-sql:4.0.8=integrationTestRuntimeClasspath +org.springframework.boot:spring-boot-starter-actuator:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-starter-data-jpa:4.0.8=integrationTestRuntimeClasspath org.springframework.boot:spring-boot-starter-flyway:4.0.8=integrationTestRuntimeClasspath org.springframework.boot:spring-boot-starter-jackson-test:4.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath @@ -172,6 +202,8 @@ org.springframework.boot:spring-boot-starter-jackson:4.0.8=integrationTestCompil org.springframework.boot:spring-boot-starter-jdbc:4.0.8=integrationTestRuntimeClasspath org.springframework.boot:spring-boot-starter-json:4.0.8=integrationTestRuntimeClasspath org.springframework.boot:spring-boot-starter-logging:4.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath +org.springframework.boot:spring-boot-starter-micrometer-metrics:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath +org.springframework.boot:spring-boot-starter-security:4.0.8=integrationTestRuntimeClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-starter-test:4.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-starter-tomcat-runtime:4.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-starter-tomcat:4.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath @@ -190,6 +222,10 @@ org.springframework.data:spring-data-commons:4.0.7=integrationTestRuntimeClasspa org.springframework.data:spring-data-jpa:4.0.7=integrationTestRuntimeClasspath org.springframework.integration:spring-integration-core:7.0.6=integrationTestRuntimeClasspath org.springframework.integration:spring-integration-jdbc:7.0.6=integrationTestRuntimeClasspath +org.springframework.security:spring-security-config:7.0.7=integrationTestRuntimeClasspath,testRuntimeClasspath +org.springframework.security:spring-security-core:7.0.7=integrationTestRuntimeClasspath,testRuntimeClasspath +org.springframework.security:spring-security-crypto:7.0.7=integrationTestRuntimeClasspath,testRuntimeClasspath +org.springframework.security:spring-security-web:7.0.7=integrationTestRuntimeClasspath,testRuntimeClasspath org.springframework:spring-aop:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework:spring-aspects:7.0.9=integrationTestRuntimeClasspath org.springframework:spring-beans:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath diff --git a/src/adapter/inbound/messaging/src/integrationTest/java/dev/caskeleton/adapter/inbound/messaging/InboxRedeliveryIntegrationTest.java b/src/adapter/inbound/messaging/src/integrationTest/java/dev/caskeleton/adapter/inbound/messaging/InboxRedeliveryIntegrationTest.java index 0670293e..527e3fe0 100644 --- a/src/adapter/inbound/messaging/src/integrationTest/java/dev/caskeleton/adapter/inbound/messaging/InboxRedeliveryIntegrationTest.java +++ b/src/adapter/inbound/messaging/src/integrationTest/java/dev/caskeleton/adapter/inbound/messaging/InboxRedeliveryIntegrationTest.java @@ -3,9 +3,15 @@ package dev.caskeleton.adapter.inbound.messaging; import static org.assertj.core.api.Assertions.assertThat; import static org.awaitility.Awaitility.await; +import dev.caskeleton.application.capability.Idempotency; +import dev.caskeleton.application.capability.RepositoryAccess; +import dev.caskeleton.application.capability.UseCaseCapability; import dev.caskeleton.application.messaging.event.ValidatedIntegrationEvent; import dev.caskeleton.application.outbox.OutboxAppendPort; import dev.caskeleton.application.outbox.testkit.OutboxEvents; +import dev.caskeleton.application.security.SystemCaller; +import dev.caskeleton.application.security.SystemInvoked; +import dev.caskeleton.application.transaction.TransactionMode; import dev.caskeleton.application.transaction.TransactionPort; import dev.caskeleton.messaging.api.MessageId; import dev.caskeleton.messaging.api.MessageType; @@ -14,8 +20,6 @@ import dev.caskeleton.messaging.api.delivery.MessageDelivery; import dev.caskeleton.messaging.api.delivery.MessageHandler; import dev.caskeleton.messaging.api.destination.DestinationName; import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.shared.concurrency.DomainContextPropagator; -import dev.caskeleton.shared.concurrency.ThreadLocalDomainContextPropagator; import java.time.Clock; import java.time.Duration; import java.time.Instant; @@ -60,8 +64,11 @@ import org.testcontainers.postgresql.PostgreSQLContainer; * from the PostgreSQL outbox and publishes it through the platform bridge; the platform consumer * decodes it and, because the destination declares {@code INBOX_TRANSACTIONAL}, runs the handler * inside the inbox this leaf assembles — the platform's JDBC inbox over the application datasource - * and a runner over the application's (JPA) transaction manager. The handler writes its effect the - * way a use case does, through the application's {@code TransactionPort}. + * and a runner over the application's (JPA) transaction manager. The handler is the shape manual 09 + * 「소비」 step 4 teaches (review 2026-11-03 RR-01-12): a {@code @SystemCaller} that constructs the + * {@code @SystemInvoked} write use case it calls, which writes through the application's {@code + * TransactionPort}. {@code composition:service-runtime} is on the classpath as it is in a service, + * so its permission guard looks at this context and lets it start: the use case is not a bean. * *

What only this lane can show is that those parts share one transaction for real: a JDBC inbox * that took a connection of its own, or a runner over a different transaction manager, passes every @@ -387,7 +394,15 @@ class InboxRedeliveryIntegrationTest { } } - /** A use case's write, in the application's transaction (joined, never opened beside it). */ + /** + * A write only the consumer calls, in the application's transaction (joined, never opened beside + * it). No permission: the consumer thread has no {@code Authentication}. + */ + @SystemInvoked(reason = "the lane's consumer calls it for each delivery") + @UseCaseCapability( + transactionMode = TransactionMode.WRITE, + idempotency = Idempotency.NOT_IDEMPOTENT, + repositoryAccess = RepositoryAccess.WRITE_REPOSITORY) static final class RecordProjection { private final TransactionPort transactions; private final JdbcTemplate jdbc; @@ -407,14 +422,18 @@ class InboxRedeliveryIntegrationTest { } } - /** The shape of an application handler: one delivery, one use case call. */ + /** + * The shape of an application handler: one delivery, one use case call. It constructs its use + * case from the ports its bean method receives, so the use case is never a bean. + */ + @SystemCaller(reason = "the platform consumer calls it for each lane delivery") static final class ProjectionHandler implements MessageHandler { private final Handled handled; private final RecordProjection useCase; - ProjectionHandler(Handled handled, RecordProjection useCase) { + ProjectionHandler(Handled handled, TransactionPort transactions, JdbcTemplate jdbc) { this.handled = handled; - this.useCase = useCase; + this.useCase = new RecordProjection(transactions, jdbc); } @Override @@ -445,14 +464,7 @@ class InboxRedeliveryIntegrationTest { @Bean ProjectionHandler projectionHandler( Handled handled, TransactionPort transactions, DataSource dataSource) { - return new ProjectionHandler( - handled, new RecordProjection(transactions, new JdbcTemplate(dataSource))); - } - - /** What composition:service-runtime gives a service; the JPA audit context reads it. */ - @Bean - DomainContextPropagator domainContextPropagator() { - return new ThreadLocalDomainContextPropagator(); + return new ProjectionHandler(handled, transactions, new JdbcTemplate(dataSource)); } @Bean diff --git a/src/adapter/inbound/messaging/src/test/java/dev/caskeleton/adapter/inbound/messaging/SystemInvokedConsumePathTest.java b/src/adapter/inbound/messaging/src/test/java/dev/caskeleton/adapter/inbound/messaging/SystemInvokedConsumePathTest.java new file mode 100644 index 00000000..040d33fd --- /dev/null +++ b/src/adapter/inbound/messaging/src/test/java/dev/caskeleton/adapter/inbound/messaging/SystemInvokedConsumePathTest.java @@ -0,0 +1,286 @@ +package dev.caskeleton.adapter.inbound.messaging; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.adapter.outbound.support.startup.StartupFailureException; +import dev.caskeleton.application.capability.Idempotency; +import dev.caskeleton.application.capability.RepositoryAccess; +import dev.caskeleton.application.capability.UseCaseCapability; +import dev.caskeleton.application.security.SystemCaller; +import dev.caskeleton.application.security.SystemInvoked; +import dev.caskeleton.application.transaction.TransactionMode; +import dev.caskeleton.application.transaction.TransactionPort; +import dev.caskeleton.application.transaction.testkit.DirectTransactionPort; +import dev.caskeleton.composition.runtime.authorization.PermissionEnforcementConfig; +import dev.caskeleton.messaging.api.ContentType; +import dev.caskeleton.messaging.api.MessageEnvelope; +import dev.caskeleton.messaging.api.MessageId; +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.ProducerId; +import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.delivery.DeliveryContext; +import dev.caskeleton.messaging.api.delivery.DeliveryMetadata; +import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.header.MessageHeaders; +import dev.caskeleton.messaging.inbox.IdempotentConsumer; +import dev.caskeleton.messaging.inbox.InboxMessageHandler; +import dev.caskeleton.messaging.reliability.InboxRepository; +import java.time.Instant; +import java.util.ArrayList; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Optional; +import java.util.Set; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionStage; +import javax.sql.DataSource; +import org.junit.jupiter.api.Test; +import org.mockito.Mockito; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.transaction.PlatformTransactionManager; +import org.springframework.transaction.TransactionDefinition; +import org.springframework.transaction.support.AbstractPlatformTransactionManager; +import org.springframework.transaction.support.DefaultTransactionStatus; + +/** + * The consume path manual 09 「소비」 step 4 teaches, assembled the way a service assembles it: this + * leaf's inbox, the permission guard every service runs ({@code composition:service-runtime}), and + * a handler that calls a write use case only the system calls (review 2026-11-03 RR-01-12, decision + * 9.1). + * + *

The use case is {@code @SystemInvoked}: a consumer thread has no {@code Authentication}, so a + * permission gate would refuse every delivery. The handler is {@code @SystemCaller} and constructs + * the use case from the ports its bean method receives; the use case is never a bean, which is the + * one condition under which the guard lets a system-invoked use case into a running service. The + * architecture rules that check the marks run over production code in {@code + * :app-bootstrap:architectureTest}; this test checks that the same shape starts and runs. + */ +class SystemInvokedConsumePathTest { + + private static final Instant NOW = Instant.parse("2026-11-04T09:00:00Z"); + private static final DestinationName ORDERS = new DestinationName("order-events"); + + private final ApplicationContextRunner runner = + new ApplicationContextRunner() + .withConfiguration( + AutoConfigurations.of( + MessagingInboxAutoConfiguration.class, PermissionEnforcementConfig.class)) + .withBean(DataSource.class, () -> Mockito.mock(DataSource.class)) + .withBean(PlatformTransactionManager.class, NoTransactions::new) + .withBean(TransactionPort.class, DirectTransactionPort::new) + .withBean(InMemoryOrderProjections.class, InMemoryOrderProjections::new) + .withPropertyValues("app.runtime.migration-on-startup=false"); + + @Test + void aHandlerThatConstructsItsSystemInvokedUseCaseStartsUnderThePermissionGuard() { + // when + runner + .withUserConfiguration(OrderConsumerConfig.class) + .run( + context -> { + // then + assertThat(context) + .hasNotFailed() + .hasSingleBean(OrderCreatedHandler.class) + .doesNotHaveBean(ProjectOrderUseCase.class); + }); + } + + @Test + void aDeliveryReachesTheSystemInvokedUseCaseOnceThroughTheInbox() { + runner + .withUserConfiguration(OrderConsumerConfig.class) + .run( + context -> { + // given: the platform's inbox around the assembled handler, over this leaf's runner + IdempotentConsumer inbox = + new IdempotentConsumer( + new InMemoryInbox(), + context.getBean(IdempotentConsumer.TransactionRunner.class)); + MessageHandler consumer = + new InboxMessageHandler<>( + context.getBean(OrderCreatedHandler.class), + inbox, + "order-projection", + () -> NOW); + MessageDelivery delivery = delivery("o-1"); + consumer.handle(delivery).toCompletableFuture().join(); + + // when + consumer.handle(delivery).toCompletableFuture().join(); + + // then + assertThat(context.getBean(InMemoryOrderProjections.class).recorded) + .containsExactly("o-1"); + }); + } + + /** The way the guard refuses: the use case as a bean, which any transport could inject. */ + @Test + void registeringTheSystemInvokedUseCaseAsABeanFailsStartup() { + // when + runner + .withUserConfiguration(UseCaseAsABeanConfig.class) + .run( + context -> { + // then + assertThat(context.getStartupFailure()) + .isInstanceOf(StartupFailureException.class) + .hasMessageContaining("never registered as a bean") + .hasMessageContaining(ProjectOrderUseCase.class.getName()); + }); + } + + private static MessageDelivery delivery(String orderId) { + return new MessageDelivery<>( + new MessageEnvelope<>( + MessageId.newId(), + new MessageType("order.created"), + new SchemaVersion(1), + NOW, + Optional.empty(), + new ProducerId("orders"), + Optional.empty(), + Optional.empty(), + ContentType.JSON, + Optional.empty(), + Optional.empty(), + Optional.empty(), + TraceContext.none(), + MessageHeaders.empty(), + orderId), + new DeliveryMetadata( + ORDERS, 1, false, Optional.empty(), Optional.empty(), Optional.empty(), NOW), + new DeliveryContext(NOW.plusSeconds(30), false, "order-projection")); + } + + /** The application's write port for the projection; this test owns it, so its fake is here. */ + interface OrderProjectionPort { + void record(String orderId); + } + + static final class InMemoryOrderProjections implements OrderProjectionPort { + private final List recorded = new ArrayList<>(); + + @Override + public void record(String orderId) { + recorded.add(orderId); + } + } + + /** A write only the consumer calls: no permission, because its caller has no Authentication. */ + @SystemInvoked(reason = "the order-events consumer calls it for each delivery") + @UseCaseCapability( + transactionMode = TransactionMode.WRITE, + idempotency = Idempotency.IDEMPOTENT, + repositoryAccess = RepositoryAccess.WRITE_REPOSITORY) + static final class ProjectOrderUseCase { + private final TransactionPort tx; + private final OrderProjectionPort projections; + + ProjectOrderUseCase(TransactionPort tx, OrderProjectionPort projections) { + this.tx = tx; + this.projections = projections; + } + + void project(String orderId) { + tx.inWrite( + () -> { + projections.record(orderId); + return null; + }); + } + } + + /** Manual 09 step 4: one delivery, one use case call; the handler owns its use case. */ + @SystemCaller(reason = "the platform consumer calls it for each order-events delivery") + static final class OrderCreatedHandler implements MessageHandler { + private final ProjectOrderUseCase useCase; + + OrderCreatedHandler(TransactionPort tx, OrderProjectionPort projections) { + this.useCase = new ProjectOrderUseCase(tx, projections); + } + + @Override + public MessageDestination destination() { + return new MessageDestination<>(ORDERS, new MessageType("order.created"), String.class); + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + useCase.project(delivery.message().payload()); + return CompletableFuture.completedFuture(HandleResult.success()); + } + } + + @Configuration(proxyBeanMethods = false) + static class OrderConsumerConfig { + @Bean + OrderCreatedHandler orderCreatedHandler( + TransactionPort tx, InMemoryOrderProjections projections) { + return new OrderCreatedHandler(tx, projections); + } + } + + @Configuration(proxyBeanMethods = false) + static class UseCaseAsABeanConfig { + @Bean + ProjectOrderUseCase projectOrderUseCase( + TransactionPort tx, InMemoryOrderProjections projections) { + return new ProjectOrderUseCase(tx, projections); + } + + @Bean + OrderCreatedHandler orderCreatedHandler( + TransactionPort tx, InMemoryOrderProjections projections) { + return new OrderCreatedHandler(tx, projections); + } + } + + /** The inbox runner's transaction manager; the mocked datasource has nothing to open. */ + private static final class NoTransactions extends AbstractPlatformTransactionManager { + + private static final long serialVersionUID = 1L; + + @Override + protected Object doGetTransaction() { + return new Object(); + } + + @Override + protected void doBegin(Object transaction, TransactionDefinition definition) {} + + @Override + protected void doCommit(DefaultTransactionStatus status) {} + + @Override + protected void doRollback(DefaultTransactionStatus status) {} + } + + private static final class InMemoryInbox implements InboxRepository { + private final Set reserved = new LinkedHashSet<>(); + + @Override + public boolean reserve(MessageId messageId, String consumerId, Instant now) { + return reserved.add(messageId.value() + "|" + consumerId); + } + + @Override + public boolean isProcessed(MessageId messageId, String consumerId) { + return reserved.contains(messageId.value() + "|" + consumerId); + } + + @Override + public int purgeProcessedBefore(Instant processedBefore, int limit) { + return 0; + } + } +} diff --git a/src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/error/EnvelopeErrorController.java b/src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/error/EnvelopeErrorController.java index 2b60ccd8..7693316a 100644 --- a/src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/error/EnvelopeErrorController.java +++ b/src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/error/EnvelopeErrorController.java @@ -28,6 +28,14 @@ import tools.jackson.databind.ObjectMapper; * status is classified by the same table the exception handler uses, and the body is the failure * envelope, with nothing from the request in it. * + *

The status sent is the code's registered status. A status in {@link HttpStatusErrorCodes} + * keeps its value; any other folds into the code of its class — {@code BAD_PARAMETER} 400 for a + * 4xx, {@code INTERNAL_ERROR} 500 for a 5xx — so the {@code error.code} a client branches on never + * disagrees with the status beside it (review 2026-11-03 RR-01-13). Rejected: keeping the + * dispatched status beside the folded code, which is what Boot did but would send {@code + * BAD_PARAMETER} with a 409 the registry does not give it. A filter that needs another status + * writes that status's code with {@link EnvelopeResponseWriter} or adds a row to the table. + * *

Written straight to the response by {@link EnvelopeResponseWriter}, like the other refusals * made outside a converter. An error dispatch has no handler of its own to negotiate for, and a * browser's {@code Accept: text/html} would otherwise turn the error into a 406 about the error. diff --git a/src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/error/EnvelopeErrorControllerTest.java b/src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/error/EnvelopeErrorControllerTest.java new file mode 100644 index 00000000..074f8f8d --- /dev/null +++ b/src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/error/EnvelopeErrorControllerTest.java @@ -0,0 +1,58 @@ +package dev.caskeleton.adapter.inbound.web.error; + +import static org.assertj.core.api.Assertions.assertThat; + +import jakarta.servlet.RequestDispatcher; +import java.util.List; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.CsvSource; +import org.springframework.mock.web.MockHttpServletRequest; +import org.springframework.mock.web.MockHttpServletResponse; +import tools.jackson.databind.ObjectMapper; + +/** + * Which status and code the container's error dispatch is answered with (review 2026-11-03 + * RR-01-13). A status in {@link HttpStatusErrorCodes} keeps its value; any other folds to the code + * of its class and that code's registered status, so the {@code error.code} a client branches on + * never disagrees with {@code docs/registries/error-codes.yaml}. + */ +class EnvelopeErrorControllerTest { + + private final ObjectMapper mapper = new ObjectMapper(); + private final EnvelopeErrorController controller = new EnvelopeErrorController(mapper); + + @DisplayName("오류 디스패치 status: 표에 있으면 그대로, 없으면 4xx는 400·5xx는 500으로 접힌다") + @ParameterizedTest(name = "{0} → {1} {2}") + @CsvSource({ + "401, 401, UNAUTHENTICATED", + "404, 404, ROUTE_NOT_FOUND", + "429, 429, RATE_LIMIT_EXCEEDED", + "409, 400, BAD_PARAMETER", + "422, 400, BAD_PARAMETER", + "500, 500, INTERNAL_ERROR", + "502, 500, INTERNAL_ERROR", + "504, 500, INTERNAL_ERROR" + }) + void anErrorDispatchKeepsATabledStatusAndFoldsAnyOtherIntoItsClassCode( + int dispatched, int answered, String code) throws Exception { + // given + MockHttpServletRequest request = new MockHttpServletRequest("GET", "/error"); + request.setAttribute(RequestDispatcher.ERROR_STATUS_CODE, dispatched); + MockHttpServletResponse response = new MockHttpServletResponse(); + + // when + controller.error(request, response); + + // then + assertThat( + List.of( + response.getStatus(), + mapper + .readTree(response.getContentAsString()) + .path("error") + .path("code") + .asString())) + .containsExactly(answered, code); + } +} diff --git a/src/adapter/outbound/cache-redis/README.md b/src/adapter/outbound/cache-redis/README.md index 99026fa9..da0bd2e8 100644 --- a/src/adapter/outbound/cache-redis/README.md +++ b/src/adapter/outbound/cache-redis/README.md @@ -215,7 +215,7 @@ typed SDK's — tagged `lane=` (`RedisLaneMetrics`; rows in `docs/registri | --- | --- | --- | | `redis.lane.acquire.wait.seconds` (`outcome` = `admitted`/`refused`) | timer, histogram | how long a borrow queued for a connection; a growing admitted p99 comes before refusals | | `redis.lane.acquire.refused.total` (`reason` = `saturated`/`closing`/`interrupted`/`connect_failed`) | counter | a borrow that got no connection, nothing sent; `saturated` on `SCRIPT` is the starvation this lane exists to prevent | -| `redis.lane.connections.opened.total` | counter | new connections; after warm-up every one is a reconnect | +| `redis.lane.connections.opened.total` | counter | new connections; after warm-up a pooled lane opens one only to replace a discarded one. A routed lease (a Cluster `TRANSACTION` borrow) is opened for each borrow and closed after it without being counted as discarded, so on that lane a steady rate is normal use, not reconnects | | `redis.lane.connections.discarded.total` (`reason` = `invalidated`/`broken`) | counter | connections closed instead of pooled — `invalidated` is a timed-out or failed call (the lease is not reused), and is what the reconnects replace | | `redis.lane.connections.active`, `redis.lane.connections.pending` | gauge | connections lent out now; borrows waiting now | @@ -283,6 +283,18 @@ A lost renewal reply is recognised by what the renewal recorded — the revision the lease it asked for: the same lease is `ALREADY_RENEWED_SAME_OPERATION` with the current handle, a different lease is `OPERATION_CONFLICT`. +A lease that runs out is taken over only from a claim that never started or an attempt marked +retryable. A record still `EXECUTING` when its lease ran out has an unknown effect: the claim +program marks it `ABANDONED` and answers `RECOVERY_REQUIRED`, and the late holder can no longer +complete it. The PostgreSQL store answers the same; the shared `IdempotencyStoreContract` holds both +to it on a real server (re-review 2026-11-04 RR-01-11 — this store used to take the record over and +run the work a second time). + +The record key lives for the processing lease plus the replay TTL (`afterLease`, kept on the record), +and a renewal stretches it to the new lease plus that margin. With the replay TTL alone, a replay +TTL shorter than the lease dropped the key of a running operation and the next claim ran it again, +and one equal to the lease left no record to answer `RECOVERY_REQUIRED` from (RR-03-11). + Redis `TIME` determines lease deadlines. A failure after a command may have executed is returned as an indeterminate outcome; the adapter does not convert an ambiguous write into a clean unavailable result that would invite duplicate execution. @@ -334,8 +346,9 @@ Useful focused checks: ./gradlew architectureCheck ``` -`redisContainerContractTest` runs the cache-region and lease port contracts (the ones the in-memory -fakes pass) and the live semantic-port tests against a Redis it starts itself, under the standalone +`redisContainerContractTest` runs the cache-region, lease and idempotency-store port contracts (the +ones the in-memory fakes pass; the idempotency contract's timed cases run only here, on the server's +own clock) and the live semantic-port tests against a Redis it starts itself, under the standalone topology's ACL file. It is in `integrationCheck` (integration-main.yml's `integration-check` job), and only there: listing it in `mainCheck` as well ran it twice on every push to main. The image is `config/testing/container-images.properties` `redis`. diff --git a/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/IdempotencyScripts.java b/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/IdempotencyScripts.java index 47e32400..7e47507c 100644 --- a/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/IdempotencyScripts.java +++ b/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/IdempotencyScripts.java @@ -30,19 +30,36 @@ public final class IdempotencyScripts { *

The fingerprint is compared before anything else. Two different requests that hash to the * same idempotency scope are a client error, and treating the second as a replay of the first * would return somebody else's response. + * + *

A lease that ran out is taken over only when nothing can have happened under it: a claim + * that never started, or an attempt its holder marked retryable. A record that was {@code + * EXECUTING} has an unknown effect, so it is abandoned and answered {@code RECOVERY_REQUIRED} — + * running it again is the duplicate this store exists to prevent, and it is what a client + * resending after a result-unknown 503 was promised would not happen (re-review 2026-11-04 + * RR-01-11). The PostgreSQL store answers the same; {@code IdempotencyStoreContract} holds both + * to it. + * + *

The key lives for the lease plus the replay TTL ({@code afterLease}), not for the replay TTL + * alone. With the replay TTL alone, a replay TTL shorter than the lease dropped the key while the + * holder still ran and the next claim acquired it fresh — a second run (RR-03-11); and a replay + * TTL equal to the lease dropped it with the lease, leaving no record to answer "recovery + * required" from. {@code afterLease} is kept on the record so {@link #RENEW} can keep the same + * margin behind a lease it extends. */ private static final String CLAIM = """ local state = redis.call('HGET', KEYS[1], 'state') local t = redis.call('TIME') local nowMillis = (tonumber(t[1]) * 1000) + math.floor(tonumber(t[2]) / 1000) + local leaseMillis = tonumber(ARGV[6]) + local afterLease = tonumber(ARGV[7]) if state == false then redis.call('HSET', KEYS[1], 'state', 'CLAIMED', 'owner', ARGV[1], 'attempt', 1, 'rev', 1, 'op', ARGV[2], 'fp', ARGV[3], 'codec', ARGV[4], 'policy', ARGV[5], - 'leaseUntil', nowMillis + tonumber(ARGV[6])) - redis.call('PEXPIRE', KEYS[1], ARGV[7]) - return {'ACQUIRED', 1, 1, ARGV[1], '', tostring(nowMillis + tonumber(ARGV[6]))} + 'leaseUntil', nowMillis + leaseMillis, 'afterLease', afterLease) + redis.call('PEXPIRE', KEYS[1], leaseMillis + afterLease) + return {'ACQUIRED', 1, 1, ARGV[1], '', tostring(nowMillis + leaseMillis)} end local fingerprint = redis.call('HGET', KEYS[1], 'fp') if fingerprint ~= ARGV[3] then @@ -68,15 +85,23 @@ public final class IdempotencyScripts { redis.call('HGET', KEYS[1], 'leaseUntil')} end local leaseUntil = tonumber(redis.call('HGET', KEYS[1], 'leaseUntil')) - if state == 'FAILED_RETRYABLE' or (leaseUntil ~= nil and leaseUntil <= nowMillis) then - -- The previous holder's processing lease expired, or they marked the attempt retryable. - -- Taking over bumps the attempt so the new holder can tell it is not the first. + local leaseOver = leaseUntil ~= nil and leaseUntil <= nowMillis + if state == 'EXECUTING' and leaseOver then + -- The holder started and its lease ran out: whether the work took effect is unknown. + -- Abandon it, so the late holder cannot complete over the verdict, and keep the key's TTL: + -- the record stays as evidence for its remaining afterLease window. + redis.call('HSET', KEYS[1], 'state', 'ABANDONED', 'rev', rev + 1) + return {'RECOVERY_REQUIRED', attempt, rev + 1, owner, '', ''} + end + if state == 'FAILED_RETRYABLE' or (state == 'CLAIMED' and leaseOver) then + -- Nothing ran under the previous holder: they never started, or they marked the attempt + -- retryable. Taking over bumps the attempt so the new holder can tell it is not the first. redis.call('HSET', KEYS[1], 'state', 'CLAIMED', 'owner', ARGV[1], 'attempt', attempt + 1, 'rev', rev + 1, - 'op', ARGV[2], 'leaseUntil', nowMillis + tonumber(ARGV[6])) - redis.call('PEXPIRE', KEYS[1], ARGV[7]) + 'op', ARGV[2], 'leaseUntil', nowMillis + leaseMillis, 'afterLease', afterLease) + redis.call('PEXPIRE', KEYS[1], leaseMillis + afterLease) return {'TAKEN_OVER', attempt + 1, rev + 1, ARGV[1], '', - tostring(nowMillis + tonumber(ARGV[6]))} + tostring(nowMillis + leaseMillis)} end return {'IN_PROGRESS', attempt, rev, owner, '', tostring(leaseUntil - nowMillis)} """; @@ -142,11 +167,13 @@ public final class IdempotencyScripts { * the same renewal; with a different lease it is a different request, because confirming it would * leave the caller believing it holds the record for a lease the record does not carry. * - *

The key's own TTL is stretched to cover the new lease. Claim sets it to the replay TTL, - * which may be shorter than a lease renewed past it; the key then expired while the holder was - * still executing, and the next request got a fresh claim and ran the side effect again (review - * 2026-11-03 RR-04-08). The TTL is only ever lengthened here — completion sets the replay TTL - * again. + *

The key's own TTL is stretched to cover the new lease and the {@code afterLease} margin the + * claim recorded behind it, so a renewed lease ends with the same evidence window as the first. + * Without it a lease renewed past the key's TTL expired the key while the holder was still + * executing, and the next request got a fresh claim and ran the side effect again (review + * 2026-11-03 RR-04-08, re-review 2026-11-04 RR-03-11). A record claimed before {@code afterLease} + * existed has none and keeps only the lease. The TTL is only ever lengthened here — completion + * sets the replay TTL again. */ private static final String RENEW = """ @@ -183,9 +210,11 @@ public final class IdempotencyScripts { redis.call('HSET', KEYS[1], 'leaseUntil', nowMillis + tonumber(ARGV[4]), 'rev', rev + 1, 'renewedFromRev', ARGV[2], 'renewedLease', ARGV[4]) + local keep = tonumber(ARGV[4]) + + tonumber(redis.call('HGET', KEYS[1], 'afterLease') or '0') local ttl = redis.call('PTTL', KEYS[1]) - if ttl >= 0 and ttl < tonumber(ARGV[4]) then - redis.call('PEXPIRE', KEYS[1], ARGV[4]) + if ttl >= 0 and ttl < keep then + redis.call('PEXPIRE', KEYS[1], keep) end return {'APPLIED', attempt, rev + 1, owner, '', ''} """; diff --git a/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/RedisIdempotencyStoreAdapter.java b/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/RedisIdempotencyStoreAdapter.java index 16af2059..c0bbb8c5 100644 --- a/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/RedisIdempotencyStoreAdapter.java +++ b/src/adapter/outbound/cache-redis/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/RedisIdempotencyStoreAdapter.java @@ -130,6 +130,7 @@ public final class RedisIdempotencyStoreAdapter implements IdempotencyStorePort request.responseCodecId(), Integer.toString(request.policyRevision()), Long.toString(request.processingLeaseTtl().toMillis()), + // How long the record outlives its lease: the key lives for both. Long.toString(request.replayTtl().toMillis())))), now), IdempotencyClaimOutcome.Unavailable::new, diff --git a/src/adapter/outbound/cache-redis/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/RedisIdempotencyStoreAdapterTest.java b/src/adapter/outbound/cache-redis/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/RedisIdempotencyStoreAdapterTest.java index 48d09e4a..7b992580 100644 --- a/src/adapter/outbound/cache-redis/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/RedisIdempotencyStoreAdapterTest.java +++ b/src/adapter/outbound/cache-redis/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/idempotency/RedisIdempotencyStoreAdapterTest.java @@ -353,8 +353,8 @@ class RedisIdempotencyStoreAdapterTest { } @Test - @DisplayName("lease를 넘겨받은 뒤 이전 owner는 renew할 수 없다") - void aTakenOverOwnerCannotRenew() { + @DisplayName("lease가 지나 다른 요청이 회복 필요로 판정한 실행은 이전 owner가 renew할 수 없다") + void anOwnerWhoseExpiredExecutionWasAbandonedCannotRenew() { // given IdempotencyOwner executing = executing(); gateway.advance(Duration.ofSeconds(60)); @@ -368,7 +368,26 @@ class RedisIdempotencyStoreAdapterTest { } @Test - @DisplayName("an expired processing lease can be taken over, and the attempt increments") + @DisplayName("lease가 지난 실행 중 기록은 넘겨주지 않고 회복 필요로 답한다") + void anExpiredExecutionIsAnsweredRecoveryRequiredNotTakenOver() { + // given + IdempotencyOwner executing = executing(); + gateway.advance(Duration.ofSeconds(60)); + + // when + IdempotencyClaimOutcome next = store.claim(claim(store.newClaimAttempt(OTHER_OPERATION))); + + // then + // TAKEN_OVER would run work whose effect is unknown a second time. The live contract proves + // the program; this proves the fixture's copy of it agrees. + assertThat(next) + .isInstanceOfSatisfying( + IdempotencyClaimOutcome.RecoveryRequired.class, + recovery -> assertThat(recovery.currentAttempt()).isEqualTo(executing.attempt())); + } + + @Test + @DisplayName("an expired claim that never started can be taken over, and the attempt increments") void anExpiredLeaseIsTakenOver() { store.claim(claim(store.newClaimAttempt(OPERATION))); diff --git a/src/adapter/outbound/cache-redis/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/ratelimit/RedisServerTimeScriptContractTest.java b/src/adapter/outbound/cache-redis/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/ratelimit/RedisServerTimeScriptContractTest.java index a678f21e..2ee0ef98 100644 --- a/src/adapter/outbound/cache-redis/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/ratelimit/RedisServerTimeScriptContractTest.java +++ b/src/adapter/outbound/cache-redis/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/ratelimit/RedisServerTimeScriptContractTest.java @@ -28,7 +28,10 @@ class RedisServerTimeScriptContractTest { String transition = script(IdempotencyScripts.class, "TRANSITION"); assertThat(claim).contains("redis.call('TIME')"); - assertThat(claim).contains("leaseUntil', nowMillis + tonumber(ARGV[6])"); + assertThat(claim).contains("local leaseMillis = tonumber(ARGV[6])"); + assertThat(claim).contains("leaseUntil', nowMillis + leaseMillis"); + // The key outlives the lease by the replay TTL (re-review 2026-11-04 RR-03-11). + assertThat(claim).contains("redis.call('PEXPIRE', KEYS[1], leaseMillis + afterLease)"); assertThat(transition).contains("redis.call('TIME')"); assertThat(transition).contains("leaseUntil', nowMillis + tonumber(ARGV[7])"); } diff --git a/src/adapter/outbound/httpclient/README.md b/src/adapter/outbound/httpclient/README.md index b80f46d8..f79afca1 100644 --- a/src/adapter/outbound/httpclient/README.md +++ b/src/adapter/outbound/httpclient/README.md @@ -43,9 +43,10 @@ no reactive (`WebClient`/Reactor Netty) surface; it was removed on 2026-09-25 (d `HttpDeadlineExceededException`. A streaming download's setup (up to the response headers) runs inside the same deadline; after hand-over each body gap is bounded by `read-idle` and nothing else, on both transports — the JDK transport arms its own per-read timer for this, because the JDK client - has no socket read timeout. A body that fails after hand-over throws the same stable exceptions as - a buffered call (`HttpResponseTimeoutException` for a stalled body, `HttpResponseTruncatedException` - for a cut one). + has no socket read timeout. A body that fails after hand-over throws `HttpBodyReadException`, an + `IOException` as `InputStream.read` promises, whose `failure()` is the same stable exception a + buffered call throws (`HttpResponseTimeoutException` for a stalled body, + `HttpResponseTruncatedException` for a cut one). - **Evidence.** Each attempt reports `NOT_SENT`, `SENT_NO_RESPONSE`, `RESPONSE_RECEIVED`, or `PARTIAL_RESPONSE`. `NOT_SENT` is only claimed when a stage failure proves it. - **Retry.** Never from the HTTP method alone. Idempotency, body replayability, evidence, deadline, diff --git a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/api/error/HttpBodyReadException.java b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/api/error/HttpBodyReadException.java new file mode 100644 index 00000000..da25f976 --- /dev/null +++ b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/api/error/HttpBodyReadException.java @@ -0,0 +1,42 @@ +package dev.caskeleton.adapter.outbound.httpclient.api.error; + +import java.io.IOException; +import java.util.Objects; + +/** + * A streamed response body failed after it was handed over; {@link #failure()} says how. + * + *

An {@link IOException}, because {@code InputStream.read} promises one: a caller that reads the + * body with {@code catch (IOException)} — to delete a partial file, to retry — still sees the + * failure. The classified failure the gateway would have thrown for a buffered call ({@link + * HttpResponseTimeoutException} for a stalled body, {@link HttpResponseTruncatedException} for a + * cut one, stage {@code RESPONSE_BODY}, evidence {@code PARTIAL_RESPONSE}) is the cause and {@link + * #failure()}, for a retry policy or a log that branches on category. + * + *

Re-review 2026-11-04 RR-03-14: the body threw the unchecked {@link HttpClientException} + * itself, which a {@code catch (IOException)} let through. Rejected: keeping the unchecked type and + * only documenting it — the {@code InputStream} contract is what every reader of a stream relies + * on, and a document does not reach a library that reads the stream for its caller. + */ +public final class HttpBodyReadException extends IOException { + + private static final long serialVersionUID = 1L; + + /** + * Wraps a classified body failure. + * + * @param failure the classified failure; its message is the caller-safe phrase + */ + public HttpBodyReadException(HttpClientException failure) { + super(Objects.requireNonNull(failure, "classified body failure").getMessage(), failure); + } + + /** + * The classified failure: category, metadata and evidence. + * + * @return the failure this exception carries + */ + public HttpClientException failure() { + return (HttpClientException) getCause(); + } +} diff --git a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/api/result/BlockingStreamingResponse.java b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/api/result/BlockingStreamingResponse.java index 260ef842..7992080f 100644 --- a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/api/result/BlockingStreamingResponse.java +++ b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/api/result/BlockingStreamingResponse.java @@ -17,6 +17,16 @@ public interface BlockingStreamingResponse extends AutoCloseable { Map> headers(); + /** + * The body, bounded after hand-over by {@code read-idle} per wait. + * + *

A read that fails throws {@link + * dev.caskeleton.adapter.outbound.httpclient.api.error.HttpBodyReadException} — an {@code + * IOException} whose {@code failure()} is the classified {@code HttpResponseTimeoutException} + * (stalled) or {@code HttpResponseTruncatedException} (cut). + * + * @return the body stream; closing the response closes it + */ InputStream body(); @Override diff --git a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/jdk/JdkStreamingClientHttpRequest.java b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/jdk/JdkStreamingClientHttpRequest.java index f38f1459..679e48be 100644 --- a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/jdk/JdkStreamingClientHttpRequest.java +++ b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/jdk/JdkStreamingClientHttpRequest.java @@ -9,6 +9,7 @@ import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; import java.time.Duration; +import java.util.Collections; import java.util.Objects; import java.util.Set; import java.util.TreeSet; @@ -19,6 +20,7 @@ import org.springframework.http.HttpHeaders; import org.springframework.http.HttpMethod; import org.springframework.http.client.AbstractBufferingClientHttpRequest; import org.springframework.http.client.ClientHttpResponse; +import org.springframework.util.StringUtils; /** * A JDK request for a streaming download, whose timeouts end at the response headers (review @@ -40,6 +42,9 @@ import org.springframework.http.client.ClientHttpResponse; */ final class JdkStreamingClientHttpRequest extends AbstractBufferingClientHttpRequest { + /** The JDK property that lets a deployment send some restricted headers after all. */ + private static final String ALLOW_RESTRICTED_HEADERS = "jdk.httpclient.allowRestrictedHeaders"; + /** Headers the JDK client refuses to set, as Spring's own JDK request lists them. */ private static final Set RESTRICTED_HEADERS = restrictedHeaders(); @@ -120,9 +125,21 @@ final class JdkStreamingClientHttpRequest extends AbstractBufferingClientHttpReq } } - private static Set restrictedHeaders() { + /** + * Spring's {@code JdkClientHttpRequest.disallowedHeaders()}, which is private, step for step: the + * five names, less any the {@code jdk.httpclient.allowRestrictedHeaders} property allows. A fixed + * copy ignored the property, so a deployment that set it sent {@code Host} on ordinary calls and + * dropped it on streamed ones (re-review 2026-11-04 RR-03-15). {@code + * JdkStreamingRestrictedHeadersTest} compares the two lists for several property values, so a + * Spring change shows up as a failing test rather than as two lists that quietly differ. + */ + static Set restrictedHeaders() { Set restricted = new TreeSet<>(String.CASE_INSENSITIVE_ORDER); restricted.addAll(Set.of("connection", "content-length", "expect", "host", "upgrade")); - return restricted; + String allowed = System.getProperty(ALLOW_RESTRICTED_HEADERS); + if (allowed != null) { + restricted.removeAll(StringUtils.commaDelimitedListToSet(allowed)); + } + return Collections.unmodifiableSet(restricted); } } diff --git a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/jdk/ReadIdleInputStream.java b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/jdk/ReadIdleInputStream.java index 02643336..a836ec1d 100644 --- a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/jdk/ReadIdleInputStream.java +++ b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/jdk/ReadIdleInputStream.java @@ -15,24 +15,48 @@ import java.util.concurrent.atomic.AtomicBoolean; * RR-04-03). * *

The JDK client has no socket read timeout, so a streamed body that stopped arriving held the - * caller's thread and the connection forever. Each read arms a timer on the JDK's shared delay - * scheduler; when it fires the stream is closed, which wakes the blocked read, and the read reports - * a {@link SocketTimeoutException} — the same exception Apache's socket timeout raises, so one - * classifier reading names both. A read that returns first disarms its timer. + * caller's thread and the connection forever. A read that has waited {@code read-idle} for the + * server closes the stream, which wakes it, and it reports a {@link SocketTimeoutException} — the + * same exception Apache's socket timeout raises, so one classifier reading names both. Only the + * time a read spends waiting counts: time the caller spends between reads is not the server's. + * + *

One watchdog per stream, not a timer per read (re-review 2026-11-04 RR-03-13). A read only + * records when it started; the watchdog wakes once per {@code read-idle}, closes the stream if the + * read in progress has waited that long, and otherwise sleeps until that read's deadline. Arming a + * timer on every read put a task on the JVM's single delay scheduler and took it off again — + * measured at about 3 µs a read, and a JVM-wide ceiling near 0.5–0.75 million reads a second shared + * with every other {@code CompletableFuture} timeout in the process. The watchdog costs a clock + * read and two volatile writes per read, and one scheduled task per {@code read-idle} while reads + * keep coming. Rejected: a transport-owned {@code ScheduledExecutorService} with a timer per read — + * it moves the queue off the shared scheduler but keeps a queue insertion and removal on every + * read. + * + *

{@link #skip} reads through this stream rather than delegating: the JDK body does not override + * {@code skip}, so {@code InputStream.skip} ran many reads inside one guarded call and a long skip + * over a steady body timed out as one long wait (re-review 2026-11-04 RR-03-12). */ final class ReadIdleInputStream extends FilterInputStream { + /** The largest scratch buffer {@link #skip} reads into, as {@code InputStream.skip} uses. */ + private static final int SKIP_BUFFER_BYTES = 2048; + + /** {@link #readStartedAt} while no read is waiting. */ + private static final long NOT_READING = Long.MIN_VALUE; + @FunctionalInterface private interface Read { long run() throws IOException; } - private final Duration readIdle; + private final long readIdleNanos; + private final AtomicBoolean watchdogArmed = new AtomicBoolean(); private final AtomicBoolean idled = new AtomicBoolean(); + private volatile long readStartedAt = NOT_READING; + private volatile boolean closed; ReadIdleInputStream(InputStream delegate, Duration readIdle) { super(Objects.requireNonNull(delegate, "body stream")); - this.readIdle = Objects.requireNonNull(readIdle, "read idle timeout"); + this.readIdleNanos = Objects.requireNonNull(readIdle, "read idle timeout").toNanos(); } @Override @@ -47,14 +71,33 @@ final class ReadIdleInputStream extends FilterInputStream { @Override public long skip(long count) throws IOException { - return guarded(() -> super.skip(count)); + if (count <= 0) { + return 0; + } + byte[] scratch = new byte[(int) Math.min(SKIP_BUFFER_BYTES, count)]; + long remaining = count; + while (remaining > 0) { + int read = read(scratch, 0, (int) Math.min(scratch.length, remaining)); + if (read < 0) { + break; + } + remaining -= read; + } + return count - remaining; + } + + @Override + public void close() throws IOException { + closed = true; + super.close(); } private long guarded(Read read) throws IOException { requireNotIdled(); - CompletableFuture timer = new CompletableFuture<>(); - CompletableFuture expiry = - timer.completeOnTimeout(null, readIdle.toNanos(), TimeUnit.NANOSECONDS).thenRun(this::idle); + readStartedAt = System.nanoTime(); + if (!watchdogArmed.get() && watchdogArmed.compareAndSet(false, true)) { + watchFrom(readStartedAt); + } try { long result = read.run(); requireNotIdled(); @@ -63,9 +106,37 @@ final class ReadIdleInputStream extends FilterInputStream { requireNotIdled(); throw failure; } finally { - // Completing first skips the idle action and lets the scheduler drop the pending task. - timer.cancel(false); - expiry.cancel(false); + readStartedAt = NOT_READING; + } + } + + /** Wakes the watchdog when a read that started at {@code started} reaches read-idle. */ + private void watchFrom(long started) { + long left = Math.max(0, readIdleNanos - (System.nanoTime() - started)); + CompletableFuture.delayedExecutor(left, TimeUnit.NANOSECONDS).execute(this::watch); + } + + private void watch() { + if (closed || idled.get()) { + return; + } + long started = readStartedAt; + if (started != NOT_READING) { + // The second read of readStartedAt: a read that ended at its deadline and was followed at + // once by another must not have the new one closed under it. + if (System.nanoTime() - started >= readIdleNanos && readStartedAt == started) { + idle(); + } else { + watchFrom(started); + } + return; + } + watchdogArmed.set(false); + // A read that started after the check above saw the watchdog armed and left it alone; look + // again, or that read would wait unwatched. + started = readStartedAt; + if (started != NOT_READING && watchdogArmed.compareAndSet(false, true)) { + watchFrom(started); } } @@ -79,7 +150,7 @@ final class ReadIdleInputStream extends FilterInputStream { } /** - * After the timer fired, whatever the read returned — data, end of stream or "closed" — is not + * After the watchdog fired, whatever the read returned — data, end of stream or "closed" — is not * the rest of the body. */ private void requireNotIdled() throws SocketTimeoutException { diff --git a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/resilience/AttemptResiliencePipeline.java b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/resilience/AttemptResiliencePipeline.java index df8c946e..e37346a4 100644 --- a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/resilience/AttemptResiliencePipeline.java +++ b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/resilience/AttemptResiliencePipeline.java @@ -23,10 +23,14 @@ import java.util.function.Supplier; * circuit error: the upstream never saw the request, and counting our own back-pressure as upstream * failure would open the breaker on a healthy dependency. * - *

An attempt its caller abandoned is not recorded either, for the same reason: an attempt that a - * caller's own deadline ended while the upstream was still answering within the profile's limits is - * the caller running out of time, not the upstream failing (review 2026-11-03 RR-04-01). Its - * permission is given back instead. + *

An attempt its caller abandoned is not recorded either: an attempt that failed after a + * caller's own deadline — set earlier than the profile's, and passed by the time it failed — is + * taken as the caller running out of time, not the upstream failing (review 2026-11-03 RR-04-01). + * Its permission is given back instead. The check does not ask whether the upstream was still + * answering within the profile's limits, and that is a blind spot: when every caller's budget is + * shorter than the stage timeouts, a hung upstream ends each attempt at the caller's deadline first + * and is never recorded, so the circuit never opens (re-review 2026-11-04 RR-03-16, + * docs/httpclient/configuration-reference.md "Two deadlines"). */ public final class AttemptResiliencePipeline { diff --git a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/restclient/ClassifiedBodyInputStream.java b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/restclient/ClassifiedBodyInputStream.java index f2597bda..aef483fb 100644 --- a/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/restclient/ClassifiedBodyInputStream.java +++ b/src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/restclient/ClassifiedBodyInputStream.java @@ -1,5 +1,6 @@ package dev.caskeleton.adapter.outbound.httpclient.restclient; +import dev.caskeleton.adapter.outbound.httpclient.api.error.HttpBodyReadException; import dev.caskeleton.adapter.outbound.httpclient.api.error.HttpClientException; import java.io.FilterInputStream; import java.io.IOException; @@ -14,7 +15,9 @@ import java.util.function.Function; *

A streamed body is read after the gateway has returned, so the gateway's own failure mapping * never sees it. Without this a read-idle timeout or a cut connection reached the caller as the * engine's bare {@code IOException} — no category, no evidence, nothing a retry policy or a log - * could branch on. Closing is the caller's own act and is passed through unclassified. + * could branch on. Each failure is thrown as {@link HttpBodyReadException}: still an {@code + * IOException}, as {@code InputStream} promises, carrying the classified failure (re-review + * 2026-11-04 RR-03-14). Closing is the caller's own act and is passed through unclassified. */ final class ClassifiedBodyInputStream extends FilterInputStream { @@ -31,7 +34,7 @@ final class ClassifiedBodyInputStream extends FilterInputStream { try { return super.read(); } catch (IOException failure) { - throw classifier.apply(failure); + throw new HttpBodyReadException(classifier.apply(failure)); } } @@ -40,7 +43,7 @@ final class ClassifiedBodyInputStream extends FilterInputStream { try { return super.read(buffer, offset, length); } catch (IOException failure) { - throw classifier.apply(failure); + throw new HttpBodyReadException(classifier.apply(failure)); } } @@ -49,7 +52,7 @@ final class ClassifiedBodyInputStream extends FilterInputStream { try { return super.skip(count); } catch (IOException failure) { - throw classifier.apply(failure); + throw new HttpBodyReadException(classifier.apply(failure)); } } @@ -58,7 +61,7 @@ final class ClassifiedBodyInputStream extends FilterInputStream { try { return super.available(); } catch (IOException failure) { - throw classifier.apply(failure); + throw new HttpBodyReadException(classifier.apply(failure)); } } } diff --git a/src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/jdk/JdkStreamingRestrictedHeadersTest.java b/src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/jdk/JdkStreamingRestrictedHeadersTest.java new file mode 100644 index 00000000..4cb25e1c --- /dev/null +++ b/src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/jdk/JdkStreamingRestrictedHeadersTest.java @@ -0,0 +1,69 @@ +package dev.caskeleton.adapter.outbound.httpclient.jdk; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.lang.reflect.Method; +import java.util.List; +import java.util.Locale; +import java.util.Set; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.parallel.ResourceLock; +import org.junit.jupiter.api.parallel.Resources; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; + +/** + * The streaming JDK request drops the same headers Spring's own JDK request drops, so a profile + * sends the same headers whether the call streams or not (re-review 2026-11-04 RR-03-15). + * + *

Spring computes its list from the {@code jdk.httpclient.allowRestrictedHeaders} system + * property; the streaming request used to keep a fixed copy that ignored it, and a deployment that + * set the property sent {@code Host} on ordinary calls and silently dropped it on streamed ones. + * Spring's list is private, so it is read reflectively: if Spring renames it, this test fails and + * says so, rather than the two lists drifting apart unseen. + */ +@ResourceLock(Resources.SYSTEM_PROPERTIES) +class JdkStreamingRestrictedHeadersTest { + + private static final String ALLOW_PROPERTY = "jdk.httpclient.allowRestrictedHeaders"; + + @ParameterizedTest + @ValueSource( + strings = {"", "host", "Host,upgrade", "connection,content-length,expect,host,upgrade"}) + @DisplayName("jdk.httpclient.allowRestrictedHeaders 값이 무엇이든 스트리밍 요청이 버리는 헤더는 Spring JDK 요청과 같다") + void theStreamingRequestDropsWhatSpringsJdkRequestDrops(String allowed) throws Exception { + // given + String previous = System.getProperty(ALLOW_PROPERTY); + if (allowed.isEmpty()) { + System.clearProperty(ALLOW_PROPERTY); + } else { + System.setProperty(ALLOW_PROPERTY, allowed); + } + try { + // when + Set streaming = JdkStreamingClientHttpRequest.restrictedHeaders(); + + // then + assertThat(lowerCased(streaming)).isEqualTo(lowerCased(springsDisallowedHeaders())); + } finally { + if (previous == null) { + System.clearProperty(ALLOW_PROPERTY); + } else { + System.setProperty(ALLOW_PROPERTY, previous); + } + } + } + + @SuppressWarnings("unchecked") + private static Set springsDisallowedHeaders() throws Exception { + Method disallowed = + Class.forName("org.springframework.http.client.JdkClientHttpRequest") + .getDeclaredMethod("disallowedHeaders"); + disallowed.setAccessible(true); + return (Set) disallowed.invoke(null); + } + + private static List lowerCased(Set names) { + return names.stream().map(name -> name.toLowerCase(Locale.ROOT)).sorted().toList(); + } +} diff --git a/src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/jdk/ReadIdleInputStreamTest.java b/src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/jdk/ReadIdleInputStreamTest.java new file mode 100644 index 00000000..6b70aa42 --- /dev/null +++ b/src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/jdk/ReadIdleInputStreamTest.java @@ -0,0 +1,152 @@ +package dev.caskeleton.adapter.outbound.httpclient.jdk; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.catchThrowable; + +import java.io.IOException; +import java.io.InputStream; +import java.net.SocketTimeoutException; +import java.time.Duration; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; + +/** + * What the JDK transport's read-idle bound promises: each read that waits longer than {@code + * read-idle} for the server ends the body, and nothing else does (re-review 2026-11-04 RR-03-12, + * RR-03-13). + */ +class ReadIdleInputStreamTest { + + private static final Duration READ_IDLE = Duration.ofMillis(300); + private static final Duration GAP = Duration.ofMillis(20); + + @Test + @DisplayName("데이터가 read-idle보다 짧은 간격으로 오면 read-idle보다 오래 걸리는 skip도 끝까지 건너뛴다") + void aSkipLongerThanReadIdleOverASteadyBodySucceeds() throws IOException { + // given + // 40 bytes at 20 ms apart: 800 ms in all, every gap well inside the 300 ms bound. + try (InputStream body = new ReadIdleInputStream(new DrippingStream(40, GAP), READ_IDLE)) { + + // when + long skipped = body.skip(40); + + // then + assertThat(skipped).isEqualTo(40); + } + } + + @Test + @DisplayName("데이터가 read-idle보다 짧은 간격으로 오면 본문 전체가 read-idle보다 오래 걸려도 끝까지 읽힌다") + void aSteadyBodyLongerThanReadIdleIsReadToTheEnd() throws IOException { + // given + try (InputStream body = new ReadIdleInputStream(new DrippingStream(40, GAP), READ_IDLE)) { + + // when + byte[] read = body.readAllBytes(); + + // then + assertThat(read).hasSize(40); + } + } + + @Test + @DisplayName("한 read가 read-idle보다 오래 서버를 기다리면 SocketTimeoutException으로 끝난다") + void aReadThatWaitsLongerThanReadIdleTimesOut() { + // given + long started = System.nanoTime(); + InputStream body = new ReadIdleInputStream(new StalledStream(), READ_IDLE); + + // when + Throwable failure = catchThrowable(body::read); + + // then + Duration waited = Duration.ofNanos(System.nanoTime() - started); + assertThat(failure).isInstanceOf(SocketTimeoutException.class); + assertThat(waited).isGreaterThanOrEqualTo(READ_IDLE).isLessThan(READ_IDLE.multipliedBy(3)); + } + + @Test + @DisplayName("호출자가 read 사이에 오래 머문 시간은 read-idle로 세지 않는다") + void timeTheCallerSpendsBetweenReadsIsNotIdle() throws Exception { + // given + try (InputStream body = new ReadIdleInputStream(new DrippingStream(2, GAP), READ_IDLE)) { + int first = body.read(); + TimeUnit.MILLISECONDS.sleep(READ_IDLE.multipliedBy(2).toMillis()); + + // when + int second = body.read(); + + // then + assertThat(new int[] {first, second}).containsExactly('x', 'x'); + } + } + + /** Gives one byte per read, each after {@code gap}. */ + private static final class DrippingStream extends InputStream { + + private final Duration gap; + private int remaining; + + private DrippingStream(int bytes, Duration gap) { + this.remaining = bytes; + this.gap = gap; + } + + @Override + public int read() throws IOException { + if (remaining == 0) { + return -1; + } + try { + TimeUnit.NANOSECONDS.sleep(gap.toNanos()); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + throw new IOException("interrupted", interrupted); + } + remaining--; + return 'x'; + } + + @Override + public int read(byte[] buffer, int offset, int length) throws IOException { + if (length == 0) { + return 0; + } + int value = read(); + if (value < 0) { + return -1; + } + buffer[offset] = (byte) value; + return 1; + } + } + + /** Never answers until closed, as a server that stopped sending does; closing wakes the read. */ + private static final class StalledStream extends InputStream { + + private final CountDownLatch closed = new CountDownLatch(1); + + @Override + public int read() throws IOException { + try { + closed.await(); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + throw new IOException("interrupted", interrupted); + } + throw new IOException("closed"); + } + + @Override + public int read(byte[] buffer, int offset, int length) throws IOException { + return read(); + } + + @Override + public void close() { + closed.countDown(); + } + } +} diff --git a/src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/restclient/StreamingBodyLifetimeTest.java b/src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/restclient/StreamingBodyLifetimeTest.java index a19d3c6f..121241d7 100644 --- a/src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/restclient/StreamingBodyLifetimeTest.java +++ b/src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/restclient/StreamingBodyLifetimeTest.java @@ -4,6 +4,8 @@ import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.catchThrowable; import dev.caskeleton.adapter.outbound.httpclient.api.OperationName; +import dev.caskeleton.adapter.outbound.httpclient.api.error.HttpBodyReadException; +import dev.caskeleton.adapter.outbound.httpclient.api.error.HttpClientException; import dev.caskeleton.adapter.outbound.httpclient.api.error.HttpResponseTimeoutException; import dev.caskeleton.adapter.outbound.httpclient.api.operation.AttemptStage; import dev.caskeleton.adapter.outbound.httpclient.api.operation.ExecutionEvidence; @@ -71,7 +73,7 @@ class StreamingBodyLifetimeTest { @EnumSource( value = TransportType.class, names = {"APACHE", "JDK"}) - @DisplayName("본문 사이 공백이 read-idle을 넘으면 분류된 응답 시간 초과로 끝난다") + @DisplayName("본문 사이 공백이 read-idle을 넘으면 IOException인 본문 읽기 실패에 분류된 응답 시간 초과를 실어 끝난다") void aBodyGapLongerThanReadIdleEndsWithAClassifiedTimeout(TransportType transport) throws Exception { try (MockHttpServer server = MockHttpServer.start(); @@ -93,9 +95,13 @@ class StreamingBodyLifetimeTest { // then Duration elapsed = Duration.ofNanos(System.nanoTime() - started); - assertThat(failure).isInstanceOf(HttpResponseTimeoutException.class); + // An IOException, as InputStream.read promises, so a caller's catch (IOException) still + // sees it; the classified failure rides on it (re-review 2026-11-04 RR-03-14). + assertThat(failure).isInstanceOf(HttpBodyReadException.class); assertThat(elapsed).isLessThan(STALL_GAP); - HttpResponseTimeoutException timeout = (HttpResponseTimeoutException) failure; + HttpClientException classified = ((HttpBodyReadException) failure).failure(); + assertThat(classified).isInstanceOf(HttpResponseTimeoutException.class); + HttpResponseTimeoutException timeout = (HttpResponseTimeoutException) classified; assertThat(timeout.metadata().stage()).isEqualTo(AttemptStage.RESPONSE_BODY); assertThat(timeout.metadata().evidence()).isEqualTo(ExecutionEvidence.PARTIAL_RESPONSE); } diff --git a/src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/platform/provider/ses/SesEventNormalizer.java b/src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/platform/provider/ses/SesEventNormalizer.java index f35bd5b5..f23507d2 100644 --- a/src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/platform/provider/ses/SesEventNormalizer.java +++ b/src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/platform/provider/ses/SesEventNormalizer.java @@ -55,9 +55,8 @@ public final class SesEventNormalizer { default -> event(NormalizedEventType.UNKNOWN, type, messageId, occurredAt); }); // One SNS delivery carries one SES event, so the delivery's id is the event's. It is what makes - // a redelivery a duplicate in the ledger: the other identity, the event fingerprint, is keyed - // by - // the active HMAC key and changes when the key rotates (review 2026-11-03 RR-07-07). + // a redelivery a duplicate in the ledger: the other identity, the event fingerprint, uses the + // active HMAC key and changes when the key rotates (review 2026-11-03 RR-07-07). return events.stream().map(event -> withEventId(event, providerEventId)).toList(); } diff --git a/src/adapter/outbound/objectstorage/README.md b/src/adapter/outbound/objectstorage/README.md index 26eed116..97d73e0b 100644 --- a/src/adapter/outbound/objectstorage/README.md +++ b/src/adapter/outbound/objectstorage/README.md @@ -105,8 +105,10 @@ testImplementation testFixtures(project(':adapter:outbound:objectstorage')) ### MinIO qualification lanes -`objectStorageMinioContractTest` and `objectStorageMinioFaultTest` (in `mainCheck`) run a real MinIO -and Toxiproxy. Their images are read from one file, +`objectStorageMinioContractTest` and `objectStorageMinioFaultTest` run a real MinIO and Toxiproxy. +They run in the root `integrationCheck` (main push and nightly) and in +`.github/workflows/object-storage-qualification.yml` on pull requests that touch object storage — +not in this leaf's `mainCheck`, which ran them a second time on every main push. Their images are read from one file, `src/test/resources/object-storage/minio-provider-evidence.json`, whose `image` digest is the release the `status`, `excludedProfiles` and `limitations` beside it were measured on. `MinioQualificationImage` (in `src/test`, checked by the leaf's `test` without Docker) reads it. diff --git a/src/adapter/outbound/objectstorage/src/main/resources/META-INF/additional-spring-configuration-metadata.json b/src/adapter/outbound/objectstorage/src/main/resources/META-INF/additional-spring-configuration-metadata.json new file mode 100644 index 00000000..2b0d80fa --- /dev/null +++ b/src/adapter/outbound/objectstorage/src/main/resources/META-INF/additional-spring-configuration-metadata.json @@ -0,0 +1,10 @@ +{ + "properties": [ + { + "name": "app.object-storage.enabled", + "type": "java.lang.Boolean", + "description": "Whether the object-storage capability is on. It is on whenever this module is on the classpath; false turns it off and leaves only the explicit legacy seam. Read by @ConditionalOnProperty, which the configuration processor cannot see, so it is declared here for the environment contract (verifyEnvKeys) and the IDE.", + "defaultValue": true + } + ] +} diff --git a/src/adapter/outbound/persistence-jpa/CLAUDE.md b/src/adapter/outbound/persistence-jpa/CLAUDE.md index a8c65e89..5e489322 100644 --- a/src/adapter/outbound/persistence-jpa/CLAUDE.md +++ b/src/adapter/outbound/persistence-jpa/CLAUDE.md @@ -38,7 +38,7 @@ packages. | `startup` | package-private startup policy (`JpaStartupAutoConfiguration`): Flyway migration strategy, datasource env/product checks, OSIV/DDL/Hikari/TLS/H2 guards, the `jpaplatform` actuator view, the readiness `db` contribution. Moved here from app-bootstrap | | `cache`, `envers`, `querydsl`, `security`, `observation` | opt-in library capabilities and the runtime-role verifier | | `experimental` | multi-tenancy, RLS, schema/database tenancy, read replica — library code a deployment constructs itself; no flag turns it on | -| `testkit` (`src/testFixtures/java`) | ArchUnit rules, fixtures, query/plan assertions, PostgreSQL matrix, failure injection, Flyway validate gate and concurrent-index inspector | +| `testkit` (`src/testFixtures/java`) | ArchUnit rules, fixtures, query/plan assertions, PostgreSQL matrix, failure injection, Flyway validate gate, concurrent-index inspector and lock-wait inspector | `springdata`, `hibernate` (`.batch`, `.bulk`, `.stateless`, statement inspector), `postgresql` (`.error`, `.lock`, `.constraint`, `.json`, `.array`, `.range`, `.copy`), `api.query`, `envers`, @@ -93,6 +93,9 @@ not grow by wiring accident. - A caller that retries re-runs the whole use case in a new transaction and a new Persistence Context. - OSIV is false in every runtime profile; Flyway owns schema change and Hibernate only validates. +- A migration that locks a table already in use starts with `SET LOCAL lock_timeout = '5s';` + (`docs/jpa/migration-guide.md` "Bound the lock wait"). `MigrationLockWaitTest` refuses one + without it; its list of migrations released before the rule only ever shrinks. - Metric tags, exception messages, and logs carry no SQL parameters, entity ids, tenant ids, or PII. - Contracts run against real PostgreSQL. H2 never satisfies one. diff --git a/src/adapter/outbound/persistence-jpa/README.md b/src/adapter/outbound/persistence-jpa/README.md index a2fc8b65..36cdf9e0 100644 --- a/src/adapter/outbound/persistence-jpa/README.md +++ b/src/adapter/outbound/persistence-jpa/README.md @@ -227,9 +227,14 @@ failure translation contract 다. 여기서는 SPI 구조와 fallback 근거만 - **PostgreSQL 전용**: data source 가 PostgreSQL 이 아니면 기동을 거부한다. `bootRunH2` 는 provider 를 `disabled` 나 `redis` 로 둔다. - **응답 크기**: 완료 응답은 8 KiB 이하만 행에 inline 저장하고, 넘으면 거부한다. -- **만료**: store 가 완료·실패 때 `expires_at` 을 앞으로 옮기고, `IdempotencyReaper` - (`IdempotencyRecordJpaRepository` 사용, 기본 10분 간격 `reaper-interval`)가 지난 행을 지운다. - reaper 는 provider 가 `postgresql` 일 때만 돈다. +- **만료**: claim 은 `expires_at` 을 처리 lease 끝 + replay TTL 로 두고, renew 는 lease 를 늘린 만큼 + `expires_at` 도 뒤로 민다. 완료·실패 때 다시 앞으로 옮긴다. `IdempotencyReaper` + (`IdempotencyRecordJpaRepository` 사용, 기본 10분 간격 `reaper-interval`)가 상태와 상관없이 지난 행을 + 지운다. 그래서 `expires_at` 이 lease 보다 먼저 오면 실행 중인 행이 지워지고 다음 claim 이 같은 작업을 + 다시 돌린다(재리뷰 2026-11-04 RR-03-11). 공유 계약의 PostgreSQL lane 은 시간을 흘린 뒤 이 reaper 를 + 실제로 돌려 이 경우를 본다. reaper 는 provider 가 `postgresql` 일 때만 돈다. +- **lease 가 지난 실행**: lease 가 지난 `EXECUTING` 행은 넘겨받지 않고 `ABANDONED` 로 바꿔 + `RecoveryRequired` 로 답한다. Redis store 도 같다(`IdempotencyStoreContract`, RR-01-11). ## outbox — `JpaOutboxStore` 외 @@ -297,8 +302,11 @@ A가 FAILED로 바꾸면 다시 발행되고(중복 전달), A의 늦은 PUBLISH owner는 비우고 token은 그대로 둔다(token은 증가만 한다). 1 row면 `APPLIED`, 0 row면 `STALE`. `findById` + setter 경로와 `@Version`은 쓰지 않는다. - **실패 원인 코드(`V22`)**: `markFailed`·`markDeferred`·`markDead`는 같은 조건부 UPDATE에서 - `last_failure_code`를 쓴다(publisher가 붙인 코드, 없으면 결과·예외 타입 이름). entity 필드는 읽기 - 전용이라 append INSERT와 claim flush는 이 열을 건드리지 않는다. `longRetries(n, now)`는 미룬 claim이 + `last_failure_code`를 쓴다(publisher가 붙인 코드, 없으면 결과·예외 타입 이름). append INSERT는 이 열을 + 쓰지 않는다. claim이 행을 DEAD로 옮길 때(`ATTEMPT_COUNTERS_INCONSISTENT`, `UNREADABLE_ROW`)는 claim의 row + lock 아래 entity flush로 그 이유를 쓴다 — 앞선 발행 실패의 코드가 남아 원인을 잘못 가리키지 않게(재리뷰 2차 + RR-02-12). 이 DEAD는 relay 결과를 거치지 않아 `outbox.publisher.published.total{outcome=DEAD}`에 안 잡히므로 + 경보는 `outbox.pending.size{status=DEAD}`로도 건다. `longRetries(n, now)`는 미룬 claim이 n번 이상인 FAILED 행 수와 가장 오래된 행의 나이를 센다(`outbox.retry.long.*` 게이지). 상태 `PARKED`는 DDL이 없다 — claim이 PENDING·FAILED·IN_FLIGHT만 고르고, FIFO 게이트는 PUBLISHED 아닌 모든 행을 막는 앞 행으로 본다. claim이 시도 횟수가 어긋난 행(`deferred_attempt_count > attempt_count`)을 만나면 DEAD로 @@ -356,6 +364,9 @@ A가 FAILED로 바꾸면 다시 발행되고(중복 전달), A의 늦은 PUBLISH `JpaOutboxStore.oldestUnpublishedAgeSecondsByEventType` 가 `HashMap` 을 쓰는 건 키가 enum 이 아니라 String(event-type 이름)이기 때문이다 — `countByStatus()` 는 키가 `OutboxEventStatus` enum 이라 `EnumMap` 을 쓴다(두 반환 타입이 의도적으로 다름). +`status` 에는 CHECK 가 없어 이 버전이 모르는 상태(뒤 버전이 쓴 상태, 손으로 친 값)가 있을 수 있다. +`countByStatus()` 는 그 상태만 빼고 세고 처음 한 번 WARN 으로 이름과 행 수를 말한다. 예전에는 그 이름 하나로 +집계 전체가 실패해 PENDING 게이지까지 꺼졌다(재리뷰 2차 RR-02-11). ### `OutboxReaper` PUBLISHED row 는 이미 전달된 terminal-success record 라 무한 보관할 필요가 없다. reaper 가 @@ -374,6 +385,12 @@ machine-readable SSOT다(개수는 registry가 소유한다). `selected` base ca 구분하며, 실제 PostgreSQL 테스트 통과만으로 immutable 운영 evidence가 필요한 R2를 주장하지 않는다. +이미 쓰이는 테이블을 잠그는 migration은 그 잠금 앞에 `SET LOCAL lock_timeout = '5s';`를 둔다. 상한이 없으면 +그 테이블을 건드린 열린 트랜잭션 하나 뒤에서 끝없이 기다리고, 그동안 뒤에 온 읽기·쓰기가 모두 migration 뒤에 줄을 +선다(V22, 재리뷰 2차 RR-02-09). `MigrationLockWaitTest`가 이 leaf DB에 도는 모든 tree를 검사하고, +`postgresqlMigrationLockWaitIntegrationTest` lane이 실제 PostgreSQL에서 상한 안에 실패하는지 본다 +(`docs/jpa/migration-guide.md` "Bound the lock wait"). + 적용된 versioned migration은 수정하지 않는다. Flyway가 기동할 때마다 checksum을 검사하므로 바꾼 파일은 이미 그것을 적용한 모든 DB의 기동을 막는다. 변경은 새 버전으로 추가하고, `src/test/resources/db/applied-migrations.sha256`에 한 줄을 더한다 diff --git a/src/adapter/outbound/persistence-jpa/build.gradle b/src/adapter/outbound/persistence-jpa/build.gradle index 9a7e7a7c..a15f558d 100644 --- a/src/adapter/outbound/persistence-jpa/build.gradle +++ b/src/adapter/outbound/persistence-jpa/build.gradle @@ -176,6 +176,7 @@ def readinessLanes = [ postgresqlSecurityBaselineIntegrationTest : ['PostgreSqlSecurityBaselineIntegrationTest', 'runtimeRoleCannotCreateInApplicationSchemaOrTempAndUsesTrustedSearchPath'], postgresqlMigrationIntegrationTest : ['PostgreSqlMigrationIntegrationTest'], + postgresqlMigrationLockWaitIntegrationTest : ['PostgreSqlMigrationLockWaitIntegrationTest'], postgresqlIntLockMigrationCompatibilityIntegrationTest : ['PostgreSqlIntLockMigrationCompatibilityIntegrationTest'], postgresqlTransactionIntegrationTest : ['PostgreSqlTransactionIntegrationTest'], postgresqlDistributedLockIntegrationTest : ['PostgreSqlDistributedLockIntegrationTest'], diff --git a/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/IdempotencyClaimGateway.java b/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/IdempotencyClaimGateway.java index 7884d1d0..e2d882d4 100644 --- a/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/IdempotencyClaimGateway.java +++ b/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/IdempotencyClaimGateway.java @@ -17,6 +17,14 @@ import org.springframework.jdbc.core.JdbcOperations; * rows back rather than an exception. That matters: a unique-violation would abort the caller's * transaction, and the loser has to be able to read the winner's row in the same transaction to * find out what happened. + * + *

A claimed row expires ({@code expires_at}, which the reaper purges on whatever the row's + * state) at the end of its processing lease plus the replay TTL — {@link #recordLifetimeMillis}. It + * used to expire at the replay TTL alone: with a replay TTL shorter than the lease the reaper + * deleted the row of a running operation and the next claim inserted a fresh one and ran it again + * (re-review 2026-11-04 RR-03-11), and with the two equal the row went with its lease, leaving + * nothing to answer "recovery required" from (RR-01-11). The Redis store keeps its key for the same + * span; {@code IdempotencyStoreContract} holds both to it. */ final class IdempotencyClaimGateway { @@ -122,7 +130,7 @@ final class IdempotencyClaimGateway { request.scope().digest(), request.scope().operationCode(), request.requestFingerprint().hex(), - request.replayTtl().toMillis(), + recordLifetimeMillis(request), request.scope().digest(), request.scope().keyDigestVersion(), request.scope().operationCode(), @@ -151,12 +159,21 @@ final class IdempotencyClaimGateway { request.processingLeaseTtl().toMillis(), request.policyRevision(), request.responseCodecId(), - request.replayTtl().toMillis(), + recordLifetimeMillis(request), request.scope().digest(), row.state().name(), row.stateRevision()); } + /** + * How long a claimed row lives: its processing lease, and then the replay TTL as the window in + * which a resend still finds it — "in progress" during the lease, "recovery required" after an + * execution outlived it. + */ + static long recordLifetimeMillis(IdempotencyClaimRequest request) { + return request.processingLeaseTtl().plus(request.replayTtl()).toMillis(); + } + /** * Abandons an execution whose lease expired, repeating the whole owner tuple. * diff --git a/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/IdempotencyTransitionGateway.java b/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/IdempotencyTransitionGateway.java index d10e2d3f..af8f021e 100644 --- a/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/IdempotencyTransitionGateway.java +++ b/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/IdempotencyTransitionGateway.java @@ -39,11 +39,22 @@ final class IdempotencyTransitionGateway { and processing_lease_until > clock_timestamp() """; + /** + * Extends the lease, and moves {@code expires_at} with it: the row keeps the time it had past its + * old lease (the replay TTL the claim gave it), now past the new one. The right-hand side reads + * the row as it was before this update. Without the move a lease renewed past {@code expires_at} + * let the reaper delete a running operation's row (re-review 2026-11-04 RR-03-11); a row claimed + * before the margin existed keeps at least its new lease. + */ private static final String RENEW_SQL = """ update idempotency_record set state_revision = state_revision + 1, processing_lease_until = clock_timestamp() + (? * interval '1 millisecond'), + expires_at = greatest( + expires_at, + clock_timestamp() + (? * interval '1 millisecond') + + greatest(expires_at - processing_lease_until, interval '0')), last_transition_operation_id = ?, last_transition_kind = 'RENEW', last_transition_result_digest = ?, @@ -151,6 +162,7 @@ final class IdempotencyTransitionGateway { return jdbc.update( RENEW_SQL, processingLeaseTtl.toMillis(), + processingLeaseTtl.toMillis(), operationId, resultDigest, owner.scope().digest(), diff --git a/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/JpaOutboxStore.java b/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/JpaOutboxStore.java index fba06bfb..3a933778 100644 --- a/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/JpaOutboxStore.java +++ b/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/JpaOutboxStore.java @@ -28,6 +28,8 @@ import java.util.List; import java.util.Map; import java.util.Objects; import java.util.Optional; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.slf4j.spi.LoggingEventBuilder; @@ -52,6 +54,9 @@ public class JpaOutboxStore private final OutboxEventJpaRepository repository; private final OutboxClaimRepository claimRepository; + /** Status names {@link #countByStatus} has met and does not know; each is reported once. */ + private final Set reportedUnknownStatuses = ConcurrentHashMap.newKeySet(); + public JpaOutboxStore( OutboxEventJpaRepository repository, OutboxClaimRepository claimRepository) { this.repository = repository; @@ -106,8 +111,11 @@ public class JpaOutboxStore *

The increment is an {@code UPDATE}, which holds the counter row's lock until the caller * commits: a second writer of the same aggregate waits, then increments the committed value, so * the numbers follow commit order. It never goes below the aggregate's highest outbox row, so a - * row an instance still on the wall-clock scheme writes during a rolling deploy comes first. The - * counter outlives the reaper's deletes, so a number is never handed out twice. + * row an instance still on the wall-clock scheme writes during a rolling deploy comes first — as + * long as it writes under the same identity. A row written under an identity this version does + * not use (a tenant scope the deploy changed) is invisible here, and the next number can be + * smaller than it; such a deploy stops the old writers first (runbook outbox-dead-letter §8, + * RR-02-10). The counter outlives the reaper's deletes, so a number is never handed out twice. * *

An aggregate's first number is inserted instead, after the larger of its highest remaining * row and the floor V21 recorded — the moment it ran, above every clock number already sent for @@ -150,7 +158,9 @@ public class JpaOutboxStore *

The WARN names the cause. A row whose deferred claims outnumber its claims is not damaged: * it is what a manual re-drive that reset {@code attempt_count} and left {@code * deferred_attempt_count} produces (re-review 2026-11-03 RR-05-01), and "cannot read" sent the - * operator looking for corruption instead of re-running the corrected runbook SQL. + * operator looking for corruption instead of re-running the corrected runbook SQL. The row says + * it too: the same code goes into {@code last_failure_code}, in place of the code of whatever + * publish failure came before, which no longer explains the row (re-review round 2, RR-02-12). */ @Override public List claimBatch( @@ -165,8 +175,7 @@ public class JpaOutboxStore entity.setLeaseOwner(owner); entity.setLeaseToken(entity.getLeaseToken() + 1); if (entity.getDeferredAttemptCount() > entity.getAttemptCount()) { - deadLetterOnClaim(entity) - .addKeyValue("error.reason", "ATTEMPT_COUNTERS_INCONSISTENT") + deadLetterOnClaim(entity, "ATTEMPT_COUNTERS_INCONSISTENT") .log( "outbox claim moved a row to DEAD: deferred_attempt_count ({}) exceeds" + " attempt_count ({}); a manual re-drive that did not reset" @@ -180,8 +189,7 @@ public class JpaOutboxStore try { event = toClaimedOutboxEvent(entity); } catch (RuntimeException unreadable) { - deadLetterOnClaim(entity) - .addKeyValue("error.reason", "UNREADABLE_ROW") + deadLetterOnClaim(entity, "UNREADABLE_ROW") .setCause(unreadable) .log("outbox claim moved a row it cannot read as an event to DEAD"); continue; @@ -191,12 +199,17 @@ public class JpaOutboxStore return List.copyOf(leases); } - /** Moves a claimed row it will not hand out to DEAD and starts the WARN that says why. */ - private static LoggingEventBuilder deadLetterOnClaim(OutboxEventEntity entity) { + /** + * Moves a claimed row it will not hand out to DEAD under {@code reason}, written to the row's + * {@code last_failure_code} and the WARN it starts. + */ + private static LoggingEventBuilder deadLetterOnClaim(OutboxEventEntity entity, String reason) { entity.setStatus(OutboxEventStatus.DEAD.name()); entity.setLeaseOwner(null); + entity.setLastFailureCode(reason); return log.atWarn() .addKeyValue("event_id", entity.getEventId()) + .addKeyValue("error.reason", reason) .addKeyValue("runbook_link", "runbook://outbox/dead-letter"); } @@ -256,17 +269,45 @@ public class JpaOutboxStore CallerTransaction.require("outbox " + operation); } + /** + * The rows per status this version knows. A status it does not know is left out and named once in + * a WARN: {@code status} has no CHECK, so a version meets a status written after it — an instance + * still on the version before PARKED during a rollout, the version an operator rolled back to, a + * value typed by hand. Failing the whole count on that name took every {@code + * outbox.pending.size} gauge down with it, the PENDING backlog the alert reads included + * (re-review round 2, RR-02-11). + */ @Override public Map countByStatus() { List rows = repository.countGroupedByStatus(); Map result = new EnumMap<>(OutboxEventStatus.class); for (Object[] row : rows) { - OutboxEventStatus status = OutboxEventStatus.valueOf((String) row[0]); - result.put(status, (Long) row[1]); + String name = (String) row[0]; + Long count = (Long) row[1]; + Optional status = knownStatus(name); + if (status.isPresent()) { + result.put(status.get(), count); + } else if (reportedUnknownStatuses.add(name)) { + log.atWarn() + .addKeyValue("outbox.status", name) + .addKeyValue("row_count", count) + .log( + "outbox_event holds rows in a status this version does not know; they are left out" + + " of outbox.pending.size. A later version or a manual UPDATE wrote it"); + } } return result; } + private static Optional knownStatus(String name) { + for (OutboxEventStatus status : OutboxEventStatus.values()) { + if (status.name().equals(name)) { + return Optional.of(status); + } + } + return Optional.empty(); + } + @Override public Map oldestUnpublishedAgeSecondsByEventType(Instant now) { List rows = repository.findOldestUnpublishedOccurredAtByEventType(); diff --git a/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/entity/OutboxEventEntity.java b/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/entity/OutboxEventEntity.java index 4ed084a8..7f27b394 100644 --- a/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/entity/OutboxEventEntity.java +++ b/src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/entity/OutboxEventEntity.java @@ -43,7 +43,7 @@ public class OutboxEventEntity implements Persistable { @Column(name = "occurred_at", nullable = false, updatable = false) private Instant occurredAt; - /** Lifecycle status string: PENDING | IN_FLIGHT | PUBLISHED | FAILED | DEAD. */ + /** Lifecycle status string: PENDING | IN_FLIGHT | PUBLISHED | FAILED | DEAD | PARKED. */ @Column(name = "status", nullable = false, length = 16) private String status; @@ -57,19 +57,19 @@ public class OutboxEventEntity implements Persistable { @Column(name = "deferred_attempt_count", nullable = false) private int deferredAttemptCount; - /** Dual-purpose per status (PENDING / IN_FLIGHT / FAILED) — see README "outbox". */ /** - * The code of the row's last failed completion (V22): the publisher's own name for the failure, - * or the outcome or exception type when it gave none. + * Why the row last failed (V22): the publisher's own name for the failure, or the outcome or + * exception type when it gave none, or the claim's reason for moving the row to DEAD. * - *

Read-only to the entity: only the relay's failed completions write it, each in its one - * fenced UPDATE ({@code OutboxEventJpaRepository#failLease} and its siblings). The append's - * INSERT and the claim's flush leave it out, so they neither clear a code nor need the column on - * a database that has not run V22 yet. + *

The relay's failed completions write it in their one fenced UPDATE ({@code + * OutboxEventJpaRepository#failLease} and its siblings); the claim writes it through this entity + * when it moves a row it will not hand out to DEAD, under the claim's row lock (re-review round + * 2, RR-02-12). The append's INSERT leaves it out, so a new row has none. */ - @Column(name = "last_failure_code", length = 120, insertable = false, updatable = false) + @Column(name = "last_failure_code", length = 120, insertable = false) private String lastFailureCode; + /** Dual-purpose per status (PENDING / IN_FLIGHT / FAILED) — see README "outbox". */ @Column(name = "next_attempt_at", nullable = false) private Instant nextAttemptAt; @@ -395,4 +395,8 @@ public class OutboxEventEntity implements Persistable { public String getLastFailureCode() { return lastFailureCode; } + + public void setLastFailureCode(String lastFailureCode) { + this.lastFailureCode = lastFailureCode; + } } diff --git a/src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V20__outbox_event_deferred_attempt_count.sql b/src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V20__outbox_event_deferred_attempt_count.sql index 1df02290..21336e91 100644 --- a/src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V20__outbox_event_deferred_attempt_count.sql +++ b/src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V20__outbox_event_deferred_attempt_count.sql @@ -12,6 +12,20 @@ -- -- The default fills existing rows without a rewrite: every earlier claim counts against the -- budget, which is what the relay assumed until now. +-- +-- This file was edited after it was first written, before any deployment applied it (the branch +-- that added it was not merged), to add the bound below. applied-migrations.sha256 carries the +-- new checksum; a development database that applied the earlier text fails Flyway validation: +-- recreate it. + +-- Bound the wait for the lock this migration takes (ACCESS EXCLUSIVE on outbox_event, held until it +-- commits). Unbounded, it queues behind any open transaction that has touched the table, and every +-- read and write of the table after it queues behind the migration — the relay's claim and every +-- business write that appends an event. Five seconds, then the migration fails, rolls back whole +-- and the database stays at V19: deploy again once that transaction has ended (re-review 2026-11-03 +-- round 2, RR-02-09; the rule is docs/jpa/migration-guide.md "Bound the lock wait"). +SET LOCAL lock_timeout = '5s'; + ALTER TABLE outbox_event ADD COLUMN deferred_attempt_count INTEGER NOT NULL DEFAULT 0 CONSTRAINT ck_outbox_event_deferred_attempt_count CHECK (deferred_attempt_count >= 0); diff --git a/src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V22__outbox_event_last_failure_code.sql b/src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V22__outbox_event_last_failure_code.sql index dbe4ed59..6215f261 100644 --- a/src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V22__outbox_event_last_failure_code.sql +++ b/src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V22__outbox_event_last_failure_code.sql @@ -17,5 +17,19 @@ -- IN_FLIGHT only (so the partial index ix_outbox_event_eligible leaves PARKED out), and the order -- gate and its index ix_outbox_event_unpublished_aggregate_order treat every status but PUBLISHED as -- blocking — which is what holding a row in its aggregate's place means. +-- +-- This file was edited after it was first written, before any deployment applied it (the branch +-- that added it was not merged), to add the bound below. applied-migrations.sha256 carries the +-- new checksum; a development database that applied the earlier text fails Flyway validation: +-- recreate it. + +-- Bound the wait for the lock this migration takes (ACCESS EXCLUSIVE on outbox_event, held until it +-- commits). Unbounded, it queues behind any open transaction that has touched the table, and every +-- read and write of the table after it queues behind the migration — the relay's claim, the gauges +-- and every business write that appends an event. Five seconds, then the migration fails, rolls +-- back whole and the database stays at V21: deploy again once that transaction has ended (re-review +-- 2026-11-03 round 2, RR-02-09; the rule is docs/jpa/migration-guide.md "Bound the lock wait"). +SET LOCAL lock_timeout = '5s'; + ALTER TABLE outbox_event ADD COLUMN last_failure_code VARCHAR(120); diff --git a/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/PostgreSqlIdempotencyStoreContractTest.java b/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/PostgreSqlIdempotencyStoreContractTest.java index 04b31d23..cac6ff47 100644 --- a/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/PostgreSqlIdempotencyStoreContractTest.java +++ b/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/idempotency/postgresql/PostgreSqlIdempotencyStoreContractTest.java @@ -1,18 +1,29 @@ package dev.caskeleton.adapter.outbound.persistence.idempotency.postgresql; +import dev.caskeleton.adapter.outbound.persistence.idempotency.jpa.IdempotencyReaper; +import dev.caskeleton.adapter.outbound.persistence.idempotency.jpa.IdempotencyRecordJpaRepository; import dev.caskeleton.adapter.outbound.persistence.platform.JpaPlatformContractSupport; import dev.caskeleton.adapter.outbound.persistence.transaction.SpringTransaction; import dev.caskeleton.application.idempotency.IdempotencyStorePort; import dev.caskeleton.application.idempotency.testkit.IdempotencyStoreContract; +import jakarta.persistence.EntityManagerFactory; +import java.time.Clock; import java.time.Duration; +import java.util.Properties; import java.util.function.Supplier; import org.flywaydb.core.Flyway; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Tag; +import org.springframework.data.jpa.repository.support.JpaRepositoryFactory; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.datasource.DataSourceTransactionManager; +import org.springframework.orm.jpa.JpaTransactionManager; +import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean; +import org.springframework.orm.jpa.SharedEntityManagerCreator; +import org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter; +import org.springframework.transaction.support.TransactionTemplate; /** * The PostgreSQL store, against a real database through the shipped migrations, held to the same @@ -20,7 +31,12 @@ import org.springframework.jdbc.datasource.DataSourceTransactionManager; * *

Every call runs in its own {@link SpringTransaction} write transaction, as the executor's * calls do; the store refuses to run outside one. Time is the database's ({@code - * clock_timestamp()}), so the timed case lets it pass for real with a one-second lease. + * clock_timestamp()}), so the timed cases let it pass for real with a one-second lease. + * + *

Letting time pass also runs the deployment's own {@link IdempotencyReaper}, through the real + * Spring Data repository. The store never expires a row itself — the reaper deletes every row past + * {@code expires_at}, whatever its state — so a contract that slept without reaping could not see a + * row that would have been purged under a live lease (re-review 2026-11-04 RR-03-11). */ @Tag("jpa-contract") class PostgreSqlIdempotencyStoreContractTest extends IdempotencyStoreContract { @@ -28,6 +44,9 @@ class PostgreSqlIdempotencyStoreContractTest extends IdempotencyStoreContract { private static JpaPlatformContractSupport support; private static JdbcTemplate jdbc; private static SpringTransaction transactions; + private static LocalContainerEntityManagerFactoryBean reaperPersistence; + private static IdempotencyReaper reaper; + private static TransactionTemplate reaperTransactions; @BeforeAll static void start() { @@ -44,10 +63,37 @@ class PostgreSqlIdempotencyStoreContractTest extends IdempotencyStoreContract { "update capability_schema_registry set lifecycle_state = 'ACTIVE' " + "where capability_id = 'jpa-idempotency-owner-safe-v2'"); transactions = new SpringTransaction(new DataSourceTransactionManager(support.dataSource())); + startReaper(); + } + + /** + * The reaper as a deployment builds it: its repository from Spring Data over the idempotency + * entity, its purge inside a JPA transaction (the {@code @Transactional} a bean proxy would add). + */ + private static void startReaper() { + reaperPersistence = new LocalContainerEntityManagerFactoryBean(); + reaperPersistence.setDataSource(support.dataSource()); + reaperPersistence.setJpaVendorAdapter(new HibernateJpaVendorAdapter()); + reaperPersistence.setPackagesToScan( + "dev.caskeleton.adapter.outbound.persistence.idempotency.jpa.entity"); + Properties properties = new Properties(); + properties.setProperty("hibernate.jdbc.time_zone", "UTC"); + reaperPersistence.setJpaProperties(properties); + reaperPersistence.afterPropertiesSet(); + EntityManagerFactory entityManagers = reaperPersistence.getObject(); + IdempotencyRecordJpaRepository repository = + new JpaRepositoryFactory( + SharedEntityManagerCreator.createSharedEntityManager(entityManagers)) + .getRepository(IdempotencyRecordJpaRepository.class); + reaper = new IdempotencyReaper(repository, Clock.systemUTC()); + reaperTransactions = new TransactionTemplate(new JpaTransactionManager(entityManagers)); } @AfterAll static void stop() { + if (reaperPersistence != null) { + reaperPersistence.destroy(); + } if (support != null) { support.close(); } @@ -81,6 +127,7 @@ class PostgreSqlIdempotencyStoreContractTest extends IdempotencyStoreContract { Thread.currentThread().interrupt(); throw new IllegalStateException("interrupted while letting the lease run", interrupted); } + reaperTransactions.executeWithoutResult(ignored -> reaper.reap()); } /** diff --git a/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/readiness/PostgreSqlMigrationLockWaitIntegrationTest.java b/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/readiness/PostgreSqlMigrationLockWaitIntegrationTest.java new file mode 100644 index 00000000..73ccad04 --- /dev/null +++ b/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/readiness/PostgreSqlMigrationLockWaitIntegrationTest.java @@ -0,0 +1,205 @@ +package dev.caskeleton.adapter.outbound.persistence.readiness; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.catchThrowable; + +import com.zaxxer.hikari.HikariDataSource; +import java.sql.Connection; +import java.sql.SQLException; +import java.sql.Statement; +import java.time.Duration; +import java.util.List; +import java.util.Locale; +import java.util.UUID; +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.ScheduledFuture; +import java.util.concurrent.TimeUnit; +import org.flywaydb.core.Flyway; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.CsvSource; +import org.springframework.jdbc.core.JdbcTemplate; + +/** + * The migrations that lock a table already in use give up within their bound instead of stopping + * the table, against real PostgreSQL (re-review 2026-11-03 round 2, RR-02-09). + * + *

Each case is the deploy the bound exists for: a database one version below the migration, and + * a transaction that has read the table and stays open — an operator's {@code BEGIN} in psql, a + * report, an instance of the old version mid-request. The migration needs a lock that conflicts + * with that read. Unbounded, it waits until the reader leaves and every read and write of the table + * queues behind it meanwhile; the reader here leaves after fifteen seconds, so an unbounded + * migration ends up applied. Bounded, it fails with {@code lock_not_available} long before that, + * Flyway rolls it back whole, and the history still ends one version below. V21 has the same case + * against a writer in {@code PostgreSqlOutboxOrderKeyRunbookIntegrationTest}. + */ +class PostgreSqlMigrationLockWaitIntegrationTest { + + private static final String LOCK_NOT_AVAILABLE = "55P03"; + private static final Duration READER_STAYS = Duration.ofSeconds(15); + private static final Duration GIVES_UP_WITHIN = Duration.ofSeconds(10); + + private static PostgreSqlReadinessSupport postgres; + + @BeforeAll + static void setup() { + postgres = PostgreSqlReadinessSupport.start(); + } + + @AfterAll + static void cleanup() { + if (postgres != null) { + postgres.close(); + } + } + + @ParameterizedTest + @CsvSource({"20, outbox_event", "22, outbox_event"}) + @DisplayName("outbox_event를 읽는 트랜잭션이 열려 있으면 그 테이블을 잠그는 outbox 마이그레이션은 상한 안에 실패하고 한 버전 아래에 남는다") + void anOutboxMigrationGivesUpOnAHeldTableAndLeavesTheVersionBelowIt(int version, String table) + throws Exception { + try (HikariDataSource database = postgres.createDatabase(databaseName())) { + // given + Flyway below = outboxStream(database, version - 1); + below.migrate(); + + // when + Attempt attempt = + whileAReaderHolds(database, table, outboxStream(database, version)::migrate); + + // then + assertThat( + List.of( + String.valueOf(attempt.sqlState()), + attempt.tookLessThan(GIVES_UP_WITHIN), + below.info().current().getVersion().getVersion())) + .containsExactly(LOCK_NOT_AVAILABLE, true, String.valueOf(version - 1)); + } + } + + @ParameterizedTest + @CsvSource({ + "11, notification_reconciliation_job", + "12, notification_suppression", + "13, notification_request" + }) + @DisplayName("알림 테이블을 읽는 트랜잭션이 열려 있으면 그 테이블을 잠그는 알림 마이그레이션은 상한 안에 실패하고 한 버전 아래에 남는다") + void aNotificationMigrationGivesUpOnAHeldTableAndLeavesTheVersionBelowIt( + int version, String table) throws Exception { + try (HikariDataSource database = postgres.createDatabase(databaseName())) { + // given + applyCore(database); + Flyway below = notificationStream(database, version - 1); + below.baseline(); + below.migrate(); + + // when + Attempt attempt = + whileAReaderHolds(database, table, notificationStream(database, version)::migrate); + + // then + assertThat( + List.of( + String.valueOf(attempt.sqlState()), + attempt.tookLessThan(GIVES_UP_WITHIN), + below.info().current().getVersion().getVersion())) + .containsExactly(LOCK_NOT_AVAILABLE, true, String.valueOf(version - 1)); + } + } + + /** + * Runs the migration while another connection holds an open transaction that has read the table, + * and that transaction ends on its own after {@link #READER_STAYS}. + */ + private static Attempt whileAReaderHolds( + HikariDataSource database, String table, Runnable migration) throws SQLException { + try (Connection reader = database.getConnection()) { + reader.setAutoCommit(false); + try (Statement statement = reader.createStatement()) { + statement.execute("SELECT count(*) FROM " + table); + } + ScheduledExecutorService release = Executors.newSingleThreadScheduledExecutor(); + ScheduledFuture released = + release.schedule( + () -> { + reader.rollback(); + return null; + }, + READER_STAYS.toSeconds(), + TimeUnit.SECONDS); + try { + long started = System.nanoTime(); + Throwable thrown = catchThrowable(migration::run); + return new Attempt(thrown, Duration.ofNanos(System.nanoTime() - started)); + } finally { + released.cancel(true); + release.shutdownNow(); + } + } + } + + private static Flyway outboxStream(HikariDataSource database, int target) { + return Flyway.configure() + .dataSource(database) + .locations("classpath:db/migration/postgresql") + .target(String.valueOf(target)) + .load(); + } + + private static Flyway notificationStream(HikariDataSource database, int target) { + PostgreSqlOptionalStreamLifecycle.Stream stream = + PostgreSqlOptionalStreamLifecycle.notificationPlatform(); + return Flyway.configure() + .dataSource(database) + .locations(stream.location()) + .table(stream.historyTable()) + .baselineVersion("0") + .baselineOnMigrate(false) + .target(String.valueOf(target)) + .load(); + } + + /** The core stream, applied and promoted: what a deployment has before any capability stream. */ + private static void applyCore(HikariDataSource database) { + Flyway core = + Flyway.configure() + .dataSource(database) + .locations("classpath:db/migration/jpa/core") + .table("flyway_jpa_core_history") + .baselineVersion("0") + .baselineOnMigrate(false) + .load(); + core.baseline(); + core.migrate(); + new JdbcTemplate(database) + .update( + "UPDATE capability_schema_registry SET lifecycle_state = 'ACTIVE'" + + " WHERE capability_id = ?", + "jpa-flyway-migration"); + } + + private static String databaseName() { + return "lockwait_" + UUID.randomUUID().toString().replace("-", "").toLowerCase(Locale.ROOT); + } + + /** What the migration did while the reader held the table. */ + private record Attempt(Throwable thrown, Duration took) { + + /** The SQLState of the first SQL failure in the cause chain, or null when it did not fail. */ + String sqlState() { + for (Throwable cause = thrown; cause != null; cause = cause.getCause()) { + if (cause instanceof SQLException sql) { + return sql.getSQLState(); + } + } + return null; + } + + boolean tookLessThan(Duration bound) { + return took.compareTo(bound) < 0; + } + } +} diff --git a/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/readiness/PostgreSqlOutboxDispositionRunbookIntegrationTest.java b/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/readiness/PostgreSqlOutboxDispositionRunbookIntegrationTest.java index 027408b4..d5b7f9f9 100644 --- a/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/readiness/PostgreSqlOutboxDispositionRunbookIntegrationTest.java +++ b/src/adapter/outbound/persistence-jpa/src/postgresqlIntegrationTest/java/dev/caskeleton/adapter/outbound/persistence/readiness/PostgreSqlOutboxDispositionRunbookIntegrationTest.java @@ -75,6 +75,30 @@ class PostgreSqlOutboxDispositionRunbookIntegrationTest { } } + /** + * A re-drive that reset {@code attempt_count} alone is refused by the next claim, which moves the + * row back to DEAD (RR-05-01). The row then says why: before, it kept the code of the publish + * failure that dead-lettered it first, and the operator went after a broker credential instead of + * re-running the corrected SQL (re-review round 2, RR-02-12). + */ + @Test + @DisplayName("deferred 횟수를 되돌리지 않은 재발행 행을 claim이 DEAD로 옮기면 행의 원인 코드도 그 이유로 바뀐다") + void theClaimThatRefusesAHalfRedrivenRowWritesWhyOnTheRow() throws SQLException { + try (Outbox outbox = Outbox.create()) { + // given: dead after a refused credential, then re-driven by SQL that left the deferred count + String redriven = outbox.append("order-" + UUID.randomUUID(), 1, OCCURRED); + outbox.setCounters(redriven, "PENDING", 0, 2); + outbox.setFailureCode(redriven, "KAFKA_AUTHENTICATIONEXCEPTION"); + + // when + List claimed = outbox.claimedEventIds(); + + // then + assertThat(List.of(claimed, outbox.status(redriven), outbox.failureCode(redriven))) + .containsExactly(List.of(), "DEAD", "ATTEMPT_COUNTERS_INCONSISTENT"); + } + } + @Test @DisplayName("runbook 구간 재발행 SQL은 그 구간의 DEAD 행만 claim 가능하게 되돌린다") void theWindowRedriveReturnsOnlyTheDeadRowsOfTheWindow() throws SQLException { @@ -253,6 +277,11 @@ class PostgreSqlOutboxDispositionRunbookIntegrationTest { "UPDATE outbox_event SET last_failure_code = ? WHERE event_id = ?", failureCode, eventId); } + String failureCode(String eventId) { + return jdbc.queryForObject( + "SELECT last_failure_code FROM outbox_event WHERE event_id = ?", String.class, eventId); + } + String status(String eventId) { return jdbc.queryForObject( "SELECT status FROM outbox_event WHERE event_id = ?", String.class, eventId); diff --git a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/AppliedMigrationImmutabilityTest.java b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/AppliedMigrationImmutabilityTest.java index cb7afea5..66f6d3e5 100644 --- a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/AppliedMigrationImmutabilityTest.java +++ b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/AppliedMigrationImmutabilityTest.java @@ -9,10 +9,8 @@ import java.nio.file.Path; import java.security.MessageDigest; import java.security.NoSuchAlgorithmException; import java.util.HexFormat; -import java.util.List; import java.util.Map; import java.util.TreeMap; -import java.util.stream.Stream; import org.junit.jupiter.api.Test; /** @@ -23,33 +21,6 @@ import org.junit.jupiter.api.Test; */ class AppliedMigrationImmutabilityTest { - /** - * Every tree whose migrations run against the database this leaf owns. - * - *

The notification capability's stream moved to {@code - * adapter:outbound:persistence-notification} with its bytes and its classpath location unchanged - * ({@code db/migration/jpa/notification-platform}), so a database that applied it still - * validates. Its pins stayed in this manifest and are checked where the files now live: moving a - * migration must not be the way it stops being frozen. - * - *

The fileserver stream ({@code db/migration/jpa/fileserver}, in {@code - * adapter:outbound:persistence-fileserver}) runs against the same database through the same - * Flyway history and was never pinned at all; it is checked here on the same terms. - * - *

The messaging platform's tree ({@code db/migration/messaging}, in {@code - * messaging-reliability-jdbc-postgresql}) runs against it too once the consume side is selected: - * {@code adapter:inbound:messaging} applies its inbox range (V11xx). The tree is pinned whole — - * the platform outbox range ships from the same location to deployments that use it, and an - * edited migration stops those just the same. - */ - private static final List MIGRATION_ROOTS = - List.of( - Path.of("src/main/resources/db/migration"), - Path.of("../persistence-notification/src/main/resources/db/migration"), - Path.of("../persistence-fileserver/src/main/resources/db/migration"), - Path.of( - "../../../messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration")); - private static final Path MANIFEST = Path.of("src/test/resources/db/applied-migrations.sha256"); @Test @@ -94,20 +65,8 @@ class AppliedMigrationImmutabilityTest { private static Map currentChecksums() throws IOException { Map checksums = new TreeMap<>(); - for (Path root : MIGRATION_ROOTS) { - try (Stream files = Files.walk(root)) { - for (Path file : - files - .filter(path -> path.getFileName().toString().matches("V\\d+__.+\\.sql")) - .toList()) { - String key = root.relativize(file).toString().replace('\\', '/'); - String previous = checksums.put(key, sha256(Files.readString(file))); - if (previous != null) { - throw new IllegalStateException( - key + " exists in two migration trees; the classpath would load one at random"); - } - } - } + for (Map.Entry migration : MigrationTrees.versionedMigrations().entrySet()) { + checksums.put(migration.getKey(), sha256(Files.readString(migration.getValue()))); } return checksums; } diff --git a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/MigrationLockWaitTest.java b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/MigrationLockWaitTest.java new file mode 100644 index 00000000..5495c145 --- /dev/null +++ b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/MigrationLockWaitTest.java @@ -0,0 +1,105 @@ +package dev.caskeleton.adapter.outbound.persistence.migration; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.adapter.outbound.persistence.testkit.migration.LockWaitMigrationInspector; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TreeMap; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; + +/** + * Every migration this leaf's database runs bounds its wait for a lock on a table already in use + * ({@link LockWaitMigrationInspector}; re-review 2026-11-03 round 2, RR-02-09). + * + *

V21 got the bound and V22, in the same commit, did not: a rule written into one file is not a + * rule for the next one. This test is. + */ +class MigrationLockWaitTest { + + /** + * Migrations released before the rule, which wait without a bound and stay that way: they are + * applied (on {@code main} when the rule came in), and an applied migration is never edited + * ({@link AppliedMigrationImmutabilityTest}). Their waits are behind every database that ran + * them. Nothing is added here — a new migration bounds its wait. + */ + private static final Set RELEASED_BEFORE_THE_RULE = + Set.of( + "jpa/core/V2__widen_capability_schema_stream.sql", + "jpa/idempotency/V2__request_hash_varchar.sql", + "postgresql/V5__int_lock_expired_after.sql", + "postgresql/V9__widen_capability_schema_stream.sql", + "postgresql/V10__idempotency_request_hash_varchar.sql", + "postgresql/V13__outbox_canonical_transport_compatibility.sql", + "postgresql/V14__validate_outbox_canonical_constraints.sql", + "postgresql/V16__outbox_canonical_hash_columns_varchar.sql", + "postgresql/V17__outbox_event_lease_fencing.sql", + "postgresql/V18__outbox_event_unpublished_aggregate_order.sql", + "postgresql/V19__outbox_event_canonical_only.sql", + "jpa/notification-platform/V4__notification_platform_alignment_and_activation.sql", + "jpa/notification-platform/V5__notification_recipient_expiry.sql", + "jpa/notification-platform/V6__notification_projection_facts.sql", + "jpa/notification-platform/V7__notification_collapse.sql", + "jpa/notification-platform/V8__notification_admin_claim.sql", + "jpa/notification-platform/V9__notification_evidence_certainty.sql", + "jpa/fileserver/V2__fileserver_recovery_and_staging_cleanup.sql", + "jpa/fileserver/V3__fileserver_fenced_cleanup_lease.sql", + "jpa/fileserver/V4__fileserver_upload_terminal_state.sql", + "jpa/fileserver/V5__fileserver_file_owner.sql", + "messaging/V1002__messaging_outbox_lease_fencing.sql", + "messaging/V1004__messaging_outbox_canonical_metadata.sql", + "messaging/V1005__messaging_outbox_ordering_key_index.sql"); + + private final LockWaitMigrationInspector inspector = new LockWaitMigrationInspector(); + + @Test + @DisplayName("규칙 이후의 모든 마이그레이션은 기존 테이블 잠금 앞에 SET LOCAL lock_timeout을 둔다") + void everyMigrationAfterTheRuleBoundsItsLockWait() throws IOException { + // given + Map migrations = MigrationTrees.versionedMigrations(); + + // when + Map> unbounded = unboundedStatements(migrations); + + // then + assertThat(unbounded.keySet()) + .as( + "these wait for a table lock without a bound: %s. Put SET LOCAL lock_timeout = '5s';" + + " first (docs/jpa/migration-guide.md, \"Bound the lock wait\")", + unbounded) + .isSubsetOf(RELEASED_BEFORE_THE_RULE); + } + + @Test + @DisplayName("규칙 이전 목록의 파일은 모두 아직 있고 실제로 상한 없이 잠금을 기다린다") + void everyMigrationReleasedBeforeTheRuleStillExistsAndStillWaitsUnbounded() throws IOException { + // given + Map migrations = MigrationTrees.versionedMigrations(); + + // when + Map> unbounded = unboundedStatements(migrations); + + // then + assertThat(unbounded.keySet()) + .as("an entry here that no longer exists or waits for nothing only hides the next file") + .containsAll(RELEASED_BEFORE_THE_RULE); + } + + private Map> unboundedStatements(Map migrations) + throws IOException { + Map> unbounded = new TreeMap<>(); + for (Map.Entry migration : migrations.entrySet()) { + List statements = + inspector.unboundedLockingStatements(Files.readString(migration.getValue())); + if (!statements.isEmpty()) { + unbounded.put(migration.getKey(), statements); + } + } + return unbounded; + } +} diff --git a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/MigrationTrees.java b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/MigrationTrees.java new file mode 100644 index 00000000..04727620 --- /dev/null +++ b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/migration/MigrationTrees.java @@ -0,0 +1,70 @@ +package dev.caskeleton.adapter.outbound.persistence.migration; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Map; +import java.util.TreeMap; +import java.util.stream.Stream; + +/** + * The migration trees that run against the database this leaf owns, read the one way every + * migration rule here reads them: {@link AppliedMigrationImmutabilityTest} pins their bytes and + * {@link MigrationLockWaitTest} holds them to the lock-wait rule. One list, so a tree added for one + * rule is not quietly missing from the other. + */ +final class MigrationTrees { + + /** + * Every tree whose migrations run against the database this leaf owns. + * + *

The notification capability's stream moved to {@code + * adapter:outbound:persistence-notification} with its bytes and its classpath location unchanged + * ({@code db/migration/jpa/notification-platform}), so a database that applied it still + * validates. Its pins stayed in this manifest and are checked where the files now live: moving a + * migration must not be the way it stops being frozen. + * + *

The fileserver stream ({@code db/migration/jpa/fileserver}, in {@code + * adapter:outbound:persistence-fileserver}) runs against the same database through the same + * Flyway history and was never pinned at all; it is checked here on the same terms. + * + *

The messaging platform's tree ({@code db/migration/messaging}, in {@code + * messaging-reliability-jdbc-postgresql}) runs against it too once the consume side is selected: + * {@code adapter:inbound:messaging} applies its inbox range (V11xx). The tree is pinned whole — + * the platform outbox range ships from the same location to deployments that use it, and an + * edited migration stops those just the same. + */ + static final List ROOTS = + List.of( + Path.of("src/main/resources/db/migration"), + Path.of("../persistence-notification/src/main/resources/db/migration"), + Path.of("../persistence-fileserver/src/main/resources/db/migration"), + Path.of( + "../../../messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration")); + + private MigrationTrees() {} + + /** + * Every versioned migration, keyed by its path below its tree ({@code postgresql/V22__...sql}), + * the key {@code applied-migrations.sha256} uses. + */ + static Map versionedMigrations() throws IOException { + Map migrations = new TreeMap<>(); + for (Path root : ROOTS) { + try (Stream files = Files.walk(root)) { + for (Path file : + files + .filter(path -> path.getFileName().toString().matches("V\\d+__.+\\.sql")) + .toList()) { + String key = root.relativize(file).toString().replace('\\', '/'); + if (migrations.put(key, file) != null) { + throw new IllegalStateException( + key + " exists in two migration trees; the classpath would load one at random"); + } + } + } + } + return migrations; + } +} diff --git a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/JpaOutboxStoreTest.java b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/JpaOutboxStoreTest.java index 6b359c1e..23b4bc5e 100644 --- a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/JpaOutboxStoreTest.java +++ b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/jpa/JpaOutboxStoreTest.java @@ -329,6 +329,27 @@ class JpaOutboxStoreTest { "runbook_link", "runbook://outbox/dead-letter"))); } + /** + * The claim moves a row to DEAD without the relay, so no failed completion writes the row's + * failure code. Without this the row kept the code of its last publish failure, and an operator + * reading {@code last_failure_code} looked for the wrong cause (re-review round 2, RR-02-12). + */ + @Test + @DisplayName("claim이 읽을 수 없는 행을 DEAD로 옮기면 그 원인 코드를 행에 남긴다") + void claimBatchLeavesTheReasonOnARowItCannotRead() { + // given + OutboxEventEntity tampered = pendingEntity("evt-tampered", "worklog-42"); + tampered.setEnvelopeSha256("2".repeat(64)); + when(claimRepo.claimEligible(eq(NOW), eq(1))).thenReturn(List.of(tampered)); + + // when + adapter.claimBatch(OWNER, 1, NOW, IN_FLIGHT_TIMEOUT); + + // then + assertThat(List.of(tampered.getStatus(), String.valueOf(tampered.getLastFailureCode()))) + .containsExactly("DEAD", "UNREADABLE_ROW"); + } + private static Map keyValues(ILoggingEvent event) { Map values = new LinkedHashMap<>(); if (event.getKeyValuePairs() != null) { @@ -421,6 +442,60 @@ class JpaOutboxStoreTest { Map.of(OutboxEventStatus.PENDING, 5L, OutboxEventStatus.IN_FLIGHT, 2L)); } + /** + * The status column has no CHECK, and a version that does not know a status meets it: an instance + * still on the version before PARKED during a rollout, the version an operator rolled back to, a + * value typed by hand. One unknown name failed the whole count, and with it every + * outbox.pending.size gauge, including the PENDING one the backlog alert reads (re-review round + * 2, RR-02-11). + */ + @Test + @DisplayName("이 버전이 모르는 상태 값이 테이블에 있어도 아는 상태의 수는 그대로 센다") + void countByStatusStillCountsTheKnownStatusesWhenTheTableHoldsAnUnknownOne() { + // given + List rows = + List.of( + new Object[] {"PENDING", 5L}, new Object[] {"ARCHIVED", 3L}, new Object[] {"DEAD", 1L}); + when(repo.countGroupedByStatus()).thenReturn(rows); + + // when + Map counts = adapter.countByStatus(); + + // then + assertThat(counts) + .containsExactlyInAnyOrderEntriesOf( + Map.of(OutboxEventStatus.PENDING, 5L, OutboxEventStatus.DEAD, 1L)); + } + + @Test + @DisplayName("모르는 상태 값은 한 번만 WARN으로 그 이름과 행 수를 말한다") + void countByStatusWarnsOnceNamingAStatusItDoesNotKnow() { + // given + List rows = List.of(new Object[] {"ARCHIVED", 3L}); + when(repo.countGroupedByStatus()).thenReturn(rows); + Logger storeLog = (Logger) LoggerFactory.getLogger(JpaOutboxStore.class); + ListAppender captured = new ListAppender<>(); + captured.start(); + storeLog.addAppender(captured); + + // when + try { + for (int poll = 0; poll < 3; poll++) { + adapter.countByStatus(); + } + } finally { + storeLog.detachAppender(captured); + } + + // then + assertThat(captured.list) + .singleElement() + .satisfies( + warn -> + assertThat(List.of(warn.getLevel().toString(), keyValues(warn))) + .containsExactly("WARN", Map.of("outbox.status", "ARCHIVED", "row_count", 3L))); + } + // ---- oldestUnpublishedAgeSecondsByEventType ---- @Test diff --git a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/JpaArchitectureRulesTest.java b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/JpaArchitectureRulesTest.java index d9405ea5..bc60ed6b 100644 --- a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/JpaArchitectureRulesTest.java +++ b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/JpaArchitectureRulesTest.java @@ -12,6 +12,8 @@ import dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity.InverseMa import dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity.InverseOneToOneEntity; import dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity.JoinColumnChildEntity; import dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity.JoinColumnParentEntity; +import dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity.MutualForeignKeyInvoiceEntity; +import dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity.MutualForeignKeyOrderEntity; import dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity.NoDefaultConstructorEntity; import dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity.OrderEntity; import dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity.OwningChildEntity; @@ -202,9 +204,32 @@ class JpaArchitectureRulesTest { .isInstanceOf(AssertionError.class) .hasMessageContaining("JoinColumnParentEntity.children") .hasMessageContaining("JoinColumnChildEntity.parent") + .hasMessageContaining("two fields map one link") .hasMessageContaining("one side only"); } + @Test + @DisplayName("서로 다른 외래 키로 서로를 가리키는 두 엔티티는 한 링크의 사본이 아니라 서로 참조하는 두 테이블로 거부된다") + void twoEntitiesThatEachHoldAForeignKeyToTheOtherAreRefusedForTheirMutualReference() { + // given + var classes = + new ClassFileImporter() + .importClasses(MutualForeignKeyOrderEntity.class, MutualForeignKeyInvoiceEntity.class); + + // when + Throwable failure = + catchThrowable( + () -> JpaArchitectureRules.associationsAreMappedFromOneSideOnly().check(classes)); + + // then + assertThat(failure) + .isInstanceOf(AssertionError.class) + .hasMessageContaining("MutualForeignKeyOrderEntity.latestInvoice") + .hasMessageContaining("MutualForeignKeyInvoiceEntity.order") + .hasMessageContaining("each holds a foreign key to the other") + .hasMessageNotContaining("two fields map one link"); + } + @Test @DisplayName("단방향 연관관계는 통과한다") void unidirectionalAssociationsPass() { diff --git a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/entity/MutualForeignKeyInvoiceEntity.java b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/entity/MutualForeignKeyInvoiceEntity.java new file mode 100644 index 00000000..60566133 --- /dev/null +++ b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/entity/MutualForeignKeyInvoiceEntity.java @@ -0,0 +1,30 @@ +package dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity; + +import jakarta.persistence.Entity; +import jakarta.persistence.FetchType; +import jakarta.persistence.Id; +import jakarta.persistence.JoinColumn; +import jakarta.persistence.ManyToOne; + +/** Refused by the association rule: the other half of {@link MutualForeignKeyOrderEntity}. */ +@Entity +public class MutualForeignKeyInvoiceEntity { + + @Id private Long id; + + @ManyToOne(fetch = FetchType.LAZY) + @JoinColumn(name = "order_id") + private MutualForeignKeyOrderEntity order; + + protected MutualForeignKeyInvoiceEntity() {} + + /** The identifier. */ + public Long id() { + return id; + } + + /** The order this invoice bills. */ + public MutualForeignKeyOrderEntity order() { + return order; + } +} diff --git a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/entity/MutualForeignKeyOrderEntity.java b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/entity/MutualForeignKeyOrderEntity.java new file mode 100644 index 00000000..8a71418e --- /dev/null +++ b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/entity/MutualForeignKeyOrderEntity.java @@ -0,0 +1,37 @@ +package dev.caskeleton.adapter.outbound.persistence.testkit.arch.entity; + +import jakarta.persistence.Entity; +import jakarta.persistence.FetchType; +import jakarta.persistence.Id; +import jakarta.persistence.JoinColumn; +import jakarta.persistence.OneToOne; + +/** + * Refused by the association rule: two entities that each hold a foreign key to the other. + * + *

{@code latest_invoice_id} here and {@link MutualForeignKeyInvoiceEntity#order}'s {@code + * order_id} are two links, not one mapped twice, so nothing falls out of step. They are refused for + * another reason: the two tables reference each other, so neither row can be inserted or deleted + * first and one key must be nullable and written by a second UPDATE (re-review round 2, RR-02-15). + */ +@Entity +public class MutualForeignKeyOrderEntity { + + @Id private Long id; + + @OneToOne(fetch = FetchType.LAZY) + @JoinColumn(name = "latest_invoice_id") + private MutualForeignKeyInvoiceEntity latestInvoice; + + protected MutualForeignKeyOrderEntity() {} + + /** The identifier. */ + public Long id() { + return id; + } + + /** The order's most recent invoice. */ + public MutualForeignKeyInvoiceEntity latestInvoice() { + return latestInvoice; + } +} diff --git a/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/migration/LockWaitMigrationInspectorTest.java b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/migration/LockWaitMigrationInspectorTest.java new file mode 100644 index 00000000..f0358905 --- /dev/null +++ b/src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/testkit/migration/LockWaitMigrationInspectorTest.java @@ -0,0 +1,89 @@ +package dev.caskeleton.adapter.outbound.persistence.testkit.migration; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.catchThrowable; + +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; + +/** + * A migration that locks a table already in use bounds how long it waits for that lock (re-review + * 2026-11-03 round 2, RR-02-09). + */ +class LockWaitMigrationInspectorTest { + + private final LockWaitMigrationInspector inspector = new LockWaitMigrationInspector(); + + @ParameterizedTest + @ValueSource( + strings = { + "ALTER TABLE outbox_event ADD COLUMN last_failure_code VARCHAR(120);", + "CREATE UNIQUE INDEX ux_poster_title ON poster (upper(title));", + "CREATE INDEX IF NOT EXISTS ix_ledger ON ONLY grpc_operation_ledger (completed_at);", + "DELETE FROM notification_suppression WHERE id = 1;", + "UPDATE outbox_event SET status = 'FAILED';", + "LOCK TABLE outbox_event IN SHARE MODE;", + "ALTER TABLE outbox_event ADD COLUMN c INT;\nSET LOCAL lock_timeout = '5s';", + "SET lock_timeout = '5s';\nALTER TABLE outbox_event ADD COLUMN c INT;" + }) + @DisplayName("이미 있는 테이블에 잠금을 거는 문장 앞에 SET LOCAL lock_timeout이 없으면 거부된다") + void aLockOnAnExistingTableWithoutABoundBeforeItIsRefused(String sql) { + // given + MigrationResource migration = new MigrationResource("V99__change.sql", sql); + + // when + Throwable refused = catchThrowable(() -> inspector.requireBoundedLockWait(migration)); + + // then + assertThat(refused) + .isInstanceOf(IllegalStateException.class) + .hasMessageContaining("V99__change.sql") + .hasMessageContaining("SET LOCAL lock_timeout"); + } + + @ParameterizedTest + @ValueSource( + strings = { + "SET LOCAL lock_timeout = '5s';\nALTER TABLE outbox_event ADD COLUMN c INT;", + "-- bound first\nset local lock_timeout = '5s';\ndelete from notification_suppression;", + "CREATE TABLE ledger (id BIGINT);\nCREATE INDEX ix_ledger ON ledger (id);\n" + + "ALTER TABLE ledger ADD COLUMN c INT;", + "CREATE TABLE IF NOT EXISTS public.ledger (id BIGINT);\nALTER TABLE ledger ADD c INT;", + "CREATE INDEX CONCURRENTLY ix_outbox ON outbox_event (status);", + "-- ALTER TABLE outbox_event ADD COLUMN c INT;\n/* DELETE FROM outbox_event; */\n" + + "COMMENT ON TABLE outbox_event IS 'update outbox_event set status';", + "CREATE TABLE poster_copy AS SELECT * FROM poster;" + }) + @DisplayName("잠금 대기에 상한을 두었거나 잠글 기존 테이블이 없으면 통과한다") + void aBoundedOrLockFreeMigrationPasses(String sql) { + // given + MigrationResource migration = new MigrationResource("V99__change.sql", sql); + + // when + Throwable refused = catchThrowable(() -> inspector.requireBoundedLockWait(migration)); + + // then + assertThat(refused).isNull(); + } + + @Test + @DisplayName("거부 메시지는 상한 없이 잠금을 기다리는 문장과 대상 테이블을 모두 말한다") + void theRefusalNamesEveryUnboundedStatementAndItsTable() { + // given + MigrationResource migration = + new MigrationResource( + "V12__suppression_unique.sql", + "DELETE FROM notification_suppression WHERE id > 1;\n" + + "ALTER TABLE notification_suppression ADD CONSTRAINT uk UNIQUE (id);"); + + // when + Throwable refused = catchThrowable(() -> inspector.requireBoundedLockWait(migration)); + + // then + assertThat(refused) + .hasMessageContaining("delete from notification_suppression") + .hasMessageContaining("alter table notification_suppression"); + } +} diff --git a/src/adapter/outbound/persistence-jpa/src/test/resources/db/applied-migrations.sha256 b/src/adapter/outbound/persistence-jpa/src/test/resources/db/applied-migrations.sha256 index bf86884e..1eb57d14 100644 --- a/src/adapter/outbound/persistence-jpa/src/test/resources/db/applied-migrations.sha256 +++ b/src/adapter/outbound/persistence-jpa/src/test/resources/db/applied-migrations.sha256 @@ -11,9 +11,9 @@ 2ba4214941a5563613b0dcc4f7cbf5f1278e342b900268721466e4bdeaa2fef2 jpa/idempotency/V1__expand_owner_safe_v2.sql 95be788589f01e32c9179acc80817e227e81805f8d518e8a9ac36e1c31b51da7 jpa/idempotency/V2__request_hash_varchar.sql 4379872989af0c97b937ff48527c788a2ba0216218d4df9435eba8c6e053ad8d jpa/notification-platform/V10__variables_payload_envelope_guard.sql -fb1eee652330995e2f761dd0c0cb9ca137b1c50dd99253aa9958c3ec130cf74b jpa/notification-platform/V11__notification_reconciliation_claim_lease.sql -a4c3b56e74bdeaa32be2e716e377aa920bc327b4517bf3d6c6c753cdbcf5c745 jpa/notification-platform/V12__notification_suppression_target_unique.sql -11bd6a456f2d0eb10c89a6f675a7a3af43f2ae1abda061a6f793f9d5a252f144 jpa/notification-platform/V13__notification_request_acceptance.sql +be51fabd07220f9ef885534ba36ada5f1491baf24cdfb54d25d93a19ddc29926 jpa/notification-platform/V11__notification_reconciliation_claim_lease.sql +43f9429f91c07528154ec01b721649b2c25fa98857ac154c7c08f4655634657f jpa/notification-platform/V12__notification_suppression_target_unique.sql +d01aaf59f66834358abf18246faa9a9eb8dfa9f42097ca9d968c44edff564630 jpa/notification-platform/V13__notification_request_acceptance.sql 1ac7cc1c2509868eb0ce202e1917bb36846f98f17ad4922abe3fac0ef128199b jpa/notification-platform/V1__notification_platform_core.sql 0add67aa23580e51712f84fabe9b09e08401ef708cfa1b3292b120fdb5c12fa2 jpa/notification-platform/V2__notification_platform_contact_template_policy.sql f5b98598a59cc4376f5d233b410f8a5ae9f3cd94156291ebdf2d0da5a66f34ef jpa/notification-platform/V3__notification_platform_inbox_admin.sql @@ -39,9 +39,9 @@ c20f1803c6afe84ac323a9fac9007f9a1676ed5ed253b72286c8fc1dee7c21d1 messaging/V100 b8f571e6cada72a8f2776f4462509ce4b535c964c75b1c3cfb34b9dc07446d59 postgresql/V17__outbox_event_lease_fencing.sql 44cacbde7d1381ca5290fb8ec28005e8127ff6b453ff84277ec8d1b3293d76c8 postgresql/V18__outbox_event_unpublished_aggregate_order.sql a0306e5b0fbcaf43984d256362dc4244aec57a0ce5031e4f5263be3762fe1caf postgresql/V19__outbox_event_canonical_only.sql -c997fa20e10b68e0ca1b30cd7275018deaadc99bb0370b7b806332e9826e5217 postgresql/V20__outbox_event_deferred_attempt_count.sql +25db296e55444636c8a017d9b736eecf4a839a9caf1733c0c4a5f37bae5a3161 postgresql/V20__outbox_event_deferred_attempt_count.sql 21744ea1aab2b29fbb3417b9a5c4389522199413048cc7035aa649ffcdd852cf postgresql/V21__outbox_aggregate_sequence.sql -459942722c69d43b15fdab34849c6e34bbca35727e800eca18fbc5b3eebe5f4c postgresql/V22__outbox_event_last_failure_code.sql +20da5649a29bf2305f0e04ab14003b21b7f120d867439fb01f1658c98179dba0 postgresql/V22__outbox_event_last_failure_code.sql 8ab106731d4d04d3719da0692069947736dc490ca46aa5a3ea9bdf65a5d956dc postgresql/V1__idempotency_record.sql 6b62a01a4eeb792b81015b4810ecc6f3f021e0a17dea50c871493f7694d4f0b9 postgresql/V3__outbox_event.sql c57ab8a35a10079190ad8a563be58fae4147c9406c5bb752debb6998324ccdf6 postgresql/V4__int_lock.sql diff --git a/src/adapter/outbound/persistence-jpa/src/testFixtures/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/SingleSidedAssociationCondition.java b/src/adapter/outbound/persistence-jpa/src/testFixtures/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/SingleSidedAssociationCondition.java index cbe87cde..592d9d62 100644 --- a/src/adapter/outbound/persistence-jpa/src/testFixtures/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/SingleSidedAssociationCondition.java +++ b/src/adapter/outbound/persistence-jpa/src/testFixtures/java/dev/caskeleton/adapter/outbound/persistence/testkit/arch/SingleSidedAssociationCondition.java @@ -36,6 +36,13 @@ import java.util.stream.Stream; * pass. * * + *

Two to-one links the other way round — {@code Order.@OneToOne latestInvoice} and {@code + * Invoice.@ManyToOne order}, each with its own foreign key — are not one link mapped twice, and the + * report says so. They are refused all the same, on purpose: the two tables reference each other, + * so neither row can be inserted or deleted first, one of the keys has to be nullable and written + * by a second UPDATE, and the entity graph is a cycle. One side holds the association and the other + * holds an ID or asks a query (re-review round 2, RR-02-15). + * *

A unidirectional {@code @OneToMany} with its own {@code @JoinColumn}, or a {@code @ManyToOne} * nobody points back at, has nothing to keep in step and passes. A self-reference is a pair only * when a second member of the same class points back the other way ({@code @ManyToOne parent} and @@ -108,13 +115,28 @@ public final class SingleSidedAssociationCondition extends ArchCondition{@code ALTER TABLE} takes the strongest lock there is, and a plain {@code CREATE INDEX} one + * that stops every write. Either waits behind any transaction that has so much as read the table, + * and while it waits, every read and write after it queues behind it: one forgotten {@code BEGIN} + * in an operator's session stops the outbox, the relay and every request that writes it for as long + * as that session stays open. {@code SET LOCAL lock_timeout} before the first such statement turns + * that into a migration that fails after the bound, rolls back whole and leaves the database on the + * version before it — a failed deploy, not an outage. {@code LOCAL}, because a plain {@code SET} + * outlives the migration on the pooled connection Flyway borrowed. + * + *

What counts as waiting for a lock: {@code ALTER TABLE}, {@code CREATE [UNIQUE] INDEX} and + * {@code DROP INDEX} without {@code CONCURRENTLY}, {@code LOCK TABLE}, {@code TRUNCATE}, {@code + * DROP TABLE}, and {@code UPDATE}/{@code DELETE} (they wait on the rows other transactions hold). A + * statement on a table this same migration created earlier waits for nothing and is left out. + * Comments and quoted literals are not read as statements. The check reads text, not a parse tree: + * it is a guard against the forgotten line, not a proof. + */ +public final class LockWaitMigrationInspector { + + private static final String NAME = + "((?:\"[^\"]+\"|[a-z_][a-z0-9_$]*)(?:\\.(?:\"[^\"]+\"|[a-z_][a-z0-9_$]*))?)"; + + private static final Pattern CREATED_TABLE = + Pattern.compile( + "\\bcreate\\s+(?:(?:temp|temporary|unlogged)\\s+)?table\\s+(?:if\\s+not\\s+exists\\s+)?" + + NAME); + + /** Each pattern's group 1, when it has one, is the table the statement locks. */ + private static final List LOCKING = + List.of( + Pattern.compile("\\balter\\s+table\\s+(?:if\\s+exists\\s+)?(?:only\\s+)?" + NAME), + Pattern.compile( + "\\bcreate\\s+(?:unique\\s+)?index\\s+(?!concurrently\\b)(?:if\\s+not\\s+exists\\s+)?" + + "(?:[a-z0-9_$\"]+\\s+)??on\\s+(?:only\\s+)?" + + NAME), + Pattern.compile("\\bdrop\\s+index\\s+(?!concurrently\\b)"), + Pattern.compile("\\block\\s+(?:table\\s+)?(?:only\\s+)?" + NAME), + Pattern.compile("\\btruncate\\s+(?:table\\s+)?(?:only\\s+)?" + NAME), + Pattern.compile("\\bdrop\\s+table\\s+(?:if\\s+exists\\s+)?" + NAME), + Pattern.compile("\\bdelete\\s+from\\s+(?:only\\s+)?" + NAME), + Pattern.compile( + "\\bupdate\\s+(?:only\\s+)?" + NAME + "(?:\\s+(?:as\\s+)?[a-z_]\\w*)?\\s+set\\b")); + + private static final Pattern BOUND = Pattern.compile("\\bset\\s+local\\s+lock_timeout\\b"); + + /** + * Fails when the migration locks a table it did not create before it bounds the wait. + * + * @throws IllegalStateException naming the migration and every statement that waits unbounded + */ + public void requireBoundedLockWait(MigrationResource migration) { + Objects.requireNonNull(migration, "migration"); + List unbounded = unboundedLockingStatements(migration.sql()); + if (unbounded.isEmpty()) { + return; + } + throw new IllegalStateException( + migration.name() + + " waits without a bound for a lock on a table already in use (" + + String.join(", ", unbounded) + + "). Put SET LOCAL lock_timeout = '5s'; before the first of them: while it waits, every" + + " read and write of the table queues behind it. LOCAL, so the setting ends with the" + + " migration's transaction (docs/jpa/migration-guide.md, \"Bound the lock wait\")"); + } + + /** + * The statements that wait for a lock on an existing table before any {@code SET LOCAL + * lock_timeout}, each as its leading words and table; empty when there are none. + */ + public List unboundedLockingStatements(String sql) { + if (sql == null) { + return List.of(); + } + String text = withoutCommentsAndLiterals(sql).toLowerCase(Locale.ROOT).replaceAll("\\s+", " "); + Map createdAt = new HashMap<>(); + Matcher created = CREATED_TABLE.matcher(text); + while (created.find()) { + createdAt.putIfAbsent(table(created.group(1)), created.start()); + } + Matcher bound = BOUND.matcher(text); + int boundAt = bound.find() ? bound.start() : Integer.MAX_VALUE; + List found = new ArrayList<>(); + for (Pattern pattern : LOCKING) { + Matcher statement = pattern.matcher(text); + while (statement.find()) { + if (statement.start() < boundAt && !createdEarlier(statement, createdAt)) { + found.add(new Located(statement.start(), statement.group().trim())); + } + } + } + return found.stream() + .sorted((left, right) -> Integer.compare(left.at(), right.at())) + .map(Located::statement) + .toList(); + } + + private static boolean createdEarlier(Matcher statement, Map createdAt) { + if (statement.groupCount() < 1 || statement.group(1) == null) { + return false; + } + Integer at = createdAt.get(table(statement.group(1))); + return at != null && at < statement.start(); + } + + /** The unqualified, unquoted table name: {@code public.ledger} and {@code ledger} are one. */ + private static String table(String name) { + String last = name.substring(name.lastIndexOf('.') + 1); + return last.replace("\"", ""); + } + + /** + * Blanks out comments and single-quoted literals, keeping every other character in place, so a + * word in a {@code COMMENT ON ... IS '...'} or a {@code -- } line is not read as a statement. + */ + private static String withoutCommentsAndLiterals(String sql) { + StringBuilder kept = new StringBuilder(sql.length()); + int i = 0; + while (i < sql.length()) { + char c = sql.charAt(i); + if (c == '-' && i + 1 < sql.length() && sql.charAt(i + 1) == '-') { + int end = sql.indexOf('\n', i); + i = end < 0 ? sql.length() : end; + kept.append(' '); + } else if (c == '/' && i + 1 < sql.length() && sql.charAt(i + 1) == '*') { + int end = sql.indexOf("*/", i + 2); + i = end < 0 ? sql.length() : end + 2; + kept.append(' '); + } else if (c == '\'') { + i++; + while (i < sql.length()) { + if (sql.charAt(i) == '\'' && i + 1 < sql.length() && sql.charAt(i + 1) == '\'') { + i += 2; + } else if (sql.charAt(i) == '\'') { + i++; + break; + } else { + i++; + } + } + kept.append("''"); + } else { + kept.append(c); + i++; + } + } + return kept.toString(); + } + + private record Located(int at, String statement) {} +} diff --git a/src/adapter/outbound/persistence-notification/CLAUDE.md b/src/adapter/outbound/persistence-notification/CLAUDE.md index 9d736619..0939743d 100644 --- a/src/adapter/outbound/persistence-notification/CLAUDE.md +++ b/src/adapter/outbound/persistence-notification/CLAUDE.md @@ -42,7 +42,10 @@ without a contract (the serving-state read model, consent, inbox), and why, are their own Flyway stream (`NotificationSchemaStream`, history table `flyway_jpa_notification_history`). An applied file is never edited: add the next `V__`, pin its SHA-256 in `applied-migrations.sha256`, and add it to `NotificationFixtures.migrations()`. - A migration that deletes rows states in a SQL comment which row survives and why. + A migration that deletes rows states in a SQL comment which row survives and why. A migration + that locks an existing table starts with `SET LOCAL lock_timeout = '5s';` + (`docs/jpa/migration-guide.md` "Bound the lock wait"; `MigrationLockWaitTest` in persistence-jpa + refuses one without it). - **Insert-or-update is decided by the database.** Never find-then-insert: insert with `ON CONFLICT () DO NOTHING`, and when it inserted nothing, update the existing row through the managed entity (`JpaSuppressionStore.upsert`, `JpaPolicyStores.Preferences.save`, diff --git a/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V11__notification_reconciliation_claim_lease.sql b/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V11__notification_reconciliation_claim_lease.sql index 2b1dc5dd..98319fd6 100644 --- a/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V11__notification_reconciliation_claim_lease.sql +++ b/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V11__notification_reconciliation_claim_lease.sql @@ -10,6 +10,19 @@ -- has passed, and every write after the claim names the fence it was claimed with, so a holder that -- outlived its lease cannot reschedule or delete a job somebody else now holds. The provider call -- stays outside any transaction, which is the reason for a lease rather than a long-held row lock. +-- +-- This file was edited after it was first written, before any deployment applied it (the branch +-- that added it was not merged), to add the bound below. applied-migrations.sha256 carries the +-- new checksum; a development database that applied the earlier text fails Flyway validation: +-- recreate it. + +-- Bound the wait for the lock this migration takes (ACCESS EXCLUSIVE on +-- notification_reconciliation_job, held until it commits). Unbounded, it queues behind any open +-- transaction that has touched the table, and every read and write of the table after it queues +-- behind the migration. Five seconds, then the migration fails, rolls back whole and the database +-- stays at V10: deploy again once that transaction has ended (re-review 2026-11-03 round 2, +-- RR-02-09; the rule is docs/jpa/migration-guide.md "Bound the lock wait"). +SET LOCAL lock_timeout = '5s'; ALTER TABLE notification_reconciliation_job ADD COLUMN IF NOT EXISTS claimed_until timestamptz, diff --git a/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V12__notification_suppression_target_unique.sql b/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V12__notification_suppression_target_unique.sql index a343067f..f595e3b0 100644 --- a/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V12__notification_suppression_target_unique.sql +++ b/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V12__notification_suppression_target_unique.sql @@ -14,6 +14,19 @@ -- 4. then the earlier created_at, then the smaller id, so the choice is deterministic. -- The other rows are deleted. A SuppressionId an operator saw for a deleted duplicate no longer -- resolves; the surviving row carries the same target and reason. +-- +-- This file was edited after it was first written, before any deployment applied it (the branch +-- that added it was not merged), to add the bound below. applied-migrations.sha256 carries the +-- new checksum; a development database that applied the earlier text fails Flyway validation: +-- recreate it. + +-- Bound the wait for the lock this migration takes (row locks for the DELETE, then ACCESS EXCLUSIVE +-- on notification_suppression, held until it commits). Unbounded, it queues behind any open +-- transaction that has touched the table, and every read and write of the table after it queues +-- behind the migration — every send checks it. Five seconds, then the migration fails, rolls back +-- whole and the database stays at V11: deploy again once that transaction has ended (re-review +-- 2026-11-03 round 2, RR-02-09; the rule is docs/jpa/migration-guide.md "Bound the lock wait"). +SET LOCAL lock_timeout = '5s'; DELETE FROM notification_suppression AS s USING ( diff --git a/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V13__notification_request_acceptance.sql b/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V13__notification_request_acceptance.sql index 9c251db8..2bcdaa7f 100644 --- a/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V13__notification_request_acceptance.sql +++ b/src/adapter/outbound/persistence-notification/src/main/resources/db/migration/jpa/notification-platform/V13__notification_request_acceptance.sql @@ -7,6 +7,19 @@ -- column records why, so the stored receipt says "dropped" rather than "accepted". -- -- Existing rows were all accepted submissions; the default records exactly that. +-- +-- This file was edited after it was first written, before any deployment applied it (the branch +-- that added it was not merged), to add the bound below. applied-migrations.sha256 carries the +-- new checksum; a development database that applied the earlier text fails Flyway validation: +-- recreate it. + +-- Bound the wait for the lock this migration takes (ACCESS EXCLUSIVE on notification_request, held +-- until it commits). Unbounded, it queues behind any open transaction that has touched the table, +-- and every read and write of the table after it queues behind the migration — every notification +-- submission writes it. Five seconds, then the migration fails, rolls back whole and the database +-- stays at V12: deploy again once that transaction has ended (re-review 2026-11-03 round 2, +-- RR-02-09; the rule is docs/jpa/migration-guide.md "Bound the lock wait"). +SET LOCAL lock_timeout = '5s'; ALTER TABLE notification_request ADD COLUMN IF NOT EXISTS acceptance varchar(40) NOT NULL DEFAULT 'ACCEPTED'; diff --git a/src/adapter/outbound/redis-runtime/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/connection/RedisRuntimeOwner.java b/src/adapter/outbound/redis-runtime/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/connection/RedisRuntimeOwner.java index 5ad8134f..9487f162 100644 --- a/src/adapter/outbound/redis-runtime/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/connection/RedisRuntimeOwner.java +++ b/src/adapter/outbound/redis-runtime/src/main/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/connection/RedisRuntimeOwner.java @@ -14,6 +14,7 @@ import java.util.Optional; import java.util.Set; import java.util.concurrent.Semaphore; import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicReference; import java.util.function.Consumer; import org.slf4j.Logger; @@ -58,6 +59,7 @@ public final class RedisRuntimeOwner implements AutoCloseable { private final Duration acquireTimeout; private final Duration drainTimeout; private final RedisLaneListener listener; + private final AtomicBoolean listenerFailureLogged = new AtomicBoolean(); private final AtomicReference state = new AtomicReference<>(State.OPEN); private final Object monitor = new Object(); @@ -345,13 +347,25 @@ public final class RedisRuntimeOwner implements AutoCloseable { /** * Tells the listener, outside the lock, and never lets it fail the borrow or the return. * + *

The first failure is logged at WARN, the rest at DEBUG. At DEBUG alone a listener that + * always fails — a meter registry that refuses the lane meters — emptied every lane meter with no + * trace in an operator's log; at WARN every time it would log on every borrow (re-review + * 2026-11-04 RR-03-17). + * * @param event what to tell it */ private void report(Consumer event) { try { event.accept(listener); } catch (RuntimeException failure) { - LOG.debug("a Redis lane listener failed; the borrow is unaffected", failure); + if (listenerFailureLogged.compareAndSet(false, true)) { + LOG.warn( + "a Redis lane listener failed; the borrow is unaffected, the lane meters may be" + + " missing, and further listener failures are logged at DEBUG", + failure); + } else { + LOG.debug("a Redis lane listener failed; the borrow is unaffected", failure); + } } } diff --git a/src/adapter/outbound/redis-runtime/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/config/RedisLaneMetricsTest.java b/src/adapter/outbound/redis-runtime/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/config/RedisLaneMetricsTest.java index 37f4bb85..02d10791 100644 --- a/src/adapter/outbound/redis-runtime/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/config/RedisLaneMetricsTest.java +++ b/src/adapter/outbound/redis-runtime/src/test/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/config/RedisLaneMetricsTest.java @@ -3,6 +3,10 @@ package dev.caskeleton.adapter.outbound.cache.redis.runtime.config; import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.catchThrowable; +import ch.qos.logback.classic.Level; +import ch.qos.logback.classic.Logger; +import ch.qos.logback.classic.spi.ILoggingEvent; +import ch.qos.logback.core.read.ListAppender; import dev.caskeleton.adapter.outbound.cache.redis.runtime.RedisDeploymentMode; import dev.caskeleton.adapter.outbound.cache.redis.runtime.command.SemanticRedisCommands; import dev.caskeleton.adapter.outbound.cache.redis.runtime.connection.RedisAccount; @@ -27,6 +31,7 @@ import java.util.Optional; import java.util.concurrent.TimeUnit; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; +import org.slf4j.LoggerFactory; /** * The lanes' own meters: how long a borrow waited, why one was refused, and how often a connection @@ -312,6 +317,39 @@ class RedisLaneMetricsTest { } } + @Test + @DisplayName("지표 기록이 처음 실패하면 WARN 한 줄을 남기고 그 뒤 실패는 WARN으로 되풀이하지 않는다") + void aFailingListenerIsReportedAtWarnOnlyTheFirstTime() { + // given + RedisLaneListener failing = + new RedisLaneListener() { + @Override + public void waited(RedisLane lane, Duration waited, boolean admitted) { + throw new IllegalStateException("meter registry rejected the sample"); + } + }; + Logger ownerLog = (Logger) LoggerFactory.getLogger(RedisRuntimeOwner.class); + ListAppender captured = new ListAppender<>(); + captured.start(); + ownerLog.addAppender(captured); + try (RedisRuntimeOwner owner = + new RedisRuntimeOwner( + client, laneLimits(1), Duration.ofMillis(50), Duration.ofSeconds(1), failing)) { + + // when + for (int borrow = 0; borrow < 3; borrow++) { + owner.borrow(ContributedLane.EXCLUSIVE).close(); + } + } finally { + ownerLog.detachAppender(captured); + } + + // then + // Silent at DEBUG, a registry conflict that empties every lane meter left no trace in an + // operator's log; at WARN on every borrow, it would flood it (re-review 2026-11-04 RR-03-17). + assertThat(captured.list).filteredOn(event -> event.getLevel() == Level.WARN).hasSize(1); + } + @Test @DisplayName("listener 없이 만든 owner는 아무 지표도 내지 않는다") void anOwnerWithoutAListenerRecordsNothing() { diff --git a/src/adapter/outbound/redis-runtime/src/testFixtures/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/command/InMemorySemanticRedisCommands.java b/src/adapter/outbound/redis-runtime/src/testFixtures/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/command/InMemorySemanticRedisCommands.java index 13d8fbb3..29e2a5d1 100644 --- a/src/adapter/outbound/redis-runtime/src/testFixtures/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/command/InMemorySemanticRedisCommands.java +++ b/src/adapter/outbound/redis-runtime/src/testFixtures/java/dev/caskeleton/adapter/outbound/cache/redis/runtime/command/InMemorySemanticRedisCommands.java @@ -223,7 +223,15 @@ public final class InMemorySemanticRedisCommands implements SemanticRedisCommand : reply("OWNER_OPERATION_CONFLICT", attempt, rev, owner, "", ""); } long leaseUntil = Long.parseLong(field(rendered, "leaseUntil")); - if ("FAILED_RETRYABLE".equals(state) || leaseUntil <= nowMillis) { + boolean leaseOver = leaseUntil <= nowMillis; + // As the program: an expired EXECUTING record has an unknown effect, so it is abandoned and + // answered RECOVERY_REQUIRED, never taken over (re-review 2026-11-04 RR-01-11). + if ("EXECUTING".equals(state) && leaseOver) { + setField(rendered, "state", "ABANDONED"); + setField(rendered, "rev", Long.toString(rev + 1)); + return reply("RECOVERY_REQUIRED", attempt, rev + 1, owner, "", ""); + } + if ("FAILED_RETRYABLE".equals(state) || ("CLAIMED".equals(state) && leaseOver)) { setField(rendered, "state", "CLAIMED"); setField(rendered, "owner", ownerToken); setField(rendered, "attempt", Long.toString(attempt + 1)); diff --git a/src/adapter/outbound/support/src/main/java/dev/caskeleton/adapter/outbound/support/startup/StartupRule.java b/src/adapter/outbound/support/src/main/java/dev/caskeleton/adapter/outbound/support/startup/StartupRule.java index 668ef1bd..9ba0fa1e 100644 --- a/src/adapter/outbound/support/src/main/java/dev/caskeleton/adapter/outbound/support/startup/StartupRule.java +++ b/src/adapter/outbound/support/src/main/java/dev/caskeleton/adapter/outbound/support/startup/StartupRule.java @@ -25,7 +25,11 @@ import org.springframework.core.env.Environment; *

The runner enforces this: when creating or running a rule brings any other singleton into * existence — a configuration class behind a non-static {@code @Bean} method, a settings bean the * method takes — the boot fails with that bean's name, because a bean created before the - * post-processors would keep unbound settings for the whole run (RR-06-01). + * post-processors would keep unbound settings for the whole run (RR-06-01). The rule itself is held + * to the same reason: a rule class with an {@code @Value} or {@code @Autowired} member, or one + * bound with {@code @ConfigurationProperties}, fails the boot by name in every profile, because + * that member would stay {@code null} (RR2-04-03). Take what the rule needs from the environment + * {@link #violations(Environment)} is given. */ public interface StartupRule { diff --git a/src/adapter/outbound/support/src/main/java/dev/caskeleton/adapter/outbound/support/startup/StartupRuleRunner.java b/src/adapter/outbound/support/src/main/java/dev/caskeleton/adapter/outbound/support/startup/StartupRuleRunner.java index 51ebba76..b6fe6e2c 100644 --- a/src/adapter/outbound/support/src/main/java/dev/caskeleton/adapter/outbound/support/startup/StartupRuleRunner.java +++ b/src/adapter/outbound/support/src/main/java/dev/caskeleton/adapter/outbound/support/startup/StartupRuleRunner.java @@ -1,16 +1,22 @@ package dev.caskeleton.adapter.outbound.support.startup; +import java.lang.reflect.AnnotatedElement; import java.util.ArrayList; import java.util.IdentityHashMap; import java.util.LinkedHashSet; import java.util.List; import java.util.Objects; +import java.util.Optional; import java.util.Set; import org.springframework.beans.factory.config.BeanFactoryPostProcessor; import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; +import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.core.Ordered; import org.springframework.core.annotation.AnnotationAwareOrderComparator; +import org.springframework.core.annotation.MergedAnnotations; import org.springframework.core.env.Environment; +import org.springframework.util.ClassUtils; +import org.springframework.util.ReflectionUtils; /** * Runs every {@link StartupRule} the context holds, once, after the bean definitions are known and @@ -44,12 +50,44 @@ import org.springframework.core.env.Environment; * whole run, and so does every other consumer of that singleton (RR-06-01). The {@code static} * declaration used to be a sentence in Javadoc; now the runner checks it. Any singleton that comes * into existence while a rule is created or run fails the boot with its name — unless it is itself - * a {@link StartupRule} or a {@link RequiredProductionSecrets}, the two kinds that are declared - * {@code static} and argument-free for exactly this reason (a non-static one creates its - * configuration class, which is reported). + * a {@link StartupRule} or a {@link RequiredProductionSecrets}, which are meant to be created here. + * + *

What a rule or a secret list may not need

+ * + *

Those two kinds are created early by design, so they must need nothing a post-processor + * provides: no {@code @Value}, {@code @Autowired}, {@code @Inject} or {@code @Resource} member, no + * {@code @PostConstruct}/{@code @PreDestroy} callback, and no {@code @ConfigurationProperties} + * binding on the class or its {@code @Bean} method. Each of those stays unset or uncalled for the + * whole run, silently: a rule then reads {@code null} and its check is off, and a settings class + * that doubles as a secret list keeps its defaults in {@code prod}, where the secret guard reads it + * early, and only there (review 2026-11-04 RR2-04-03). The runner refuses such a bean by name + * before any rule runs, in every profile, and again by its runtime class when it is created. + * + *

Checked by what the bean needs rather than by how it is declared. Requiring a {@code static} + * factory method was the other way, and it checks the wrong thing: it refuses a rule registered + * with an instance supplier, which needs nothing, and passes a {@code static @Bean} that returns a + * class with an {@code @Value} field, which is the same silent {@code null}. */ public final class StartupRuleRunner implements BeanFactoryPostProcessor, Ordered { + /** The kinds the runner creates before any bean post-processor exists. */ + private static final List> EARLY_KINDS = + List.of(StartupRule.class, RequiredProductionSecrets.class); + + /** + * What the post-processors that do not exist yet act on: member injection ({@code + * AutowiredAnnotationBeanPostProcessor}, {@code CommonAnnotationBeanPostProcessor}) and lifecycle + * callbacks. Named, not imported, so the check needs no Jakarta API on the classpath. + */ + private static final List POST_PROCESSED_MEMBER_ANNOTATIONS = + List.of( + "org.springframework.beans.factory.annotation.Value", + "org.springframework.beans.factory.annotation.Autowired", + "jakarta.inject.Inject", + "jakarta.annotation.Resource", + "jakarta.annotation.PostConstruct", + "jakarta.annotation.PreDestroy"); + private final Environment environment; /** @@ -63,6 +101,14 @@ public final class StartupRuleRunner implements BeanFactoryPostProcessor, Ordere @Override public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) { + // Before anything is created, and whether or not this profile reads them: the secret guard + // creates every RequiredProductionSecrets bean under prod only, so a bean that cannot be + // created early must fail where it is written, not on the first production boot. + for (Class earlyKind : EARLY_KINDS) { + for (String name : beanFactory.getBeanNamesForType(earlyKind, true, false)) { + refuseUnprocessedMembers(beanFactory, name, beanFactory.getType(name, false)); + } + } IdentityHashMap names = new IdentityHashMap<>(); List rules = new ArrayList<>(); for (String name : beanFactory.getBeanNamesForType(StartupRule.class, true, false)) { @@ -95,7 +141,9 @@ public final class StartupRuleRunner implements BeanFactoryPostProcessor, Ordere /** * Fails when creating or running the rule {@code ruleName} brought a singleton into existence * other than a rule or a {@link RequiredProductionSecrets}: that bean was built before the bean - * post-processors, so its settings are unbound and it is not proxied, for the rest of the run. + * post-processors, so its settings are unbound and it is not proxied, for the rest of the run. A + * rule or secret list it created is held to {@link #refuseUnprocessedMembers} by its runtime + * class, which a {@code @Bean} method's declared return type may hide. */ private static void refusePulledForward( ConfigurableListableBeanFactory beanFactory, @@ -104,9 +152,15 @@ public final class StartupRuleRunner implements BeanFactoryPostProcessor, Ordere Set before) { Set pulled = new LinkedHashSet<>(); for (String name : beanFactory.getSingletonNames()) { - if (!before.contains(name) - && !beanFactory.isTypeMatch(name, StartupRule.class) - && !beanFactory.isTypeMatch(name, RequiredProductionSecrets.class)) { + if (before.contains(name)) { + continue; + } + if (EARLY_KINDS.stream().anyMatch(kind -> beanFactory.isTypeMatch(name, kind))) { + Object singleton = beanFactory.getSingleton(name); + if (singleton != null) { + refuseUnprocessedMembers(beanFactory, name, ClassUtils.getUserClass(singleton)); + } + } else { pulled.add(name); } } @@ -130,6 +184,49 @@ public final class StartupRuleRunner implements BeanFactoryPostProcessor, Ordere + " bean is not a rule: keep it a bean, ordered before what it protects."); } + /** + * Fails when the rule or secret list {@code name} needs a bean post-processor: it is created + * before any exists, so what one would inject, bind or call never happens, for the whole run. + */ + private static void refuseUnprocessedMembers( + ConfigurableListableBeanFactory beanFactory, String name, Class type) { + List needs = new ArrayList<>(); + if (beanFactory.findAnnotationOnBean(name, ConfigurationProperties.class, false) != null) { + needs.add("@ConfigurationProperties binding"); + } + if (type != null) { + ReflectionUtils.doWithFields( + type, field -> postProcessedBy(field).ifPresent(by -> needs.add(field.getName() + by))); + ReflectionUtils.doWithMethods( + type, + method -> + postProcessedBy(method).ifPresent(by -> needs.add(method.getName() + "()" + by))); + } + if (needs.isEmpty()) { + return; + } + throw StartupFailures.envValidation( + "'" + + name + + "' (" + + (type == null ? "unknown type" : type.getName()) + + ") is a StartupRule or RequiredProductionSecrets, which are created before any bean" + + " post-processor exists, but it needs one: " + + needs + + ". None of that happens for the whole run, so the rule reads null or the settings" + + " keep their defaults, silently. Read the value from the Environment the rule or" + + " the secret list is given, or pass it to the constructor in a static @Bean method;" + + " a settings class is not a secret list."); + } + + private static Optional postProcessedBy(AnnotatedElement member) { + MergedAnnotations annotations = MergedAnnotations.from(member); + return POST_PROCESSED_MEMBER_ANNOTATIONS.stream() + .filter(annotations::isPresent) + .findFirst() + .map(annotation -> " (@" + annotation.substring(annotation.lastIndexOf('.') + 1) + ")"); + } + /** * Runs one rule and raises its violations as its phase's typed failure. * diff --git a/src/adapter/outbound/support/src/test/java/dev/caskeleton/adapter/outbound/support/startup/StartupRuleRunnerTest.java b/src/adapter/outbound/support/src/test/java/dev/caskeleton/adapter/outbound/support/startup/StartupRuleRunnerTest.java index 398c108d..6e51a550 100644 --- a/src/adapter/outbound/support/src/test/java/dev/caskeleton/adapter/outbound/support/startup/StartupRuleRunnerTest.java +++ b/src/adapter/outbound/support/src/test/java/dev/caskeleton/adapter/outbound/support/startup/StartupRuleRunnerTest.java @@ -8,12 +8,14 @@ import java.util.concurrent.atomic.AtomicInteger; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; import org.springframework.beans.factory.ObjectProvider; +import org.springframework.beans.factory.annotation.Value; import org.springframework.boot.autoconfigure.AutoConfigurations; import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.boot.context.properties.EnableConfigurationProperties; import org.springframework.boot.test.context.runner.ApplicationContextRunner; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; import org.springframework.core.env.Environment; class StartupRuleRunnerTest { @@ -163,6 +165,81 @@ class StartupRuleRunnerTest { }); } + @Test + @DisplayName("비밀 목록을 겸하는 JavaBean 설정 클래스는 일찍 만들어지면 바인딩되지 않으므로 그 bean 이름을 대며 기동을 실패시킨다") + void aSettingsClassThatIsAlsoAProductionSecretsListFailsTheBoot() { + // given + BindingSecretsSettings.SEEN.clear(); + + // when + runner + .withPropertyValues("probe2.name=bound") + .withUserConfiguration(SecretReadingRuleConfig.class, BindingSecretsConfig.class) + .run( + context -> { + // then + assertThat(BindingSecretsSettings.SEEN).doesNotContainNull(); + assertThat(context.getStartupFailure()) + .isInstanceOf(StartupFailureException.class) + .hasMessageContaining("probe2-") + .hasMessageContaining("@ConfigurationProperties"); + }); + } + + @Test + @DisplayName("비밀 목록을 겸하는 JavaBean 설정 클래스는 그것을 읽는 규칙이 없어도 기동을 실패시킨다") + void aSettingsClassThatIsAlsoAProductionSecretsListFailsTheBootInEveryProfile() { + // when + runner + .withPropertyValues("probe2.name=bound") + .withUserConfiguration(BindingSecretsConfig.class) + .run( + context -> { + // then + assertThat(context.getStartupFailure()) + .isInstanceOf(StartupFailureException.class) + .hasMessageContaining("probe2-"); + }); + } + + @Test + @DisplayName("@Value 필드를 가진 규칙 클래스는 필드가 주입되지 않으므로 그 필드 이름을 대며 기동을 실패시킨다") + void aRuleClassWithAValueFieldFailsTheBootNamingTheField() { + // given + FieldInjectedRule.SEEN.clear(); + + // when + runner + .withPropertyValues("probe3.mode=strict") + .withUserConfiguration(FieldInjectedRuleImport.class) + .run( + context -> { + // then + assertThat(FieldInjectedRule.SEEN).doesNotContainNull(); + assertThat(context.getStartupFailure()) + .isInstanceOf(StartupFailureException.class) + .hasMessageContaining("FieldInjectedRule") + .hasMessageContaining("mode"); + }); + } + + @Test + @DisplayName("static @Bean이 돌려준 규칙이라도 @Value 필드가 있으면 기동을 실패시킨다") + void aStaticBeanRuleWhoseClassHasAValueFieldFailsTheBoot() { + // when + runner + .withPropertyValues("probe3.mode=strict") + .withUserConfiguration(StaticFieldInjectedRuleConfig.class) + .run( + context -> { + // then + assertThat(context.getStartupFailure()) + .isInstanceOf(StartupFailureException.class) + .hasMessageContaining("fieldInjectedRule") + .hasMessageContaining("mode"); + }); + } + private static StartupRule rule(StartupPhase phase, String summary, List violations) { return new StartupRule() { @Override @@ -257,6 +334,67 @@ class StartupRuleRunnerTest { } } + /** RR2-04-03 2-a: a JavaBean settings class that is also the module's secret list. */ + @ConfigurationProperties("probe2") + public static class BindingSecretsSettings implements RequiredProductionSecrets { + static final List SEEN = new ArrayList<>(); + private String name; + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + @Override + public List requiredSecrets(Environment environment) { + SEEN.add(name); + return List.of(); + } + } + + @Configuration(proxyBeanMethods = false) + @EnableConfigurationProperties(BindingSecretsSettings.class) + static class BindingSecretsConfig {} + + /** RR2-04-03 2-b: the most common way to write a Spring component, as a rule. */ + public static class FieldInjectedRule implements StartupRule { + static final List SEEN = new ArrayList<>(); + + @Value("${probe3.mode}") + private String mode; + + @Override + public StartupPhase phase() { + return StartupPhase.ENV_VALIDATION; + } + + @Override + public String summary() { + return "field"; + } + + @Override + public List violations(Environment environment) { + SEEN.add(mode); + return List.of(); + } + } + + @Configuration(proxyBeanMethods = false) + @Import(FieldInjectedRule.class) + static class FieldInjectedRuleImport {} + + @Configuration(proxyBeanMethods = false) + static class StaticFieldInjectedRuleConfig { + @Bean + static StartupRule fieldInjectedRule() { + return new FieldInjectedRule(); + } + } + @Configuration(proxyBeanMethods = false) static class NonStaticSecretsConfig { @Bean diff --git a/src/app-bootstrap/CLAUDE.md b/src/app-bootstrap/CLAUDE.md index 4d816bcb..6056b12f 100644 --- a/src/app-bootstrap/CLAUDE.md +++ b/src/app-bootstrap/CLAUDE.md @@ -89,7 +89,10 @@ configurations, Redis as one 376-line class. and the one `StartupRuleRunner` raises them before any other singleton exists (declare it with a `static` `@Bean` method that takes only the `Environment`, the bean factory or an `ObjectProvider` of static `RequiredProductionSecrets`; the runner fails the boot naming any other bean a rule - creates). A guard that needs another bean stays a bean. A + creates, and naming any rule or `RequiredProductionSecrets` that has an `@Value`/`@Autowired`/ + `@Inject`/`@Resource` member, a `@PostConstruct`/`@PreDestroy` callback or a + `@ConfigurationProperties` binding — those are created before the post-processors and would stay + unset). A guard that needs another bean stays a bean. A capability's production secrets are the capability's: its module contributes a `RequiredProductionSecrets` bean, and `SecretSourceValidator` checks the names without knowing which roles exist. diff --git a/src/app-bootstrap/README.md b/src/app-bootstrap/README.md index 3f6fc3ab..e4b52f5d 100644 --- a/src/app-bootstrap/README.md +++ b/src/app-bootstrap/README.md @@ -129,11 +129,14 @@ README의 "startup" 절에 있다. ### SecretSourceValidator - **prod 에서 깨지면 부팅 자체를 거부하는 fail-fast 가드다.** 두 가지 prod 불변식을 context refresh 완료 전에 강제한다. (1) `__LOCAL_DEV_` 접두사를 가진 dev/local 가짜 자격증명 값이 prod 프로파일에 - 도달하는 것은 잘못 로드된 사고이므로 부팅을 중단시킨다. `fixture-` 접두사 값도 같다. 이 저장소가 + 도달하는 것은 잘못 로드된 사고이므로 부팅을 중단시킨다. 비밀값이 `fixture-`로 시작해도 같다. 이 저장소가 lane용으로 적어 둔 공개 비밀번호(`infra/redis-sdk/acl`의 Redis ACL fixture, compose lane의 `REDIS_LANE_*`)라서, lane 환경이나 lane ACL 파일을 베낀 배포가 그대로 운영에 뜨는 것을 막는다 - (review 2026-11-03 RR-07-04). 비밀이 아닌 설정값이라도 prod에서 `fixture-`로 시작하면 거부되니 이름을 - 바꾼다. 실패 메시지는 변수 이름만 적고 값은 적지 않는다. (2) 레지스트리에서 `secret` + + (review 2026-11-03 RR-07-04). 이 검사는 비밀값만 본다 — 레지스트리의 `secret` 행 + (`CLASSIFIED_SECRETS`), 모듈이 기여한 비밀, `secret://…/` 참조가 가리키는 변수 + (review 2026-11-04 RR2-04-01). `fixture-`는 평범한 단어라서 `fixture-api` Deployment의 pod 이름 + (`HOSTNAME`)이나 앱 이름까지 거부하면 운영에서만 기동이 막혔다. `__LOCAL_DEV_` 표시는 정당한 값이 가질 + 일이 없으니 모든 값을 본다. 실패 메시지는 변수 이름만 적고 값은 적지 않는다. (2) 레지스트리에서 `secret` + `prod_default: null`로 분류된 모든 키는 prod 에서 비어 있지 않게 주입돼야 한다 — 빈 시크릿으로 부팅하는 것은 금지다. dev/local 프로파일은 두 검사를 건너뛴다(`__LOCAL_DEV_*`가 의도된 로컬 폴백이므로). diff --git a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ApplicationBoundaryArchitectureTest.java b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ApplicationBoundaryArchitectureTest.java index d64f500f..664b4f1b 100644 --- a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ApplicationBoundaryArchitectureTest.java +++ b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ApplicationBoundaryArchitectureTest.java @@ -25,6 +25,7 @@ import dev.caskeleton.application.capability.UseCaseCapability; import dev.caskeleton.application.messaging.event.IntegrationEventDraft; import dev.caskeleton.application.messaging.event.IntegrationEventDrafts; import dev.caskeleton.application.security.RequiresPermission; +import dev.caskeleton.application.security.SystemCaller; import dev.caskeleton.application.security.SystemInvoked; import dev.caskeleton.application.usecase.CommandUseCase; import dev.caskeleton.application.usecase.QueryUseCase; @@ -447,23 +448,109 @@ class ApplicationBoundaryArchitectureTest { .allowEmptyShould(true); /** - * A {@link SystemInvoked} use case is a write with no permission check, so no transport may reach - * it. A bean of one fails startup ({@code PermissionEnforcementGuard}); this closes the other - * path, an inbound adapter that constructs or calls one itself (review 2026-11-03 RR-01-02). + * The inbound packages the system drives rather than a user: a broker delivery, a schedule. Only + * a {@link SystemCaller} may live here and reach a {@link SystemInvoked} use case; every other + * inbound package — web, GraphQL, gRPC, WebSocket, and any transport added later — is a way a + * user request comes in. A new system-driven inbound family joins this list, nowhere else. + */ + private static final String[] SYSTEM_DRIVEN_INBOUND_PACKAGES = { + "..adapter.inbound.messaging..", "..adapter.inbound.scheduler.." + }; + + /** + * What makes a class a way in for a user request: a user-transport adapter package or the + * framework types an endpoint is built from. {@code org.springframework.messaging} is named only + * for STOMP ({@code simp}, {@code @MessageMapping}): a broker listener's {@code @Payload} and + * {@code @Header} live beside them and are a system consumer's, not a user's. + */ + private static final DescribedPredicate ARE_USER_TRANSPORT_TYPES = + JavaClass.Predicates.resideInAnyPackage( + "..adapter.inbound.web..", + "..adapter.inbound.graphql..", + "..adapter.inbound.grpc..", + "..adapter.inbound.websocket..", + "org.springframework.web..", + "org.springframework.graphql..", + "org.springframework.messaging.simp..", + "io.grpc..", + "jakarta.servlet..", + "jakarta.websocket..") + .or( + com.tngtech.archunit.core.domain.properties.HasName.Predicates.name( + "org.springframework.stereotype.Controller")) + .or( + com.tngtech.archunit.core.domain.properties.HasName.Predicates.name( + "org.springframework.messaging.handler.annotation.MessageMapping")) + .as("user-transport types (web, GraphQL, gRPC, WebSocket/STOMP endpoints)"); + + /** + * A class marked {@link SystemCaller}, or nested in one: a handler's anonymous or inner helper + * calls the use case on the handler's behalf. + */ + private static final DescribedPredicate ARE_SYSTEM_CALLERS = + new DescribedPredicate<>("are @SystemCaller classes or nested in one") { + @Override + public boolean test(JavaClass candidate) { + for (JavaClass type = candidate; + type != null; + type = type.getEnclosingClass().orElse(null)) { + if (type.isAnnotatedWith(SystemCaller.class)) { + return true; + } + } + return false; + } + }; + + /** + * A {@link SystemInvoked} use case is a write with no permission check, so no user request may + * reach it. A user comes in through a transport; the system comes in through a delivery or a tick + * (decision 9.1, review 2026-11-03 RR-01-12). An inbound adapter may depend on one only when it + * says it is the second kind — {@link SystemCaller} — and {@code + * SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS} checks that claim. The rule names who may, + * not who may not, so a transport added later is refused without editing it. A bean of one fails + * startup ({@code PermissionEnforcementGuard}). */ @ArchTest - static final ArchRule INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES = + static final ArchRule ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES = noClasses() .that() .resideInAPackage("..adapter.inbound..") + .and(DescribedPredicate.not(ARE_SYSTEM_CALLERS)) .should() .dependOnClassesThat() .areAnnotatedWith(SystemInvoked.class) .as( - "INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES: a @SystemInvoked use case " + "ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES: a @SystemInvoked use case " + "declares no permission because only the system calls it; an inbound adapter " - + "that reaches it runs a write with no authorization check. Its composition " - + "constructs it and hands it to the scheduler that calls it.") + + "a user request reaches (web, GraphQL, gRPC, WebSocket) must not depend on it. " + + "A message handler or scheduled job marks itself @SystemCaller(reason = ...) " + + "and constructs the use case it calls (manual 09 「소비」 step 4); a scheduler " + + "assembled in a composition module gets it the way the outbox relay does.") + .allowEmptyShould(true); + + /** + * {@link SystemCaller} is a claim, so it is checked: the marked class lives in a system-driven + * inbound package and is no endpoint a user can call — a controller, a GraphQL or STOMP mapping, + * a gRPC service. Without this the mark would be the next way to silence the rule above (review + * 2026-11-03 RR-01-12). + */ + @ArchTest + static final ArchRule SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS = + classes() + .that() + .areAnnotatedWith(SystemCaller.class) + .should() + .resideInAnyPackage(SYSTEM_DRIVEN_INBOUND_PACKAGES) + .andShould( + com.tngtech.archunit.lang.conditions.ArchConditions.never( + com.tngtech.archunit.lang.conditions.ArchConditions.dependOnClassesThat( + ARE_USER_TRANSPORT_TYPES))) + .as( + "SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS: a @SystemCaller lives in a " + + "system-driven inbound package (..adapter.inbound.messaging.., " + + "..adapter.inbound.scheduler..) and depends on no user-transport type — the mark " + + "says no user request reaches the class, so a user transport cannot carry it") .allowEmptyShould(true); /** diff --git a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ArchRuleDiscoveryContractTest.java b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ArchRuleDiscoveryContractTest.java index b4da42bc..e959f0e7 100644 --- a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ArchRuleDiscoveryContractTest.java +++ b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ArchRuleDiscoveryContractTest.java @@ -114,7 +114,6 @@ class ArchRuleDiscoveryContractTest { "IDENTIFIER_ADAPTER_DOES_NOT_DEPEND_ON_OTHER_ADAPTERS_OR_BOOTSTRAP", "INBOUND_ADAPTERS_DEPEND_ON_PORTS_NOT_APPLICATION_SERVICES", "INBOUND_ADAPTERS_DO_NOT_DEPEND_ON_OUTBOUND_ADAPTERS", - "INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES", "INBOUND_PORT_IMPLEMENTATIONS_DECLARE_A_CAPABILITY", "INBOUND_PORT_IMPLEMENTATIONS_DECLARE_CAPABILITY", "INBOUND_PORT_IMPLEMENTATIONS_END_WITH_USE_CASE", @@ -144,6 +143,7 @@ class ArchRuleDiscoveryContractTest { "NO_UUID_RANDOM_IN_CONTROLLER", "NO_VARCHAR_255_FOR_ID_COLUMN", "NO_WEBSOCKET_HANDLER", + "ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES", "ONLY_THE_PERSISTENCE_ADAPTER_HOLDS_A_MONGO_HANDLE", "ONLY_THE_SECURITY_PACKAGE_HANDLES_KEY_MATERIAL", "OPTIONAL_ADAPTER_BEANS_ARE_GATED_BY_CONDITIONAL_ON_PROPERTY", @@ -164,6 +164,7 @@ class ArchRuleDiscoveryContractTest { "REQUEST_DTOS_DO_NOT_SILENCE_UNKNOWN_FIELDS", "SHARED_CONTRACT_CONTAINS_ONLY_OPERATIONAL_CONTRACT_PACKAGES", "STARTUP_FAILURES_ARE_RAISED_THROUGH_THE_TYPED_FACTORY", + "SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS", "SYSTEM_INVOKED_USE_CASES_DECLARE_NO_PERMISSION", "TEMPLATE_PACKAGES_DO_NOT_DEPEND_ON_FEATURE_PACKAGES", "THROWABLES_ARE_NAMED_EXCEPTION", diff --git a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ArchitectureViolationFixtureTest.java b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ArchitectureViolationFixtureTest.java index 2eb2d35f..cab3c152 100644 --- a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ArchitectureViolationFixtureTest.java +++ b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/ArchitectureViolationFixtureTest.java @@ -6,8 +6,12 @@ import com.tngtech.archunit.core.domain.JavaClasses; import com.tngtech.archunit.core.importer.ClassFileImporter; import com.tngtech.archunit.lang.EvaluationResult; import dev.caskeleton.application.architecture.violations.ApplicationDiagnosticFrameworkViolation; +import dev.caskeleton.bootstrap.architecture.allowed.adapter.inbound.messaging.SystemCallerHandlerFixture; import dev.caskeleton.bootstrap.architecture.allowed.application.CleanProjectionQueryPort; import dev.caskeleton.bootstrap.architecture.fixtures.application.RootWriteTransactionBoundaryUseCase; +import dev.caskeleton.bootstrap.architecture.violations.adapter.inbound.messaging.SystemCallerRestControllerFixture; +import dev.caskeleton.bootstrap.architecture.violations.adapter.inbound.messaging.UnmarkedSystemInvokedHandlerFixture; +import dev.caskeleton.bootstrap.architecture.violations.adapter.inbound.web.controller.SystemCallerControllerFixture; import dev.caskeleton.bootstrap.architecture.violations.adapter.inbound.web.controller.SystemInvokedCallingControllerFixture; import dev.caskeleton.bootstrap.architecture.violations.adapter.outbound.RawExternalResponseFixture; import dev.caskeleton.bootstrap.architecture.violations.adapter.outbound.messaging.GenericTypeLeakingAdapterFixture; @@ -158,25 +162,90 @@ class ArchitectureViolationFixtureTest { .importPackages("dev.caskeleton.bootstrap.architecture.violations.secrets"); // Review 2026-11-03 RR-01-02: the controller and the use case it reaches, and nothing else, so - // the rule is proven on the one dependency it forbids. + // the rule is proven on the one dependency it forbids. RR-01-12 narrowed the rule to user + // transports; this is the path it must still refuse. private static final JavaClasses SYSTEM_INVOKED_FROM_TRANSPORT_FIXTURE_ONLY = new ClassFileImporter() .importClasses( SystemInvokedCallingControllerFixture.class, SystemInvokedFixtureUseCase.class); + // Review 2026-11-03 RR-01-12: the consume path manual 09 teaches — a marked handler and the use + // case it constructs — and the same handler without the mark. + private static final JavaClasses SYSTEM_CALLER_HANDLER_FIXTURE_ONLY = + new ClassFileImporter() + .importClasses(SystemCallerHandlerFixture.class, SystemInvokedFixtureUseCase.class); + + private static final JavaClasses UNMARKED_HANDLER_FIXTURE_ONLY = + new ClassFileImporter() + .importClasses( + UnmarkedSystemInvokedHandlerFixture.class, SystemInvokedFixtureUseCase.class); + + // A user transport that carries the mark: by its package, and by being an HTTP endpoint inside a + // system-driven package. + private static final JavaClasses SYSTEM_CALLER_IN_A_USER_TRANSPORT_PACKAGE_FIXTURE_ONLY = + new ClassFileImporter().importClasses(SystemCallerControllerFixture.class); + + private static final JavaClasses SYSTEM_CALLER_THAT_IS_AN_HTTP_ENDPOINT_FIXTURE_ONLY = + new ClassFileImporter().importClasses(SystemCallerRestControllerFixture.class); + // Review 2026-11-03 RR-01-03: one class that builds the draft by hand. private static final JavaClasses HAND_BUILT_DRAFT_FIXTURE_ONLY = new ClassFileImporter().importClasses(HandBuiltIntegrationEventDraftFixture.class); @Test - void anInboundAdapterThatCallsASystemInvokedUseCaseIsCaught() { + void aWebAdapterThatCallsASystemInvokedUseCaseIsCaught() { EvaluationResult result = - ApplicationBoundaryArchitectureTest.INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES + ApplicationBoundaryArchitectureTest.ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES .evaluate(SYSTEM_INVOKED_FROM_TRANSPORT_FIXTURE_ONLY); assertThat(result.hasViolation()).isTrue(); } + @Test + void aMessageHandlerMarkedAsASystemCallerMayReachASystemInvokedUseCase() { + EvaluationResult result = + ApplicationBoundaryArchitectureTest.ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES + .evaluate(SYSTEM_CALLER_HANDLER_FIXTURE_ONLY); + + assertThat(result.getFailureReport().getDetails()).isEmpty(); + } + + @Test + void aMessageHandlerThatReachesASystemInvokedUseCaseWithoutTheMarkIsCaught() { + EvaluationResult result = + ApplicationBoundaryArchitectureTest.ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES + .evaluate(UNMARKED_HANDLER_FIXTURE_ONLY); + + assertThat(result.hasViolation()).isTrue(); + } + + @Test + void aMessageHandlerMarkedAsASystemCallerIsASystemDrivenInboundAdapter() { + EvaluationResult result = + ApplicationBoundaryArchitectureTest.SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS + .evaluate(SYSTEM_CALLER_HANDLER_FIXTURE_ONLY); + + assertThat(result.getFailureReport().getDetails()).isEmpty(); + } + + @Test + void aWebAdapterMarkedAsASystemCallerIsCaught() { + EvaluationResult result = + ApplicationBoundaryArchitectureTest.SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS + .evaluate(SYSTEM_CALLER_IN_A_USER_TRANSPORT_PACKAGE_FIXTURE_ONLY); + + assertThat(result.hasViolation()).isTrue(); + } + + @Test + void anHttpEndpointInAMessagingPackageMarkedAsASystemCallerIsCaught() { + EvaluationResult result = + ApplicationBoundaryArchitectureTest.SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS + .evaluate(SYSTEM_CALLER_THAT_IS_AN_HTTP_ENDPOINT_FIXTURE_ONLY); + + assertThat(result.hasViolation()).isTrue(); + } + @Test void aFeatureThatBuildsAnIntegrationEventDraftByHandIsCaught() { EvaluationResult result = diff --git a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/allowed/adapter/inbound/messaging/SystemCallerHandlerFixture.java b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/allowed/adapter/inbound/messaging/SystemCallerHandlerFixture.java new file mode 100644 index 00000000..0a5067a8 --- /dev/null +++ b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/allowed/adapter/inbound/messaging/SystemCallerHandlerFixture.java @@ -0,0 +1,18 @@ +package dev.caskeleton.bootstrap.architecture.allowed.adapter.inbound.messaging; + +import dev.caskeleton.application.security.SystemCaller; +import dev.caskeleton.bootstrap.architecture.violations.application.SystemInvokedFixtureUseCase; + +/** + * The shape manual 09 teaches for a consumer: a handler the broker consumer drives, marked as a + * system caller, that constructs the system-invoked use case it calls and keeps it to itself. + */ +@SystemCaller(reason = "fixture: the broker consumer calls it for each delivery") +public final class SystemCallerHandlerFixture { + + private final SystemInvokedFixtureUseCase useCase = new SystemInvokedFixtureUseCase(); + + public void handle() { + useCase.handle(); + } +} diff --git a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/allowed/adapter/inbound/messaging/package-info.java b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/allowed/adapter/inbound/messaging/package-info.java new file mode 100644 index 00000000..e2765663 --- /dev/null +++ b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/allowed/adapter/inbound/messaging/package-info.java @@ -0,0 +1,9 @@ +/** + * "Allowed-as-data" fixture for the system-caller rules (review 2026-11-03 RR-01-12): a message + * handler that the broker consumer drives, marked {@code @SystemCaller}, calling a + * {@code @SystemInvoked} use case it constructed itself. {@code + * ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES} and {@code + * SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS} must NOT flag it — it is the one legitimate + * way a consumer reaches a system write (manual 09 「소비」 step 4). + */ +package dev.caskeleton.bootstrap.architecture.allowed.adapter.inbound.messaging; diff --git a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/messaging/SystemCallerRestControllerFixture.java b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/messaging/SystemCallerRestControllerFixture.java new file mode 100644 index 00000000..81383c14 --- /dev/null +++ b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/messaging/SystemCallerRestControllerFixture.java @@ -0,0 +1,23 @@ +package dev.caskeleton.bootstrap.architecture.violations.adapter.inbound.messaging; + +import dev.caskeleton.application.security.SystemCaller; +import dev.caskeleton.bootstrap.architecture.violations.application.SystemInvokedFixtureUseCase; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RestController; + +/** + * Negative fixture for {@code SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS}: an HTTP endpoint + * placed in a system-driven inbound package and marked as a system caller. The package is right; + * the class is still a user transport (review 2026-11-03 RR-01-12). + */ +@SystemCaller(reason = "fixture: an HTTP endpoint hiding in the messaging package") +@RestController +public class SystemCallerRestControllerFixture { + + private final SystemInvokedFixtureUseCase useCase = new SystemInvokedFixtureUseCase(); + + @PostMapping("/fixture/system-write") + public void trigger() { + useCase.handle(); + } +} diff --git a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/messaging/UnmarkedSystemInvokedHandlerFixture.java b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/messaging/UnmarkedSystemInvokedHandlerFixture.java new file mode 100644 index 00000000..cb8c183f --- /dev/null +++ b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/messaging/UnmarkedSystemInvokedHandlerFixture.java @@ -0,0 +1,17 @@ +package dev.caskeleton.bootstrap.architecture.violations.adapter.inbound.messaging; + +import dev.caskeleton.bootstrap.architecture.violations.application.SystemInvokedFixtureUseCase; + +/** + * Negative fixture for {@code ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES}: a handler in a + * system-driven inbound package that reaches a {@code @SystemInvoked} use case without saying it is + * a system caller. The package alone is not the mark (review 2026-11-03 RR-01-12). + */ +public final class UnmarkedSystemInvokedHandlerFixture { + + private final SystemInvokedFixtureUseCase useCase = new SystemInvokedFixtureUseCase(); + + public void handle() { + useCase.handle(); + } +} diff --git a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/web/controller/SystemCallerControllerFixture.java b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/web/controller/SystemCallerControllerFixture.java new file mode 100644 index 00000000..a2ca9337 --- /dev/null +++ b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/web/controller/SystemCallerControllerFixture.java @@ -0,0 +1,19 @@ +package dev.caskeleton.bootstrap.architecture.violations.adapter.inbound.web.controller; + +import dev.caskeleton.application.security.SystemCaller; +import dev.caskeleton.bootstrap.architecture.violations.application.SystemInvokedFixtureUseCase; + +/** + * Negative fixture for {@code SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS}: a web adapter + * that claims to be a system caller to reach a {@code @SystemInvoked} use case. A user transport + * cannot carry the mark (review 2026-11-03 RR-01-12). + */ +@SystemCaller(reason = "fixture: claims a system caller from a user transport") +public class SystemCallerControllerFixture { + + private final SystemInvokedFixtureUseCase useCase = new SystemInvokedFixtureUseCase(); + + public void trigger() { + useCase.handle(); + } +} diff --git a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/web/controller/SystemInvokedCallingControllerFixture.java b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/web/controller/SystemInvokedCallingControllerFixture.java index dfe5f538..4d0d9f3d 100644 --- a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/web/controller/SystemInvokedCallingControllerFixture.java +++ b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/adapter/inbound/web/controller/SystemInvokedCallingControllerFixture.java @@ -3,9 +3,9 @@ package dev.caskeleton.bootstrap.architecture.violations.adapter.inbound.web.con import dev.caskeleton.bootstrap.architecture.violations.application.SystemInvokedFixtureUseCase; /** - * Negative fixture for {@code INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES}: an inbound - * adapter that calls a {@code @SystemInvoked} use case, a write with no permission gate, straight - * from a transport (review 2026-11-03 RR-01-02). + * Negative fixture for {@code ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES}: a user transport + * that calls a {@code @SystemInvoked} use case, a write with no permission gate (review 2026-11-03 + * RR-01-02; still refused after RR-01-12 let system callers through). */ public class SystemInvokedCallingControllerFixture { diff --git a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/application/SystemInvokedFixtureUseCase.java b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/application/SystemInvokedFixtureUseCase.java index 00e5fc4d..b01bb447 100644 --- a/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/application/SystemInvokedFixtureUseCase.java +++ b/src/app-bootstrap/src/architectureTest/java/dev/caskeleton/bootstrap/architecture/violations/application/SystemInvokedFixtureUseCase.java @@ -3,9 +3,10 @@ package dev.caskeleton.bootstrap.architecture.violations.application; import dev.caskeleton.application.security.SystemInvoked; /** - * A use case only the system calls. Not a violation on its own: it is what {@code - * SystemInvokedCallingControllerFixture} reaches, the transport path {@code - * INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES} forbids. + * A use case only the system calls. Not a violation on its own: it is what the inbound fixtures + * reach — {@code SystemInvokedCallingControllerFixture}, the user-transport path {@code + * ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES} forbids, and {@code + * SystemCallerHandlerFixture}, the consume path it allows. */ @SystemInvoked(reason = "fixture: a scheduler calls it") public final class SystemInvokedFixtureUseCase { diff --git a/src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/runtime/SecretSourceValidator.java b/src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/runtime/SecretSourceValidator.java index 7d4c1662..d781a4b5 100644 --- a/src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/runtime/SecretSourceValidator.java +++ b/src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/runtime/SecretSourceValidator.java @@ -4,14 +4,14 @@ import dev.caskeleton.adapter.outbound.support.runtime.RuntimeProfileSnapshot; import dev.caskeleton.adapter.outbound.support.startup.RequiredProductionSecrets; import dev.caskeleton.adapter.outbound.support.startup.StartupPhase; import dev.caskeleton.adapter.outbound.support.startup.StartupRule; +import dev.caskeleton.application.secret.SecretReference; import dev.caskeleton.application.secret.SecretSourcePort; import java.util.ArrayList; -import java.util.LinkedHashMap; import java.util.LinkedHashSet; import java.util.List; -import java.util.Map; import java.util.Objects; import java.util.Set; +import java.util.function.BiConsumer; import java.util.function.Supplier; import org.springframework.core.env.ConfigurableEnvironment; import org.springframework.core.env.EnumerablePropertySource; @@ -38,20 +38,47 @@ public final class SecretSourceValidator implements StartupRule { * Prefix of the passwords this repository writes down for its lanes: the Redis ACL fixtures in * {@code infra/redis-sdk/acl} and the Compose lanes' {@code REDIS_LANE_*} values that connect * with them. They are public — anyone who can read the repository has them — so under {@code - * prod} they are refused like the sentinel (review 2026-11-03 RR-07-04). A deployment that copied - * a lane's environment — which is also what an application needs to authenticate against a Redis - * that loaded the lane ACL file — would start with credentials it does not own; this is the one - * place that sees every value before anything connects. It cannot see a Redis server that loaded - * that file while the application uses real passwords; the ACL README says why no deployment - * loads it. + * prod} a secret holding one is refused (review 2026-11-03 RR-07-04). A deployment that copied a + * lane's environment — which is also what an application needs to authenticate against a Redis + * that loaded the lane ACL file — would start with credentials it does not own. It cannot see a + * Redis server that loaded that file while the application uses real passwords; the ACL README + * says why no deployment loads it. + * + *

Only secrets are read for it (review 2026-11-04 RR2-04-01, owner decision): {@code fixture-} + * is an ordinary word before it is this repository's password prefix, and a pod named after a + * {@code fixture-api} Deployment carries it in {@code HOSTNAME}. Checking every value refused + * that deployment in production only, with no way around it short of renaming the service. The + * {@code __LOCAL_DEV_} sentinel is still looked for in every value: no legitimate value carries + * an invented marker, so the wide scan costs nothing there. */ static final String REPOSITORY_FIXTURE_PREFIX = "fixture-"; - /** Values that are public by construction and so can never be a production secret. */ - private static final Map PUBLIC_CREDENTIAL_PREFIXES = - Map.of( - LOCAL_DEV_SENTINEL_PREFIX, "sentinel credential", - REPOSITORY_FIXTURE_PREFIX, "repository fixture password (public, infra/redis-sdk/acl)"); + /** + * The keys {@code docs/registries/secrets-classification.yaml} classifies {@code secret}: the + * values the {@code fixture-} check reads, together with what a credential reference names and + * what a module contributes. {@code SecretsClassificationRegistryTest} holds this list to the + * registry 1:1, the way it holds {@link #REQUIRED_PROD_SECRETS}. + */ + public static final List CLASSIFIED_SECRETS = + List.of( + "APP_DATASOURCE_PASSWORD", + "SPRING_MAIL_PASSWORD", + "APP_CACHE_REDIS_PASSWORD", + "APP_CACHE_REDIS_KEY_HMAC_SECRET", + "APP_RATE_LIMIT_REDIS_PASSWORD", + "APP_RATE_LIMIT_REDIS_KEY_HMAC_SECRET", + "APP_SESSION_REDIS_PASSWORD", + "APP_SESSION_REDIS_KEY_HMAC_SECRET", + "APP_IDEMPOTENCY_REDIS_KEY_HMAC_SECRET", + "APP_LEASE_REDIS_KEY_HMAC_SECRET", + "APP_PRIVACY_PSEUDONYMIZATION_SALT", + "APP_NOTIFICATION_PLATFORM_CONTACT_ENCRYPTION_KEY", + "APP_NOTIFICATION_PLATFORM_CONTACT_LOOKUP_HMAC_KEY", + "APP_NOTIFICATION_PLATFORM_CALLBACK_SIGNING_KEY", + "APP_NOTIFICATION_PLATFORM_PROVIDER_CREDENTIAL_KEY", + "APP_NOTIFICATION_PLATFORM_PAYLOAD_ENCRYPTION_KEY", + "APP_NOTIFICATION_PLATFORM_PROVIDER_REQUEST_LOOKUP_HMAC_KEY", + "APP_NOTIFICATION_PLATFORM_CALLBACK_FINGERPRINT_HMAC_KEY"); /** * Secret keys that must be injected under the {@code prod} profile whatever the deployment @@ -89,8 +116,8 @@ public final class SecretSourceValidator implements StartupRule { @Override public String remedy() { return "inject real non-blank secrets from the secret manager / mounted environment; " - + "local/dev sentinel credentials and the repository's fixture-* passwords are forbidden" - + " in prod."; + + "local/dev sentinel credentials, and the repository's fixture-* lane passwords in a" + + " secret, are forbidden in prod."; } @Override @@ -98,53 +125,100 @@ public final class SecretSourceValidator implements StartupRule { if (!RuntimeProfileSnapshot.from(environment).isProduction()) { return List.of(); } - List violations = new ArrayList<>(); - collectPublicCredentialViolations(environment, violations); - collectMissingSecretViolations(environment, violations); - return violations; - } - - /** - * Names every property whose value is public by construction. Every property source is read, the - * process environment included, because a credential reference ({@code - * secret://@environment/}) may point at any variable name. The value itself is - * never put in the message. - */ - private static void collectPublicCredentialViolations( - Environment environment, List violations) { - if (!(environment instanceof ConfigurableEnvironment configurable)) { - return; - } - Map offenders = new LinkedHashMap<>(); - for (PropertySource source : configurable.getPropertySources()) { - if (source instanceof EnumerablePropertySource enumerable) { - for (String name : enumerable.getPropertyNames()) { - if (enumerable.getProperty(name) instanceof String text) { - PUBLIC_CREDENTIAL_PREFIXES.keySet().stream() - .filter(text::startsWith) - .findFirst() - .ifPresent(prefix -> offenders.putIfAbsent(name, prefix)); - } - } - } - } - offenders.forEach( - (offender, prefix) -> - violations.add( - "'" - + prefix - + "' " - + PUBLIC_CREDENTIAL_PREFIXES.get(prefix) - + " is configured for " - + offender)); - } - - private void collectMissingSecretViolations(Environment environment, List violations) { SecretSourcePort secretSource = SecretSourceFactory.fromEnvironment(environment); Set required = new LinkedHashSet<>(REQUIRED_PROD_SECRETS); // Each module names the secrets its selected capabilities need. This guard does not know which // capabilities a module has or how they are selected — it only checks the names resolve. contributed.get().forEach(module -> required.addAll(module.requiredSecrets(environment))); + List violations = new ArrayList<>(); + collectSentinelViolations(environment, violations); + collectFixturePasswordViolations(environment, secretSource, required, violations); + collectMissingSecretViolations(secretSource, required, violations); + return violations; + } + + /** + * Names every property whose value carries the dev/local sentinel. Every property source is read, + * the process environment included, because a credential reference ({@code + * secret://@environment/}) may point at any variable name. The value itself is + * never put in the message. + */ + private static void collectSentinelViolations(Environment environment, List violations) { + Set offenders = new LinkedHashSet<>(); + forEachStringValue( + environment, + (name, value) -> { + if (value.startsWith(LOCAL_DEV_SENTINEL_PREFIX)) { + offenders.add(name); + } + }); + offenders.forEach( + offender -> + violations.add( + "'" + + LOCAL_DEV_SENTINEL_PREFIX + + "' sentinel credential is configured for " + + offender)); + } + + /** + * Names every secret that holds a lane fixture password: the keys the registry classifies {@code + * secret}, the ones a module contributed, and every variable a credential reference names. Read + * through the same backend the application resolves them with. The value is never put in the + * message. + */ + private static void collectFixturePasswordViolations( + Environment environment, + SecretSourcePort secretSource, + Set required, + List violations) { + Set secrets = new LinkedHashSet<>(CLASSIFIED_SECRETS); + secrets.addAll(required); + forEachStringValue( + environment, + (name, value) -> { + if (SecretReference.isReference(value)) { + try { + secrets.add(SecretReference.parse(value).name()); + } catch (IllegalArgumentException ignored) { + // Malformed: the capability that reads this reference refuses it with its own + // message. + } + } + }); + for (String secret : secrets) { + secretSource + .resolve(secret) + .filter(value -> value.startsWith(REPOSITORY_FIXTURE_PREFIX)) + .ifPresent( + value -> + violations.add( + "secret " + + secret + + " holds a lane fixture password ('" + + REPOSITORY_FIXTURE_PREFIX + + "…', public in infra/redis-sdk/acl)")); + } + } + + private static void forEachStringValue( + Environment environment, BiConsumer action) { + if (!(environment instanceof ConfigurableEnvironment configurable)) { + return; + } + for (PropertySource source : configurable.getPropertySources()) { + if (source instanceof EnumerablePropertySource enumerable) { + for (String name : enumerable.getPropertyNames()) { + if (enumerable.getProperty(name) instanceof String text) { + action.accept(name, text); + } + } + } + } + } + + private static void collectMissingSecretViolations( + SecretSourcePort secretSource, Set required, List violations) { for (String key : required) { if (secretSource.resolve(key).isEmpty()) { violations.add(key + " is missing/blank"); diff --git a/src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/SecretsClassificationRegistryTest.java b/src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/SecretsClassificationRegistryTest.java index 5e8c33bf..6dd3794f 100644 --- a/src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/SecretsClassificationRegistryTest.java +++ b/src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/SecretsClassificationRegistryTest.java @@ -12,6 +12,7 @@ import java.util.List; import java.util.Map; import java.util.Set; import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Tag; import org.junit.jupiter.api.Test; import org.yaml.snakeyaml.Yaml; @@ -79,6 +80,30 @@ class SecretsClassificationRegistryTest { .isEqualTo(new LinkedHashSet<>(SecretSourceValidator.REQUIRED_PROD_SECRETS)); } + @Test + @DisplayName("prod fixture 비밀번호 검사가 읽는 비밀 목록은 레지스트리의 secret 분류 행과 1:1로 같다") + void classifiedSecretRowsMatchTheFixtureCheckListOneToOne() { + // given + Set registrySecrets = new LinkedHashSet<>(); + for (Map row : registryRows) { + if ("secret".equals(row.get("classification"))) { + registrySecrets.add((String) row.get("name")); + } + } + + // when + Set checked = new LinkedHashSet<>(SecretSourceValidator.CLASSIFIED_SECRETS); + + // then + assertThat(checked) + .as( + "SecretSourceValidator.CLASSIFIED_SECRETS is what the prod fixture-password check reads" + + " (review 2026-11-04 RR2-04-01: secrets only, not every value). A row classified" + + " 'secret' in docs/registries/secrets-classification.yaml that the list misses is" + + " a secret the check never looks at.") + .isEqualTo(registrySecrets); + } + @Test void everyRegistryRowUsesAValidClassificationTier() { for (Map row : registryRows) { diff --git a/src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/runtime/SecretSourceValidatorTest.java b/src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/runtime/SecretSourceValidatorTest.java index aaeabb20..db81d0a3 100644 --- a/src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/runtime/SecretSourceValidatorTest.java +++ b/src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/runtime/SecretSourceValidatorTest.java @@ -8,6 +8,7 @@ import dev.caskeleton.adapter.outbound.support.startup.StartupFailureException; import dev.caskeleton.adapter.outbound.support.startup.StartupRuleAutoConfiguration; import java.util.Arrays; import java.util.List; +import java.util.Map; import java.util.concurrent.atomic.AtomicInteger; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; @@ -15,6 +16,7 @@ import org.springframework.boot.autoconfigure.AutoConfigurations; import org.springframework.boot.test.context.runner.ApplicationContextRunner; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.core.env.SystemEnvironmentPropertySource; /** * Contract test for the secret/config-source startup fail-fast guard @@ -106,16 +108,20 @@ class SecretSourceValidatorTest { // ---- repository fixture passwords (review 2026-11-03 RR-07-04) -------------------------- @Test - @DisplayName("prod에서 비밀값이 저장소 fixture 비밀번호(fixture-)면 그 변수를 짚어 기동을 거부한다") + @DisplayName("prod에서 비밀 참조가 가리키는 변수가 저장소 fixture 비밀번호(fixture-)면 그 변수를 짚어 기동을 거부한다") void prodProfileWithARepositoryFixturePasswordFailsStartup() { // given ApplicationContextRunner prodWithLaneFixture = runner .withInitializer(ctx -> ctx.getEnvironment().setActiveProfiles("prod")) .withPropertyValues(allRequiredSecretsPresent()) - // The Compose Redis lanes' value: a public password from infra/redis-sdk/acl, which a - // deployment that copied the lane's environment would carry into production. - .withPropertyValues("REDIS_LANE_APPLICATION_PASSWORD=fixture-application"); + // The Compose Redis lanes' shape: a credential reference, and the variable it names + // holding a public password from infra/redis-sdk/acl. A deployment that copied the + // lane's environment would carry both into production. + .withPropertyValues( + "APP_REDIS_AUTHENTICATION_CREDENTIAL_REFERENCE=" + + "secret://ca-skeleton-application@environment/REDIS_LANE_APPLICATION_PASSWORD", + "REDIS_LANE_APPLICATION_PASSWORD=fixture-application"); // when prodWithLaneFixture.run( @@ -133,6 +139,55 @@ class SecretSourceValidatorTest { }); } + @Test + @DisplayName("prod에서 비밀로 분류된 키의 값이 저장소 fixture 비밀번호(fixture-)면 그 키를 짚어 기동을 거부한다") + void prodProfileWithAFixturePasswordInAClassifiedSecretFailsStartup() { + // given + ApplicationContextRunner prodWithFixtureSecret = + runner + .withInitializer(ctx -> ctx.getEnvironment().setActiveProfiles("prod")) + .withPropertyValues(allRequiredSecretsPresent()) + .withPropertyValues("APP_CACHE_REDIS_PASSWORD=fixture-application"); + + // when + prodWithFixtureSecret.run( + context -> { + // then + assertThat(context.getStartupFailure()) + .isInstanceOf(StartupFailureException.class) + .hasMessageContaining("APP_CACHE_REDIS_PASSWORD") + .hasMessageContaining("lane fixture password"); + }); + } + + @Test + @DisplayName("prod에서 비밀이 아닌 값(HOSTNAME·앱 이름·topic)이 fixture-로 시작해도 기동한다") + void prodStartsWhenOnlyNonSecretValuesStartWithTheFixturePrefix() { + // given + ApplicationContextRunner prodWithFixtureNamedService = + runner + .withInitializer( + ctx -> { + ctx.getEnvironment().setActiveProfiles("prod"); + // Kubernetes names the pod after its Deployment and hands it in as HOSTNAME. + ctx.getEnvironment() + .getPropertySources() + .addLast( + new SystemEnvironmentPropertySource( + "podEnvironment", Map.of("HOSTNAME", "fixture-api-6d5f7c9b8-x2k4q"))); + }) + .withPropertyValues(allRequiredSecretsPresent()) + .withPropertyValues( + "spring.application.name=fixture-service", "app.messaging.topic=fixture-updates"); + + // when + prodWithFixtureNamedService.run( + context -> { + // then + assertThat(context).hasNotFailed(); + }); + } + @Test @DisplayName("prod가 아니면 저장소 fixture 비밀번호(fixture-)로도 기동한다") void nonProdProfileWithARepositoryFixturePasswordStarts() { diff --git a/src/application-core/CLAUDE.md b/src/application-core/CLAUDE.md index 93780a35..5a7b5071 100644 --- a/src/application-core/CLAUDE.md +++ b/src/application-core/CLAUDE.md @@ -130,7 +130,7 @@ goes. `docs/manual/03-placement.md` points here. | `usecase`, `command`, `query` | application-core | inbound port base types and the command/query markers | | `capability` | application-core | `@UseCaseCapability` and its enums | | `transaction` | application-core | `TransactionPort` and its modes | -| `security` | application-core | authorization port, `@RequiresPermission`, `@SystemInvoked`, `ObjectAccessPolicy` | +| `security` | application-core | authorization port, `@RequiresPermission`, `@SystemInvoked`, `@SystemCaller`, `ObjectAccessPolicy` | | `tenant` | application-core | `TenantId` | | `observability` | application-core | `CorrelationIdPort`, `UserPrincipalPseudonymizerPort` | | `secret` | application-core | `SecretSourcePort`, `SecretReference` | @@ -200,10 +200,14 @@ application-core never self-registers with a DI framework. It is **not final**: declares `@RequiresPermission`, the web adapter enforces it through a CGLIB proxy, and CGLIB cannot subclass a final class (`RequiresPermissionProxyabilityTest`). It is registered as a bean for the same reason — a use case built with `new` is never checked. A use case only the system calls (a -scheduler, a relay) carries `@SystemInvoked(reason = ...)` instead of a permission and is not a bean: -it is a write with no permission check, so a transport must not reach it. Both halves are enforced — -a `@SystemInvoked` bean fails startup, and an inbound adapter that depends on one fails -`INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES`. A composition with `@RequiresPermission` +scheduler, a relay, a message consumer) carries `@SystemInvoked(reason = ...)` instead of a +permission and is not a bean: it is a write with no permission check, so no user request may reach +it. Its system caller constructs it — a composition module for a scheduler, or a +`@SystemCaller(reason = ...)` message handler in `..adapter.inbound.messaging..` (manual 09 +「소비」 step 4). Both halves are enforced — a `@SystemInvoked` bean fails startup, and an inbound +adapter that depends on one without being a `@SystemCaller` fails +`ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES` (`SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS` +keeps the mark off user transports). A composition with `@RequiresPermission` beans but no enforcing advisor fails startup too (`composition:service-runtime` `PermissionEnforcementGuard`, both checks). @@ -255,13 +259,17 @@ generation; the V1 `IdempotencyScope`/`tryBegin` API is gone. `app.capabilities.idempotency.provider` (`redis` → `cache-redis`, `postgresql` → `persistence-jpa`; default `disabled`). In-memory prod storage is forbidden. - **Executor**: built once by `composition:idempotency` `IdempotencyAutoConfiguration` from - `IdempotencySettings` (processing lease 30s, replay TTL `APP_IDEMPOTENCY_TTL` 24h ≤72h, - failure retention 24h). Do not construct it in a use case or a sample config. + `IdempotencySettings` (processing lease 30s, replay TTL `APP_IDEMPOTENCY_TTL` 24h ≤72h and not + shorter than the lease, failure retention 24h). Do not construct it in a use case or a sample config. - The action returns `IdempotentAction.Outcome` (`Success` / `RetryableNoEffect` / `EffectUnknown`); a thrown exception counts as effect-unknown. In progress → 409 `IDEMPOTENT_IN_FLIGHT`, fingerprint mismatch (SHA-256) → 422 `IDEMPOTENT_REQUEST_MISMATCH`; `IdempotencyUnavailableException` and `IdempotencyRecoveryRequiredException` mean the store could not prove the outcome, so do not blindly retry. +- Every store answers an execution whose processing lease ran out with `RecoveryRequired` and never + takes it over; only a claim that never started or a retryable failure is taken over. A record lives + for the lease plus the replay TTL. `IdempotencyStoreContract` holds the fake, Redis and PostgreSQL + to both on real servers (re-review 2026-11-04 RR-01-11, RR-03-11). The former ArchUnit freeze rule `inbound_port_implementations_do_not_declare_keyed_idempotency` and its fixture were removed when this branch merged. diff --git a/src/application-core/README.md b/src/application-core/README.md index b86059ed..3a71c1db 100644 --- a/src/application-core/README.md +++ b/src/application-core/README.md @@ -454,10 +454,11 @@ claim → 트랜잭션 밖에서 발행 → at-least-once 보장. `@RequiresPermission("outbox:relay")`는 D4 규칙을 통과시키려고 붙인 장식이었다(빈이 아니라 한 번도 검사되지 않았고, 빈이었다면 매 tick 거부됐다). ArchUnit D4는 `@SystemInvoked`를 권한 선언 대신 받고, `SYSTEM_INVOKED_USE_CASES_DECLARE_NO_PERMISSION`이 둘을 함께 다는 것을 막는다. - 면제의 전제(전송 계층이 닿지 않는다)도 코드가 지킨다: `@SystemInvoked` 빈은 - `PermissionEnforcementGuard`가 기동을 실패시키고, inbound adapter가 그 클래스에 의존하면 - `INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES`가 빌드를 실패시킨다(review 2026-11-03 - RR-01-02). + 면제의 전제(사용자 요청이 닿지 않는다)도 코드가 지킨다: `@SystemInvoked` 빈은 + `PermissionEnforcementGuard`가 기동을 실패시키고, `@SystemCaller`가 아닌 inbound adapter가 그 클래스에 + 의존하면 `ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES`가 빌드를 실패시킨다(review 2026-11-03 + RR-01-02, RR-01-12). 메시지 소비자는 `@SystemCaller` handler가 use case를 직접 만든다(매뉴얼 09장 + 「소비」 4단계). ### OutboxBackoffPolicy diff --git a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyClaimOutcome.java b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyClaimOutcome.java index a23fc9d2..64eb0246 100644 --- a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyClaimOutcome.java +++ b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyClaimOutcome.java @@ -24,6 +24,13 @@ public sealed interface IdempotencyClaimOutcome { } } + /** + * This caller now holds a record somebody else held before: a claim whose lease ran out before it + * started executing, or an attempt marked retryable without effect. Nothing ran under the + * previous holder, so running the action is safe. A record whose execution had started is never + * taken over — its effect is unknown, and the claim answers {@link RecoveryRequired} instead + * (re-review 2026-11-04 RR-01-11, {@code IdempotencyStoreContract}). + */ record TakenOverClaimed(IdempotencyOwner owner, Instant processingLeaseUntil) implements IdempotencyClaimOutcome { public TakenOverClaimed { @@ -50,6 +57,11 @@ public sealed interface IdempotencyClaimOutcome { } } + /** + * An earlier attempt may have taken effect and nobody can say whether it did: it was marked + * abandoned with an unknown effect, or its processing lease ran out while it was executing. The + * action must not run again under this key; the outcome is settled from business data. + */ record RecoveryRequired(long currentAttempt) implements IdempotencyClaimOutcome { public RecoveryRequired { if (currentAttempt < 1) { diff --git a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyCompletionUnknownException.java b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyCompletionUnknownException.java index 2d067c1b..13448bcc 100644 --- a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyCompletionUnknownException.java +++ b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyCompletionUnknownException.java @@ -8,13 +8,18 @@ import dev.caskeleton.shared.error.OperationalError; * The action ran, and the store did not answer when its result was being recorded. * *

Not {@link IdempotencyUnavailableException}: that one is raised before the action, so the - * request certainly did not apply. Here it may have. Effects inside the action's own transaction - * roll back with this exception; anything else the action did — a call to another system, a message - * — has happened. Telling the client "not applied" would invite a resend under a new key, which is - * the duplicate the key exists to prevent (re-review 2026-11-03, RR-02-03). + * request certainly did not apply. Here it may have. When {@code execute} was called inside the + * caller's write transaction, the action's database writes roll back with this exception; an action + * that committed a transaction of its own has kept them, and anything else it did — a call to + * another system, a message — has happened. Telling the client "not applied" would invite a resend + * under a new key, which is the duplicate the key exists to prevent (re-review 2026-11-03, + * RR-02-03). * - *

A resend with the same key is safe: the server judges it as a replay when the result was in - * fact recorded, as in-flight while the attempt's lease lasts, and as recovery-required after that. + *

A resend with the same key does not run the action again: the server judges it as a replay + * when the result was in fact recorded, as in-flight while the attempt's processing lease lasts, + * and as recovery-required after that. Both stores answer so ({@code IdempotencyStoreContract}, + * re-review 2026-11-04 RR-01-11). The record lasts the lease plus the replay TTL; a resend after + * that finds no record and is a new request. */ public final class IdempotencyCompletionUnknownException extends RuntimeException implements ApiErrorCarrier { diff --git a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyExecutor.java b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyExecutor.java index 6dae5a62..4ca9e5f5 100644 --- a/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyExecutor.java +++ b/src/application-core/src/main/java/dev/caskeleton/application/idempotency/IdempotencyExecutor.java @@ -133,6 +133,9 @@ public final class IdempotencyExecutor { startAndRun(request, acquired.owner(), action, codec, false); case IdempotencyClaimOutcome.ReplayedAcquire replayed -> startAndRun(request, replayed.owner(), action, codec, false); + // Only a claim that never started or failed without effect is taken over, so running here is + // the first run that can have an effect. An execution whose lease ran out is not: the store + // answers RecoveryRequired for it above (IdempotencyStoreContract, RR-01-11). case IdempotencyClaimOutcome.TakenOverClaimed takenOver -> startAndRun(request, takenOver.owner(), action, codec, false); }; diff --git a/src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAggregateSequencePort.java b/src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAggregateSequencePort.java index 73a17469..b1eac0c8 100644 --- a/src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAggregateSequencePort.java +++ b/src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAggregateSequencePort.java @@ -27,10 +27,14 @@ import dev.caskeleton.application.messaging.event.AggregateIdentity; * for events the retention reaper has since deleted, and greater than any number an outbox * row of the aggregate carries when the call is made. *

  • Where an implementation replaced an earlier numbering (the sample's wall clock), greater - * than any number that numbering could have sent for the aggregate — also for rows deleted - * since, or written under an identity that has since changed. The first number is therefore - * not necessarily 1: the JPA counter starts above the moment its migration ran, in epoch - * microseconds (review RR-03-02). + * than any number that numbering sent for the aggregate before the replacement took effect — + * also for rows deleted since, or written then under an identity that has since changed. The + * first number is therefore not necessarily 1: the JPA counter starts above the moment its + * migration ran, in epoch microseconds (review RR-03-02). A number an instance still on the + * earlier numbering writes after that is covered only under the same identity (the item + * above): under an identity this call no longer uses it is invisible, and the new number can + * be smaller. A deploy that changes the identity therefore stops every old writer before the + * new version writes (runbook outbox-dead-letter §8; re-review round 2, RR-02-10). *
  • Two first calls for one aggregate that overlap both succeed, with consecutive numbers. *
  • Independent per aggregate identity: one aggregate's numbers say nothing about another's. * diff --git a/src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxStorePort.java b/src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxStorePort.java index b96ac1da..2a590b14 100644 --- a/src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxStorePort.java +++ b/src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxStorePort.java @@ -103,9 +103,12 @@ public interface OutboxStorePort { * {@code outbox.pending.size} metric. Called outside a transaction (read-only). * *

    PUBLISHED is never counted: those rows wait on nothing, and counting them would scan the - * whole retention window on every poll. + * whole retention window on every poll. A status this version has no constant for (written by a + * later version, or by hand) is left out rather than failing the whole count, so the gauges of + * the statuses it does know stay up. * - * @return map from status to row count; PUBLISHED and statuses with zero rows are absent + * @return map from status to row count; PUBLISHED, unknown statuses and statuses with zero rows + * are absent */ Map countByStatus(); diff --git a/src/application-core/src/main/java/dev/caskeleton/application/security/SystemCaller.java b/src/application-core/src/main/java/dev/caskeleton/application/security/SystemCaller.java new file mode 100644 index 00000000..851dfebe --- /dev/null +++ b/src/application-core/src/main/java/dev/caskeleton/application/security/SystemCaller.java @@ -0,0 +1,34 @@ +package dev.caskeleton.application.security; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Marks an inbound adapter that the system drives — a message handler the broker consumer calls, a + * scheduled job — and that therefore may call a {@link SystemInvoked} use case. + * + *

    A {@code @SystemInvoked} use case is a write with no permission check, so no user request may + * reach it. A user reaches the service through a transport (web, GraphQL, gRPC, WebSocket); the + * system reaches it through a delivery or a tick. This marker is how an inbound adapter of the + * second kind says so, in code, next to the reason. Without it an inbound adapter that depends on a + * {@code @SystemInvoked} use case fails {@code ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES}; + * with it, {@code SYSTEM_CALLERS_ARE_SYSTEM_DRIVEN_INBOUND_ADAPTERS} checks the claim: the class + * lives in a system-driven inbound package ({@code ..adapter.inbound.messaging..}, {@code + * ..adapter.inbound.scheduler..}) and depends on no user-transport type. A controller cannot carry + * it. + * + *

    The marked adapter constructs the use case it calls with {@code new} from the ports it is + * given and keeps it to itself: the use case is never a bean ({@code PermissionEnforcementGuard} + * refuses one), so nothing but this adapter holds it. Review 2026-11-03 RR-01-12, decision 9.1. + */ +@Documented +@Retention(RetentionPolicy.RUNTIME) +@Target(ElementType.TYPE) +public @interface SystemCaller { + + /** What drives this adapter and why no user request reaches it. */ + String reason(); +} diff --git a/src/application-core/src/main/java/dev/caskeleton/application/security/SystemInvoked.java b/src/application-core/src/main/java/dev/caskeleton/application/security/SystemInvoked.java index 54b9cb18..ab646e1a 100644 --- a/src/application-core/src/main/java/dev/caskeleton/application/security/SystemInvoked.java +++ b/src/application-core/src/main/java/dev/caskeleton/application/security/SystemInvoked.java @@ -7,22 +7,25 @@ import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; /** - * Marks a mutating use case that only the system calls — a scheduler, a relay — and that therefore - * carries no {@link RequiresPermission}. + * Marks a mutating use case that only the system calls — a scheduler, a relay, a message consumer — + * and that therefore carries no {@link RequiresPermission}. * *

    A permission gate on such a use case is decoration: the method-security advisor checks the - * caller's {@code Authentication}, a scheduler thread has none, so the gate either never runs (the - * use case is not a bean) or refuses every call (it is). This marker states that instead, and the - * architecture rule that otherwise requires a permission on every {@code WRITE_REPOSITORY} use case - * accepts it in place of one. + * caller's {@code Authentication}, a scheduler or consumer thread has none, so the gate either + * never runs (the use case is not a bean) or refuses every call (it is). This marker states that + * instead, and the architecture rule that otherwise requires a permission on every {@code + * WRITE_REPOSITORY} use case accepts it in place of one. * - *

    A use case marked this way is a write with no permission check, so a transport must never - * reach it. The composition that runs it constructs it directly and hands it to its system caller. - * Both ways a transport could reach it are refused: a bean of it fails startup ({@code + *

    A use case marked this way is a write with no permission check, so no user request may reach + * it. Its system caller constructs it directly and keeps it: a composition module hands it to a + * scheduler (the outbox relay), or a {@link SystemCaller} inbound adapter — a message handler, a + * scheduled job — constructs it from the ports its bean method receives (manual 09 「소비」 step 4). + * Both ways a user could reach it are refused: a bean of it fails startup ({@code * composition:service-runtime} {@code PermissionEnforcementGuard}), and an inbound adapter that - * depends on it fails the architecture rule {@code - * INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES}. It is not a way to silence the rule that - * a mutating use case declares a permission: a write a user triggers declares one. + * depends on it without being a {@link SystemCaller} fails the architecture rule {@code + * ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES} (review 2026-11-03 RR-01-02, RR-01-12). It is + * not a way to silence the rule that a mutating use case declares a permission: a write a user + * triggers declares one, and a write both a user and the system trigger is two use cases. */ @Documented @Retention(RetentionPolicy.RUNTIME) diff --git a/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/IdempotencyStoreContract.java b/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/IdempotencyStoreContract.java index 29bfef2f..ce58c450 100644 --- a/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/IdempotencyStoreContract.java +++ b/src/application-core/src/testFixtures/java/dev/caskeleton/application/idempotency/testkit/IdempotencyStoreContract.java @@ -31,14 +31,28 @@ import org.junit.jupiter.api.Test; * against the in-memory fake and every store: the fake, the Redis store over the command fake and * over a real server, and the PostgreSQL store over a real database. * - *

    Mostly time-free: expiry and take-over of an abandoned lease depend on whose clock the store - * trusts, and those cases stay with each store's own tests. The one timed case is the one a caller - * relies on whichever store is composed — {@code renew} really extends the lease (review 05-02: the - * Redis store answered a renew as already applied and let the lease run out). A store reaches time - * through {@link #elapse(Duration)}. A store that cannot move its clock — the in-memory fake, which - * models no expiry, and the Redis store over the command fake — keeps the default, which aborts the - * timed case so the report shows it skipped instead of passing a case that never ran its clock - * (review 2026-11-03 RR-01-06). + *

    Mostly time-free. The timed cases are the ones a caller relies on whichever store is composed, + * because the stores used to differ on them and every gate stayed green: + * + *

      + *
    • {@code renew} really extends the lease (review 05-02: the Redis store answered a renew as + * already applied and let the lease run out); + *
    • a lease that runs out takes over only what never ran or failed without effect. A record + * whose execution had started is answered "recovery required" and never run again: its effect + * is unknown, and a client resending the same key after a result-unknown 503 was told exactly + * that (re-review 2026-11-04 RR-01-11 — the Redis store took it over and ran it a second + * time, the PostgreSQL store did not); + *
    • a record outlives its processing lease by the replay TTL, whatever the two are set to. A + * record that expired first — Redis at its key TTL, PostgreSQL at the reaper — let the next + * claim of a running operation acquire it fresh (re-review 2026-11-04 RR-03-11), and one that + * expired with its lease left no record to answer "recovery required" from. + *
    + * + *

    A store reaches time through {@link #elapse(Duration)}, which also runs whatever expires the + * store's records (Redis does it itself; the PostgreSQL contract runs the reaper). A store that + * cannot move its clock — the in-memory fake, which models no expiry, and the Redis store over the + * command fake — keeps the default, which aborts the timed case so the report shows it skipped + * instead of passing a case that never ran its clock (review 2026-11-03 RR-01-06). * *

    Every call goes through {@link #inTransaction(Supplier)}, the way the store's callers make it: * the PostgreSQL store refuses to run outside an application-owned write transaction. @@ -85,9 +99,11 @@ public abstract class IdempotencyStoreContract { } /** - * Lets {@code duration} pass on the clock the store trusts. The default aborts the calling test: - * a store without a clock the contract can move cannot run a timed case, and a no-op here would - * let that case pass without time ever passing. + * Lets {@code duration} pass on the clock the store trusts, and then lets the store expire what + * it expires on its own — so a record the store would have dropped by then is gone when the next + * call arrives. The default aborts the calling test: a store without a clock the contract can + * move cannot run a timed case, and a no-op here would let that case pass without time ever + * passing. * * @param duration how long to let pass */ @@ -107,8 +123,26 @@ public abstract class IdempotencyStoreContract { private static IdempotencyClaimRequest claim( IdempotencyClaimAttempt attempt, RequestFingerprint fingerprint, Duration processingLease) { + return claim(attempt, fingerprint, processingLease, Duration.ofHours(24)); + } + + private static IdempotencyClaimRequest claim( + IdempotencyClaimAttempt attempt, + RequestFingerprint fingerprint, + Duration processingLease, + Duration replayTtl) { return new IdempotencyClaimRequest( - SCOPE, fingerprint, attempt, processingLease, Duration.ofHours(24), "json-v2", 1); + SCOPE, fingerprint, attempt, processingLease, replayTtl, "json-v2", 1); + } + + /** Claims and starts the operation with the given lease and replay TTL. */ + private static IdempotencyOwner executingWith( + IdempotencyStorePort store, Duration processingLease, Duration replayTtl) { + IdempotencyOwner claimed = + ownerOf( + store.claim( + claim(store.newClaimAttempt(OPERATION), FINGERPRINT, processingLease, replayTtl))); + return store.markExecutionStarted(claimed, OPERATION).owner().orElseThrow(); } private static IdempotencyOwner acquire(IdempotencyStorePort store) { @@ -392,6 +426,93 @@ public abstract class IdempotencyStoreContract { assertThat(other).isInstanceOf(IdempotencyClaimOutcome.InProgress.class); } + @Test + @DisplayName("처리 lease가 지난 실행 전 claim은 다음 요청이 attempt 2로 넘겨받는다") + void aClaimThatNeverStartedIsTakenOverWhenItsLeaseRunsOut() { + // given + IdempotencyStorePort store = subject(); + Duration lease = processingLease(); + store.claim(claim(store.newClaimAttempt(OPERATION), FINGERPRINT, lease)); + elapse(lease.multipliedBy(2)); + + // when + IdempotencyClaimOutcome next = + store.claim(claim(store.newClaimAttempt(OTHER_OPERATION), FINGERPRINT, lease)); + + // then + assertThat(next).isInstanceOf(IdempotencyClaimOutcome.TakenOverClaimed.class); + assertThat(ownerOf(next).attempt()).isEqualTo(2); + } + + @Test + @DisplayName("실행을 시작한 기록의 처리 lease가 지나면 다음 요청은 작업을 다시 돌리지 않고 회복 필요를 받는다") + void anExecutionWhoseLeaseRanOutRequiresRecoveryInsteadOfASecondRun() { + // given + IdempotencyStorePort store = subject(); + Duration lease = processingLease(); + executingWith(store, lease, Duration.ofHours(24)); + elapse(lease.multipliedBy(2)); + + // when + IdempotencyClaimOutcome next = + store.claim(claim(store.newClaimAttempt(OTHER_OPERATION), FINGERPRINT, lease)); + + // then + assertThat(next).isInstanceOf(IdempotencyClaimOutcome.RecoveryRequired.class); + } + + @Test + @DisplayName("lease가 지나 회복 필요로 판정된 실행은 원래 holder가 늦게 complete해도 기록되지 않는다") + void anExecutionFoundExpiredCannotBeCompletedByItsLateHolder() { + // given + IdempotencyStorePort store = subject(); + Duration lease = processingLease(); + IdempotencyOwner late = executingWith(store, lease, Duration.ofHours(24)); + elapse(lease.multipliedBy(2)); + store.claim(claim(store.newClaimAttempt(OTHER_OPERATION), FINGERPRINT, lease)); + + // when + IdempotencyCompleteOutcome outcome = + store.complete(late, new StoredResponse("{}"), Duration.ofHours(1), OPERATION); + + // then + assertThat(outcome).isEqualTo(IdempotencyCompleteOutcome.NOT_OWNER); + } + + @Test + @DisplayName("replay TTL보다 긴 처리 lease로 claim해도 lease가 끝날 때까지 기록이 남아 다른 요청은 진행 중으로 막힌다") + void aLeaseLongerThanTheReplayTtlKeepsTheRecordForTheWholeLease() { + // given + IdempotencyStorePort store = subject(); + Duration replayTtl = processingLease(); + executingWith(store, replayTtl.multipliedBy(3), replayTtl); + elapse(replayTtl.multipliedBy(2)); + + // when + IdempotencyClaimOutcome other = + store.claim(claim(store.newClaimAttempt(OTHER_OPERATION), FINGERPRINT, processingLease())); + + // then + assertThat(other).isInstanceOf(IdempotencyClaimOutcome.InProgress.class); + } + + @Test + @DisplayName("replay TTL이 처리 lease와 같아도 lease가 지난 실행 기록은 남아 다음 요청에 회복 필요로 답한다") + void anExecutionIsRememberedPastItsLeaseEvenWhenTheReplayTtlIsNoLonger() { + // given + IdempotencyStorePort store = subject(); + Duration lease = processingLease(); + executingWith(store, lease, lease); + elapse(lease.multipliedBy(3).dividedBy(2)); + + // when + IdempotencyClaimOutcome next = + store.claim(claim(store.newClaimAttempt(OTHER_OPERATION), FINGERPRINT, lease)); + + // then + assertThat(next).isInstanceOf(IdempotencyClaimOutcome.RecoveryRequired.class); + } + /** Routes every call of the store through {@link #inTransaction(Supplier)}. */ private final class EachCallInTransaction implements IdempotencyStorePort { diff --git a/src/application-core/src/testFixtures/java/dev/caskeleton/application/transaction/testkit/DirectTransactionPort.java b/src/application-core/src/testFixtures/java/dev/caskeleton/application/transaction/testkit/DirectTransactionPort.java index fd0e237b..5ecafe46 100644 --- a/src/application-core/src/testFixtures/java/dev/caskeleton/application/transaction/testkit/DirectTransactionPort.java +++ b/src/application-core/src/testFixtures/java/dev/caskeleton/application/transaction/testkit/DirectTransactionPort.java @@ -3,7 +3,10 @@ package dev.caskeleton.application.transaction.testkit; import dev.caskeleton.application.transaction.NestedRootTransactionRejectedException; import dev.caskeleton.application.transaction.TransactionPort; import java.util.ArrayList; +import java.util.Collections; import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; import java.util.function.Supplier; /** @@ -30,7 +33,11 @@ import java.util.function.Supplier; * * *

    {@link TransactionPortContract} holds it to the same cases as the Spring adapter on a real - * database. Single-threaded: one instance per test. + * database. A transaction belongs to the thread that opened it, as a connection does: a boundary on + * another thread opens a transaction of its own rather than joining this one, so an overlapping- + * writes case ({@link OverlappingWrites}) runs two transactions here too (review 2026-11-03 + * RR-01-16). {@link #active()}, {@link #readOnly()} and {@link #onRollback} answer for the calling + * thread. One instance per test. */ public final class DirectTransactionPort implements TransactionPort { @@ -43,8 +50,10 @@ public final class DirectTransactionPort implements TransactionPort { } private final boolean rejectNesting; - private final List committed = new ArrayList<>(); - private Transaction current; + private final List committed = Collections.synchronizedList(new ArrayList<>()); + // Per thread, keyed by the thread itself; an entry lives only while that thread's outermost + // boundary runs. + private final Map current = new ConcurrentHashMap<>(); public DirectTransactionPort() { this(false); @@ -59,19 +68,22 @@ public final class DirectTransactionPort implements TransactionPort { return new DirectTransactionPort(true); } - /** Boundaries whose work committed, in completion order. */ + /** Boundaries whose work committed, on any thread, in completion order. */ public List committed() { - return List.copyOf(committed); + synchronized (committed) { + return List.copyOf(committed); + } } - /** Whether an action is running right now. */ + /** Whether an action is running on this thread right now. */ public boolean active() { - return current != null; + return current.containsKey(Thread.currentThread()); } - /** Whether the transaction running right now is a read-only one, opened by {@link #inRead}. */ + /** Whether this thread's running transaction is a read-only one, opened by {@link #inRead}. */ public boolean readOnly() { - return current != null && current.readOnly; + Transaction transaction = current.get(Thread.currentThread()); + return transaction != null && transaction.readOnly; } /** @@ -80,10 +92,11 @@ public final class DirectTransactionPort implements TransactionPort { * @throws IllegalStateException when no transaction is active */ public void onRollback(Runnable undo) { - if (current == null) { + Transaction transaction = current.get(Thread.currentThread()); + if (transaction == null) { throw new IllegalStateException("no transaction is active to roll back"); } - current.undo.add(undo); + transaction.undo.add(undo); } @Override @@ -93,7 +106,7 @@ public final class DirectTransactionPort implements TransactionPort { @Override public T inRootWrite(Supplier action) { - if (current != null) { + if (current.containsKey(Thread.currentThread())) { throw new NestedRootTransactionRejectedException(); } return run(Boundary.ROOT_WRITE, action); @@ -110,15 +123,15 @@ public final class DirectTransactionPort implements TransactionPort { } private T run(Boundary boundary, Supplier action) { - if (current != null && rejectNesting) { + Transaction suspended = current.get(Thread.currentThread()); + if (suspended != null && rejectNesting) { throw new IllegalStateException(boundary + " transaction opened inside another"); } - if (current != null && boundary != Boundary.NEW) { - return join(boundary, action); + if (suspended != null && boundary != Boundary.NEW) { + return join(suspended, boundary, action); } - Transaction suspended = current; Transaction transaction = new Transaction(boundary == Boundary.READ); - current = transaction; + current.put(Thread.currentThread(), transaction); try { T result; try { @@ -132,16 +145,21 @@ public final class DirectTransactionPort implements TransactionPort { throw new IllegalStateException( boundary + " transaction rolled back: a boundary that joined it failed"); } - committed.addAll(transaction.joined); - committed.add(boundary); + synchronized (committed) { + committed.addAll(transaction.joined); + committed.add(boundary); + } return result; } finally { - current = suspended; + if (suspended == null) { + current.remove(Thread.currentThread()); + } else { + current.put(Thread.currentThread(), suspended); + } } } - private T join(Boundary boundary, Supplier action) { - Transaction joined = current; + private static T join(Transaction joined, Boundary boundary, Supplier action) { try { T result = action.get(); joined.joined.add(boundary); diff --git a/src/application-core/src/testFixtures/java/dev/caskeleton/application/transaction/testkit/TransactionPortContract.java b/src/application-core/src/testFixtures/java/dev/caskeleton/application/transaction/testkit/TransactionPortContract.java index 69b7d2d8..1856dc51 100644 --- a/src/application-core/src/testFixtures/java/dev/caskeleton/application/transaction/testkit/TransactionPortContract.java +++ b/src/application-core/src/testFixtures/java/dev/caskeleton/application/transaction/testkit/TransactionPortContract.java @@ -6,6 +6,11 @@ import static org.assertj.core.api.Assertions.catchThrowable; import dev.caskeleton.application.transaction.NestedRootTransactionRejectedException; import dev.caskeleton.application.transaction.TransactionPort; import java.util.List; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicBoolean; import java.util.function.Supplier; import org.junit.jupiter.api.DisplayName; @@ -294,6 +299,55 @@ public abstract class TransactionPortContract { .containsExactly("runnable-new", "runnable-root", "runnable-write"); } + /** + * A transaction belongs to the thread that opened it, as a database connection does. A boundary + * opened on another thread while this one is open is a transaction of its own: it commits when it + * returns and this thread's rollback does not take its write. A port that shared one open + * transaction across threads would make every overlapping-writes contract case two calls in one + * transaction instead of two transactions (review 2026-11-03 RR-01-16). + */ + @Test + @DisplayName("다른 스레드의 경계는 이 스레드의 열린 트랜잭션에 합류하지 않고 따로 커밋한다") + void aBoundaryOnAnotherThreadIsATransactionOfItsOwn() { + // given + TransactionPort transactions = transactions(); + TransactionProbe probe = probe(); + ExecutorService otherThread = Executors.newSingleThreadExecutor(); + + // when + Throwable thrown; + try { + thrown = + catchThrowable( + () -> + transactions.inWrite( + () -> { + probe.write("this-thread"); + awaitOn( + otherThread, + () -> transactions.inWrite(() -> probe.write("other-thread"))); + throw new IllegalStateException("this thread rolls back"); + })); + } finally { + otherThread.shutdownNow(); + } + + // then + assertThat(List.of(thrown.getMessage(), probe.committed())) + .containsExactly("this thread rolls back", List.of("other-thread")); + } + + private static void awaitOn(ExecutorService thread, Runnable work) { + try { + thread.submit(work).get(30, TimeUnit.SECONDS); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + throw new IllegalStateException(interrupted); + } catch (ExecutionException | TimeoutException failed) { + throw new IllegalStateException("the other thread's boundary failed", failed); + } + } + private static Supplier written(TransactionProbe probe, String mark, T value) { return () -> { probe.write(mark); diff --git a/src/application-fileserver/src/test/java/dev/caskeleton/application/fileserver/testkit/InMemoryQuotaLedgerContractTest.java b/src/application-fileserver/src/test/java/dev/caskeleton/application/fileserver/testkit/InMemoryQuotaLedgerContractTest.java index 2494b443..c96285d9 100644 --- a/src/application-fileserver/src/test/java/dev/caskeleton/application/fileserver/testkit/InMemoryQuotaLedgerContractTest.java +++ b/src/application-fileserver/src/test/java/dev/caskeleton/application/fileserver/testkit/InMemoryQuotaLedgerContractTest.java @@ -12,6 +12,7 @@ import java.time.Instant; import java.util.Optional; import java.util.OptionalLong; import java.util.UUID; +import java.util.function.Supplier; class InMemoryQuotaLedgerContractTest extends FileQuotaLedgerContract { @@ -50,6 +51,11 @@ class InMemoryQuotaLedgerContractTest extends FileQuotaLedgerContract { return FileserverFixtures.session(fileId, 1_000, 1_000); } + @Override + protected T inTransaction(Supplier work) { + return ledger.inTransaction(work); + } + @Override protected void elapse(Duration duration) { now = now.plus(duration); diff --git a/src/application-fileserver/src/testFixtures/java/dev/caskeleton/application/fileserver/testkit/FakeFileQuotaService.java b/src/application-fileserver/src/testFixtures/java/dev/caskeleton/application/fileserver/testkit/FakeFileQuotaService.java index 7a5e189b..018f6317 100644 --- a/src/application-fileserver/src/testFixtures/java/dev/caskeleton/application/fileserver/testkit/FakeFileQuotaService.java +++ b/src/application-fileserver/src/testFixtures/java/dev/caskeleton/application/fileserver/testkit/FakeFileQuotaService.java @@ -16,6 +16,7 @@ import java.util.List; import java.util.Map; import java.util.Optional; import java.util.UUID; +import java.util.function.Supplier; /** * The in-memory quota ledger: the fake of {@link FileQuotaPort}, and the ledger that {@link @@ -45,6 +46,27 @@ public final class FakeFileQuotaService implements FileQuotaPort { this.now = instant; } + /** + * Runs {@code work} as one caller transaction over the ledger's rows: what it wrote is undone if + * it throws, as the reservation table is when the JPA adapters' caller rolls back. The recorded + * calls ({@link #reservedScopes()} and the like) are not undone. For {@link + * FileQuotaLedgerContract}, whose rollback case holds the fake and PostgreSQL to the same answer + * (review 2026-11-03 RR-01-17). + */ + public T inTransaction(Supplier work) { + Map before = new LinkedHashMap<>(); + rows.forEach((id, row) -> before.put(id, row.copy())); + long writesBefore = writes; + try { + return work.get(); + } catch (RuntimeException | Error failure) { + rows.clear(); + rows.putAll(before); + writes = writesBefore; + throw failure; + } + } + /** Books usage the scope already holds, as if earlier uploads had committed it. */ public void alreadyHolds(QuotaScope scope, long bytes) { Row row = newRow(scope, bytes, now); @@ -215,6 +237,13 @@ public final class FakeFileQuotaService implements FileQuotaPort { this.updatedAt = createdAt; } + private Row copy() { + Row copy = new Row(id, scope, reservedBytes, expiresAt, updatedAt); + copy.committedBytes = committedBytes; + copy.status = status; + return copy; + } + private boolean isLiveAt(Instant instant) { return status == QuotaReservationStatus.RESERVED && expiresAt.isAfter(instant); } diff --git a/src/application-fileserver/src/testFixtures/java/dev/caskeleton/application/fileserver/testkit/FileQuotaLedgerContract.java b/src/application-fileserver/src/testFixtures/java/dev/caskeleton/application/fileserver/testkit/FileQuotaLedgerContract.java index b217dad1..546dba28 100644 --- a/src/application-fileserver/src/testFixtures/java/dev/caskeleton/application/fileserver/testkit/FileQuotaLedgerContract.java +++ b/src/application-fileserver/src/testFixtures/java/dev/caskeleton/application/fileserver/testkit/FileQuotaLedgerContract.java @@ -48,10 +48,11 @@ public abstract class FileQuotaLedgerContract { /** Lets {@code duration} pass on the clock the ledger reads. */ protected abstract void elapse(Duration duration); - /** Runs one ledger call inside the caller transaction the adapters require, and commits it. */ - protected T inTransaction(Supplier work) { - return work.get(); - } + /** + * Runs ledger calls inside the caller transaction the adapters require: commits when {@code work} + * returns, rolls back what it wrote when it throws. + */ + protected abstract T inTransaction(Supplier work); @Test @DisplayName("살아 있는 예약은 그 scope의 사용량에 들어간다") @@ -307,6 +308,38 @@ public abstract class FileQuotaLedgerContract { assertThat(refused).isInstanceOf(IllegalArgumentException.class); } + /** + * A move books the file's bytes on its target and reclaims them from its source inside the + * caller's transaction that also relocates the record ({@code MoveFileApplicationUseCase}). When + * that transaction rolls back — the relocate lost a version race — neither namespace may keep its + * half. So every write of the ledger joins the caller's transaction; a port moved into a + * transaction of its own fails here (review 2026-11-03 RR-01-17). + */ + @Test + @DisplayName("호출자 트랜잭션이 롤백되면 그 안의 예약·확정·회수가 양쪽 scope에서 모두 사라진다") + void ledgerWritesInACallerTransactionThatRollsBackLeaveBothScopesAsTheyWere() { + // given + QuotaScope source = newScope(); + QuotaScope target = newScope(); + commit(source, 400); + + // when + Throwable thrown = + catchThrowable( + () -> + inTransaction( + () -> { + QuotaReservation booked = quota().reserve(target, 400, TTL); + quota().commit(booked, 400); + reclaims().reclaim(source, 400); + throw new IllegalStateException("the relocate lost its version race"); + })); + + // then + assertThat(List.of(thrown.getMessage(), usedBytes(source), usedBytes(target))) + .containsExactly("the relocate lost its version race", 400L, 0L); + } + private QuotaReservation reserve(QuotaScope scope, long bytes) { return inTransaction(() -> quota().reserve(scope, bytes, TTL)); } diff --git a/src/build-logic/README.md b/src/build-logic/README.md index 97afbfdb..bfdb77e9 100644 --- a/src/build-logic/README.md +++ b/src/build-logic/README.md @@ -16,7 +16,7 @@ capability, and the files it reads are declared as that test task's inputs. | `ca.java-library` / `ca.spring-library` / `ca.spring-config` | leaf-level composition of the Java/quality conventions for plain and Spring libraries | | `ca.platform-module` / `ca.grpc-platform-module` | what a vendored platform leaf (`messaging:*`, `grpc:*`, `grpc-advanced:*`) is: `java-library`, plus the gRPC BOM for the leaves that had it | | `ca.strict-test-lane` | the one "named tests ran, none skipped" mechanism: tagged / named / own-source-set `Test` lanes that fail when they execute nothing, miss a `requires` class, or (with `rejectSkipped`) skip anything. `mainStage()` moves a tagged lane out of `test` into the leaf's `mainCheck`; `explicitOnly(reason)` keeps a lane that needs an outside input out of every aggregate, and the aggregate prints the reason. Extra test source sets are Gradle `jvm-test-suite` suites declared in the leaf (`testing { suites { … } }`); a lane named after its suite makes the suite's own `Test` task strict, and `ca.java-conventions` gives every suite the BOM's JUnit | -| `ca.strict-lane-stages` (root of each build with strict lanes: the main build and optional-platforms) | the other half of the lane mechanism: `verifyStrictLaneStages --dry-run` fails when a strict lane of that build that is not `explicitOnly` is in the task plan of no declared stage that a workflow executes, or when a declared stage is executed by no workflow. A `--dry-run`, a command for another build (`workflowProjectDirectory` names the `-p` value that starts this one), and a command under `continue-on-error` or an `if:` other than `always()`/`success()`/`!cancelled()` do not count, and a lane the command removes with `-x` is not covered. The plan is Gradle's own; the workflows are read for their `run:` steps with YAML folding applied (review 2026-11-01 08-01, 2026-11-03 RR-06-03/04) | +| `ca.strict-lane-stages` (root of each build with strict lanes: the main build and optional-platforms) | the other half of the lane mechanism: `verifyStrictLaneStages --dry-run` fails when a strict lane of that build that is not `explicitOnly` is in the task plan of no declared stage that a workflow executes, or when a declared stage is executed by no workflow. A `--dry-run`, a command for another build (the build a command starts is its step's `working-directory`, or the job's or workflow's `defaults.run.working-directory`, joined with `-p`; the plugin reads this build's own directory from where it sits under the repository that holds `.github/workflows`), a command under `continue-on-error` or an `if:` other than `always()`/`success()`/`!cancelled()`, and a command of a workflow that starts only on `workflow_dispatch` do not count, and a lane the command removes with `-x` is not covered — matched the way Gradle matches task and project names, abbreviations included (`TaskSelectors`, a copy of Gradle's internal `NameMatcher`). A `run:` input of a `uses:` step is not a command. The plan is Gradle's own; the workflows are read for their `run:` steps with YAML folding applied (review 2026-11-01 08-01, 2026-11-03 RR-06-03/04, 2026-11-04 RR2-04-04) | | `ca.redis-topology-lane` | the Redis topology lane (mode selection, `-P` property validation, per-mode required classes) the redis-sdk-topology workflow drives | | `ca.api-surface` | read-only public API surface verification with a separate update task, wired into the leaf's `check` (owner decision 20); optionally holds other leaves' imports to a reviewed adopter subset (`adopterBaseline`) | | `ca.config-contract` | app-bootstrap's `verifyEnvKeys` (moved from the deleted build-tools included build; owner decision 4 keeps the task in app-bootstrap) | diff --git a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/ConfigContractPlugin.java b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/ConfigContractPlugin.java index 19aa949a..bf045466 100644 --- a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/ConfigContractPlugin.java +++ b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/ConfigContractPlugin.java @@ -45,8 +45,10 @@ public final class ConfigContractPlugin implements Plugin { // What an operator edits or runs: the root Compose files, the Logback // configuration, and the Compose lane contract with the smoke wrapper that drives it // (each lane's activationEnv, the wrapper's CAPABILITIES_OFF). Their APP_ names must - // be registered and read (07-10; RR-06-07: an unread CASKELETON_GRPC_ENABLED-style - // switch in a lane was found by hand, not by this task). + // be registered and read (07-10, RR-06-07), and a lane's other variables must be read + // too — a Spring Boot namespace or a secret://…/ target — so an unread + // CASKELETON_GRPC_ENABLED-style switch fails here (RR2-04-05) instead of being found + // by hand. task.getOperatorFiles() .from( root.fileTree( diff --git a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/EnvContractVerifier.java b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/EnvContractVerifier.java index c6cd6192..c3ef68b9 100644 --- a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/EnvContractVerifier.java +++ b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/EnvContractVerifier.java @@ -42,6 +42,17 @@ public final class EnvContractVerifier { * argument the wrapper passes on) is not a key and is not matched. */ private static final Pattern JSON_KEY = Pattern.compile("\"(APP_[A-Z][A-Z0-9_]*)\"\\s*:"); + /** Every variable a JSON object sets, {@code APP_} or not. */ + private static final Pattern JSON_VARIABLE_KEY = Pattern.compile("\"([A-Z][A-Z0-9_]*)\"\\s*:"); + /** The variable a credential reference names: the last segment of {@code secret://…/}. */ + private static final Pattern REFERENCED_VARIABLE = + Pattern.compile("secret://[^\\s\"']*/([A-Z][A-Z0-9_]*)"); + /** + * Spring Boot's own namespaces: a variable under one is bound to its {@code spring.*}, {@code + * management.*} or {@code server.*} property by relaxed binding, so the framework reads it. + */ + private static final List FRAMEWORK_VARIABLE_PREFIXES = + List.of("SPRING_", "MANAGEMENT_", "SERVER_"); /** * A list element ({@code APP_HTTPCLIENT_CLIENTS_0_NAME}). Its field template is registered by the * owning module's field registry ({@code docs/httpclient/env-fields.yaml}, held by that module's @@ -92,6 +103,46 @@ public final class EnvContractVerifier { Collection metadataScopes, Collection enforcedPrefixes, Collection operatorFiles) { + Set declared = new TreeSet<>(); + for (MetadataScope scope : metadataScopes) { + if (scope.metadataFile().exists()) { + parseMetadata(scope.metadataFile()).properties().stream() + .map(SpringConfigurationProperty::name) + .forEach(declared::add); + } + } + return verify( + envFile, + configurationFiles, + registryFile, + productionSources, + metadataScopes, + enforcedPrefixes, + operatorFiles, + declared); + } + + /** + * Verifies the contract, with every configuration property the composition declares. + * + *

    An operator-file key whose registry row names a {@code property:} counts as read only when + * that property is declared — a registry row alone is what the operator file was already trusted + * for (review 2026-11-04 RR2-04-05). A non-{@code APP_} key of a JSON object counts as read only + * under a Spring Boot namespace or as the variable a {@code secret://…/} reference names: + * the {@code CASKELETON_GRPC_ENABLED} switch that started RR-06-07 was neither, and nothing read + * it. + * + * @param declaredProperties every property name the composition's configuration metadata declares + */ + public static EnvContractResult verify( + File envFile, + Collection configurationFiles, + File registryFile, + Collection productionSources, + Collection metadataScopes, + Collection enforcedPrefixes, + Collection operatorFiles, + Collection declaredProperties) { requireFile( envFile, "missing " @@ -209,22 +260,28 @@ public final class EnvContractVerifier { consumed.addAll(scanProductionSourceConsumers(productionSources)); Set operatorReferences = new TreeSet<>(); + Set unreadOperatorVariables = new TreeSet<>(); for (File operatorFile : operatorFiles) { requireFile(operatorFile, "missing operator file " + operatorFile); - Matcher matcher = operatorKeyPattern(operatorFile.getName()).matcher(read(operatorFile)); + String content = read(operatorFile); + Matcher matcher = operatorKeyPattern(operatorFile.getName()).matcher(content); while (matcher.find()) { if (!INDEXED_KEY.matcher(matcher.group(1)).matches()) { operatorReferences.add(matcher.group(1)); } } + if (operatorKeyPattern(operatorFile.getName()) == JSON_KEY) { + unreadOperatorVariables.addAll(unreadNonApplicationVariables(content)); + } } failUnlessEmpty( difference(operatorReferences, registryAppKeys), "operator files (Compose, logback-spring.xml, the Compose lane contract, the smoke wrapper)" + " set APP_ keys absent from docs/registries/env-keys.yaml: "); Set typedRows = new TreeSet<>(); + Set declared = new HashSet<>(declaredProperties); registryRows.stream() - .filter(row -> row.property().isPresent()) + .filter(row -> row.property().filter(declared::contains).isPresent()) .forEach(row -> typedRows.add(row.name())); Set unreadOperatorReferences = new TreeSet<>(); for (String name : operatorReferences) { @@ -237,6 +294,12 @@ public final class EnvContractVerifier { "operator files (Compose, logback-spring.xml, the Compose lane contract, the smoke" + " wrapper) set APP_ keys nothing reads — no shipped configuration placeholder," + " production source or typed property: "); + failUnlessEmpty( + unreadOperatorVariables, + "the Compose lane contract or the smoke wrapper sets variables outside APP_ that nothing" + + " reads — not a Spring Boot namespace (SPRING_, MANAGEMENT_, SERVER_) and not named" + + " by a secret://…/ reference: ", + " — a switch spelled this way changes nothing, and the lane can pass for another reason."); Set unconsumed = new TreeSet<>(); for (EnvRegistryRow row : registryRows) { @@ -278,6 +341,29 @@ public final class EnvContractVerifier { warnings); } + /** + * The non-{@code APP_} variables a JSON object in {@code content} sets that nothing reads: outside + * Spring Boot's namespaces and named by no {@code secret://} reference in the same file. + */ + private static Set unreadNonApplicationVariables(String content) { + Set referenced = new TreeSet<>(); + Matcher reference = REFERENCED_VARIABLE.matcher(content); + while (reference.find()) { + referenced.add(reference.group(1)); + } + Set unread = new TreeSet<>(); + Matcher variable = JSON_VARIABLE_KEY.matcher(content); + while (variable.find()) { + String name = variable.group(1); + if (!name.startsWith("APP_") + && FRAMEWORK_VARIABLE_PREFIXES.stream().noneMatch(name::startsWith) + && !referenced.contains(name)) { + unread.add(name); + } + } + return unread; + } + private static Pattern operatorKeyPattern(String fileName) { if (fileName.endsWith(".xml")) { return LOGBACK_SOURCE; diff --git a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/VerifyEnvKeysTask.java b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/VerifyEnvKeysTask.java index 9ff45763..9e98f3c8 100644 --- a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/VerifyEnvKeysTask.java +++ b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/config/VerifyEnvKeysTask.java @@ -83,7 +83,8 @@ public abstract class VerifyEnvKeysTask extends DefaultTask { getProductionSources().getFiles(), scopes, ENFORCED_PREFIXES, - getOperatorFiles().getFiles()); + getOperatorFiles().getFiles(), + ConfigurationMetadataClasspath.propertiesUnder(getMetadataClasspath().getFiles(), "")); } catch (IllegalStateException invalidContract) { throw new GradleException(invalidContract.getMessage(), invalidContract); } diff --git a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/convention/SpringConfigConventionPlugin.java b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/convention/SpringConfigConventionPlugin.java index f6b98a04..40a641eb 100644 --- a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/convention/SpringConfigConventionPlugin.java +++ b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/convention/SpringConfigConventionPlugin.java @@ -2,12 +2,43 @@ package dev.caskeleton.buildlogic.convention; import org.gradle.api.Plugin; import org.gradle.api.Project; +import org.gradle.api.tasks.PathSensitivity; public final class SpringConfigConventionPlugin implements Plugin { + /** + * How a module declares a property the processor cannot see — a switch read only by + * {@code @ConditionalOnProperty} — so the environment contract finds it in the metadata. + */ + private static final String ADDITIONAL_METADATA = + "src/main/resources/META-INF/additional-spring-configuration-metadata.json"; + @Override public void apply(Project project) { project .getDependencies() .add("annotationProcessor", "org.springframework.boot:spring-boot-configuration-processor"); + // The processor merges the additional metadata from the processed resources, so compileJava + // runs after processResources and reruns when that one file changes. Without this it could run + // first, or stay up to date when only that file changed, and the declared switch was missing + // from the metadata verifyEnvKeys reads (review 2026-11-04 RR2-04-05). Only that file is an + // input: every other resource leaves compileJava's cache key alone. + project + .getPluginManager() + .withPlugin( + "java", + ignored -> + project + .getTasks() + .named( + "compileJava", + compile -> { + compile.dependsOn(project.getTasks().named("processResources")); + // files(), not file(): most modules have no such file. + compile + .getInputs() + .files(project.file(ADDITIONAL_METADATA)) + .withPropertyName("additionalConfigurationMetadata") + .withPathSensitivity(PathSensitivity.RELATIVE); + })); } } diff --git a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesExtension.java b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesExtension.java index ac09e803..6a140481 100644 --- a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesExtension.java +++ b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesExtension.java @@ -2,7 +2,6 @@ package dev.caskeleton.buildlogic.strictlane; import org.gradle.api.file.DirectoryProperty; import org.gradle.api.provider.ListProperty; -import org.gradle.api.provider.Property; /** * The stages a strict lane may belong to, and where the workflows that run them live. @@ -14,15 +13,14 @@ import org.gradle.api.provider.Property; public abstract class StrictLaneStagesExtension { public abstract ListProperty getStages(); - public abstract DirectoryProperty getWorkflowDirectory(); - /** - * The {@code -p} directory a workflow step passes to start this build, relative to the step's - * working directory — {@code optional-platforms} for the optional build, which steps run as - * {@code ./gradlew -p optional-platforms } from {@code src}. Unset for the build a step - * starts in its working directory. Only commands that start this build execute its stages. + * The repository's {@code .github/workflows}. Its grandparent is the repository root, and a + * command executes this build's stages only when it starts in this build's directory from there + * — its step's {@code working-directory} joined with its {@code -p}. The directory is read from + * where the build is, so no build states it (review 2026-11-04 RR2-04-04: a {@code -p} value + * alone missed {@code working-directory: src/optional-platforms} with {@code ../gradlew}). */ - public abstract Property getWorkflowProjectDirectory(); + public abstract DirectoryProperty getWorkflowDirectory(); public void stage(String... taskPaths) { getStages().addAll(taskPaths); diff --git a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesPlugin.java b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesPlugin.java index 64bcef82..0cf9de12 100644 --- a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesPlugin.java +++ b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesPlugin.java @@ -30,16 +30,19 @@ import org.gradle.api.execution.TaskExecutionGraph; * *

      *
    1. every strict lane that is not {@code explicitOnly} is in the task plan of a declared stage - * as a workflow runs it — a lane the command removes with {@code -x} is not covered; + * as a workflow runs it — a lane the command removes with {@code -x} is not covered, matched + * the way Gradle matches it, abbreviations included ({@link TaskSelectors}); *
    2. every declared stage is executed by a workflow — a {@code --dry-run}, a command for another - * build, and a command under {@code continue-on-error} or an {@code if:} other than a status - * function do not count (review 2026-11-03 RR-06-04). + * build, a command under {@code continue-on-error} or an {@code if:} other than a status + * function, and a command of a workflow that starts only on {@code workflow_dispatch} do not + * count (review 2026-11-03 RR-06-04, 2026-11-04 RR2-04-04). *
    * *

    Each build that holds strict lanes applies it at its own root: the main build, and the - * optional-platforms build, whose stages workflows start with {@code -p optional-platforms} - * ({@link StrictLaneStagesExtension#getWorkflowProjectDirectory()}). Applied to the main build only, - * it said "every strict lane" while the optional build's performance lane ran nowhere (RR-06-03).

    + * optional-platforms build. A command executes this build when it starts in this build's directory + * — its step's {@code working-directory} joined with its {@code -p}, from the repository root, + * which is where GitHub keeps {@code .github/workflows}. Applied to the main build only, it said + * "every strict lane" while the optional build's performance lane ran nowhere (RR-06-03).

    * *

    The task plan is Gradle's own: {@code verifyStrictLaneStages} depends on the stages, so a * {@code --dry-run} of it builds the real plan, and the check reads it when the graph is ready. It @@ -85,13 +88,14 @@ public final class StrictLaneStagesPlugin implements Plugin { + extension.getStages().get() + " and would execute all of them; run it with --dry-run."); } + File workflowDirectory = extension.getWorkflowDirectory().get().getAsFile(); List problems = problems( project, graph, extension.getStages().get(), - extension.getWorkflowDirectory().get().getAsFile(), - extension.getWorkflowProjectDirectory().getOrNull()); + workflowDirectory, + buildDirectory(project, workflowDirectory)); if (!problems.isEmpty()) { throw new GradleException( TASK_NAME @@ -119,6 +123,10 @@ public final class StrictLaneStagesPlugin implements Plugin { List invocations = invocations(workflowDirectory); // Only an executed stage covers a lane — a stage no workflow runs is the same gap one level // up — and only as the command runs it: what its -x removes is not covered. + Map> tasksByProject = new LinkedHashMap<>(); + root.getAllprojects() + .forEach( + project -> tasksByProject.put(project.getPath(), Set.copyOf(project.getTasks().getNames()))); Map> plans = new LinkedHashMap<>(); for (String stage : stages) { List executions = @@ -131,17 +139,20 @@ public final class StrictLaneStagesPlugin implements Plugin { + stage + " is executed by no workflow in " + workflowDirectory.getName() - + " (a --dry-run, a command for another build, and a command under" - + " continue-on-error or an if: other than always()/success()/!cancelled() do not" - + " count)"); + + " for the build in " + + buildDirectory + + " (a --dry-run, a command started in another directory, a command under" + + " continue-on-error or an if: other than always()/success()/!cancelled(), and a" + + " workflow that starts only on workflow_dispatch do not count)"); continue; } Task stageTask = root.getTasks().findByPath(stage); int run = 0; for (WorkflowGradleInvocations.Invocation execution : executions) { + Set excluded = + TaskSelectors.excludedPaths(execution.excludedTasks(), tasksByProject); plans.put( - stage + "#" + run++, - stageTask == null ? Set.of() : plan(graph, stageTask, execution)); + stage + "#" + run++, stageTask == null ? Set.of() : plan(graph, stageTask, excluded)); } } List lanes = new ArrayList<>(); @@ -168,14 +179,13 @@ public final class StrictLaneStagesPlugin implements Plugin { return problems; } - /** The tasks {@code stage} reaches as {@code execution} runs it: what its -x removes is cut off. */ - private static Set plan( - TaskExecutionGraph graph, Task stage, WorkflowGradleInvocations.Invocation execution) { + /** The tasks {@code stage} reaches as a command runs it: what its -x removes is cut off. */ + private static Set plan(TaskExecutionGraph graph, Task stage, Set excluded) { Set seen = new HashSet<>(); Deque pending = new ArrayDeque<>(List.of(stage)); while (!pending.isEmpty()) { Task task = pending.pop(); - if (!execution.excludes(task.getPath()) && seen.add(task.getPath())) { + if (!excluded.contains(task.getPath()) && seen.add(task.getPath())) { pending.addAll(graph.getDependencies(task)); } } @@ -222,6 +232,22 @@ public final class StrictLaneStagesPlugin implements Plugin { return invocations; } + /** + * This build's directory from the repository root, which holds {@code .github/workflows}: {@code + * src} for the main build, {@code src/optional-platforms} for the optional one. + */ + private static String buildDirectory(Project root, File workflowDirectory) { + File repository = workflowDirectory.getAbsoluteFile().getParentFile().getParentFile(); + String relative = + repository + .toPath() + .normalize() + .relativize(root.getRootDir().getAbsoluteFile().toPath().normalize()) + .toString() + .replace(File.separatorChar, '/'); + return WorkflowGradleInvocations.Invocation.normalized(relative); + } + private static String taskPath(Project project, String name) { return project == project.getRootProject() ? ":" + name : project.getPath() + ":" + name; } diff --git a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/TaskSelectors.java b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/TaskSelectors.java new file mode 100644 index 00000000..6dad58eb --- /dev/null +++ b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/TaskSelectors.java @@ -0,0 +1,214 @@ +package dev.caskeleton.buildlogic.strictlane; + +import java.util.Collection; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Set; +import java.util.TreeSet; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +/** + * Which tasks a command line's {@code -x} selectors remove, matched the way Gradle matches them. + * + *

    Gradle accepts an abbreviation wherever it accepts a task or project name: {@code -x + * grpcPerformanceT} removes {@code grpcPerformanceTest}, and {@code :g-t:gPT} a task of {@code + * :grpc-testkit}. The workflow reader compared names exactly, so an abbreviated {@code -x} — the + * form people type in a terminal and paste into a workflow — removed a lane from Gradle's plan while + * the stage check still counted it (review 2026-11-04 RR2-04-04). + * + *

    The matching is a copy of Gradle's {@code org.gradle.util.internal.NameMatcher} (9.0): exact, + * then case-insensitive, then camel-case, then kebab-case abbreviation. It is copied rather than + * called because that class is internal API. Where Gradle would refuse the selector as ambiguous, + * every candidate counts as removed: the command cannot run, so it covers none of them. + */ +final class TaskSelectors { + private static final Pattern CAMEL_CASE_BOUNDARY = + Pattern.compile( + "((^|\\p{Punct})\\p{javaLowerCase}+)|((\\p{javaUpperCase}|\\p{Digit})\\p{javaLowerCase}*)"); + private static final Pattern KEBAB_BOUNDARY = + Pattern.compile("((^|\\p{Punct})\\p{javaLowerCase}+)|(\\p{javaUpperCase}\\p{javaLowerCase}*)"); + + private TaskSelectors() {} + + /** + * The paths of the tasks the selectors remove from a command started at the root project. + * + * @param selectors the {@code -x} values, as written + * @param tasksByProject each project's path ({@code :} for the root) and its task names + * @return the removed task paths + */ + static Set excludedPaths( + List selectors, Map> tasksByProject) { + Set excluded = new LinkedHashSet<>(); + for (String selector : selectors) { + excluded.addAll(selected(selector, tasksByProject)); + } + return excluded; + } + + private static Set selected( + String selector, Map> tasksByProject) { + Set paths = new LinkedHashSet<>(); + if (!selector.contains(":")) { + // A bare name is matched against every task name in the build and removes the task of that + // name in every project. + Set names = new TreeSet<>(); + tasksByProject.values().forEach(names::addAll); + for (String name : matches(selector, names)) { + tasksByProject.forEach( + (project, tasks) -> { + if (tasks.contains(name)) { + paths.add(taskPath(project, name)); + } + }); + } + return paths; + } + String relative = selector.startsWith(":") ? selector.substring(1) : selector; + String[] segments = relative.split(":", -1); + Set projects = Set.of(":"); + for (int index = 0; index < segments.length - 1; index++) { + Set next = new LinkedHashSet<>(); + for (String project : projects) { + for (String child : matches(segments[index], children(project, tasksByProject.keySet()))) { + next.add(project.equals(":") ? ":" + child : project + ":" + child); + } + } + projects = next; + } + String taskName = segments[segments.length - 1]; + for (String project : projects) { + Collection tasks = tasksByProject.get(project); + for (String name : matches(taskName, tasks == null ? List.of() : tasks)) { + paths.add(taskPath(project, name)); + } + } + return paths; + } + + private static Set children(String project, Set projects) { + String prefix = project.equals(":") ? ":" : project + ":"; + Set children = new TreeSet<>(); + for (String candidate : projects) { + if (candidate.startsWith(prefix) && candidate.length() > prefix.length()) { + String rest = candidate.substring(prefix.length()); + if (!rest.contains(":")) { + children.add(rest); + } + } + } + return children; + } + + private static String taskPath(String project, String name) { + return project.equals(":") ? ":" + name : project + ":" + name; + } + + /** + * The names {@code pattern} selects among {@code items}: one when Gradle would accept it, several + * when Gradle would refuse it as ambiguous, none when nothing matches. + */ + static Set matches(String pattern, Collection items) { + if (items.contains(pattern)) { + return Set.of(pattern); + } + if (pattern.isEmpty()) { + return Set.of(); + } + Pattern camelCase = Pattern.compile(camelCasePattern(pattern)); + Pattern camelCaseIgnoringCase = + Pattern.compile(camelCasePattern(pattern), Pattern.CASE_INSENSITIVE | Pattern.UNICODE_CASE); + String kebabCasePattern = kebabCasePattern(pattern); + Pattern kebabCase = Pattern.compile(kebabCasePattern); + Pattern kebabCasePrefix = Pattern.compile(kebabCasePattern + "[\\p{javaLowerCase}\\p{Digit}-]*"); + Set ignoringCase = new TreeSet<>(); + Set camelCaseWhole = new TreeSet<>(); + Set camelCaseStart = new TreeSet<>(); + Set camelCaseStartIgnoringCase = new TreeSet<>(); + Set kebabCaseWhole = new TreeSet<>(); + Set kebabCaseStart = new TreeSet<>(); + for (String candidate : items) { + if (candidate.equalsIgnoreCase(pattern)) { + ignoringCase.add(candidate); + } + if (camelCase.matcher(candidate).matches()) { + camelCaseWhole.add(candidate); + } + if (camelCase.matcher(candidate).lookingAt()) { + camelCaseStart.add(candidate); + } + if (camelCaseIgnoringCase.matcher(candidate).lookingAt()) { + camelCaseStartIgnoringCase.add(candidate); + } + if (kebabCase.matcher(candidate).matches()) { + kebabCaseWhole.add(candidate); + } + if (kebabCasePrefix.matcher(candidate).matches()) { + kebabCaseStart.add(candidate); + } + } + Set matches = new TreeSet<>(); + if (!ignoringCase.isEmpty()) { + matches.addAll(ignoringCase); + } else if (!camelCaseWhole.isEmpty()) { + matches.addAll(camelCaseWhole); + } else if (!camelCaseStart.isEmpty()) { + matches.addAll(camelCaseStart); + } else if (kebabCaseWhole.isEmpty() && kebabCaseStart.isEmpty()) { + matches.addAll(camelCaseStartIgnoringCase); + } + if (!kebabCaseWhole.isEmpty()) { + matches.addAll(kebabCaseWhole); + } else if (!kebabCaseStart.isEmpty()) { + matches.addAll(kebabCaseStart); + } + return matches; + } + + private static String camelCasePattern(String name) { + return pattern( + name, + CAMEL_CASE_BOUNDARY, + (builder, position, part) -> + builder.append(Pattern.quote(part)).append("[\\p{javaLowerCase}]*")); + } + + private static String kebabCasePattern(String name) { + return pattern( + name, + KEBAB_BOUNDARY, + (builder, position, part) -> { + if (position > 0) { + builder.append('-'); + } + builder + .append(Pattern.quote(part.toLowerCase(Locale.ROOT))) + .append("[\\p{javaLowerCase}\\p{Digit}]*"); + }); + } + + private interface PartWriter { + void write(StringBuilder builder, int position, String part); + } + + private static String pattern(String name, Pattern boundary, PartWriter writer) { + Matcher matcher = boundary.matcher(name); + int position = 0; + StringBuilder builder = new StringBuilder(); + while (matcher.find()) { + String prefix = name.substring(position, matcher.start()); + if (!prefix.isEmpty()) { + builder.append(Pattern.quote(prefix)); + } + writer.write(builder, position, matcher.group()); + position = matcher.end(); + } + if (position < name.length()) { + builder.append(Pattern.quote(name.substring(position))); + } + return builder.toString(); + } +} diff --git a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/WorkflowGradleInvocations.java b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/WorkflowGradleInvocations.java index 22dc2291..b4d0e0d8 100644 --- a/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/WorkflowGradleInvocations.java +++ b/src/build-logic/src/main/java/dev/caskeleton/buildlogic/strictlane/WorkflowGradleInvocations.java @@ -1,6 +1,9 @@ package dev.caskeleton.buildlogic.strictlane; +import java.util.ArrayDeque; import java.util.ArrayList; +import java.util.Deque; +import java.util.LinkedHashSet; import java.util.List; import java.util.Locale; import java.util.Set; @@ -19,7 +22,14 @@ import java.util.regex.Pattern; *

    A command is read with what decides whether it gates anything (review 2026-11-03 RR-06-04): * its {@code -x} exclusions, and the {@code continue-on-error} and {@code if:} of its step and its * job. A command whose failure cannot fail the run, or that runs only under a condition other than - * a status function, is still returned, with the reason it does not count. + * a status function, is still returned, with the reason it does not count. So is a command of a + * workflow that starts only on {@code workflow_dispatch}: nothing runs it unless someone asks, the + * same reason an {@code if:} on the event name does not count (review 2026-11-04 RR2-04-04). + * + *

    Which build a command starts is where it starts: the step's {@code working-directory} (or the + * job's or the workflow's {@code defaults.run.working-directory}) joined with its {@code -p}, as a + * path from the repository root. A {@code run:} that is an input of a {@code uses:} step ({@code + * with: run:}) is not a shell command and is not read. */ public final class WorkflowGradleInvocations { private static final Pattern RUN_KEY = Pattern.compile("^(\\s*)(?:-\\s+)?run:(.*)$"); @@ -27,6 +37,8 @@ public final class WorkflowGradleInvocations { private static final Pattern MAPPING_KEY = Pattern.compile("^(\\s*)(?:-\\s+)?([A-Za-z0-9_-]+):(?:\\s+(.*?))?\\s*$"); private static final Pattern JOB_KEY = Pattern.compile("^(\\s*)([A-Za-z0-9_-]+):\\s*(#.*)?$"); + /** The top-level {@code on:} key, which YAML 1.1 readers want quoted. */ + private static final Pattern TRIGGER_KEY = Pattern.compile("^(?:on|\"on\"|'on'):(?:\\s+(.*?))?\\s*$"); /** Conditions under which a step still runs on every run whose earlier steps passed. */ private static final Set STATUS_CONDITIONS = Set.of("always()", "success()", "!cancelled()", "true"); @@ -54,6 +66,8 @@ public final class WorkflowGradleInvocations { /** * One Gradle command. * + * @param workingDirectory the directory the step runs in, from the repository root, or {@code + * null} for the repository root itself * @param projectDir the {@code -p} directory, or {@code null} for the build in the step's working * directory * @param tasks the task selectors it requests, in order @@ -63,6 +77,7 @@ public final class WorkflowGradleInvocations { * continue-on-error}, or a condition other than a status function — or {@code null} */ public record Invocation( + String workingDirectory, String projectDir, List tasks, boolean dryRun, @@ -74,50 +89,43 @@ public final class WorkflowGradleInvocations { } /** - * Whether this command executes the task at {@code taskPath} (for example {@code :ci}) in the - * build its step starts in. {@code ci} and {@code :ci} name the same root task. + * The directory of the build this command starts, from the repository root: the step's working + * directory joined with {@code -p}. {@code .} is the repository root. */ - public boolean executesInStartBuild(String taskPath) { - return executes(taskPath, null); + public String buildDirectory() { + return normalized( + (workingDirectory == null ? "." : workingDirectory) + + "/" + + (projectDir == null ? "." : projectDir)); } /** - * Whether this command executes, and gates on, the task at {@code taskPath} of the build a - * workflow starts with {@code -p buildDirectory} ({@code null}: the build in the step's working - * directory). + * Whether this command executes, and gates on, the task at {@code taskPath} (for example {@code + * :ci}; {@code ci} names the same root task) of the build in {@code buildDirectory}, a path from + * the repository root such as {@code src}. */ public boolean executes(String taskPath, String buildDirectory) { String wanted = absolute(taskPath); - return sameDirectory(projectDir, buildDirectory) + return buildDirectory().equals(normalized(buildDirectory)) && !dryRun && uncountedReason == null && tasks.stream().map(Invocation::absolute).anyMatch(wanted::equals); } - /** - * Whether the task at {@code taskPath} is one this command's {@code -x} removes. An absolute - * or project-qualified selector names one task; a bare name removes that task in every project, - * as Gradle does for a command started at the root. - */ - public boolean excludes(String taskPath) { - String name = taskPath.substring(taskPath.lastIndexOf(':') + 1); - for (String selector : excludedTasks) { - if (selector.contains(":") ? absolute(selector).equals(taskPath) : selector.equals(name)) { - return true; + /** A relative path with {@code .} and {@code ..} resolved; {@code .} for the root itself. */ + static String normalized(String directory) { + Deque parts = new ArrayDeque<>(); + for (String part : directory.split("/")) { + if (part.isEmpty() || part.equals(".")) { + continue; + } + if (part.equals("..") && !parts.isEmpty() && !parts.peekLast().equals("..")) { + parts.removeLast(); + } else { + parts.addLast(part); } } - return false; - } - - private static boolean sameDirectory(String left, String right) { - return left == null || right == null - ? left == null && right == null - : normalized(left).equals(normalized(right)); - } - - private static String normalized(String directory) { - String plain = directory.startsWith("./") ? directory.substring(2) : directory; - return plain.endsWith("/") ? plain.substring(0, plain.length() - 1) : plain; + return parts.isEmpty() ? "." : String.join("/", parts); } private static String absolute(String selector) { @@ -125,8 +133,11 @@ public final class WorkflowGradleInvocations { } } - /** The script of one {@code run:} step, and why its commands gate nothing, if they do not. */ - private record RunStep(String script, String uncountedReason) {} + /** + * The script of one {@code run:} step, the directory it runs in, and why its commands gate + * nothing, if they do not. + */ + private record RunStep(String script, String workingDirectory, String uncountedReason) {} private WorkflowGradleInvocations() {} @@ -134,7 +145,8 @@ public final class WorkflowGradleInvocations { List invocations = new ArrayList<>(); for (RunStep step : runSteps(workflow)) { for (String command : shellCommands(step.script())) { - Invocation invocation = gradleInvocation(command, step.uncountedReason()); + Invocation invocation = + gradleInvocation(command, step.workingDirectory(), step.uncountedReason()); if (invocation != null) { invocations.add(invocation); } @@ -157,7 +169,7 @@ public final class WorkflowGradleInvocations { List problems = new ArrayList<>(); for (int index = 0; index < lines.size(); index++) { Matcher run = RUN_KEY.matcher(lines.get(index)); - if (!run.matches()) { + if (!run.matches() || !isStepRun(lines, index)) { continue; } Matcher header = BLOCK_HEADER.matcher(run.group(2).trim()); @@ -197,17 +209,22 @@ public final class WorkflowGradleInvocations { private static List runSteps(String workflow) { List lines = workflow.lines().toList(); List scripts = new ArrayList<>(); + String dispatchOnly = dispatchOnlyReason(lines); for (int index = 0; index < lines.size(); index++) { Matcher run = RUN_KEY.matcher(lines.get(index)); - if (!run.matches()) { + if (!run.matches() || !isStepRun(lines, index)) { continue; } int keyIndent = run.group(1).length(); String value = run.group(2).trim(); String uncounted = uncountedReason(lines, index); + if (uncounted == null) { + uncounted = dispatchOnly; + } + String directory = workingDirectory(lines, index); Matcher header = BLOCK_HEADER.matcher(value); if (!header.matches()) { - scripts.add(new RunStep(stripScalarComment(value), uncounted)); + scripts.add(new RunStep(stripScalarComment(value), directory, uncounted)); continue; } List block = new ArrayList<>(); @@ -224,25 +241,182 @@ public final class WorkflowGradleInvocations { block.stream().map(line -> line.isBlank() ? "" : line.substring(blockIndent)).toList(); scripts.add( new RunStep( - header.group(1).equals("|") ? String.join("\n", content) : fold(content), uncounted)); + header.group(1).equals("|") ? String.join("\n", content) : fold(content), + directory, + uncounted)); } return scripts; } /** - * Why the {@code run:} key at {@code runIndex} gates nothing: a {@code continue-on-error} other - * than {@code false}, or an {@code if:} other than a status function, on its step or its job. + * Whether the {@code run:} key at {@code index} is a step's: a key of a {@code steps:} item, not + * an input under a {@code uses:} step's {@code with:} or a {@code defaults.run} mapping. */ - private static String uncountedReason(List lines, int runIndex) { - Matcher run = RUN_KEY.matcher(lines.get(runIndex)); - if (!run.matches()) { + private static boolean isStepRun(List lines, int index) { + String line = lines.get(index); + if (line.trim().startsWith("-")) { + return true; + } + int keyIndent = indentOf(line); + for (int previous = index - 1; previous >= 0; previous--) { + String candidate = lines.get(previous); + if (candidate.isBlank() || candidate.trim().startsWith("#")) { + continue; + } + if (indentOf(candidate) < keyIndent) { + return candidate.trim().startsWith("-") && keyColumn(candidate) == keyIndent; + } + } + return false; + } + + /** + * Why every command of the workflow gates nothing, when it starts only on {@code + * workflow_dispatch}; {@code null} when it has another trigger or none is written. + */ + private static String dispatchOnlyReason(List lines) { + Set triggers = new LinkedHashSet<>(); + for (int index = 0; index < lines.size(); index++) { + Matcher on = TRIGGER_KEY.matcher(lines.get(index)); + if (!on.matches()) { + continue; + } + String inline = + on.group(1) == null || on.group(1).startsWith("#") ? "" : stripScalarComment(on.group(1)); + if (!inline.isEmpty()) { + String list = + inline.startsWith("[") && inline.endsWith("]") + ? inline.substring(1, inline.length() - 1) + : inline; + for (String trigger : list.split(",")) { + if (!trigger.isBlank()) { + triggers.add(stripScalarComment(trigger.trim())); + } + } + break; + } + int childIndent = -1; + for (int next = index + 1; next < lines.size(); next++) { + String line = lines.get(next); + if (line.isBlank() || line.trim().startsWith("#")) { + continue; + } + int indent = indentOf(line); + if (indent == 0) { + break; + } + if (childIndent < 0) { + childIndent = indent; + } + if (indent == childIndent) { + String key = line.trim(); + key = key.startsWith("-") ? key.substring(1).trim() : key; + int colon = key.indexOf(':'); + triggers.add(stripScalarComment(colon < 0 ? key : key.substring(0, colon))); + } + } + break; + } + return !triggers.isEmpty() && triggers.stream().allMatch("workflow_dispatch"::equals) + ? "its workflow starts only on workflow_dispatch, so nothing runs it unless asked" + : null; + } + + /** + * The directory the {@code run:} at {@code runIndex} runs in: its step's {@code + * working-directory}, else its job's, else the workflow's {@code defaults.run.working-directory}; + * {@code null} for the repository root. + */ + private static String workingDirectory(List lines, int runIndex) { + StepScope step = stepScope(lines, runIndex); + String directory = mappingValue(lines, step.start(), step.keyIndent(), "working-directory"); + if (directory != null) { + return directory; + } + int jobHeader = jobHeader(lines, runIndex); + if (jobHeader >= 0) { + int jobKeyIndent = childIndent(lines, jobHeader); + if (jobKeyIndent > indentOf(lines.get(jobHeader))) { + directory = defaultsWorkingDirectory(lines, jobHeader + 1, jobKeyIndent); + if (directory != null) { + return directory; + } + } + } + return defaultsWorkingDirectory(lines, 0, 0); + } + + /** {@code defaults.run.working-directory} of the mapping whose keys sit at {@code keyIndent}. */ + private static String defaultsWorkingDirectory(List lines, int start, int keyIndent) { + int defaults = mappingKey(lines, start, keyIndent, "defaults"); + if (defaults < 0) { return null; } + int runIndent = childIndent(lines, defaults); + int run = runIndent > keyIndent ? mappingKey(lines, defaults + 1, runIndent, "run") : -1; + if (run < 0) { + return null; + } + int directoryIndent = childIndent(lines, run); + return directoryIndent > runIndent + ? mappingValue(lines, run + 1, directoryIndent, "working-directory") + : null; + } + + /** The value of {@code key} in the mapping at {@code keyIndent} starting at {@code start}. */ + private static String mappingValue(List lines, int start, int keyIndent, String key) { + int index = mappingKey(lines, start, keyIndent, key); + if (index < 0) { + return null; + } + Matcher matcher = MAPPING_KEY.matcher(lines.get(index)); + return matcher.matches() && matcher.group(3) != null + ? stripScalarComment(matcher.group(3)) + : null; + } + + /** The line of {@code key} in the mapping at {@code keyIndent} starting at {@code start}, or -1. */ + private static int mappingKey(List lines, int start, int keyIndent, String key) { + for (int index = start; index < lines.size(); index++) { + String line = lines.get(index); + if (line.isBlank() || line.trim().startsWith("#")) { + continue; + } + if (index > start && indentOf(line) < keyIndent) { + break; + } + if (index > start && keyColumn(line) == keyIndent && line.trim().startsWith("-")) { + break; + } + Matcher matcher = MAPPING_KEY.matcher(line); + if (matcher.matches() && keyColumn(line) == keyIndent && matcher.group(2).equals(key)) { + return index; + } + } + return -1; + } + + /** The indentation of the first content line after {@code index}. */ + private static int childIndent(List lines, int index) { + for (int next = index + 1; next < lines.size(); next++) { + String line = lines.get(next); + if (!line.isBlank() && !line.trim().startsWith("#")) { + return indentOf(line); + } + } + return -1; + } + + /** Where the step holding the {@code run:} at {@code runIndex} starts, and its key column. */ + private record StepScope(int start, int keyIndent) {} + + private static StepScope stepScope(List lines, int runIndex) { + Matcher run = RUN_KEY.matcher(lines.get(runIndex)); boolean itemLine = lines.get(runIndex).trim().startsWith("-"); int stepKeyIndent = itemLine ? lines.get(runIndex).indexOf("run:") - : run.group(1).length(); + : run.matches() ? run.group(1).length() : indentOf(lines.get(runIndex)); int stepStart = runIndex; if (!itemLine) { while (stepStart > 0) { @@ -256,7 +430,16 @@ public final class WorkflowGradleInvocations { } } } - String step = gatingProblem(lines, stepStart, stepKeyIndent, "step"); + return new StepScope(stepStart, stepKeyIndent); + } + + /** + * Why the {@code run:} key at {@code runIndex} gates nothing: a {@code continue-on-error} other + * than {@code false}, or an {@code if:} other than a status function, on its step or its job. + */ + private static String uncountedReason(List lines, int runIndex) { + StepScope scope = stepScope(lines, runIndex); + String step = gatingProblem(lines, scope.start(), scope.keyIndent(), "step"); if (step != null) { return step; } @@ -264,14 +447,7 @@ public final class WorkflowGradleInvocations { if (jobHeader < 0) { return null; } - int jobKeyIndent = -1; - for (int next = jobHeader + 1; next < lines.size(); next++) { - String line = lines.get(next); - if (!line.isBlank() && !line.trim().startsWith("#")) { - jobKeyIndent = indentOf(line); - break; - } - } + int jobKeyIndent = childIndent(lines, jobHeader); return jobKeyIndent <= indentOf(lines.get(jobHeader)) ? null : gatingProblem(lines, jobHeader + 1, jobKeyIndent, "job"); @@ -397,7 +573,8 @@ public final class WorkflowGradleInvocations { return commands; } - private static Invocation gradleInvocation(String command, String uncountedReason) { + private static Invocation gradleInvocation( + String command, String workingDirectory, String uncountedReason) { List tokens = new ArrayList<>(); for (String token : command.trim().split("\\s+")) { if (token.startsWith("#")) { @@ -443,7 +620,7 @@ public final class WorkflowGradleInvocations { tasks.add(token); } } - return new Invocation(projectDir, tasks, dryRun, excluded, uncountedReason); + return new Invocation(workingDirectory, projectDir, tasks, dryRun, excluded, uncountedReason); } private static String unquote(String token) { diff --git a/src/build-logic/src/test/java/dev/caskeleton/buildlogic/config/EnvContractVerifierTest.java b/src/build-logic/src/test/java/dev/caskeleton/buildlogic/config/EnvContractVerifierTest.java index a14483c5..a70e2ade 100644 --- a/src/build-logic/src/test/java/dev/caskeleton/buildlogic/config/EnvContractVerifierTest.java +++ b/src/build-logic/src/test/java/dev/caskeleton/buildlogic/config/EnvContractVerifierTest.java @@ -273,6 +273,58 @@ class EnvContractVerifierTest { assertTrue(!failure.getMessage().contains("APP_NAME"), failure.getMessage()); } + @Test + @DisplayName("lane activationEnv의 APP_ 밖 키는 Spring 이름공간도 비밀 참조 대상도 아니면 아무도 읽지 않으므로 실패한다") + void aLaneActivationKeyOutsideEveryReadNamespaceFails(@TempDir Path temp) throws Exception { + // given + Fixture fixture = fixture(temp); + Path contracts = fixture.repositoryRoot().resolve("compose-profile-contracts.json"); + Files.writeString( + contracts, + "{\"lanes\":[{\"id\":\"grpc\",\"activationEnv\":{" + + "\"CASKELETON_GRPC_ENABLED\":\"true\"," + + "\"SPRING_MAIL_HOST\":\"mailpit\"," + + "\"APP_REDIS_HOST\":\"secret://lane@environment/REDIS_LANE_APPLICATION_PASSWORD\"," + + "\"REDIS_LANE_APPLICATION_PASSWORD\":\"fixture-application\"}}]}"); + + // when + IllegalStateException failure = + assertThrows( + IllegalStateException.class, + () -> verifyWithOperatorFiles(fixture, contracts)); + + // then + assertTrue(failure.getMessage().contains("CASKELETON_GRPC_ENABLED"), failure.getMessage()); + assertTrue(!failure.getMessage().contains("SPRING_MAIL_HOST"), failure.getMessage()); + assertTrue( + !failure.getMessage().contains("REDIS_LANE_APPLICATION_PASSWORD"), failure.getMessage()); + } + + @Test + @DisplayName("레지스트리 행의 property가 어떤 설정 metadata에도 없으면 그 키는 읽히는 것으로 치지 않는다") + void anOperatorKeyWhoseRegisteredPropertyNothingDeclaresFails(@TempDir Path temp) + throws Exception { + // given + Fixture fixture = fixture(temp); + Files.writeString( + fixture.envFile(), Files.readString(fixture.envFile()) + "APP_GHOST_ENABLED=true\n"); + Files.writeString( + fixture.registry(), + Files.readString(fixture.registry()) + + "- name: APP_GHOST_ENABLED\n property: app.ghost.enabled\n type: boolean\n"); + Path compose = fixture.repositoryRoot().resolve("docker-compose.yml"); + Files.writeString( + compose, "services:\n app:\n environment:\n APP_GHOST_ENABLED: \"true\"\n"); + + // when + IllegalStateException failure = + assertThrows(IllegalStateException.class, () -> verifyWithOperatorFiles(fixture, compose)); + + // then + assertTrue(failure.getMessage().contains("nothing reads"), failure.getMessage()); + assertTrue(failure.getMessage().contains("APP_GHOST_ENABLED"), failure.getMessage()); + } + private static EnvContractResult verifyWithOperatorFiles(Fixture fixture, Path operatorFile) { return EnvContractVerifier.verify( fixture.envFile().toFile(), diff --git a/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesFunctionalTest.java b/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesFunctionalTest.java index 9c038036..3540d55a 100644 --- a/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesFunctionalTest.java +++ b/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/StrictLaneStagesFunctionalTest.java @@ -102,6 +102,9 @@ class StrictLaneStagesFunctionalTest { """); workflow( """ + defaults: + run: + working-directory: src jobs: main: steps: @@ -197,6 +200,9 @@ class StrictLaneStagesFunctionalTest { """); workflow( """ + defaults: + run: + working-directory: src jobs: main: steps: @@ -223,6 +229,9 @@ class StrictLaneStagesFunctionalTest { """); workflow( """ + defaults: + run: + working-directory: src jobs: gate: steps: @@ -258,6 +267,9 @@ class StrictLaneStagesFunctionalTest { """); workflow( """ + defaults: + run: + working-directory: src jobs: gate: steps: @@ -284,17 +296,37 @@ class StrictLaneStagesFunctionalTest { } @Test - @DisplayName("다른 빌드로 실행되는 빌드는 workflow가 -p 그 디렉터리로 부른 단계만 실행으로 센다") - void aBuildStartedWithDashPCountsOnlyCommandsThatNameItsDirectory() throws Exception { + @DisplayName("다른 빌드로 실행되는 빌드는 workflow가 그 디렉터리에서 시작한 단계만 실행으로 센다") + void aBuildStartedWithDashPCountsOnlyCommandsThatStartInItsDirectory() throws Exception { // given - leafLanes( + Path optional = gradleRoot.resolve("optional"); + Files.createDirectories(optional.resolve("leaf/src/test/java")); + Files.writeString( + optional.resolve("settings.gradle"), "rootProject.name = 'optional'\ninclude 'leaf'\n"); + Files.writeString( + optional.resolve("build.gradle"), """ - lane('databaseLane') { integration(); tag = 'db'; description = 'Real database.' } + plugins { id 'ca.strict-lane-stages' } + tasks.register('ci') { dependsOn ':leaf:check' } + tasks.register('integrationCheck') { dependsOn ':leaf:integrationTestLaneCheck' } + tasks.register('performanceCheck') { dependsOn ':leaf:check' } + strictLaneStages { + stage ':ci', ':integrationCheck', ':performanceCheck' + workflowDirectory = layout.projectDirectory.dir('../../.github/workflows') + } """); Files.writeString( - gradleRoot.resolve("build.gradle"), - Files.readString(gradleRoot.resolve("build.gradle")) - + "strictLaneStages { workflowProjectDirectory = 'optional' }\n"); + optional.resolve("leaf/build.gradle"), + """ + plugins { + id 'java' + id 'ca.strict-test-lane' + } + strictTestLanes { + lane('databaseLane') { integration(); tag = 'db'; description = 'Real database.' } + } + """); + Files.writeString(optional.resolve("leaf/src/test/java/LaneTest.java"), "class LaneTest {}\n"); workflow( """ jobs: @@ -302,6 +334,74 @@ class StrictLaneStagesFunctionalTest { steps: - working-directory: src run: ./gradlew -p optional ci --stacktrace + - working-directory: src/optional + run: ../gradlew performanceCheck + - working-directory: src + run: ./gradlew :integrationCheck + """); + + // when + BuildResult result = + GradleRunner.create() + .withProjectDir(optional.toFile()) + .withPluginClasspath() + .withArguments(":verifyStrictLaneStages", "--dry-run") + .buildAndFail(); + + // then + assertThat(result.getOutput()) + .contains("stage :integrationCheck is executed by no workflow") + .doesNotContain("stage :ci is executed by no workflow") + .doesNotContain("stage :performanceCheck is executed by no workflow"); + } + + @Test + @DisplayName("축약한 -x로 뺀 lane도 Gradle처럼 맞춰 보아 덮이지 않은 것으로 실패한다") + void aLaneExcludedWithAnAbbreviatedDashXIsNotCovered() throws Exception { + // given + leafLanes( + """ + lane('databaseLane') { integration(); tag = 'db'; description = 'Real database.' } + lane('cacheLane') { integration(); tag = 'cache'; description = 'Real cache.' } + """); + workflow( + """ + defaults: + run: + working-directory: src + jobs: + main: + steps: + - run: ./gradlew :ci --stacktrace + - run: ./gradlew :integrationCheck -x databaseL --continue + """); + + // when + BuildResult result = runner(":verifyStrictLaneStages", "--dry-run").buildAndFail(); + + // then + assertThat(result.getOutput()) + .contains("strict lane :leaf:databaseLane is in no executed stage") + .doesNotContain("strict lane :leaf:cacheLane"); + } + + @Test + @DisplayName("workflow_dispatch로만 시작하는 workflow가 부르는 단계는 실행으로 세지 않는다") + void aStageRunOnlyByADispatchOnlyWorkflowIsNotAnExecution() throws Exception { + // given + leafLanes( + """ + lane('databaseLane') { integration(); tag = 'db'; description = 'Real database.' } + """); + workflow(EXECUTING_WORKFLOW.replace("./gradlew :integrationCheck", "./gradlew :ci")); + Files.writeString( + repository.resolve(".github/workflows/manual.yml"), + """ + on: + workflow_dispatch: + jobs: + manual: + steps: - working-directory: src run: ./gradlew :integrationCheck """); @@ -312,7 +412,33 @@ class StrictLaneStagesFunctionalTest { // then assertThat(result.getOutput()) .contains("stage :integrationCheck is executed by no workflow") - .doesNotContain("stage :ci is executed by no workflow"); + .contains("strict lane :leaf:databaseLane is in no executed stage"); + } + + @Test + @DisplayName("다른 빌드의 디렉터리를 working-directory로 둔 gradlew는 이 빌드의 단계 실행으로 세지 않는다") + void aCommandStartedInAnotherBuildsDirectoryIsNotThisBuildsExecution() throws Exception { + // given + leafLanes( + """ + lane('databaseLane') { integration(); tag = 'db'; description = 'Real database.' } + """); + workflow( + """ + jobs: + main: + steps: + - working-directory: src + run: ./gradlew :ci --stacktrace + - working-directory: src/optional + run: ../gradlew :integrationCheck + """); + + // when + BuildResult result = runner(":verifyStrictLaneStages", "--dry-run").buildAndFail(); + + // then + assertThat(result.getOutput()).contains("stage :integrationCheck is executed by no workflow"); } private void leafLanes(String lanes) throws Exception { diff --git a/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/TaskSelectorsTest.java b/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/TaskSelectorsTest.java new file mode 100644 index 00000000..76a189dc --- /dev/null +++ b/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/TaskSelectorsTest.java @@ -0,0 +1,71 @@ +package dev.caskeleton.buildlogic.strictlane; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.util.List; +import java.util.Map; +import java.util.Set; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; + +class TaskSelectorsTest { + + private static final Map> BUILD = + Map.of( + ":", Set.of("ci", "performanceCheck", "test"), + ":grpc", Set.of("test"), + ":grpc:grpc-testkit", + Set.of("test", "testClasses", "grpcPerformanceTest", "grpcPerformanceTestClasses"), + ":leaf", Set.of("test", "databaseLane", "databaseLoadTest")); + + @Test + @DisplayName("이름이 정확히 같은 태스크가 있으면 그 이름만 모든 프로젝트에서 뺀다") + void anExactNameRemovesOnlyThatNameInEveryProject() { + // when + Set excluded = TaskSelectors.excludedPaths(List.of("test"), BUILD); + + // then + assertThat(excluded) + .containsExactlyInAnyOrder(":test", ":grpc:test", ":grpc:grpc-testkit:test", ":leaf:test"); + } + + @Test + @DisplayName("camel-case 축약 이름은 Gradle처럼 그 태스크를 뺀다") + void aCamelCaseAbbreviationRemovesTheTaskGradleWouldRemove() { + // when + Set excluded = TaskSelectors.excludedPaths(List.of("grpcPerformanceT"), BUILD); + + // then + assertThat(excluded).containsExactly(":grpc:grpc-testkit:grpcPerformanceTest"); + } + + @Test + @DisplayName("경로의 프로젝트 이름도 축약할 수 있다") + void aQualifiedSelectorMayAbbreviateItsProjectNames() { + // when + Set excluded = TaskSelectors.excludedPaths(List.of(":grpc:grpcT:gPT"), BUILD); + + // then + assertThat(excluded).containsExactly(":grpc:grpc-testkit:grpcPerformanceTest"); + } + + @Test + @DisplayName("Gradle이 모호하다고 거부할 축약은 후보 전부를 뺀 것으로 본다") + void anAmbiguousAbbreviationRemovesEveryCandidate() { + // when + Set excluded = TaskSelectors.excludedPaths(List.of("database"), BUILD); + + // then + assertThat(excluded).containsExactlyInAnyOrder(":leaf:databaseLane", ":leaf:databaseLoadTest"); + } + + @Test + @DisplayName("맞는 태스크가 없는 선택자는 아무것도 빼지 않는다") + void aSelectorThatMatchesNothingRemovesNothing() { + // when + Set excluded = TaskSelectors.excludedPaths(List.of(":leaf:cacheLane"), BUILD); + + // then + assertThat(excluded).isEmpty(); + } +} diff --git a/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/WorkflowGradleInvocationsTest.java b/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/WorkflowGradleInvocationsTest.java index 3bd5f940..81530956 100644 --- a/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/WorkflowGradleInvocationsTest.java +++ b/src/build-logic/src/test/java/dev/caskeleton/buildlogic/strictlane/WorkflowGradleInvocationsTest.java @@ -4,6 +4,8 @@ import static org.assertj.core.api.Assertions.assertThat; import dev.caskeleton.buildlogic.strictlane.WorkflowGradleInvocations.Invocation; import java.util.List; +import java.util.Map; +import java.util.Set; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; @@ -30,7 +32,7 @@ class WorkflowGradleInvocationsTest { assertThat(invocations).singleElement().satisfies( invocation -> { assertThat(invocation.tasks()).containsExactly(":mainCheck"); - assertThat(invocation.executesInStartBuild(":mainCheck")).isTrue(); + assertThat(invocation.executes(":mainCheck", "src")).isTrue(); }); } @@ -53,7 +55,7 @@ class WorkflowGradleInvocationsTest { // then assertThat(invocations) .isNotEmpty() - .noneMatch(invocation -> invocation.executesInStartBuild(":integrationCheck")); + .noneMatch(invocation -> invocation.executes(":integrationCheck", ".")); } @Test @@ -73,8 +75,8 @@ class WorkflowGradleInvocationsTest { // then assertThat(invocations) - .noneMatch(invocation -> invocation.executesInStartBuild(":integrationCheck")) - .anyMatch(invocation -> invocation.executesInStartBuild(":ci")); + .noneMatch(invocation -> invocation.executes(":integrationCheck", ".")) + .anyMatch(invocation -> invocation.executes(":ci", ".")); } @Test @@ -93,7 +95,8 @@ class WorkflowGradleInvocationsTest { assertThat(invocations).singleElement().satisfies( invocation -> { assertThat(invocation.projectDir()).isEqualTo("optional-platforms"); - assertThat(invocation.executesInStartBuild(":integrationCheck")).isFalse(); + assertThat(invocation.executes(":integrationCheck", ".")).isFalse(); + assertThat(invocation.executes(":integrationCheck", "optional-platforms")).isTrue(); }); } @@ -140,8 +143,8 @@ class WorkflowGradleInvocationsTest { // then assertThat(invocations).singleElement().satisfies( invocation -> { - assertThat(invocation.executesInStartBuild(":integrationCheck")).isTrue(); - assertThat(invocation.executesInStartBuild(":sample-portfolio:check")).isTrue(); + assertThat(invocation.executes(":integrationCheck", ".")).isTrue(); + assertThat(invocation.executes(":sample-portfolio:check", ".")).isTrue(); assertThat(invocation.tasks()).doesNotContain("*Contract*"); }); } @@ -211,10 +214,16 @@ class WorkflowGradleInvocationsTest { Invocation invocation = WorkflowGradleInvocations.read(workflow).get(0); // then - assertThat(invocation.excludes(":leaf:databaseLane")).isTrue(); - assertThat(invocation.excludes(":other:test")).isTrue(); - assertThat(invocation.excludes(":leaf:cacheLane")).isFalse(); - assertThat(invocation.executesInStartBuild(":integrationCheck")).isTrue(); + assertThat(invocation.excludedTasks()).containsExactly(":leaf:databaseLane", "test"); + assertThat( + TaskSelectors.excludedPaths( + invocation.excludedTasks(), + Map.of( + ":", Set.of("integrationCheck"), + ":leaf", Set.of("databaseLane", "cacheLane", "test"), + ":other", Set.of("test")))) + .containsExactlyInAnyOrder(":leaf:databaseLane", ":leaf:test", ":other:test"); + assertThat(invocation.executes(":integrationCheck", ".")).isTrue(); } @Test @@ -249,4 +258,124 @@ class WorkflowGradleInvocationsTest { reason -> assertThat(reason).contains("step runs only if: github.event_name"), reason -> assertThat(reason).isNull()); } + + @Test + @DisplayName("workflow_dispatch로만 시작하는 workflow의 명령은 아무 실행도 막지 못하므로 세지 않는다") + void aWorkflowThatOnlyRunsOnDispatchGatesNothing() { + // given + String workflow = + """ + on: + workflow_dispatch: + inputs: + lane: + type: string + jobs: + manual: + steps: + - working-directory: src + run: ./gradlew :integrationCheck + """; + + // when + List invocations = WorkflowGradleInvocations.read(workflow); + + // then + assertThat(invocations) + .singleElement() + .extracting(Invocation::uncountedReason) + .asString() + .contains("workflow_dispatch"); + } + + @Test + @DisplayName("workflow_dispatch 말고 다른 트리거도 있으면 명령을 실행으로 센다") + void aWorkflowWithAnotherTriggerBesidesDispatchCounts() { + // given + String workflow = + """ + on: [push, workflow_dispatch] + jobs: + main: + steps: + - working-directory: src + run: ./gradlew :integrationCheck + """; + + // when + List invocations = WorkflowGradleInvocations.read(workflow); + + // then + assertThat(invocations).singleElement().extracting(Invocation::uncountedReason).isNull(); + } + + @Test + @DisplayName("working-directory는 -p처럼 읽어서, 하위 디렉터리에서 부른 gradlew는 그 디렉터리의 빌드를 실행한다") + void workingDirectoryIsReadLikeDashP() { + // given + String workflow = + """ + jobs: + optional: + steps: + - working-directory: src/optional-platforms + run: ../gradlew ci --stacktrace + """; + + // when + List invocations = WorkflowGradleInvocations.read(workflow); + + // then + assertThat(invocations).singleElement().satisfies( + invocation -> { + assertThat(invocation.executes(":ci", "src/optional-platforms")).isTrue(); + assertThat(invocation.executes(":ci", "src")).isFalse(); + }); + } + + @Test + @DisplayName("job의 defaults.run.working-directory도 그 job 단계의 시작 디렉터리로 읽는다") + void jobDefaultsWorkingDirectoryIsTheStepsStartDirectory() { + // given + String workflow = + """ + jobs: + release: + defaults: + run: + working-directory: src + steps: + - run: ./gradlew -p optional-platforms releaseCheck + - run: ./gradlew :releaseCheck + """; + + // when + List invocations = WorkflowGradleInvocations.read(workflow); + + // then + assertThat(invocations).hasSize(2); + assertThat(invocations.get(0).executes(":releaseCheck", "src/optional-platforms")).isTrue(); + assertThat(invocations.get(1).executes(":releaseCheck", "src")).isTrue(); + } + + @Test + @DisplayName("uses 단계의 with 입력에 적힌 run은 셸 명령이 아니므로 읽지 않는다") + void aRunInputOfAnActionIsNotAShellCommand() { + // given + String workflow = + """ + jobs: + main: + steps: + - uses: some/action@v1 + with: + run: ./gradlew :integrationCheck + """; + + // when + List invocations = WorkflowGradleInvocations.read(workflow); + + // then + assertThat(invocations).isEmpty(); + } } diff --git a/src/composition/idempotency/README.md b/src/composition/idempotency/README.md index 50a362e9..5ce585b7 100644 --- a/src/composition/idempotency/README.md +++ b/src/composition/idempotency/README.md @@ -45,7 +45,7 @@ executor, 설정 기본값이 함께 빠진다(그때 KEYED use case는 executor | 언제 | 키 | | --- | --- | | 켤 때 | `APP_IDEMPOTENCY_PROVIDER` (`redis` 또는 `postgresql`), 그리고 그 provider 모듈의 줄과 env | -| 조정할 때 | `APP_IDEMPOTENCY_PROCESSING_LEASE`(≤1h), `APP_IDEMPOTENCY_TTL`(≤72h), `APP_IDEMPOTENCY_FAILURE_RETENTION`(≤30d) | +| 조정할 때 | `APP_IDEMPOTENCY_PROCESSING_LEASE`(≤1h), `APP_IDEMPOTENCY_TTL`(≤72h, processing lease 이상), `APP_IDEMPOTENCY_FAILURE_RETENTION`(≤30d) | ## 설계 결정 참조 (app-bootstrap에서 옮김) @@ -73,7 +73,7 @@ executor, 설정 기본값이 함께 빠진다(그때 KEYED use case는 executor | --- | --- | --- | --- | | `provider` | `APP_IDEMPOTENCY_PROVIDER` | `disabled` | `disabled` \| `redis` \| `postgresql` | | `processing-lease` | `APP_IDEMPOTENCY_PROCESSING_LEASE` | 30s | 1h | - | `replay-ttl` | `APP_IDEMPOTENCY_TTL` | 24h | 72h | + | `replay-ttl` | `APP_IDEMPOTENCY_TTL` | 24h | 72h, 그리고 `processing-lease` 이상 | | `failure-retention` | `APP_IDEMPOTENCY_FAILURE_RETENTION` | 24h | 30d | | `response-codec-id` | — | `json-v2` | 1–64자 | | `policy-revision` | — | 2 | 양수 | @@ -81,6 +81,12 @@ executor, 설정 기본값이 함께 빠진다(그때 KEYED use case는 executor Redis store 튜닝값 `key-version`, `command-timeout`(`APP_IDEMPOTENCY_REDIS_COMMAND_TIMEOUT`)도 같은 prefix 아래 있지만 lifecycle 정책이 아니라서 이 record 에 두지 않고 `cache-redis`가 읽는다. +- **`replay-ttl` 이 `processing-lease` 보다 짧으면 기동을 거부한다.** 같은 키로 다시 보내라는 답을 받은 + 클라이언트는 lease 안 어느 때든 돌아올 수 있다. 재생 창이 그보다 먼저 닫히면 그 재전송은 작업을 한 번 + 더 돌린다(재리뷰 2026-11-04 RR-03-11). 두 store 는 이 관계와 상관없이 기록을 lease + replay TTL 동안 + 둔다 — 이 검사는 store 를 지키는 것이 아니라 재생을 무력하게 만드는 정책을 막는다. + `IdempotencyClaimRequest` 에는 두지 않았다: 포트는 어떤 값 쌍으로도 쓸 수 있고, store 가 모든 쌍에서 + 안전한지는 공유 계약이 본다. - **상한 검사는 컴팩트 생성자에서 fail-fast 로 던진다.** `Duration`의 상한은 JSR-303(`@Valid`) 애너테이션으로 표현할 수 없어 생성자에서 직접 검증한다. 검증을 느슨하게 두면 과도하게 긴 TTL 이 저장소를 조용히 부풀리고 멱등성 키 추측 공격의 유효 시간 창을 넓히기 때문에, 부팅 시점에 즉시 diff --git a/src/composition/idempotency/src/main/java/dev/caskeleton/composition/idempotency/IdempotencySettings.java b/src/composition/idempotency/src/main/java/dev/caskeleton/composition/idempotency/IdempotencySettings.java index e56a0ccc..19bf60f7 100644 --- a/src/composition/idempotency/src/main/java/dev/caskeleton/composition/idempotency/IdempotencySettings.java +++ b/src/composition/idempotency/src/main/java/dev/caskeleton/composition/idempotency/IdempotencySettings.java @@ -21,7 +21,8 @@ import org.springframework.core.env.Environment; * @param processingLease how long one attempt holds a claim before another may take it over; longer * than the in-flight 409's {@code Retry-After} ({@link * IdempotencyExecutor#IN_FLIGHT_RETRY_AFTER}) - * @param replayTtl how long a completed response is replayable (at most 72h, D6) + * @param replayTtl how long a completed response is replayable (at most 72h, D6), and no shorter + * than {@code processingLease} * @param failureRetention how long a failed attempt's evidence is kept * @param responseCodecId identifier of the codec the stored payload is written with * @param policyRevision replay policy revision written into and compared on each record @@ -63,6 +64,20 @@ public record IdempotencySettings( + processingLease); } replayTtl = bounded("replay-ttl", orDefault(replayTtl, Duration.ofHours(24)), MAX_REPLAY_TTL); + // A duplicate told to resend with the same key may come back anywhere inside the lease. A + // replay window shorter than that closes before such a resend arrives, and the resend runs the + // operation a second time (re-review 2026-11-04 RR-03-11). Both stores also keep a record for + // the lease plus the replay TTL whatever the pair, so this refuses a policy that defeats the + // replay, not a store hazard. Kept here, not in IdempotencyClaimRequest: the port stays usable + // with any pair, and the stores are held to staying safe for every pair by their contract. + if (replayTtl.compareTo(processingLease) < 0) { + throw StartupFailures.envValidation( + PREFIX + + ".replay-ttl must not be shorter than processing-lease (" + + processingLease + + "), was " + + replayTtl); + } failureRetention = bounded( "failure-retention", diff --git a/src/composition/idempotency/src/main/resources/config/module-defaults/idempotency.yml b/src/composition/idempotency/src/main/resources/config/module-defaults/idempotency.yml index 1a4a1b19..7ba85134 100644 --- a/src/composition/idempotency/src/main/resources/config/module-defaults/idempotency.yml +++ b/src/composition/idempotency/src/main/resources/config/module-defaults/idempotency.yml @@ -12,6 +12,9 @@ app: # the lifecycle keys below, whichever provider is selected. disabled binds and builds nothing. provider: ${APP_IDEMPOTENCY_PROVIDER:disabled} # Lifecycle policy (IdempotencySettings), the same whichever provider holds the records. + # replay-ttl must not be shorter than processing-lease (startup fails otherwise): a duplicate + # told to resend with the same key may come back anywhere inside the lease. A record lives + # for the lease plus the replay TTL in both stores. processing-lease: ${APP_IDEMPOTENCY_PROCESSING_LEASE:30s} replay-ttl: ${APP_IDEMPOTENCY_TTL:24h} failure-retention: ${APP_IDEMPOTENCY_FAILURE_RETENTION:24h} diff --git a/src/composition/idempotency/src/test/java/dev/caskeleton/composition/idempotency/IdempotencySettingsTest.java b/src/composition/idempotency/src/test/java/dev/caskeleton/composition/idempotency/IdempotencySettingsTest.java index 1c573f2d..a4ebe984 100644 --- a/src/composition/idempotency/src/test/java/dev/caskeleton/composition/idempotency/IdempotencySettingsTest.java +++ b/src/composition/idempotency/src/test/java/dev/caskeleton/composition/idempotency/IdempotencySettingsTest.java @@ -176,6 +176,49 @@ class IdempotencySettingsTest { "app.capabilities.idempotency.processing-lease must be longer than PT2S"); } + /** + * A completed response must stay replayable at least as long as one attempt may hold its claim: a + * client told to resend with the same key may arrive anywhere inside the lease, and a replay + * window closed by then turns that resend into a second run (re-review 2026-11-04 RR-03-11). + */ + @ParameterizedTest + @CsvSource({"PT30S, PT10S", "PT1H, PT59M"}) + @DisplayName("replay-ttl이 processing-lease보다 짧으면 기동을 거부한다") + void aReplayTtlShorterThanTheProcessingLeaseIsRefused(Duration lease, Duration replay) { + // when + Throwable thrown = + catchThrowable( + () -> + new IdempotencySettings( + IdempotencySettings.Provider.REDIS, lease, replay, null, null, null, null)); + + // then + assertThat(thrown) + .isInstanceOfSatisfying( + StartupFailureException.class, + e -> assertThat(e.errorCode()).isEqualTo(StartupErrorCode.STARTUP_VALIDATION_FAILED)) + .hasMessageContaining( + "app.capabilities.idempotency.replay-ttl must not be shorter than processing-lease"); + } + + @Test + @DisplayName("replay-ttl이 processing-lease와 같으면 받아들인다") + void aReplayTtlEqualToTheProcessingLeaseIsAccepted() { + // when + IdempotencySettings settings = + new IdempotencySettings( + IdempotencySettings.Provider.REDIS, + Duration.ofMinutes(5), + Duration.ofMinutes(5), + null, + null, + null, + null); + + // then + assertThat(settings.replayTtl()).isEqualTo(settings.processingLease()); + } + @ParameterizedTest @CsvSource({"'', response-codec-id must contain 1-64 characters", "' ', response-codec-id"}) void blankCodecIdIsRefused(String codecId, String message) { diff --git a/src/composition/outbox-relay/README.md b/src/composition/outbox-relay/README.md index 87cbc7b8..2c7ddc2b 100644 --- a/src/composition/outbox-relay/README.md +++ b/src/composition/outbox-relay/README.md @@ -118,7 +118,9 @@ poll 주기·batch 크기·in-flight timeout·보존 기간은 env 키 없이 `a `outbox.retry.long.size`는 미룬 claim이 `LONG_RETRY_DEFERRED_CLAIMS`(10)번 이상인 FAILED 행 수, `outbox.retry.long.age`는 그중 가장 오래된 행의 나이다. 10번은 backoff 상한 5분 기준으로 적어도 약 35분 동안 실패했다는 뜻이라 broker 재시작·credential 교체는 넘긴다. publish-failed runbook §4.1의 조회가 같은 - 기준을 쓴다. + 기준을 쓴다. runbook SQL에는 숫자가 그대로 적혀 있어서(운영자가 psql에 붙여 넣는다) 상수만 바꾸면 경보가 센 행과 + 조회가 보여 주는 행이 갈라진다. `OutboxLongRetryRunbookContractTest`(lane `outboxRelayDocumentContractTest`, + `mainCheck`)가 runbook 블록의 숫자와 상수를 비교한다(재리뷰 2차 RR-02-14). - **MultiGauge 갱신 시 `overwrite=true`로 등록한다.** 매 스케줄러 틱마다 store 를 다시 조회해 게이지를 갱신하는데, 이전 틱에 있었지만 지금은 사라진 event type 같은 오래된 time-series 태그를 덮어써 제거하기 위함이다. diff --git a/src/composition/outbox-relay/build.gradle b/src/composition/outbox-relay/build.gradle index 0a700d77..0b9916bd 100644 --- a/src/composition/outbox-relay/build.gradle +++ b/src/composition/outbox-relay/build.gradle @@ -41,3 +41,21 @@ dependencies { testImplementation 'ch.qos.logback:logback-classic' testRuntimeOnly 'org.yaml:snakeyaml' } + +// OutboxLongRetryRunbookContractTest holds the long-retry gauges' threshold to the number written into +// the publish-failed runbook's query (re-review 2026-11-03 round 2, RR-02-14). A comparison with a +// repository document, so it runs in this module's `mainCheck`, not in the pull-request `test`; the +// runbook is its input, so editing the number reruns it. +strictTestLanes { + lane('outboxRelayDocumentContractTest') { + tag = 'document-contract' + description = 'Holds the long-retry threshold to docs/runbooks/outbox-publish-failed.md.' + rejectSkipped = true + mainStage() + } +} +tasks.named('outboxRelayDocumentContractTest', Test) { + inputs.file(new File(rootProject.projectDir.parentFile, 'docs/runbooks/outbox-publish-failed.md')) + .withPathSensitivity(PathSensitivity.RELATIVE) + .withPropertyName('publishFailedRunbook') +} diff --git a/src/composition/outbox-relay/src/test/java/dev/caskeleton/composition/outbox/OutboxLongRetryRunbookContractTest.java b/src/composition/outbox-relay/src/test/java/dev/caskeleton/composition/outbox/OutboxLongRetryRunbookContractTest.java new file mode 100644 index 00000000..17becf0c --- /dev/null +++ b/src/composition/outbox-relay/src/test/java/dev/caskeleton/composition/outbox/OutboxLongRetryRunbookContractTest.java @@ -0,0 +1,61 @@ +package dev.caskeleton.composition.outbox; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Tag; +import org.junit.jupiter.api.Test; + +/** + * The long-retry gauges and the runbook query that lists the rows behind them use one threshold + * (re-review 2026-11-03 round 2, RR-02-14). + * + *

    The gauge reads {@link OutboxMetrics#LONG_RETRY_DEFERRED_CLAIMS}; the publish-failed runbook's + * {@code long-retries} block has the number written into its SQL, because an operator pastes it + * into psql. Changing one alone made the alert count rows the runbook query did not show. The + * runbook lane in persistence-jpa runs the SQL against PostgreSQL; this test holds its number to + * the constant. + */ +@Tag("document-contract") +class OutboxLongRetryRunbookContractTest { + + private static final Path RUNBOOK = Path.of("../../../docs/runbooks/outbox-publish-failed.md"); + private static final Pattern LONG_RETRIES_BLOCK = + Pattern.compile("\\s*```sql\\n(.*?)```", Pattern.DOTALL); + private static final Pattern THRESHOLD = + Pattern.compile("deferred_attempt_count\\s*>=\\s*(\\d+)"); + + @Test + @DisplayName("runbook의 오래 재시도 조회는 게이지와 같은 미룬 claim 기준을 쓴다") + void theRunbookQueryUsesTheThresholdTheGaugeCounts() throws IOException { + // given + String runbook = Files.readString(RUNBOOK, StandardCharsets.UTF_8); + + // when + List thresholds = thresholdsIn(runbook); + + // then + assertThat(thresholds) + .as("the long-retries block of %s", RUNBOOK) + .containsExactly(OutboxMetrics.LONG_RETRY_DEFERRED_CLAIMS); + } + + private static List thresholdsIn(String runbook) { + Matcher block = LONG_RETRIES_BLOCK.matcher(runbook); + if (!block.find()) { + return List.of(); + } + return THRESHOLD + .matcher(block.group(1)) + .results() + .map(match -> Integer.valueOf(match.group(1))) + .toList(); + } +} diff --git a/src/composition/service-runtime/README.md b/src/composition/service-runtime/README.md index 46112736..dd2d6b4c 100644 --- a/src/composition/service-runtime/README.md +++ b/src/composition/service-runtime/README.md @@ -30,7 +30,9 @@ composition:service-runtime "어떤 advisor든 pointcut이 맞으면 통과"라 인가가 아닌 advisor를 구분하지 못한다. `new`로 만든 유스케이스는 검사 대상이 아니다(시스템 호출용은 `@SystemInvoked`). 반대로 `@SystemInvoked` 유스케이스가 빈으로 등록돼 있으면 같은 가드가 기동을 멈춘다 — 권한 검사가 없는 쓰기를 전송 계층이 - 닿는 곳에 두는 조립이기 때문이다(review 2026-11-03 RR-01-02). + 닿는 곳에 두는 조립이기 때문이다(review 2026-11-03 RR-01-02). 가드가 허용하는 경우는 하나다: 시스템 + 호출자의 bean이 자기가 `new`로 만든 use case를 필드로 든다(스케줄러에 넘기는 조립 모듈, 또는 + `@SystemCaller` 메시지 handler — RR-01-12). 가드는 bean 타입만 보므로 둘 다 기동한다. - 모두 이 모듈의 `AutoConfiguration.imports`로 등록된다(설정 브리지와 tracing 스위치는 `spring.factories`의 `EnvironmentPostProcessor`·`AutoConfigurationImportFilter`). 구성 루트의 diff --git a/src/composition/service-runtime/src/main/java/dev/caskeleton/composition/runtime/authorization/PermissionEnforcementGuard.java b/src/composition/service-runtime/src/main/java/dev/caskeleton/composition/runtime/authorization/PermissionEnforcementGuard.java index c671f8b8..f15207dd 100644 --- a/src/composition/service-runtime/src/main/java/dev/caskeleton/composition/runtime/authorization/PermissionEnforcementGuard.java +++ b/src/composition/service-runtime/src/main/java/dev/caskeleton/composition/runtime/authorization/PermissionEnforcementGuard.java @@ -28,11 +28,16 @@ import org.springframework.util.ClassUtils; * whose pointcut happens to match; it closes the "adapter missing" gap, not every misconfiguration. * *

    A {@code @SystemInvoked} use case carries no permission by design — only the system calls it, - * on a thread with no {@code Authentication} — and its composition constructs it with {@code new} - * and hands it to that caller. Registered as a bean, it is a write any transport can reach with no - * check, whatever advisor the context holds, so its presence fails startup (review 2026-11-03 - * RR-01-02). {@code INBOUND_ADAPTERS_DO_NOT_REACH_SYSTEM_INVOKED_USE_CASES} closes the other path, - * an inbound adapter that constructs one itself. + * on a thread with no {@code Authentication}. Registered as a bean, it is a write any transport can + * reach with no check, whatever advisor the context holds, so its presence fails startup (review + * 2026-11-03 RR-01-02), with no exception. The system caller that runs it holds it instead, and + * that is the one condition this guard allows: the use case is a field of a bean, never a bean. + * There are two such callers (decision 9.1, RR-01-12): a scheduler, to which a composition module + * hands the use case it constructed (the outbox relay), and a {@code @SystemCaller} message + * handler, which constructs the use case from the ports its own bean method receives. The guard + * looks at bean types only, so both start. {@code + * ONLY_SYSTEM_CALLERS_REACH_SYSTEM_INVOKED_USE_CASES} closes the static path, a user transport that + * constructs one itself. */ final class PermissionEnforcementGuard implements SmartInitializingSingleton, BeanFactoryAware { @@ -64,8 +69,9 @@ final class PermissionEnforcementGuard implements SmartInitializingSingleton, Be if (!systemInvoked.isEmpty()) { throw StartupFailures.requiredAdapterDisabled( "@SystemInvoked use cases are registered as beans, where a transport can reach a write" - + " that has no permission check — a system-invoked use case must be constructed by" - + " its composition, not registered as a bean: " + + " that has no permission check — a system-invoked use case is constructed by the" + + " system caller that runs it (a scheduler's composition, a @SystemCaller message" + + " handler) and never registered as a bean: " + String.join(", ", systemInvoked)); } if (!unenforced.isEmpty()) { diff --git a/src/composition/service-runtime/src/test/java/dev/caskeleton/composition/runtime/authorization/PermissionEnforcementGuardTest.java b/src/composition/service-runtime/src/test/java/dev/caskeleton/composition/runtime/authorization/PermissionEnforcementGuardTest.java index e8c1acf8..95993ba4 100644 --- a/src/composition/service-runtime/src/test/java/dev/caskeleton/composition/runtime/authorization/PermissionEnforcementGuardTest.java +++ b/src/composition/service-runtime/src/test/java/dev/caskeleton/composition/runtime/authorization/PermissionEnforcementGuardTest.java @@ -4,6 +4,7 @@ import static org.assertj.core.api.Assertions.assertThat; import dev.caskeleton.adapter.outbound.support.startup.StartupFailureException; import dev.caskeleton.application.security.RequiresPermission; +import dev.caskeleton.application.security.SystemCaller; import dev.caskeleton.application.security.SystemInvoked; import org.aopalliance.intercept.MethodInterceptor; import org.junit.jupiter.api.Test; @@ -52,10 +53,26 @@ class PermissionEnforcementGuardTest { context -> assertThat(context.getStartupFailure()) .isInstanceOf(StartupFailureException.class) - .hasMessageContaining("must be constructed by its composition") + .hasMessageContaining("never registered as a bean") .hasMessageContaining(SystemInvokedUseCase.class.getName())); } + /** + * The one shape the guard allows for a system-invoked use case: a system caller's bean holds the + * use case it constructed, so the use case is a field, never a bean (review 2026-11-03 RR-01-12). + */ + @Test + void aSystemCallerBeanHoldingTheSystemInvokedUseCaseItConstructedStarts() { + runner + .withUserConfiguration(SystemCallerConfig.class) + .run( + context -> + assertThat(context) + .hasNotFailed() + .hasSingleBean(SystemCallerHandler.class) + .doesNotHaveBean(SystemInvokedUseCase.class)); + } + @Test void aContextWithoutGatedBeansNeedsNoAdvisor() { runner.run(context -> assertThat(context).hasNotFailed()); @@ -87,6 +104,27 @@ class PermissionEnforcementGuardTest { } } + @SystemCaller(reason = "the broker consumer calls it") + public static class SystemCallerHandler { + private final SystemInvokedUseCase useCase; + + SystemCallerHandler() { + this.useCase = new SystemInvokedUseCase(); + } + + public void handle() { + useCase.handle(); + } + } + + @Configuration(proxyBeanMethods = false) + static class SystemCallerConfig { + @Bean + SystemCallerHandler systemCallerHandler() { + return new SystemCallerHandler(); + } + } + @Configuration(proxyBeanMethods = false) static class EnforcingAdvisorConfig { @Bean diff --git a/src/config/spotbugs/exclude.xml b/src/config/spotbugs/exclude.xml index 2a7d1e5e..d1a4707a 100644 --- a/src/config/spotbugs/exclude.xml +++ b/src/config/spotbugs/exclude.xml @@ -473,9 +473,11 @@ + unchanged to /error. CSRF protection guards a state change made with ambient credentials; + this method makes none — it writes a fixed failure envelope, with nothing from the request in + it. A direct request to /error is answered as an unknown route and still goes through the + authentication chain (a token under the JWT chain, which keeps CSRF off because it carries + no ambient credential). Exact to the one method (review 2026-11-04 RR2-04-06). --> diff --git a/src/grpc/grpc-core-api/src/main/java/dev/caskeleton/grpc/api/ledger/GrpcClaimToken.java b/src/grpc/grpc-core-api/src/main/java/dev/caskeleton/grpc/api/ledger/GrpcClaimToken.java index 7b5fe358..269b00ed 100644 --- a/src/grpc/grpc-core-api/src/main/java/dev/caskeleton/grpc/api/ledger/GrpcClaimToken.java +++ b/src/grpc/grpc-core-api/src/main/java/dev/caskeleton/grpc/api/ledger/GrpcClaimToken.java @@ -12,7 +12,12 @@ package dev.caskeleton.grpc.api.ledger; * that still holds the claim. * *

    A counter rather than the claim moment: it increases by one with every hand-over, so two - * attempts never share a value, whatever the clocks of the processes that claimed it. + * attempts of one record never share a value, whatever the clocks of the processes that claimed it. + * The counter lives in the record: once the record is deleted ({@code release}, or the purge of a + * finished record), the next claim of the key starts again at {@link #FIRST}. Both deletions are + * therefore limited to records no live attempt can still write to — a released claim's owner is + * known to be gone, and a purged record finished longer ago than any retry (review 2026-11-03 + * RR-01-15). * * @param attempt one for the first claim of an operation, one more for each hand-over */ diff --git a/src/grpc/grpc-core-api/src/main/java/dev/caskeleton/grpc/api/ledger/GrpcOperationLedger.java b/src/grpc/grpc-core-api/src/main/java/dev/caskeleton/grpc/api/ledger/GrpcOperationLedger.java index 381979a7..19c239c7 100644 --- a/src/grpc/grpc-core-api/src/main/java/dev/caskeleton/grpc/api/ledger/GrpcOperationLedger.java +++ b/src/grpc/grpc-core-api/src/main/java/dev/caskeleton/grpc/api/ledger/GrpcOperationLedger.java @@ -135,10 +135,22 @@ public interface GrpcOperationLedger { int purgeCompletedBefore(Instant completedBefore, int limit); /** - * Releases a claim that will never complete, so the caller may try again. + * Releases a claim that will never complete, so the caller may try again: deletes the record if + * it is still {@code IN_PROGRESS} and {@code claimToken} still holds it. * *

    Only for a claim whose owner is known to be gone. Releasing one whose owner is merely slow - * re-admits the duplicate. + * re-admits the duplicate. Fenced like every other transition (review 2026-11-03 RR-01-15): an + * attempt that lost the claim cannot delete the claim of the attempt that took it over, and a + * finished record — the answer a retry replays — is never released. + * + *

    The record is gone afterwards, so the next claim of the key starts again at {@link + * GrpcClaimToken#FIRST}. An earlier attempt that lost the claim and is still running would then + * hold a token equal to the new claim's; releasing is safe only when no such attempt can still + * record an outcome — which is what "the owner is known to be gone" has to cover. + * + * @param claimToken the token of the attempt that holds the claim + * @return true when the record was deleted; false when it is missing, finished, or held by + * another attempt, in which case nothing changed */ - void release(GrpcOperationIdentity identity); + boolean release(GrpcOperationIdentity identity, GrpcClaimToken claimToken); } diff --git a/src/grpc/grpc-core-api/src/testFixtures/java/dev/caskeleton/grpc/api/ledger/GrpcOperationLedgerContract.java b/src/grpc/grpc-core-api/src/testFixtures/java/dev/caskeleton/grpc/api/ledger/GrpcOperationLedgerContract.java index d91f8de1..ac84f97f 100644 --- a/src/grpc/grpc-core-api/src/testFixtures/java/dev/caskeleton/grpc/api/ledger/GrpcOperationLedgerContract.java +++ b/src/grpc/grpc-core-api/src/testFixtures/java/dev/caskeleton/grpc/api/ledger/GrpcOperationLedgerContract.java @@ -429,6 +429,60 @@ public abstract class GrpcOperationLedgerContract { .isEqualTo(takenOver); } + /** + * Release deletes the row, so it is fenced like every other transition: only the attempt that + * holds an unfinished claim may give it up (review 2026-11-03 RR-01-15). + */ + @Test + @DisplayName("진행 중 claim 을 가진 시도가 release 하면 키가 비고 다음 claim 이 첫 시도가 된다") + void releaseByTheAttemptHoldingAnUnfinishedClaimFreesTheKey() { + // given + ledger().claim(ID, FINGERPRINT, NOW); + + // when + boolean released = ledger().release(ID, GrpcClaimToken.FIRST); + + // then + assertThat(List.of(released, ledger().claim(ID, FINGERPRINT, NOW.plusSeconds(1)).isEmpty())) + .containsExactly(true, true); + } + + @Test + @DisplayName("claim 을 넘겨준 옛 시도의 release 는 무시되고 새 시도의 claim 이 남는다") + void releaseByTheAttemptThatLostTheClaimChangesNothing() { + // given + ledger().claim(ID, FINGERPRINT, NOW); + ledger().reclaimStale(ID, FINGERPRINT, CUTOFF, LATER); + + // when + boolean released = ledger().release(ID, GrpcClaimToken.FIRST); + + // then + assertThat(released).isFalse(); + assertThat(ledger().find(ID)) + .get() + .extracting(GrpcOperationLedgerRecord::state, GrpcOperationLedgerRecord::claimToken) + .containsExactly(GrpcOperationLedgerState.IN_PROGRESS, GrpcClaimToken.FIRST.next()); + } + + @Test + @DisplayName("끝난 claim 은 release 로 지워지지 않는다 — 재시도에 결과를 돌려줄 기록이다") + void releaseOfAFinishedClaimChangesNothing() { + // given + ledger().claim(ID, FINGERPRINT, NOW); + ledger().markCommitted(ID, GrpcClaimToken.FIRST, OUTCOME, NOW.plusSeconds(1)); + + // when + boolean released = ledger().release(ID, GrpcClaimToken.FIRST); + + // then + assertThat(released).isFalse(); + assertThat(ledger().find(ID)) + .get() + .extracting(GrpcOperationLedgerRecord::state) + .isEqualTo(GrpcOperationLedgerState.COMMITTED); + } + /** * The re-review probe's sequence (2026-11-03, RR-02-04): A claims and is slow, B takes the claim * over after the TTL, then A reports a failure. Recording it would mark B's running attempt diff --git a/src/grpc/grpc-core-api/src/testFixtures/java/dev/caskeleton/grpc/api/ledger/InMemoryOperationLedger.java b/src/grpc/grpc-core-api/src/testFixtures/java/dev/caskeleton/grpc/api/ledger/InMemoryOperationLedger.java index bc26942d..7aa38665 100644 --- a/src/grpc/grpc-core-api/src/testFixtures/java/dev/caskeleton/grpc/api/ledger/InMemoryOperationLedger.java +++ b/src/grpc/grpc-core-api/src/testFixtures/java/dev/caskeleton/grpc/api/ledger/InMemoryOperationLedger.java @@ -156,8 +156,12 @@ public final class InMemoryOperationLedger implements GrpcOperationLedger { } @Override - public void release(GrpcOperationIdentity identity) { - records.remove(identity.storageKey()); + public boolean release(GrpcOperationIdentity identity, GrpcClaimToken claimToken) { + GrpcOperationLedgerRecord existing = records.get(identity.storageKey()); + return existing != null + && existing.state() == GrpcOperationLedgerState.IN_PROGRESS + && existing.claimToken().equals(claimToken) + && records.remove(identity.storageKey(), existing); } // An attempt that lost the claim must not commit beside the attempt that holds it; checked diff --git a/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerMigrationLockWaitIT.java b/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerMigrationLockWaitIT.java new file mode 100644 index 00000000..15fc7416 --- /dev/null +++ b/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerMigrationLockWaitIT.java @@ -0,0 +1,155 @@ +package dev.caskeleton.grpc.ledger; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.catchThrowable; + +import java.sql.Connection; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.sql.Statement; +import java.time.Duration; +import java.util.List; +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.ScheduledFuture; +import java.util.concurrent.TimeUnit; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; +import org.postgresql.ds.PGSimpleDataSource; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.postgresql.PostgreSQLContainer; + +/** + * The ledger migrations after V001 lock a table already in use and give up within their bound + * instead of stopping it (re-review 2026-11-03 round 2, RR-02-09: the rule V21 follows). + * + *

    A transaction that has written the ledger stays open for fifteen seconds — a claim in flight, + * an operator's {@code BEGIN}. A write is the holder both migrations wait for: V003's ALTER TABLE + * would wait for a read too, V002's plain CREATE INDEX only for a write. The migration runs the way + * Flyway runs it, in one transaction. Unbounded it waits until the writer leaves, with every claim + * queued behind it, and then applies; bounded it fails with {@code lock_not_available} first and + * its rollback leaves the ledger as it was. + */ +@Testcontainers(disabledWithoutDocker = true) +class GrpcOperationLedgerMigrationLockWaitIT { + + private static final String LOCK_NOT_AVAILABLE = "55P03"; + private static final Duration WRITER_STAYS = Duration.ofSeconds(15); + private static final Duration GIVES_UP_WITHIN = Duration.ofSeconds(10); + + @SuppressWarnings("resource") + @Container + private static final PostgreSQLContainer POSTGRES = + new PostgreSQLContainer(System.getProperty("ca.test.image.postgres-16")) + .withDatabaseName("grpc") + .withUsername("grpc") + .withPassword("grpc"); + + @ParameterizedTest + @ValueSource( + strings = { + "V002__index_grpc_operation_ledger_finished_rows.sql", + "V003__add_grpc_operation_ledger_claim_attempt.sql" + }) + @DisplayName("ledger에 쓴 트랜잭션이 열려 있으면 ledger를 잠그는 마이그레이션은 상한 안에 실패하고 아무것도 바꾸지 않는다") + void aLedgerMigrationGivesUpOnAnOpenWriterAndChangesNothing(String migration) throws Exception { + PGSimpleDataSource dataSource = dataSource(); + try (Connection setup = dataSource.getConnection(); + Statement statement = setup.createStatement()) { + // given + statement.execute("DROP TABLE IF EXISTS grpc_operation_ledger"); + statement.execute(GrpcOperationLedgerMigrations.before(migration)); + String schemaBefore = schema(statement); + + // when + Attempt attempt = + whileAWriterHolds(dataSource, GrpcOperationLedgerMigrations.named(migration)); + + // then + assertThat( + List.of( + String.valueOf(attempt.sqlState()), + attempt.took().compareTo(GIVES_UP_WITHIN) < 0, + schema(statement))) + .containsExactly(LOCK_NOT_AVAILABLE, true, schemaBefore); + } + } + + /** Runs the migration in one transaction while another one holds a write to the ledger. */ + private static Attempt whileAWriterHolds(PGSimpleDataSource dataSource, String migration) + throws SQLException { + try (Connection writer = dataSource.getConnection(); + Connection migrator = dataSource.getConnection()) { + writer.setAutoCommit(false); + try (Statement write = writer.createStatement()) { + write.execute("LOCK TABLE grpc_operation_ledger IN ROW EXCLUSIVE MODE"); + } + ScheduledExecutorService release = Executors.newSingleThreadScheduledExecutor(); + ScheduledFuture released = + release.schedule( + () -> { + writer.rollback(); + return null; + }, + WRITER_STAYS.toSeconds(), + TimeUnit.SECONDS); + try { + migrator.setAutoCommit(false); + long started = System.nanoTime(); + Throwable thrown = + catchThrowable( + () -> { + try (Statement statement = migrator.createStatement()) { + statement.execute(migration); + } + migrator.commit(); + }); + Duration took = Duration.ofNanos(System.nanoTime() - started); + migrator.rollback(); + return new Attempt(thrown, took); + } finally { + released.cancel(true); + release.shutdownNow(); + } + } + } + + /** The ledger's columns, indexes and constraints, enough to see whether a migration landed. */ + private static String schema(Statement statement) throws SQLException { + try (ResultSet row = + statement.executeQuery( + "SELECT (SELECT string_agg(column_name, ',' ORDER BY column_name)" + + " FROM information_schema.columns" + + " WHERE table_name = 'grpc_operation_ledger')" + + " || '|' || (SELECT string_agg(indexname, ',' ORDER BY indexname)" + + " FROM pg_indexes WHERE tablename = 'grpc_operation_ledger')" + + " || '|' || (SELECT string_agg(conname, ',' ORDER BY conname) FROM pg_constraint" + + " WHERE conrelid = 'grpc_operation_ledger'::regclass)")) { + row.next(); + return row.getString(1); + } + } + + private static PGSimpleDataSource dataSource() { + PGSimpleDataSource dataSource = new PGSimpleDataSource(); + dataSource.setUrl(POSTGRES.getJdbcUrl()); + dataSource.setUser(POSTGRES.getUsername()); + dataSource.setPassword(POSTGRES.getPassword()); + return dataSource; + } + + /** What the migration did while the writer held the ledger. */ + private record Attempt(Throwable thrown, Duration took) { + + String sqlState() { + for (Throwable cause = thrown; cause != null; cause = cause.getCause()) { + if (cause instanceof SQLException sql) { + return sql.getSQLState(); + } + } + return null; + } + } +} diff --git a/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerMigrations.java b/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerMigrations.java index d84695d5..749302b1 100644 --- a/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerMigrations.java +++ b/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerMigrations.java @@ -23,16 +23,38 @@ final class GrpcOperationLedgerMigrations { /** Every migration's SQL, concatenated in version order. */ static String all() throws IOException { + return before(null); + } + + /** The SQL of every migration before the one named, concatenated in version order. */ + static String before(String fileName) throws IOException { StringBuilder sql = new StringBuilder(); for (String path : IN_ORDER) { - try (InputStream stream = - GrpcOperationLedgerMigrations.class.getClassLoader().getResourceAsStream(path)) { - if (stream == null) { - throw new IOException("gRPC operation-ledger migration is missing: " + path); - } - sql.append(new String(stream.readAllBytes(), StandardCharsets.UTF_8)).append('\n'); + if (path.endsWith("/" + fileName)) { + break; } + sql.append(read(path)).append('\n'); } return sql.toString(); } + + /** One migration's SQL, by its file name. */ + static String named(String fileName) throws IOException { + for (String path : IN_ORDER) { + if (path.endsWith("/" + fileName)) { + return read(path); + } + } + throw new IOException("no gRPC operation-ledger migration is named " + fileName); + } + + private static String read(String path) throws IOException { + try (InputStream stream = + GrpcOperationLedgerMigrations.class.getClassLoader().getResourceAsStream(path)) { + if (stream == null) { + throw new IOException("gRPC operation-ledger migration is missing: " + path); + } + return new String(stream.readAllBytes(), StandardCharsets.UTF_8); + } + } } diff --git a/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/JpaGrpcOperationLedgerContractIT.java b/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/JpaGrpcOperationLedgerContractIT.java index ec357db4..b4b645d2 100644 --- a/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/JpaGrpcOperationLedgerContractIT.java +++ b/src/grpc/grpc-operation-ledger-jpa/src/integrationTest/java/dev/caskeleton/grpc/ledger/JpaGrpcOperationLedgerContractIT.java @@ -233,8 +233,9 @@ class JpaGrpcOperationLedgerContractIT extends GrpcOperationLedgerContract { } @Override - public void release(GrpcOperationIdentity identity) { - transactions.executeWithoutResult(status -> delegate.release(identity)); + public boolean release(GrpcOperationIdentity identity, GrpcClaimToken claimToken) { + return Boolean.TRUE.equals( + transactions.execute(status -> delegate.release(identity, claimToken))); } } } diff --git a/src/grpc/grpc-operation-ledger-jpa/src/main/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerRepository.java b/src/grpc/grpc-operation-ledger-jpa/src/main/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerRepository.java index 9a043670..c24507f0 100644 --- a/src/grpc/grpc-operation-ledger-jpa/src/main/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerRepository.java +++ b/src/grpc/grpc-operation-ledger-jpa/src/main/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerRepository.java @@ -228,8 +228,25 @@ public interface GrpcOperationLedgerRepository /** Reads a claim by its derived storage key. */ Optional findById(String storageKey); - /** Removes a claim. Only for one whose owner is known to be gone. */ - void deleteById(String storageKey); + /** + * Deletes the claim if attempt {@code claimAttempt} still holds it and it has not finished; the + * same fencing as {@link #tryMarkFailed}. Only for a claim whose owner is known to be gone + * (review 2026-11-03 RR-01-15). + * + * @return one when the claim was deleted, zero otherwise + */ + @Modifying(flushAutomatically = true, clearAutomatically = true) + @Transactional + @Query( + value = + """ + DELETE FROM grpc_operation_ledger + WHERE storage_key = :storageKey + AND state = 'IN_PROGRESS' + AND claim_attempt = :claimAttempt + """, + nativeQuery = true) + int tryRelease(@Param("storageKey") String storageKey, @Param("claimAttempt") long claimAttempt); /** Whether a claim exists. */ boolean existsById(String storageKey); diff --git a/src/grpc/grpc-operation-ledger-jpa/src/main/java/dev/caskeleton/grpc/ledger/JpaGrpcOperationLedger.java b/src/grpc/grpc-operation-ledger-jpa/src/main/java/dev/caskeleton/grpc/ledger/JpaGrpcOperationLedger.java index 9db0c3d8..5901103a 100644 --- a/src/grpc/grpc-operation-ledger-jpa/src/main/java/dev/caskeleton/grpc/ledger/JpaGrpcOperationLedger.java +++ b/src/grpc/grpc-operation-ledger-jpa/src/main/java/dev/caskeleton/grpc/ledger/JpaGrpcOperationLedger.java @@ -201,9 +201,10 @@ public final class JpaGrpcOperationLedger implements GrpcOperationLedger { } @Override - public void release(GrpcOperationIdentity identity) { + public boolean release(GrpcOperationIdentity identity, GrpcClaimToken claimToken) { requireIdentity(identity); - repository.deleteById(identity.storageKey()); + requireToken(claimToken); + return repository.tryRelease(identity.storageKey(), claimToken.attempt()) == 1; } private static void requireToken(GrpcClaimToken claimToken) { diff --git a/src/grpc/grpc-operation-ledger-jpa/src/main/resources/db/migration/grpc/V002__index_grpc_operation_ledger_finished_rows.sql b/src/grpc/grpc-operation-ledger-jpa/src/main/resources/db/migration/grpc/V002__index_grpc_operation_ledger_finished_rows.sql index a0c6d7d5..3c6adef4 100644 --- a/src/grpc/grpc-operation-ledger-jpa/src/main/resources/db/migration/grpc/V002__index_grpc_operation_ledger_finished_rows.sql +++ b/src/grpc/grpc-operation-ledger-jpa/src/main/resources/db/migration/grpc/V002__index_grpc_operation_ledger_finished_rows.sql @@ -11,6 +11,19 @@ -- A plain CREATE INDEX, not CONCURRENTLY: Flyway runs each migration in a transaction, and the -- platform is not adopted anywhere yet, so the table is empty when this runs. An adopter that -- applies it to a large existing ledger should run the CONCURRENTLY form by hand first. +-- +-- This file was edited after it was first written, before any deployment applied it (the branch +-- that added it was not merged, and the ledger is adopted nowhere yet), to add the bound below. A +-- development database that applied the earlier text fails Flyway validation: recreate it. + +-- Bound the wait for the lock this migration takes (SHARE on grpc_operation_ledger, held until it +-- commits; it stops writes, not reads). Unbounded, it queues behind any open transaction that has +-- written to the table, and every write of the table after it queues behind the migration — every +-- claim writes it. Five seconds, then the migration fails, rolls back whole and the database stays +-- at V001: deploy again once that transaction has ended (re-review 2026-11-03 round 2, RR-02-09; +-- the rule is docs/jpa/migration-guide.md "Bound the lock wait"). +SET LOCAL lock_timeout = '5s'; + CREATE INDEX ix_grpc_operation_ledger_finished_completed_at ON grpc_operation_ledger (completed_at) WHERE state IN ('COMMITTED', 'FAILED_TERMINAL'); diff --git a/src/grpc/grpc-operation-ledger-jpa/src/main/resources/db/migration/grpc/V003__add_grpc_operation_ledger_claim_attempt.sql b/src/grpc/grpc-operation-ledger-jpa/src/main/resources/db/migration/grpc/V003__add_grpc_operation_ledger_claim_attempt.sql index 23b03687..89007ad5 100644 --- a/src/grpc/grpc-operation-ledger-jpa/src/main/resources/db/migration/grpc/V003__add_grpc_operation_ledger_claim_attempt.sql +++ b/src/grpc/grpc-operation-ledger-jpa/src/main/resources/db/migration/grpc/V003__add_grpc_operation_ledger_claim_attempt.sql @@ -13,6 +13,19 @@ -- -- Existing rows are their first attempt: no row has been handed over under a token yet. The table is -- small and the platform is not adopted anywhere, so a plain ALTER with a default is enough. +-- +-- This file was edited after it was first written, before any deployment applied it (the branch +-- that added it was not merged, and the ledger is adopted nowhere yet), to add the bound below. A +-- development database that applied the earlier text fails Flyway validation: recreate it. + +-- Bound the wait for the lock this migration takes (ACCESS EXCLUSIVE on grpc_operation_ledger, held +-- until it commits). Unbounded, it queues behind any open transaction that has touched the table, +-- and every read and write of the table after it queues behind the migration — every claim reads +-- and writes it. Five seconds, then the migration fails, rolls back whole and the database stays at +-- V002: deploy again once that transaction has ended (re-review 2026-11-03 round 2, RR-02-09; the +-- rule is docs/jpa/migration-guide.md "Bound the lock wait"). +SET LOCAL lock_timeout = '5s'; + ALTER TABLE grpc_operation_ledger ADD COLUMN claim_attempt BIGINT NOT NULL DEFAULT 1; diff --git a/src/grpc/grpc-operation-ledger-jpa/src/test/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerRepositoryTest.java b/src/grpc/grpc-operation-ledger-jpa/src/test/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerRepositoryTest.java index f2f90758..37c04781 100644 --- a/src/grpc/grpc-operation-ledger-jpa/src/test/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerRepositoryTest.java +++ b/src/grpc/grpc-operation-ledger-jpa/src/test/java/dev/caskeleton/grpc/ledger/GrpcOperationLedgerRepositoryTest.java @@ -16,6 +16,7 @@ import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Path; import java.time.Instant; +import java.util.List; import java.util.Optional; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentMap; @@ -120,8 +121,13 @@ class GrpcOperationLedgerRepositoryTest { } @Override - public void deleteById(String storageKey) { - rows.remove(storageKey); + public int tryRelease(String storageKey, long claimAttempt) { + GrpcOperationLedgerEntity row = rows.get(storageKey); + boolean held = + row != null + && row.getState() == GrpcOperationLedgerState.IN_PROGRESS + && row.getClaimToken().equals(new GrpcClaimToken(claimAttempt)); + return held && rows.remove(storageKey, row) ? 1 : 0; } @Override @@ -320,18 +326,24 @@ class GrpcOperationLedgerRepositoryTest { assertThat(stored.sameRequestAs(OTHER_FINGERPRINT)).isFalse(); } + /** + * A failed record is finished: a retry re-claims it with a new token ({@code reclaimFailed}), so + * release, which deletes and resets the token, leaves it (review 2026-11-03 RR-01-15). + */ @Test - @DisplayName("a failed claim is terminal, and releasing removes it") - void failureIsTerminalAndReleaseRemoves() { + @DisplayName("실패한 claim 은 끝난 기록이라 release 로 지워지지 않는다") + void aFailedClaimIsTerminalAndReleaseLeavesIt() { + // given GrpcOperationIdentity id = identity("tenant-1.actor-1"); ledger.claim(id, FINGERPRINT, NOW); - ledger.markFailed(id, GrpcClaimToken.FIRST, NOW.plusSeconds(1)); - assertThat(ledger.find(id).orElseThrow().state()) - .isEqualTo(GrpcOperationLedgerState.FAILED_TERMINAL); - ledger.release(id); - assertThat(ledger.find(id)).isEmpty(); + // when + boolean released = ledger.release(id, GrpcClaimToken.FIRST); + + // then + assertThat(List.of(released, ledger.find(id).orElseThrow().state())) + .containsExactly(false, GrpcOperationLedgerState.FAILED_TERMINAL); } @Test diff --git a/src/messaging/messaging-spring-boot-starter-kafka/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantineTest.java b/src/messaging/messaging-spring-boot-starter-kafka/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantineTest.java new file mode 100644 index 00000000..35c6d145 --- /dev/null +++ b/src/messaging/messaging-spring-boot-starter-kafka/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantineTest.java @@ -0,0 +1,82 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.error.FailureCategory; +import dev.caskeleton.messaging.api.header.ReservedHeaders; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.List; +import java.util.stream.StreamSupport; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.producer.MockProducer; +import org.apache.kafka.clients.producer.ProducerRecord; +import org.apache.kafka.common.header.Header; +import org.apache.kafka.common.serialization.ByteArraySerializer; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.EnumSource; + +/** + * The dead-letter copy says which of the two quarantine paths wrote it (review 2026-11-01 06-14, + * re-review round 2 RR-02-16): a record that could not be rebuilt at all and one whose handler + * spent its attempt budget leave different failure-category bytes, so a redrive can tell a message + * that will never decode from one an outage kept failing. + */ +class KafkaDeadLetterQuarantineTest { + + private static final String DEAD_LETTER_TOPIC = "orders.dlq"; + + @ParameterizedTest + @EnumSource( + value = FailureCategory.class, + names = {"DESERIALIZATION", "POISON_MESSAGE"}) + @DisplayName("격리 사본은 원래 header를 지키고 failure-category를 호출자가 준 분류 하나로 쓴다") + void theCopyCarriesTheCategoryTheCallerGaveInPlaceOfAnyEarlierOne(FailureCategory category) { + // given + MockProducer producer = + new MockProducer<>(true, null, new ByteArraySerializer(), new ByteArraySerializer()); + KafkaDeadLetterQuarantine quarantine = + new KafkaDeadLetterQuarantine(producer, DEAD_LETTER_TOPIC, Duration.ofSeconds(1)); + ConsumerRecord record = + new ConsumerRecord<>("orders", 0, 7L, bytes("order-7"), bytes("{\"broken\"")); + record.headers().add("trace-id", bytes("trace-1")); + record.headers().add(ReservedHeaders.FAILURE_CATEGORY, bytes("STALE_FROM_AN_EARLIER_HOP")); + + // when + boolean confirmed = quarantine.quarantine(record, category, new IllegalStateException("x")); + + // then + ProducerRecord copy = producer.history().getFirst(); + assertThat( + List.of( + confirmed, + copy.topic(), + text(copy.value()), + values(copy, "trace-id"), + values(copy, ReservedHeaders.FAILURE_CATEGORY), + values(copy, ReservedHeaders.FAILURE_CODE))) + .containsExactly( + true, + DEAD_LETTER_TOPIC, + "{\"broken\"", + List.of("trace-1"), + List.of(category.name()), + List.of("IllegalStateException")); + } + + private static List values(ProducerRecord record, String header) { + return StreamSupport.stream(record.headers().headers(header).spliterator(), false) + .map(Header::value) + .map(KafkaDeadLetterQuarantineTest::text) + .toList(); + } + + private static byte[] bytes(String value) { + return value.getBytes(StandardCharsets.UTF_8); + } + + private static String text(byte[] value) { + return new String(value, StandardCharsets.UTF_8); + } +} diff --git a/src/optional-platforms/build.gradle b/src/optional-platforms/build.gradle index 1c5ff1b0..ecdce3e0 100644 --- a/src/optional-platforms/build.gradle +++ b/src/optional-platforms/build.gradle @@ -75,9 +75,9 @@ tasks.register('releaseCheck') { } // The stages this build's strict lanes may belong to, executed by workflows as -// `./gradlew -p optional-platforms ` from `src`. +// `./gradlew -p optional-platforms ` from `src`. A command counts when it starts in this +// directory (working-directory joined with -p), which the plugin reads from where this build is. strictLaneStages { stage ':ci', ':integrationCheck', ':performanceCheck', ':releaseCheck' workflowDirectory = layout.projectDirectory.dir('../../.github/workflows') - workflowProjectDirectory = 'optional-platforms' } diff --git a/src/sample-portfolio/README.md b/src/sample-portfolio/README.md index eb63c231..c67df5e1 100644 --- a/src/sample-portfolio/README.md +++ b/src/sample-portfolio/README.md @@ -54,7 +54,9 @@ 누구나 관리자로 들어올 수 있기 때문입니다. 거부 목록(`prod`)이 아니라 허용 목록인 이유는 프로필 이름이 하나만 달라도 계정이 열리지 않게 하려는 것입니다. `./gradlew :sample-portfolio:bootRun`은 `SPRING_PROFILES_ACTIVE`가 비어 있으면 `local`을 넘기고, jar·이미지로 띄울 때는 -`SPRING_PROFILES_ACTIVE=local`(또는 `test`·`demo`)을 직접 줍니다. 운영으로 +`SPRING_PROFILES_ACTIVE=demo`를 직접 줍니다(`src/Dockerfile.sample` 머리말의 `docker run` 예시). 세 프로필 +모두 같은 데모 기본값(object storage 끔 등)을 `application.yml`에서 받고, 각 프로필로 실제로 뜨는지는 +`SampleDemoProfilesStartupSystemTest`가 확인합니다. 운영으로 가져가려면 `SampleDemoSecurityConfig`·`SampleDemoAccounts`를 지우고 `SamplePortfolioApplication`에서 `WebSecurityAutoConfiguration` 제외를 풉니다. 이 규칙은 데모 설정 옆에 선언되어 있어 함께 지워집니다. diff --git a/src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoAccountsProductionGuard.java b/src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoAccountsProductionGuard.java index 943e2542..df0f73bf 100644 --- a/src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoAccountsProductionGuard.java +++ b/src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoAccountsProductionGuard.java @@ -23,6 +23,11 @@ import org.springframework.core.env.Environment; * other name a deployment happened to use: one differently spelled profile opened an admin account. * Here a deployment is refused unless every active profile is a demo profile, so no profile at all * — the sample's image sets none — is refused too. + * + *

    Each profile listed here is one the sample starts under as shipped: {@code application.yml} + * gives the same three the lean demo defaults, and {@code SampleDemoProfilesStartupSystemTest} + * starts the application under each (review 2026-11-04 RR2-04-02). {@code demo} is the one for the + * image, {@code local} for {@code bootRun}, {@code test} for the sample's own tests. */ final class SampleDemoAccountsProductionGuard implements StartupRule { @@ -41,7 +46,8 @@ final class SampleDemoAccountsProductionGuard implements StartupRule { @Override public String remedy() { - return "run the sample with SPRING_PROFILES_ACTIVE=local (or test or demo); a real service" + return "run the sample with SPRING_PROFILES_ACTIVE=demo (local for bootRun, test for its" + + " tests); a real service" + " authenticates through the JWT chain — delete SampleDemoSecurityConfig and" + " SampleDemoAccounts and stop excluding WebSecurityAutoConfiguration in" + " SamplePortfolioApplication"; diff --git a/src/sample-portfolio/src/main/resources/application.yml b/src/sample-portfolio/src/main/resources/application.yml index 4ce4bf16..d77e0a28 100644 --- a/src/sample-portfolio/src/main/resources/application.yml +++ b/src/sample-portfolio/src/main/resources/application.yml @@ -213,14 +213,20 @@ app: enabled: false --- +# Every demo profile, the same list SampleDemoAccountsProductionGuard.DEMO_PROFILES allows: a +# profile the guard lets the sample start under must also start it. This document used to name +# `local` alone, so `test` and `demo` were allowed and stopped on an object-storage binding with no +# destination (review 2026-11-04 RR2-04-02). SampleDemoProfilesStartupSystemTest starts the sample +# under each listed profile with nothing but this file and a database address. spring: config: activate: - on-profile: local + on-profile: local | test | demo -# The poster image endpoint is off locally: its only mode is canonical publication +# The poster image endpoint is off in the demo: its only mode is canonical publication # (`app.poster-image.api.mode=publication`), which needs an `app.object-storage` destination and the -# idempotency key ring. The whole-byte legacy upload and its storage seam are gone. +# idempotency key ring. The object-storage module is on whenever it is on the classpath, so it is +# switched off here rather than left to fail on a binding nobody gave it. app: object-storage: enabled: false diff --git a/src/sample-portfolio/src/main/resources/db/sample-migration/V8_1__poster_title_unique.sql b/src/sample-portfolio/src/main/resources/db/sample-migration/V8_1__poster_title_unique.sql index 7ff4d46b..40d48883 100644 --- a/src/sample-portfolio/src/main/resources/db/sample-migration/V8_1__poster_title_unique.sql +++ b/src/sample-portfolio/src/main/resources/db/sample-migration/V8_1__poster_title_unique.sql @@ -14,6 +14,18 @@ -- one Flyway location list, and the production stream already owns the integer versions from 9 -- up; a fractional version cannot collide with them. The sample's own history table -- (flyway_sample_schema_history) orders it right after V8. +-- +-- This file was edited after it was first written, before any deployment applied it (the branch +-- that added it was not merged), to add the bound below. A development database that applied the +-- earlier text fails Flyway validation: recreate it. + +-- Bound the wait for the lock this migration takes (SHARE on poster for the unique index, held +-- until it commits; it stops writes, not reads). Unbounded, it queues behind any open transaction +-- that has written to the table, and every write of the table after it queues behind the migration +-- — every poster create and update. Five seconds, then the migration fails, rolls back whole and +-- the database stays at V8: deploy again once that transaction has ended (re-review 2026-11-03 +-- round 2, RR-02-09; the rule is docs/jpa/migration-guide.md "Bound the lock wait"). +SET LOCAL lock_timeout = '5s'; DO $$ DECLARE diff --git a/src/sample-portfolio/src/systemTest/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoProfilesStartupSystemTest.java b/src/sample-portfolio/src/systemTest/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoProfilesStartupSystemTest.java new file mode 100644 index 00000000..9d415c70 --- /dev/null +++ b/src/sample-portfolio/src/systemTest/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoProfilesStartupSystemTest.java @@ -0,0 +1,65 @@ +package dev.caskeleton.sample.portfolio.bootstrap.security; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.sample.portfolio.SamplePortfolioApplication; +import dev.caskeleton.sample.portfolio.testinfra.PostgreSqlTestContainer; +import java.util.stream.Stream; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; +import org.springframework.boot.builder.SpringApplicationBuilder; +import org.springframework.context.ConfigurableApplicationContext; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.postgresql.PostgreSQLContainer; + +/** + * Every profile the demo guard allows starts the sample as its jar and image do: {@link + * SamplePortfolioApplication} with the shipped configuration and nothing but a database address + * (review 2026-11-04 RR2-04-02). + * + *

    The allow list is a promise — "run the sample under one of these" — and the guard's remedy and + * the sample README repeat it. Every other sample test either runs under {@code local} or under a + * {@code test} profile whose test resources used to switch off what the shipped configuration left + * on, so {@code test} and {@code demo} were allowed and did not start. This test is what keeps the + * list and the shipped configuration together: a profile added to {@link + * SampleDemoAccountsProductionGuard#DEMO_PROFILES} must start here. Under {@code test} this source + * set's {@code application-test.yml} loads as well; it no longer touches object storage, so what + * starts the capability set is the shipped file in every case. + */ +@Testcontainers(disabledWithoutDocker = true) +class SampleDemoProfilesStartupSystemTest { + + @Container static final PostgreSQLContainer POSTGRES = PostgreSqlTestContainer.create(); + + static Stream demoProfiles() { + return SampleDemoAccountsProductionGuard.DEMO_PROFILES.stream().sorted(); + } + + @ParameterizedTest(name = "{0}") + @MethodSource("demoProfiles") + @DisplayName("허용 목록의 각 데모 프로필로 샘플이 출하 설정 그대로 기동한다") + void theSampleStartsUnderEveryDemoProfile(String profile) { + // given + SpringApplicationBuilder sample = + new SpringApplicationBuilder(SamplePortfolioApplication.class).profiles(profile); + + // when + // Command-line arguments, so the address wins over this source set's application-test.yml + // under `test` as well; the ports are free ones so the three starts do not collide with each + // other or with a running sample. + try (ConfigurableApplicationContext context = + sample.run( + "--spring.datasource.url=" + POSTGRES.getJdbcUrl(), + "--spring.datasource.username=" + POSTGRES.getUsername(), + "--spring.datasource.password=" + POSTGRES.getPassword(), + "--server.port=0", + "--management.server.port=0")) { + + // then + assertThat(context.isRunning()).isTrue(); + assertThat(context.getEnvironment().getActiveProfiles()).containsExactly(profile); + } + } +} diff --git a/src/sample-portfolio/src/systemTest/resources/application-test.yml b/src/sample-portfolio/src/systemTest/resources/application-test.yml index faaef3e2..23017f05 100644 --- a/src/sample-portfolio/src/systemTest/resources/application-test.yml +++ b/src/sample-portfolio/src/systemTest/resources/application-test.yml @@ -173,8 +173,8 @@ app: enabled: false publication: enabled: false - object-storage: - enabled: false + # No app.object-storage switch here: the shipped application.yml turns it off for every demo + # profile, and a copy here hid that `test` did not start without it (RR2-04-02). cache: redis: enabled: false diff --git a/src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/bootstrap/persistence/SampleMigrationLockWaitTest.java b/src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/bootstrap/persistence/SampleMigrationLockWaitTest.java new file mode 100644 index 00000000..36eb0a89 --- /dev/null +++ b/src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/bootstrap/persistence/SampleMigrationLockWaitTest.java @@ -0,0 +1,63 @@ +package dev.caskeleton.sample.portfolio.bootstrap.persistence; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.adapter.outbound.persistence.testkit.migration.LockWaitMigrationInspector; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TreeMap; +import java.util.stream.Stream; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; + +/** + * The sample's own stream follows the rule the template's streams follow: a migration that locks a + * table already in use bounds the wait first ({@code LockWaitMigrationInspector}, re-review + * 2026-11-03 round 2, RR-02-09). A service stream copies this test with its own directory. + */ +class SampleMigrationLockWaitTest { + + private static final Path STREAM = Path.of("src/main/resources/db/sample-migration"); + + /** Released before the rule and applied, so never edited again; nothing is added here. */ + private static final Set RELEASED_BEFORE_THE_RULE = + Set.of("V8__poster_image_publication.sql"); + + private final LockWaitMigrationInspector inspector = new LockWaitMigrationInspector(); + + @Test + @DisplayName("샘플 스트림의 규칙 이후 마이그레이션은 기존 테이블 잠금 앞에 SET LOCAL lock_timeout을 둔다") + void everySampleMigrationAfterTheRuleBoundsItsLockWait() throws IOException { + // given + Map migrations = versionedMigrations(); + + // when + Map> unbounded = new TreeMap<>(); + for (Map.Entry migration : migrations.entrySet()) { + List statements = + inspector.unboundedLockingStatements(Files.readString(migration.getValue())); + if (!statements.isEmpty()) { + unbounded.put(migration.getKey(), statements); + } + } + + // then + assertThat(unbounded.keySet()) + .as("these wait for a table lock without a bound: %s", unbounded) + .isEqualTo(RELEASED_BEFORE_THE_RULE); + } + + private static Map versionedMigrations() throws IOException { + Map migrations = new TreeMap<>(); + try (Stream files = Files.list(STREAM)) { + files + .filter(path -> path.getFileName().toString().matches("V[0-9_]+__.+\\.sql")) + .forEach(path -> migrations.put(path.getFileName().toString(), path)); + } + return migrations; + } +} diff --git a/src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoSecurityConfigTest.java b/src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoSecurityConfigTest.java index e5b8f976..62e15481 100644 --- a/src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoSecurityConfigTest.java +++ b/src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/bootstrap/security/SampleDemoSecurityConfigTest.java @@ -54,10 +54,7 @@ import org.springframework.test.web.servlet.ResultActions; properties = { "app.persistence-jpa.enabled=false", // The PostgreSQL idempotency store needs the relational module this test leaves off. - "app.capabilities.idempotency.provider=disabled", - // Object storage is on while present and has no destination here; the `local` profile used to - // switch it off by default, and this test runs under `test` (RR-07-02). - "app.object-storage.enabled=false" + "app.capabilities.idempotency.provider=disabled" }) @AutoConfigureMockMvc // The demo accounts start only under a demo profile (SampleDemoAccountsProductionGuard). diff --git a/src/shared-contract/src/main/java/dev/caskeleton/shared/error/OperationalError.java b/src/shared-contract/src/main/java/dev/caskeleton/shared/error/OperationalError.java index db6e1d52..9325e740 100644 --- a/src/shared-contract/src/main/java/dev/caskeleton/shared/error/OperationalError.java +++ b/src/shared-contract/src/main/java/dev/caskeleton/shared/error/OperationalError.java @@ -47,7 +47,7 @@ public enum OperationalError implements ApiErrorCode { IDEMPOTENT_IN_FLIGHT(Category.CONFLICT, 409, true), IDEMPOTENT_REQUEST_MISMATCH(Category.VALIDATION, 422, false), IDEMPOTENCY_UNAVAILABLE(Category.TRANSIENT_DEPENDENCY, 503, true), - // the action ran and recording its result failed; a resend with the same key is safe + // the action ran and recording its result failed; a resend with the same key never runs it again IDEMPOTENCY_COMPLETION_UNKNOWN(Category.TRANSIENT_DEPENDENCY, 503, true), IDEMPOTENCY_RECOVERY_REQUIRED(Category.DATA_INTEGRITY, 500, false), // adapter runtime lifecycle