# Notification Production Capability Implementation Plan > **Execution workflow:** 구현 시 `superpowers:subagent-driven-development`, > `superpowers:test-driven-development`, `superpowers:verification-before-completion`, > `superpowers:requesting-code-review`를 순서에 맞게 사용한다. 현재 세션에는 해당 skill package가 > 노출되지 않았으므로 이 문서는 저장소의 기존 Redis/Fileserver/HTTP Client 계획 형식과 동일한 > RED/GREEN/verification 규칙을 수동으로 명시한다. - 작성일: 2026-07-28 - 상태: Wave A/B 및 Task 8 구현 checkpoint를 main 통합. Task 9 이후는 미착수 - 원 작업 위치: `codex/notification-production-capability` 격리 worktree - 통합 위치: `/home/donghyeon/workspace/clean-architecture-backend-template` (`main`) - 현재 checkpoint: Wave A application/transaction foundation, Wave B catalog/render/provider fake protocol, Task 8 local crypto primitive 구현. Task 9의 migration/persistence model과 이후 provider/runtime 범위는 미착수다. 아래 isolated-worktree 검증 기록과 main 통합 검증 결과는 서로 다른 증거로 구분한다. - 설계 정본: [Notification Production Capability Deep Design](../specs/2026-07-28-notification-production-capability-design.md) - 비교한 실행 계획: [Redis Foundation](2026-07-28-redis-production-capability-foundation.md), [Redis Runtime](2026-07-28-redis-runtime-cache.md), [HTTP Client Foundation](2026-07-28-httpclient-production-capability-foundation.md), [HTTP Client Total Deadline](2026-07-28-httpclient-total-deadline.md), [Fileserver Foundation](2026-07-28-fileserver-production-capability-foundation.md), [Fileserver Durable Recovery](2026-07-28-fileserver-durable-recovery.md), [Fileserver R2](2026-07-28-fileserver-r2-control-plane-provider-selection.md) Repository commit policy는 모든 플랫폼에서 `human-only`다. 이 계획에는 `git add`, `git commit`, `git amend`, `git push` 단계가 없다. ## 1. Goal provider-neutral application intent를 다음 두 실행 모드로 안전하게 처리하는 production Notification capability를 구현한다. - `BEST_EFFORT_INLINE`: root business transaction의 physical commit 뒤 한정된 provider attempt를 수행하고 직교 outcome을 호출자에게 반환한다. - `DURABLE_ASYNC`: business state와 같은 PostgreSQL transaction에서 한 logical recipient의 intent와 frozen provider leg를 append하고, 별도 dispatcher가 claim, `WIRE_AUTHORIZED`, provider call, terminal-once result, receipt/reconciliation을 수행한다. 초기 qualification 대상은 다음 exact card 세 개뿐이다. ```text slack-web-api-inline-single-local-v1 slack-web-api-durable-single-local-v1 aws-ses-v2-durable-single-local-sns-v1 ``` Slack은 Web API `chat.postMessage`, email은 Amazon SES v2 `SendEmail`을 사용한다. SES feedback topology는 `configuration set -> SNS HTTPS adapter-inbound-web -> DLQ`로 고정한다. 이 계획은 외부 provider와 local DB 사이 exactly-once, inbox placement, read receipt 또는 production topology R3를 주장하지 않는다. ## 2. Architecture and fixed decisions ```text feature application policy -> NotificationKindPolicy -> BEST_EFFORT_INLINE -> TransactionPort.inRootWrite(business write) -> physical commit -> InlineNotificationAttemptPort -> DURABLE_ASYNC -> TransactionPort.inWrite(business write + NotificationIntentAppendPort) -> same PostgreSQL commit NotificationDispatchUseCase -> short claim transaction -> short ATTEMPT_RESERVED/WIRE_AUTHORIZED transaction -> render + exactly one authorized provider call outside DB transaction -> terminal-once result/frozen projection transaction SES SNS HTTPS callback -> inbound signature/account/topic verification -> NormalizedNotificationReceiptCommand -> application receipt reducer -> PostgreSQL receipt/orphan/suppression projection ``` 고정 결정: 1. source business state와 notification journal은 같은 PostgreSQL transaction manager에 참여한다. 2. intent 하나는 logical recipient 한 명이고 delivery row는 provider leg다. 3. mode와 admission class는 application `NotificationKindPolicy`만 결정한다. 4. config의 `expected-mode`는 assertion이며 mode override가 아니다. 5. provider call은 DB transaction 밖에서만 수행한다. 6. claim owner token과 immutable attempt execution token을 분리한다. 7. `WIRE_AUTHORIZED` commit을 local linearization point로 사용한다. 8. transmission certainty, retry disposition, fault scope를 한 enum으로 합치지 않는다. 9. binding/account fault는 PostgreSQL shared admission gate를 `PARKED`로 만들고 initial fallback을 자동 활성화하지 않는다. 10. PII payload는 `DIRECT_AEAD_AES_256_GCM_V1`, lookup/dedupe는 purpose-separated versioned HMAC을 사용한다. 11. 기존 `slack-webhook`, `google-email`, raw `NotificationPort`는 R0 legacy path로 동결한 뒤 route cutover가 증명되면 제거한다. 12. legacy→canonical 전환은 PostgreSQL fence, bounded permit, append-only operation/ attestation sequence, retained signed-evidence header와 proof registry로 수행한다. 13. BEGIN은 독립 infrastructure issuer가 서명한 complete old-node inventory header와 node rows를 server-side 검증·동결한다. `QUIESCENCE_REQUIRED` COMPLETE는 그 BEGIN에 귀속된 exact signed per-node irreversible deployment-generation tombstone, legacy credential-or-egress revocation, ACTIVE permit 0과 provider-call-ledger open-count 0의 durable proof를 요구한다. 이 사실은 monotonic/irreversible하므로 application pre-commit freshness나 DEFERRABLE trigger를 COMPLETE authority로 사용하지 않는다. 14. cutover causality는 post-lock shared DB sequence와 explicit BEGIN FK가 SSOT다. timestamp는 post-lock `clock_timestamp()` 보조 evidence이고 transaction-start 시각은 사용하지 않는다. 15. V8은 complete canonical upgrade history를 검증·보존하고 discriminator를 `UPGRADE_VALIDATED`+validated-history digest로 닫으며, exact empty database는 `AWAITING_SIGNED_FRESH_PROVISIONING`+두 arm field null로 남긴다. fresh canonical seed는 migration 안에서 만들지 않는다. 별도 final-artifact `notificationFreshProvisioning` Gradle/CLI가 independent infrastructure issuer의 signed DB-birth certificate와 이미 committed irreversible no-legacy-authority fence를 Java로 검증한 뒤, exact two-method/ two-function PostgreSQL provisioning port를 통해 provenance, `INITIALIZE_CANONICAL_FRESH`, 모든 canonical fence와 `FRESH_PROVISIONED`+fresh token을 한 transaction으로 생성한다. runtime은 이 commit과 retained Java 재검증 전까지 dark다. 16. 정확히 세 role만 둔다. `notification_migrator`는 Flyway/schema owner, `notification_runtime`은 non-owner runtime role, `notification_provisioner`는 fresh provisioning exact two-function operation 전용 role이며 그 밖의 제4 notification role은 만들지 않는다. raw database credential은 각 전용 reference로만 해석하고 production artifact에는 issuer private key를 넣지 않는다. ## 3. Scope boundary and owner leaves 정확한 leaf와 production dependency edge는 `src/config/architecture/modules.json`에서 파생한다. 이 계획은 registry edge를 추가하지 않는다. | 책임 | owner leaf | Gradle path | 기존 허용 edge | | --- | --- | --- | --- | | semantic values, policy, port, use case | `application-core` | `:application-core` | `domain-core`, `shared-contract` | | provider catalog/render/SPI/Slack/SES | `adapter-outbound-notification` | `:adapter:outbound:notification` | `domain-core`, `application-core`, `shared-contract`, `adapter-outbound-support` | | transaction/crypto/schema/store/claim | `adapter-outbound-persistence-jpa` | `:adapter:outbound:persistence-jpa` | `domain-core`, `application-core`, `shared-contract` | | SNS HTTPS verification/transport mapping | `adapter-inbound-web` | `:adapter:inbound:web` | `domain-core`, `application-core`, `shared-contract` | | canonical graph/composition/scheduler/readiness | `app-bootstrap` | `:app-bootstrap` | registry에 등록된 runtime leaves | 금지: - `domain-core`에 notification framework/transport/persistence 개념을 추가하지 않는다. - notification leaf가 persistence, inbound-web, httpclient sibling leaf를 의존하지 않는다. - inbound-web가 notification outbound adapter 타입을 import하지 않는다. - app-bootstrap settings/configuration에 mode, retry, fallback, consent 같은 정책을 구현하지 않는다. - sample WorkLog를 production Notification consumer로 만들지 않는다. ## 4. Evidence ladder and claim rule | evidence | 허용되는 주장 | | --- | --- | | application unit/contract | framework-free semantic/state policy가 정의됨 | | adapter fake/loopback protocol | local render와 provider request/outcome mapping이 정의됨 | | real PostgreSQL concurrency/fault | same-DB append와 provider-neutral durable protocol의 local evidence | | Slack sandbox | exact Slack card의 provider evidence | | SES sandbox + actual SNS callback | exact SES/SNS card의 provider/feedback evidence | | privacy/load/rotation/rollout drill | selected card의 operational R2 | 낮은 row의 evidence를 높은 row나 다른 provider/account/region/workspace/mode로 일반화하지 않는다. 실 provider lane이 실행되지 않으면 코드는 구현될 수 있어도 해당 exact card는 `NOT_QUALIFIED`다. ## 5. Execution rules 1. 모든 checkbox는 구현 시작 시 `[ ]`에서 시작한다. 2. 각 behavior task는 먼저 명시한 test를 작성하고 같은 focused command로 RED와 GREEN을 확인한다. 3. RED가 예상 원인이 아니라 compilation drift, 외부 환경 또는 unrelated dirty change로 실패하면 구현하지 말고 원인을 먼저 분리한다. 4. RED가 처음부터 통과하면 기존 coverage 또는 plan drift를 조사하고 test를 강화한다. 5. ordinary `test`/`check`에는 실제 network, credential, account 또는 skip 기반 성공을 넣지 않는다. 6. task가 끝날 때 focused test, owner leaf test/check, 그 task가 건드린 boundary gate 순으로 검증한다. 7. migration은 expand-first다. 새 worker와 provider는 canonical binding 전까지 dark/disabled다. 8. canonical disabled + legacy absent인 `PURE_DISABLED`에서 provider client, thread, scheduler, probe, callback subscription, operator와 application/runtime table DML·scan은 0이어야 한다. PRE legacy-only bridge는 별도 closed state다. expand-first V7 DDL, V8 structural validation과 explicitly invoked final `notificationFreshProvisioning`은 runtime zero-resource 계수에서 제외한다. exact empty FINAL startup은 `AWAITING_SIGNED_FRESH_PROVISIONING`으로 liveness만 유지하고 provider/worker/admission/DML은 0이다. 9. rollback 시 accepted/indeterminate intent를 legacy path로 자동 resend하지 않는다. 10. active/retained row가 참조하는 template, binding, renderer, AEAD/HMAC revision을 제거하지 않는다. 11. shared workspace의 기존 Fileserver/JPA/Object Storage 및 file-publication 변경을 덮어쓰지 않는다. 각 task 시작 전 `git status --short`로 overlap을 다시 확인한다. 12. 새 타입이나 파일이 이 계획의 surface 밖에 필요하면 조용히 확장하지 말고 plan을 먼저 갱신한다. 13. 각 Wave exit에서 LLM Wiki branch-note를 갱신해 files, decisions, commands/results, evidence grade와 blocker를 남긴다. 파생 raw 문서가 없으면 cluster에 “없음”을 명시한다. ## 6. Stop conditions 다음 중 하나라도 확인되면 해당 wave를 중단하고 설계/계획을 수정한다. - business DB와 notification journal이 같은 transaction manager에 참여하지 못한다. - `TransactionPort.inRootWrite`가 ambient actual transaction을 side effect 전에 거부하거나 physical commit-before-return을 보장하지 못한다. - Slack/AWS SDK의 hidden retry를 끄거나 실제 physical attempt를 journal에 계수할 수 없다. - SES configuration set, SNS TopicArn, HTTPS ingress, DLQ topology를 exact profile로 묶을 수 없다. - provider callback signature 검증에 outbound notification adapter 의존이 필요하다. - migration `V7`이 실행 시점에 이미 다른 의미로 사용 중이다. - canonical/legacy activation을 동시에 허용해야만 rollout이 가능하다. - independent issuer가 complete old-node inventory와 per-node irreversible deployment/credential/egress fence를 서명·검증할 수 없다. - BEGIN inventory 또는 quiescence evidence의 canonical signed payload, signature, issuer/trust snapshot, issued/expires/verified time, environment/DB/artifact, consumer inventory identity, provider-call-ledger identity/snapshot과 zero-node authority를 retained header로 보존하고 Java write/startup에서 재검증할 수 없다. - COMPLETE 뒤 paused old node나 revoked legacy credential/egress identity가 provider I/O 0임을 증명하지 못한다. - V8이 exact empty를 `AWAITING_SIGNED_FRESH_PROVISIONING`으로 분류하거나 nonempty missing-fence database를 mutation 0으로 거부하지 못한다. - independent infrastructure issuer가 committed irreversible no-legacy-authority fence를 먼저 확인한 signed DB-birth authorization과 final-artifact `notificationFreshProvisioning` one-transaction/two-function protocol을 제공하지 못한다. - `notification_migrator`/`notification_runtime`/`notification_provisioner` role, safe `SECURITY DEFINER` ownership/grant closure 또는 FINAL DML/sequence/execute revoke를 증명하지 못한다. - real-provider evidence가 없는데 R2/production-ready 표현이 필요하다. --- ## Wave A — Truth freeze and application foundation ### Task 1: Freeze current R0 truth and protect unrelated changes **Owner leaf:** documentation + existing notification/bootstrap tests **Depends on:** approved deep design **Behavior change:** none **Files:** - Modify: `src/adapter/outbound/notification/README.md` - Modify: `src/adapter/outbound/notification/CLAUDE.md` - Test: `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/core/NotificationAdapterTest.java` - Test: `src/app-bootstrap/src/test/java/dev/caskeleton/adapter/outbound/OptionalAdapterBeanGatingTest.java` - [x] Record the starting `git status --short`, current branch, registry edges and current provider dependencies in the branch-note. - [x] Re-run the existing R0 behavior without changing it: ```bash cd src && ./gradlew :adapter:outbound:notification:test \ --tests '*NotificationAdapterTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*OptionalAdapterBeanGatingTest' --console=plain ``` - [x] Document one truth table covering raw `NotificationPort`, fan-out routing, global fail-open, fake-only `google-email`/`slack-webhook`, selector drift and production consumer count 0. - [x] Mark all current provider seams `R0 legacy`; do not call them Slack/Email integration. - [x] Preserve a deletion inventory for Wave G rather than adding behavior to legacy classes. - [x] Acceptance: the baseline is reproducible, no source behavior changes, no unrelated dirty file changes. **Rollback checkpoint:** documentation-only changes may be reverted independently; legacy tests remain the executable baseline until the canonical-only cutover and deletion in Task 21. ### Task 2: Add the physical root-write transaction contract **Owner leaves:** `application-core` (`:application-core`), then `adapter-outbound-persistence-jpa` (`:adapter:outbound:persistence-jpa`) **Depends on:** Task 1 **Files:** - Create: `src/application-core/src/main/java/dev/caskeleton/application/transaction/NestedRootTransactionRejectedException.java` - Modify: `src/application-core/src/main/java/dev/caskeleton/application/transaction/TransactionPort.java` - Modify: `src/application-core/src/test/java/dev/caskeleton/application/transaction/TransactionPortTest.java` - Modify: `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/transaction/SpringTransactionPort.java` - Modify: `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/transaction/SpringTransactionPortTest.java` - Modify: `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/CleanArchitectureTest.java` - Modify: `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/ArchitectureViolationFixtureTest.java` - Modify: `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/violations/application/MissingTransactionBoundaryUseCase.java` - Create: `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/fixtures/application/RootWriteTransactionBoundaryUseCase.java` - Modify only as mechanical interface implementers: `src/application-core/src/test/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCaseTest.java`, `src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/authz/WorkLogAuthorizationE2ETest.java`, `src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/authz/WorkLogAuthorizationContractTest.java`, `src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/ListRecentWorkLogSummariesUseCaseTest.java`, `src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogOutboxTest.java`, `src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/WorkLogUseCasesTest.java` - Modify: `src/application-core/README.md`, `src/application-core/CLAUDE.md`, `src/adapter/outbound/persistence-jpa/README.md`, `src/adapter/outbound/persistence-jpa/CLAUDE.md` - [x] Write RED tests proving: `inRootWrite` is part of the framework-free contract; ambient actual transaction is rejected before action/TM side effects; root execution is `WRITE + REQUIRED + READ_COMMITTED`; return occurs after commit; commit failure propagates and no caller-visible committed result is produced. - [x] Verify RED: ```bash cd src && ./gradlew :application-core:test \ --tests '*TransactionPortTest' --console=plain cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*SpringTransactionPortTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*CleanArchitectureTest' \ --tests '*ArchitectureViolationFixtureTest' \ --console=plain ``` Expected failure: `inRootWrite`/typed rejection and adapter behavior do not exist. - [x] Add an abstract `inRootWrite` contract and update all current fake implementations explicitly; do not provide a default that silently delegates to join-capable `inWrite`. - [x] Implement the adapter precondition with actual transaction state inspection before `TransactionTemplate.execute`. - [x] Reuse a prebuilt `WRITE + REQUIRED + READ_COMMITTED` template. Do not add `NEVER` propagation or a new `TransactionMode`. - [x] Extend the transaction fitness rule so a `WRITE_REPOSITORY + WRITE` use case may directly call either join-capable `inWrite` or root-only `inRootWrite`, while READ/REQUIRES_NEW mappings stay unchanged. Add positive and negative fixtures so this is not a broad transaction bypass. - [x] Verify GREEN with the same three commands. - [x] Verify the architecture RED/GREEN with the third command too; the violation fixture must fail for a declared WRITE boundary that calls neither `inWrite` nor `inRootWrite`, while the positive root-write fixture passes. - [x] Run compatibility regression: ```bash cd src && ./gradlew :application-core:test \ :adapter:outbound:persistence-jpa:test \ :sample-portfolio:test --console=plain ``` - [x] Acceptance: nested use fails before action/provider call, root return is post-commit, existing `inWrite`/`inRead`/`inNew` semantics are unchanged. **Rollback checkpoint:** this public contract cannot be rolled back after Task 4 callers use it. Before that point, revert the interface and all mechanical fake changes together. ### Task 3: Introduce bounded application notification values and policy **Owner leaf:** `application-core` (`:application-core`) **Depends on:** Task 2 **Files — create under** `src/application-core/src/main/java/dev/caskeleton/application/notification/`: - `NotificationChannel.java` - `NotificationIntentId.java` - `NotificationDeliveryId.java` - `NotificationAttemptId.java` - `NotificationReceiptEventId.java` - `NotificationKindId.java` - `NotificationRouteId.java` - `NotificationTemplateRef.java` - `NotificationMode.java` - `NotificationAdmissionClass.java` - `NotificationRouteStrategy.java` - `ConsentCheckMode.java` - `NotificationRecipientReference.java` - `EmailRecipientReference.java` - `SlackAudienceReference.java` - `NotificationTemplateValue.java` - `NotificationTemplateParameters.java` - `NotificationKindPolicy.java` - `NotificationFrozenPlan.java` - `NotificationIntentDraft.java` - `SubmissionCertainty.java` - `RetryDisposition.java` - `NotificationFaultScope.java` - `NotificationReasonCode.java` - `ProviderAttemptOutcome.java` - `TargetAttemptOutcome.java` - `NotificationRequestResult.java` **Tests — create:** - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationValueContractTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationKindPolicyTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationRequestResultTest.java` - [x] Write RED tests for bounded/nonblank IDs, one-recipient typing, closed template scalar types, locale/time bounds, immutable collections, redacted `toString`, and exact orthogonal outcome axes. - [x] Write RED policy tests proving mode/admission are code-owned, config cannot strengthen or weaken them, and a critical kind cannot bind `BEST_EFFORT_INLINE`. - [x] Verify RED: ```bash cd src && ./gradlew :application-core:test \ --tests '*NotificationValueContractTest' \ --tests '*NotificationKindPolicyTest' \ --tests '*NotificationRequestResultTest' \ --console=plain ``` Expected failure: the new semantic types and invariants do not exist. - [x] Implement only Java 21/framework-free records, sealed interfaces and immutable collections. - [x] Do not add provider IDs, AWS/Slack types, raw JSON, `Map`, inbound DTOs, raw HTML or arbitrary address/channel union types. - [x] Keep feature-specific factory examples in test fixtures; do not add WorkLog or password-reset business concepts to production packages. - [x] Verify GREEN with the same command and then: ```bash cd src && ./gradlew :application-core:test --console=plain ``` - [x] Acceptance claim: application semantic contract R1 only; no provider or durable evidence yet. **Rollback checkpoint:** no external side effects/schema. Revert this whole value cluster before ports in Task 4 depend on it. ### Task 4: Add application ports, dispatcher and receipt reducer contracts **Owner leaf:** `application-core` (`:application-core`) **Depends on:** Task 3 **Files — create under** `src/application-core/src/main/java/dev/caskeleton/application/notification/`: - `InlineNotificationAttemptPort.java` - `NotificationIntentAppendPort.java` - `NotificationPlanPort.java` - `NotificationPlanningResult.java` - `NotificationAppendResult.java` - `NotificationDeliveryStorePort.java` - `NotificationProviderAttemptPort.java` - `NotificationTechnicalSuppressionPort.java` - `NotificationReceiptStorePort.java` - `NotificationMaintenanceStorePort.java` - `NotificationReconciliationPort.java` - `NotificationAdmissionReadinessPort.java` - `NotificationCanonicalWriterFencePort.java` - `NotificationCanonicalWriterRouteSet.java` - `NotificationWriterRouteSet.java` - `NotificationWriterCutoverPort.java` - `NotificationWriterQuiescenceAttestationPort.java` - `NotificationWriterInventoryEvidenceVerifierPort.java` - `NotificationWriterInventoryEvidence.java` - `NotificationSignedEvidenceHeader.java` - `NotificationEvidenceTrustSnapshot.java` - `SignedNotificationWriterInventoryManifest.java` - `SignedNotificationWriterQuiescenceManifest.java` - `InitializeNotificationWriterFencesCommand.java` - `InitializeNotificationWriterFencesResult.java` - `InitializeNotificationWriterFencesOperation.java` - `InitializeNotificationWriterFencesUseCase.java` - `NotificationCanonicalWriterFenceGuard.java` - `NotificationLegacyWriterPermitCommand.java` - `NotificationLegacyWriterPermitResult.java` - `NotificationLegacyWriterPermitUseCase.java` - `TerminalizeExpiredNotificationWriterPermitsCommand.java` - `TerminalizeExpiredNotificationWriterPermitsResult.java` - `TerminalizeExpiredNotificationWriterPermitsOperation.java` - `TerminalizeExpiredNotificationWriterPermitsUseCase.java` - `RecordNotificationWriterQuiescenceAttestationCommand.java` - `RecordNotificationWriterQuiescenceAttestationResult.java` - `RecordNotificationWriterQuiescenceAttestationOperation.java` - `RecordNotificationWriterQuiescenceAttestationUseCase.java` - `SwitchNotificationWriterOwnershipCommand.java` - `SwitchNotificationWriterOwnershipResult.java` - `SwitchNotificationWriterOwnershipOperation.java` - `SwitchNotificationWriterOwnershipUseCase.java` - `NotificationWriterOwnership.java` - `NotificationDispatchCommand.java` - `NotificationDispatchResult.java` - `NotificationDispatchUseCase.java` - `NormalizedNotificationReceiptCommand.java` - `NotificationReceiptFact.java` - `NotificationReceiptProjection.java` - `ApplyNotificationReceiptCommand.java` - `ApplyNotificationReceiptResult.java` - `ApplyNotificationReceiptUseCase.java` - `NotificationAdmissionGateCommand.java` - `NotificationAdmissionGateUseCase.java` - `ReconcileNotificationDeliveriesCommand.java` - `ReconcileNotificationDeliveriesResult.java` - `ReconcileNotificationDeliveriesUseCase.java` - `NotificationMaintenanceCommand.java` - `NotificationMaintenanceResult.java` - `NotificationMaintenanceUseCase.java` - `NotificationProviderCapabilityDescriptor.java` - `NotificationStoreCapabilityDescriptor.java` - `NotificationReceiptIngressCapabilityDescriptor.java` - `NotificationCapabilityCompatibilityValidator.java` - `NotificationOperationsSnapshotPort.java` - `NotificationOperationsSnapshot.java` - `NotificationOperationsSnapshotQuery.java` - `NotificationOperationsSnapshotUseCase.java` - `NotificationApplicationException.java` **Tests — create:** - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationPortBoundaryTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationPlanningBoundaryTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationDispatchUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationReceiptReducerTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationAdmissionGateUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationCanonicalWriterFenceGuardTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/InitializeNotificationWriterFencesUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationLegacyWriterPermitUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/TerminalizeExpiredNotificationWriterPermitsUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/RecordNotificationWriterQuiescenceAttestationUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/SwitchNotificationWriterOwnershipUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/ReconcileNotificationDeliveriesUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationMaintenanceUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationCapabilityCompatibilityValidatorTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationOperationsSnapshotUseCaseTest.java` - [x] Execute Task 4 as five sequential RED/GREEN subcycles, never as one large implementation: (1) port/planning boundary, (2) dispatch state machine, (3) receipt reducer, (4) admission/reconciliation/maintenance, and (5) compatibility descriptors/validator. Run only the named focused test(s) for a subcycle before starting the next, then run the combined command below. - [x] Write RED port tests proving append joins caller transaction semantics, provider SDK/entity/DTO types are absent, and no giant send/store/receipt port is introduced. - [x] Write RED planning handoff tests: feature factory + `NotificationKindPolicy` produce a draft; `NotificationPlanPort` returns only application-owned `NotificationFrozenPlan`; append and inline ports consume that frozen plan; adapter compiled binding/profile types never cross into application or persistence. - [x] Write RED dispatcher tests for the sequence: short claim transaction -> short reserve/authorize transaction -> provider outside transaction -> terminal-once result transaction. - [x] Cover stale claim token, distinct attempt execution token, late exact result, fallback only on `DEFINITELY_NOT_APPLIED`, and terminal indeterminate without blind retry. - [ ] Cover `PARK_BINDING`: gate CAS by scope/generation, parked leg not hot-looping, audited resume rechecking expiry/cancel/suppression and not activating initial fallback. - [ ] Cover a route-specific single-writer fence with an exact database generation and owner (`LEGACY` or `CANONICAL`). Both legacy admission and canonical intent admission must present the expected generation; stale/mismatched ownership fails closed before append or provider I/O. Split the transaction contracts: `NotificationCanonicalWriterFenceGuard` asserts canonical ownership inside the caller's business-write/intent-append transaction and holds a tested share lock that conflicts with `BEGIN_DRAIN` until physical commit/rollback; `NotificationLegacyWriterPermitUseCase` physically commits a bounded acquire before provider I/O, returns DB-time acquired/wire-deadline/expiry facts, and releases afterward. The wrapper/ client refuses network start after the committed absolute wire deadline and enforces the smaller of its monotonic elapsed budget and DB interval; `TerminalizeExpiredNotificationWriterPermitsUseCase` is a distinct PRE-only authenticated root-write operation that, only during exact DRAINING, terminalizes bounded DB-time-expired ACTIVE permits across all generations with registry-bound token/rowVersion CAS; `SwitchNotificationWriterOwnershipUseCase` physically commits the audited `BEGIN_DRAIN`, `COMPLETE_SWITCH` or `ABORT_DRAIN` CAS. `BEGIN_DRAIN` verifies an independently signed, short-lived exact environment/DB/route/PRE-artifact complete old-writer inventory through `NotificationWriterInventoryEvidenceVerifierPort`, then atomically freezes its canonical node row set/count/digest while closing new legacy acquire; caller-authored node digests have no authority. The operations query polls active count/max expiry outside a transaction; `COMPLETE_SWITCH` refuses unsafe legacy permits. Freeze the only transition matrix: `ACTIVE/LEGACY@g -> DRAINING/LEGACY@g` (BEGIN), unchanged DRAINING (terminalize), `DRAINING/LEGACY@g -> ACTIVE/CANONICAL@g+1` (COMPLETE), or `DRAINING/LEGACY@g -> ACTIVE/LEGACY@g+1` (ABORT). Requests never supply a target owner; every action from CANONICAL and every reverse transition fails with mutation 0. Crashed permits become `EXPIRED_PROVEN` only for a transport profile with tested hard bounds; current R0 timeouts become `TIMED_OUT_UNPROVEN`. For an unproven profile, an authenticated, immutable, fresh route/generation quiescence attestation is mandatory even when the unproven set is empty. Its independently signed manifest must exactly match the complete BEGIN node inventory and bind per-node retired/quiesced facts plus deployment-generation tombstones and legacy credential/egress revocation that make resume impossible, production consumer inventory/count 0 and provider-call ledger identity/open-count 0. ACTIVE permit 0 remains mandatory for every profile and cannot be overridden by attestation. It includes the bounded canonical digest of every generation/profile `TIMED_OUT_UNPROVEN` `(token,generation,profile,state,rowVersion)` permit tuple, the distinct permit-holder set and the persisted transport-proof registry; COMPLETE locks/recomputes the exact sets and records its token/digest. Every permit holder must be in the BEGIN inventory. The route-level proof requirement is `QUIESCENCE_REQUIRED` when any current/retiring persisted registry profile is `QUIESCENCE_REQUIRED`; only an all-`HARD_BOUND_PROVEN` registry may use the hard-bound path. Every switch operation carries authenticated actor, reason and reviewed target generation. For every COMPLETE arm, application validation alone is insufficient: the persistence port must lock the retained BEGIN inventory header/children and pass its canonical payload/signature/SPKI/trust snapshot back through the Java verifier before the ownership mutation. It rejects a missing, altered, unverifiable or semantically mismatched BEGIN even when the permit/registry rows are structurally valid. QUIESCENCE_REQUIRED additionally locks and revalidates the committed signed quiescence header, exact per-node tombstone/revocation rows, ACTIVE permit 0 and provider-ledger open-count 0. HARD_BOUND_PROVEN forbids a quiescence header/children but still requires the verified signed BEGIN plus the exact all-hard-bound registry/evidence revision, safe terminal permits and ACTIVE permit 0. The ownership transaction returns only after physical commit. Manifest expiry gates admission into an immutable evidence header; once verified and committed, the signed deployment/credential/egress fences are durable monotonic facts and are not converted into a pre-commit TTL guard. No use case sleeps or holds a DB transaction while waiting. The route's exact transport profile/proof/evidence registry comes from `NotificationWriterRouteSet`, not permit rows or request data; therefore a `QUIESCENCE_REQUIRED` route with permit count 0 still requires attestation. - [ ] Cover audited batch `InitializeNotificationWriterFencesUseCase`: it rejects ambient transactions and root-commits the bounded ordered reviewed route set as `ACTIVE/LEGACY@predecessor` only when the command set/digest exactly equals `NotificationWriterRouteSet` derived from the compiled cutover route catalog and every notification control/data-plane journal table is empty. It inserts all fences, one immutable operation header, all route-result children and the exact route/profile/ admission-role/proof-class/evidence-revision registry snapshot atomically; partial or sequential route initialization is forbidden. Initialization and every later switch append immutable operation history in the same root transaction as fence mutation. Replay of any old same-token/same-input route set returns its stored committed result; token reuse with a different route set/action/input and existing/mismatched/nonempty state fail without mutation. A mutable `last_operation_token` fence field is never the audit or replay SSOT. - [ ] Cover `TerminalizeExpiredNotificationWriterPermitsUseCase`: it is not a scheduler, snapshot query or COMPLETE side effect. An authenticated PRE operator supplies exact route/drain generation, bounded batch (`<=100`), reason and opaque operation token. The root transaction locks `DRAINING/LEGACY`, the immutable persisted registry and DB-time-expired ACTIVE permits across all historical generations in canonical order. It CASes exact token/rowVersion to `EXPIRED_PROVEN` only for HARD_BOUND_PROVEN or `TIMED_OUT_UNPROVEN` only for QUIESCENCE_REQUIRED, then appends affected count/set digest and actor/reason in the operation journal before commit. Same-token/same-input replay returns the stored result; changed input, non-DRAINING fence, unknown/drifted profile, nonexpired row or commit failure changes nothing. Idempotency lookup precedes set selection, so replay still returns the original affected result after those rows are terminal. The affected set is a derived result, not caller input, and its digest covers the sorted immutable post-CAS tuple. The operation journal persists requested batch bound and a server-canonical request-input digest so changed-input token reuse fails. It performs provider I/O 0 and never updates the fence's latest-mutation pointer. - [ ] Cover `RecordNotificationWriterQuiescenceAttestationUseCase`: only a method-security operator path may root-commit an immutable exact route/draining-generation/transport-profile set attestation after BEGIN_DRAIN. It accepts a bounded signed quiescence manifest, verifies it through the trusted issuer-key port, and server-derives the bounded sorted blocking permit `(token,generation,profile,state,rowVersion)` set/count/digest, distinct holder set, frozen BEGIN old-node set, consumer inventory/count 0 and provider-call-ledger identity/open-count 0. Exact node-set equality, holder subset, per-node retired/quiesced + restart/credential/egress revocation facts, environment/DB/artifact/ generation identity and bounded freshness are mandatory; caller-provided node/zero-fact digest is never authoritative. Same-token/same-input replay is idempotent and mismatch is rejected. `COMPLETE_SWITCH` for an unproven transport requires the attestation token and fails on missing/stale/wrong-route/wrong-generation, partial multi-profile/node coverage, omitted/extra node or permit holder, changed permit-set digest, registry mismatch, unsigned/unknown-issuer evidence or nonzero facts. After initialization, proof data comes from the immutable persisted registry/BEGIN inventory; caller data and permit rows cannot invent it. Both BEGIN and attestation persist a first-class immutable signed-evidence header containing exact canonical payload bytes/profile, signature bytes/digest, algorithm, issuer key ID, bounded canonical issuer public-key SPKI plus its digest and trust-catalog revision/historical-key-status snapshot, issued/expires/verified DB times, profile-pinned `allowedClockSkew` and `acceptanceMargin`, environment/DB/artifact identity, consumer-inventory identity, provider-call-ledger identity/ snapshot and canonical child count/set digest. A zero-node manifest still creates one header, so issuer authority is never hidden in absent child rows. Java write validation and canonical startup reverify stored payload/signature/SPKI, header/child exact equality and that the historical key digest remains allowed/non-revoked in the current closed catalog. Admission requires `issuedAt - allowedClockSkew <= serverVerifiedAt <= expiresAt - acceptanceMargin`; after acceptance, expiry does not reverse the recorded irreversible facts. SQL owns only structural FK/digest/state/window-shape constraints and never claims Ed25519 verification. - [ ] Cover the stale-node safety proof that closes the paused-node race: pause an old bridge node after its last permitted local step, commit BEGIN, signed per-node deployment-generation tombstone plus legacy credential/egress revocation, ACTIVE permit 0, provider-ledger 0 and COMPLETE, then resume that exact process. Its legacy client must fail before provider network I/O, the revoked credential/egress identity must record provider-call count 0, and canonical ownership must remain the only admitted writer. Repeat for a cached credential and an already constructed client/connection. If this cannot be proven, keep the route DRAINING and `NOT_QUALIFIED`. - [x] Cover periodic reconciliation as a separate use case: bounded claim transaction -> provider reconciliation outside transaction -> token/version guarded result transaction; orphan attach without provider I/O stays in the bounded store transaction. A scheduler must not call store/provider ports itself. - [x] Cover receipt fact permutations so `SEND/DELIVERY/BOUNCE/COMPLAINT/DELIVERY_DELAY` produce the same orthogonal projection independent of order; accepted fact is never erased. - [x] Keep technical-suppression policy in the receipt reducer/application use case: hard bounce and complaint may emit an explicit suppression mutation; transient/delayed/soft bounce does not. Business consent/unsubscribe remains outside this capability. - [x] Cover the pure compatibility validator: application policy mode/admission, provider/store/ ingress descriptors, exact receipt requirement and frozen revision availability. Concrete adapter settings/types must not enter the validator. - [x] Cover the operational read boundary: bootstrap never calls `NotificationOperationsSnapshotPort` directly. A concrete `NotificationOperationsSnapshotUseCase implements QueryUseCase` invokes the port only inside `TransactionPort.inRead` and returns bounded, non-sensitive values. - [x] Verify RED: ```bash cd src && ./gradlew :application-core:test \ --tests '*NotificationPortBoundaryTest' \ --tests '*NotificationPlanningBoundaryTest' \ --tests '*NotificationDispatchUseCaseTest' \ --tests '*NotificationReceiptReducerTest' \ --tests '*NotificationAdmissionGateUseCaseTest' \ --tests '*NotificationCanonicalWriterFenceGuardTest' \ --tests '*InitializeNotificationWriterFencesUseCaseTest' \ --tests '*NotificationLegacyWriterPermitUseCaseTest' \ --tests '*TerminalizeExpiredNotificationWriterPermitsUseCaseTest' \ --tests '*RecordNotificationWriterQuiescenceAttestationUseCaseTest' \ --tests '*SwitchNotificationWriterOwnershipUseCaseTest' \ --tests '*ReconcileNotificationDeliveriesUseCaseTest' \ --tests '*NotificationMaintenanceUseCaseTest' \ --tests '*NotificationCapabilityCompatibilityValidatorTest' \ --tests '*NotificationOperationsSnapshotUseCaseTest' \ --console=plain ``` Expected failure: ports/use cases/state transitions do not exist. - [x] Implement `NotificationDispatchUseCase` as a manually wired `CommandUseCase` with `WRITE`, `IDEMPOTENT`, `WRITE_REPOSITORY`, `externalOutboundAllowed=true` capability metadata. - [x] Make receipt apply, admission operation, writer-fence initialization, legacy writer permit, expired-permit terminalization, quiescence attestation, writer ownership switch and maintenance concrete `CommandUseCase` implementations too. The canonical guard is an internal application policy collaborator invoked only from an existing application write use case, not a bootstrap-callable `*UseCase`. Annotate every concrete use case with exact existing capability vocabulary and a type-level permission: dispatch `notification:dispatch`, receipt apply `notification:receipt`, admission operation `notification:operate`, fence initialization and ownership switch `notification:cutover`, legacy writer permit `notification:cutover-admit`, expired-permit terminalization `notification:cutover-terminalize`, quiescence attestation `notification:cutover-attest`, maintenance `notification:maintain`. Receipt apply uses `WRITE + WRITE_REPOSITORY + IDEMPOTENT` and calls `inRootWrite`; dispatch/maintenance use `externalOutboundAllowed=true` only when they actually call provider/reconciliation ports. Do not add a `capabilities.yaml` row because no new capability attribute is introduced. - [x] Freeze the exact capability matrix: | use case | transaction/repository | idempotency | external | direct boundary | | --- | --- | --- | --- | --- | | dispatch | `WRITE` / `WRITE_REPOSITORY` | `IDEMPOTENT` | true | `inWrite` claim/authorize/finalize | | receipt apply | `WRITE` / `WRITE_REPOSITORY` | `IDEMPOTENT` | false | `inRootWrite` | | admission operate | `WRITE` / `WRITE_REPOSITORY` | `IDEMPOTENT` | true | probe outside, then `inWrite` | | writer fence initialize | `WRITE` / `WRITE_REPOSITORY` | `IDEMPOTENT` | false | `inRootWrite` | | legacy writer permit | `WRITE` / `WRITE_REPOSITORY` | `IDEMPOTENT` | false | `inRootWrite` acquire/release | | expired permit terminalize | `WRITE` / `WRITE_REPOSITORY` | `IDEMPOTENT` | false | bounded `inRootWrite` | | writer quiescence attest | `WRITE` / `WRITE_REPOSITORY` | `IDEMPOTENT` | false | `inRootWrite` | | writer ownership switch | `WRITE` / `WRITE_REPOSITORY` | `IDEMPOTENT` | false | `inRootWrite` | | reconcile | `WRITE` / `WRITE_REPOSITORY` | `IDEMPOTENT` | true | `inWrite`, provider outside | | maintenance | `WRITE` / `WRITE_REPOSITORY` | `IDEMPOTENT` | false | bounded `inWrite` | | operations snapshot | `READ_ONLY` / `READ_REPOSITORY` | `IDEMPOTENT` | false | `inRead` | `sensitiveRead=true` for dispatch and reconcile because their safe application models still carry decrypted recipient/template data or opaque provider references; receipt/admission/ maintenance may remain false only when tests prove their application values contain digest/ciphertext/closed reason fields rather than plaintext. `bulkWrite=false` is valid only because every claim/receipt/maintenance batch is validated `<=100`; raising that cap requires `bulkWrite=true`. Initial infrastructure dispatch, provider/account admission, reconcile and retention sweeps span tenant partitions and therefore declare `crossTenantAdmin=true`; single-correlated-receipt apply remains false. A future tenant-partitioned command may lower that flag only with query/fitness evidence. Exact permission tokens are `notification:dispatch`, `notification:receipt`, `notification:operate`, `notification:cutover-admit`, `notification:cutover-terminalize`, `notification:cutover-attest`, `notification:cutover`, `notification:reconcile`, `notification:maintain`, `notification:observe`. Operations snapshot is `sensitiveRead=false`, `bulkWrite=false`, `crossTenantAdmin=true` because it returns only bounded infrastructure aggregates across partitions. All writer cutover operations are `sensitiveRead=false`, `bulkWrite=false` and `crossTenantAdmin=true`. Initialization requires a bounded ordered route/initial-generation set exactly equal to the application-owned route set derived from the compiled cutover catalog, its digest, actor/reason/token, absent fences and empty control/data-plane journals. BEGIN requires a trusted signed complete old-node inventory; quiescence attestation requires exact route/draining generation, a signed exact inventory/quiescence/consumer/ledger manifest and token. Actor, post-lock DB-time validity, immutable persisted current+retiring proof registry, permit/holder sets and node inventory digests are server-derived. Permit acquire/release require exact route, LEGACY owner/generation and opaque token; ownership switch requires exact route/expected generation/token and derives owner/result from the closed action matrix. Switch additionally requires exact `BEGIN_DRAIN|COMPLETE_SWITCH|ABORT_DRAIN` action plus audited actor/reason. Initialization, attestation, terminalization, acquire, release and every switch action reject ambient transactions and return success only after `inRootWrite` physical commit. A every COMPLETE additionally succeeds only when the persistence adapter locks and Java re-verifies the retained signed BEGIN inventory. QUIESCENCE_REQUIRED also revalidates its signed durable quiescence proof and monotonic zero/irreversible facts. HARD_BOUND forbids quiescence evidence and accepts only that verified BEGIN, registry-qualified safe terminal permits and ACTIVE permit 0. - [x] Keep retry/fallback/admission state policy in application, not mapper/config/scheduler. - [ ] Make maintenance/reconciliation/retention schedulers call application use cases; app-bootstrap must not call repositories or persistence entities directly. - [x] Use injected `Clock`; use bounded batch/deadline/count values; do not sleep inside the use case. - [x] Verify GREEN with the same command and run: ```bash cd src && ./gradlew :application-core:check --console=plain ``` - [x] Acceptance claim: pure orchestration/state model is proven with fakes; PostgreSQL/provider R2 is not yet proven. **Rollback checkpoint:** Task 4 is the public port boundary. Later adapters may be rolled back by removing bindings, but these types must remain while compiled consumers exist. ### Wave A exit gate - [x] Run: ```bash cd src && ./gradlew :application-core:check \ :adapter:outbound:persistence-jpa:check \ verifyCleanArchitectureDependencies \ verifyPublicPathSnapshot \ --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*CleanArchitectureTest' --console=plain ``` Isolated-worktree evidence: both commands pass, including the wildcard architecture test. - [x] Confirm application bytecode/import scan contains no Spring, JPA, Slack, AWS, JSON or HTTP provider type. - [x] Request an application/transaction boundary review before Wave B. - [x] Update the LLM Wiki branch-note with Wave A evidence and an explicit derived-document decision. --- ## Wave B — Notification-local catalog, rendering and provider protocol ### Task 5: Build provider/template/route descriptors and binding compiler **Owner leaf:** `adapter-outbound-notification` (`:adapter:outbound:notification`) **Depends on:** Task 4 **Files — create:** - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationProviderDescriptor.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationProviderCapabilityCard.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationTemplateDescriptor.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationRouteDescriptor.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationCanonicalRouteCatalog.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationCutoverRouteCatalog.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationProviderRuntimeProfile.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/CompiledNotificationBinding.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationBindingCompiler.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationPlanAdapter.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationProviderCapabilityDescriptorSource.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationCatalogException.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationBindingCompilerTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationCanonicalRouteCatalogTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationCutoverRouteCatalogTest.java` - [x] RED cases: unknown/blank/duplicate local catalog entry; channel/provider mismatch; durable + legacy/fail-open; receipt-required + unsupported provider; unsafe fallback after indeterminate; target/retry/reconcile/amplification bound; unsupported non-`SINGLE` binding. Initial R1 intentionally compiles only the three reviewed `SINGLE` cards. - [ ] RED cutover-catalog cases: `NotificationCanonicalRouteCatalog` is the retained key-only canonical SSOT and maps with the trusted runtime target config to application-owned `NotificationCanonicalWriterRouteSet`. PRE-only `NotificationCutoverRouteCatalog` decorates exactly those keys with legacy aliases and transport proof metadata and maps to transitional `NotificationWriterRouteSet`; it cannot add/remove keys. the sorted route-revision key set, optional legacy alias and each route's bounded current+retiring legacy transport profile registry are one checked-in PRE SSOT. The registry marks one active admission profile and, for every revision, proof class (`HARD_BOUND_PROVEN|QUIESCENCE_REQUIRED`) plus evidence revision; its key set exactly equals the canonical binding graph and reviewed V8 provenance-bound seed manifest. Runtime cutover target generations are a separate exact config/evidence revision and may not add or remove catalog keys. PRE bridge possible routes must equal this set even when production consumer count and current legacy route settings are 0. Missing/extra/duplicate alias, route-key drift and an R0 mapping/profile outside the catalog, an omitted historical blocking profile, absent proof evidence or a profile marked HARD_BOUND without the reviewed integration evidence revision fail closed. A retiring profile cannot be removed while any permit/attestation/ operation/persisted-registry history references it. The registry digest covers the sorted route/profile/admission-role/proof-class/evidence-revision tuple set and changes on any drift. A catalog route with no live legacy consumer is initialized as closed LEGACY predecessor and switched through the audited protocol; it is never directly seeded canonical in PRE. - [x] Verify RED: ```bash cd src && ./gradlew :adapter:outbound:notification:test \ --tests '*NotificationBindingCompilerTest' \ --tests '*NotificationCanonicalRouteCatalogTest' \ --tests '*NotificationCutoverRouteCatalogTest' \ --console=plain ``` Expected failure: canonical descriptors/compiler do not exist. - [x] Implement a pure, deterministic compiler over explicit input; do not inspect Spring beans, application context, persistence schema or inbound adapters. - [x] Keep `expected-state`, exact actual/expected binding IDs, application mode/admission matching, store capability and ingress topology out of this sibling-local compiler. Task 17 passes provider-neutral descriptors to the application compatibility validator for those checks. - [x] Register only the three initial card IDs. Legacy descriptors must explicitly advertise R0, no durable/receipt capability. - [x] Emit a sorted immutable binding graph and manifest digest; unknown inputs fail closed. - [ ] Emit one immutable `NotificationCutoverRouteCatalog` and digest from the same route descriptor inputs over the retained immutable `NotificationCanonicalRouteCatalog`. Bootstrap converts the canonical catalog to `NotificationCanonicalWriterRouteSet` and the PRE decorator to `NotificationWriterRouteSet`; neither legacy settings nor a request may invent/remove route revisions. Bootstrap combines that key set with the exact reviewed runtime target-generation config; only a reviewed config revision may change target values after ABORT, and it invalidates qualification evidence. Bootstrap also maps the catalog's immutable current+retiring transport proof registry into the application route set; permit acquire uses only the active profile, while timeout, attestation and COMPLETE must recognize every referenced current/retiring profile even when a route has zero permits. `NotificationCutoverRouteCatalogTest` freezes the digest algorithm and proves the exact registry value that batch initialization must persist; after initialization, PRE composition rejects any persisted/catalog mismatch instead of silently refreshing it. Static catalog/digest/key-set validation is implemented; persisted-registry comparison and bootstrap enforcement remain explicitly deferred to Task 17. - [x] Implement `NotificationPlanPort` by converting the selected adapter-local compiled binding into an application-owned `NotificationFrozenPlan`. The conversion freezes policy/route/template/ renderer/provider-leg revisions and contains no credential, SDK, settings or adapter type. - [x] Derive the application-owned provider capability descriptor from the actual compiled cards, renderer and client capabilities. Do not reconstruct “actual” provider facts from the expected bootstrap settings. - [x] Verify GREEN with the same command. - [x] Acceptance claim: local graph compatibility only, not actual composition/readiness. **Rollback checkpoint:** compiler can coexist dark with the legacy router until Task 18 canonical composition succeeds. ### Task 6: Implement immutable local template manifests and bounded renderers **Owner leaf:** `adapter-outbound-notification` (`:adapter:outbound:notification`) **Depends on:** Task 5 **Files — create:** - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/template/NotificationTemplateCatalog.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/template/NotificationTemplateManifest.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/template/NotificationTemplateRenderer.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/template/RenderedNotification.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/template/LocalEmailRenderer.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/template/SlackBlockKitRenderer.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/template/TemplateRenderingException.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/template/NotificationTemplateRendererTest.java` - `src/adapter/outbound/notification/src/test/resources/notification/templates/email/contract-v1.subject.txt` - `src/adapter/outbound/notification/src/test/resources/notification/templates/email/contract-v1.text.txt` - `src/adapter/outbound/notification/src/test/resources/notification/templates/email/contract-v1.html` - `src/adapter/outbound/notification/src/test/resources/notification/templates/slack/contract-v1.txt` - [x] RED cases: checksum/revision drift; missing/unknown/unused parameter; exact locale fallback independent of JVM default; email header CR/LF; HTML text/attribute/URL escaping; Slack mrkdwn/plain-text/mention escaping; output byte/block/depth limits; no file/network/reflection include; redacted failures. - [x] Verify RED: ```bash cd src && ./gradlew :adapter:outbound:notification:test \ --tests '*NotificationTemplateRendererTest' --console=plain ``` - [x] Implement checked-in resource loading by exact manifest/checksum. Keep business-specific assets out of production main resources until a consuming project supplies a reviewed catalog; use test resources only for the generic contract proof. - [x] Produce local email subject/text/HTML and Slack Block Kit through typed builders; never accept caller-supplied arbitrary JSON or provider block objects. - [x] Verify GREEN with the same command. - [x] Acceptance claim: deterministic local render R1; no provider call. **Rollback checkpoint:** retained intent template revisions prevent later asset deletion. Before durable append, this task is independently reversible. ### Task 7: Define the adapter-internal one-authorized-attempt SPI **Owner leaf:** `adapter-outbound-notification` (`:adapter:outbound:notification`) **Depends on:** Tasks 5–6 **Files — create:** - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationProviderAttemptClient.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/PreparedNotificationAttempt.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationAttemptContext.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/AttemptCorrelationId.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/ProviderMessageReference.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/ReconciliationLookupMode.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationProviderAttemptAdapter.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/InlineNotificationAttemptAdapter.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationReconciliationAdapter.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationProviderSecretMaterialProvider.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationSecretMaterialHandle.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationProviderReadinessProbe.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationProviderReadinessSnapshot.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationProviderRateAdmission.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationAdmissionReadinessAdapter.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationProviderAttemptContractTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationSecretMaterialHandleTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/provider/InlineNotificationAttemptAdapterTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationReconciliationAdapterTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/provider/NotificationAdmissionReadinessAdapterTest.java` - [x] RED cases: `prepare` has no I/O; one authorization invokes client exactly once; deadline is absolute/bounded; pre-wire validation maps to definitely-not-applied; possible write timeout maps indeterminate; accepted response stores only opaque provider reference; SDK exceptions never escape to application. - [x] Verify RED: ```bash cd src && ./gradlew :adapter:outbound:notification:test \ --tests '*NotificationProviderAttemptContractTest' \ --tests '*NotificationSecretMaterialHandleTest' \ --tests '*InlineNotificationAttemptAdapterTest' \ --tests '*NotificationReconciliationAdapterTest' \ --tests '*NotificationAdmissionReadinessAdapterTest' \ --console=plain ``` - [x] Keep this SPI adapter-internal. Implement the application `NotificationProviderAttemptPort` with compiled binding + renderer + internal client lookup. - [x] Keep secret material resolution, control-plane readiness and provider-local rate admission behind adapter-owned interfaces. Profiles contain secret references/generations only; readiness snapshots contain bounded non-secret identity/capability facts. - [x] Secret acquisition returns a versioned `AutoCloseable` mutable byte/char handle. Acquire it per provider operation, close it on success/exception/cancellation, wipe on close, reject use after close, and redact `toString`/exceptions. Never store the raw token in an adapter-owned record/String/settings field; the wipe claim covers only the adapter-facing mutable copy. - [x] Implement the application-owned `NotificationAdmissionReadinessPort` with the adapter-internal readiness probes. Application admission use cases must never import the internal probe type. - [x] Freeze the outbound binding matrix: `NotificationPlanPort -> NotificationPlanAdapter`, `InlineNotificationAttemptPort -> InlineNotificationAttemptAdapter`, `NotificationProviderAttemptPort -> NotificationProviderAttemptAdapter`, `NotificationReconciliationPort -> NotificationReconciliationAdapter`, `NotificationAdmissionReadinessPort -> NotificationAdmissionReadinessAdapter`. Every implementation has a focused contract test before composition. - [x] Keep attempt correlation, optional provider operation key and post-response message reference as distinct types. - [x] Verify GREEN with the same command and: ```bash cd src && ./gradlew :adapter:outbound:notification:check --console=plain ``` - [x] Acceptance claim: deterministic fake protocol R1, no exact provider card qualification. **Rollback checkpoint:** no network resources are created until a canonical profile is bound in Task 18. ### Wave B exit gate - [x] Run: ```bash cd src && ./gradlew :application-core:check \ :adapter:outbound:notification:check \ verifyCleanArchitectureDependencies \ --console=plain ``` - [x] Verify the notification leaf has no project dependency on persistence, inbound-web or httpclient. - [x] Request catalog/template/provider-SPI review. Final independent re-review: Blocker 0 / High 0. - [x] Update the LLM Wiki branch-note with Wave B evidence and an explicit derived-document decision. --- ## Wave C — PostgreSQL durable kernel and cryptography ### Task 8: Add adapter-owned direct AEAD and versioned HMAC primitives **Owner leaf:** `adapter-outbound-persistence-jpa` (`:adapter:outbound:persistence-jpa`) **Depends on:** Task 4 **Files — create:** - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/crypto/NotificationKeyMaterialProvider.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/crypto/NotificationKeyMaterialHandle.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/crypto/NotificationCiphertext.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/crypto/DirectAeadNotificationPayloadCrypto.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/crypto/NotificationHmacDigester.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/crypto/NotificationCryptoException.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/crypto/NotificationPayloadCryptoTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/crypto/NotificationHmacDigesterTest.java` - [ ] RED cases: AES-256-GCM profile/version, fresh 96-bit nonce per field, context-bound AAD, ciphertext swapping failure, wrong key/revision failure, purpose-separated length-prefixed HMAC, current + bounded retiring keys, no key/plaintext in `toString`/exception. - [ ] Fix the AEAD contract to a 128-bit GCM tag and canonical length-prefixed AAD tuple: `(schema/table, record ID, notification ID, optional delivery ID, optional attempt ID, field purpose, provider binding revision, crypto profile version)`. This is the exact approved design §24.2 hierarchy; key reference/version remain stored non-secret ciphertext metadata but are not substitutes for the notification/delivery/attempt and binding coordinates. Any tuple-field swap must fail authentication. - [ ] Make key acquisition a versioned `AutoCloseable` mutable handle with close-time wipe and use-after-close failure. Test success, exception and cancellation paths; never retain key bytes in adapter-owned immutable records/Strings. - [ ] Verify RED: ```bash cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*NotificationPayloadCryptoTest' \ --tests '*NotificationHmacDigesterTest' \ --console=plain ``` - [ ] Use JCA primitives directly; zero temporary mutable key buffers where feasible and never place material in settings/application records. - [ ] Do not claim envelope encryption. Persist algorithm/key reference/version/nonce/AAD revision with ciphertext. - [ ] Verify GREEN with the same command. - [ ] Acceptance claim: local cryptographic contract; external key management/rotation readiness is not yet proven. **Rollback checkpoint:** once Task 10 persists ciphertext, old key/AAD/canonicalization revisions cannot be removed by code rollback. ### Task 9: Add the additive Notification journal migration and persistence model **Owner leaf:** `adapter-outbound-persistence-jpa` (`:adapter:outbound:persistence-jpa`) **Depends on:** Task 8 **Files — create:** - `src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V7__notification_delivery_journal.sql` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationIntentEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationDeliveryLegEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationAttemptEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationReceiptEventEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationTechnicalSuppressionEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationAdmissionGateEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationRouteWriterFenceEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationWriterOperationEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationWriterOperationRouteEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationWriterTransportProofRegistryEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationRouteWriterPermitEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationWriterQuiescenceAttestationEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationWriterInventoryManifestEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationWriterDrainNodeInventoryEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationWriterQuiescenceManifestEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationWriterQuiescenceNodeEvidenceEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationWriterEvidenceTrustSnapshotEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationFreshInstallationProvenanceEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationWriterFinalizationDiscriminatorEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/entity/NotificationHmacAliasEntity.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationIntentJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationDeliveryLegJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationAttemptJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationReceiptEventJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationTechnicalSuppressionJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationAdmissionGateJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationRouteWriterFenceJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterOperationJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterOperationRouteJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterTransportProofRegistryJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationRouteWriterPermitJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterQuiescenceAttestationJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterInventoryManifestJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterDrainNodeInventoryJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterQuiescenceManifestJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterQuiescenceNodeEvidenceJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterEvidenceTrustSnapshotJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationFreshInstallationProvenanceJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterFinalizationDiscriminatorJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationHmacAliasJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationJournalMigrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationWriterFinalizationDiscriminatorIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationDatabaseRoleIsolationIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationRetainedEvidenceNoSaveArchitectureTest.java` - `docs/runbooks/notification-database-role-bootstrap.md` - [ ] Before editing, scan every Flyway location. If any `V7` exists, stop and reserve the next global version instead of creating a collision. - [ ] Add Testcontainers PostgreSQL dependencies to `src/adapter/outbound/persistence-jpa/build.gradle` and update its lockfile only when the RED test requires them. - [ ] After adding only the test harness dependencies, regenerate and review the leaf lock before the behavior RED: ```bash cd src && ./gradlew :adapter:outbound:persistence-jpa:resolveAndLockAll \ --write-locks --console=plain cd src && ./gradlew :adapter:outbound:persistence-jpa:verifyDependencyLocks \ --console=plain ``` - [ ] RED migration tests for all PK/FK/unique/partial-unique/check constraints and eligible/stale lease/orphan lookup indexes from design §16.8, including one route writer-fence row and globally unique append-only writer-operation header token/history, unique DB-assigned `operation_sequence` plus attestation sequence from the same post-lock cutover sequence, cross-table collision rejection, composite route-child identity, route-set digest/action/expected/result ownership constraints including bounded `TERMINALIZE_EXPIRED_PERMITS` requested batch bound and affected count/set digest. Store a server-canonical header `request_input_digest`; header `route_set_digest` must equal the sorted exact child set, and request digest must recompute from action-specific persisted header/child input. Freeze domain-separated length-prefixed SHA-256 profiles `writer-operation-route-set-v1`/`writer-operation-input-v1` and prove delimiter/order permutations differ without including PII/secrets. Reject orphan header/child and empty child sets. Continue with writer-permit token uniqueness, route/generation/owner scope, `ACTIVE/RELEASED/EXPIRED_PROVEN/TIMED_OUT_UNPROVEN` checks and blocking-permit lookup index. Add the retained immutable transport-proof registry with composite `(route_revision, transport_profile_revision)` PK, exactly one ACTIVE admission profile per route, one canonical digest per route, initialization-operation-child composite FK and UPDATE/DELETE rejection. Permit rows freeze `(route_revision, transport_profile_revision, transport_proof_class, transport_proof_evidence_revision)` and reference the exact registry row; operation children and attestations carry the matching registry digest. Add cross-state CHECKs: `EXPIRED_PROVEN => HARD_BOUND_PROVEN` and `TIMED_OUT_UNPROVEN => QUIESCENCE_REQUIRED`; `ACTIVE|RELEASED` allow either class. The two timeout states require a terminalization-operation FK and DB timestamp; ACTIVE/RELEASED forbid that FK. Persist DB-time `wire_deadline_at` and enforce the reviewed `acquired_at <= wire_deadline_at < expires_at` shape; the catalog evidence revision supplies the stricter finalize-margin proof. Include globally unique immutable quiescence-attestation token, exact route/draining-generation/BEGIN-operation scope, transport-profile/blocking-permit/permit-holder/ old-node count/set digests, signed evidence identity, zero/true fact constraints, observed/expiry bounds and the COMPLETE operation-child attestation-token/set-digest FK. Add an immutable per-BEGIN node inventory row set whose exact count/digest equals the BEGIN child and whose node set covers every distinct route permit holder. Caller-written inventory digests are not accepted. Add immutable per-attestation node evidence rows whose node keys exactly equal that BEGIN inventory and which retain each deployment-generation tombstone plus legacy credential-or-egress revocation digest; the canonical row-set digest must equal the attestation summary. Add immutable inventory-manifest and quiescence-manifest headers plus an immutable trust snapshot. Each header retains canonical domain-separated payload bytes/profile, signature, algorithm, issuer key ID, bounded canonical issuer public-key SPKI/digest, trust catalog revision, historical-key allow/revocation snapshot and validity profile, signed issued/expires facts, DB `verified_at`, profile-pinned `allowedClockSkew` and `acceptanceMargin`, environment/DB/artifact identity, consumer-inventory identity, provider-ledger identity/snapshot, exact route/drain generation, and canonical child count/set digest. Header-to-child exact equality is structural and a zero-node inventory still has exactly one authoritative header. Store no private key or raw credential. SQL checks bytes/digests/FK/cardinality/state only; application write/startup Java verifies Ed25519 from the stored payload/signature/SPKI, requires the historical key digest to remain allowed/non-revoked in the current closed catalog, and enforces `issuedAt - allowedClockSkew <= serverVerifiedAt <= expiresAt - acceptanceMargin`. Add the retained singleton table `notification_writer_finalization_discriminator`, whose eventual row has the exact closed states `AWAITING_SIGNED_FRESH_PROVISIONING|FRESH_PROVISIONED|UPGRADE_VALIDATED`. Its structural XOR is authoritative: `FRESH_PROVISIONED` has exactly one fresh provisioning token and no validated-history digest; `UPGRADE_VALIDATED` has exactly one validated complete-history digest and no fresh token; awaiting has neither. No state permits both, an unknown state, or a reverse transition. The fresh token must equal the provenance initialization token and `INITIALIZE_CANONICAL_FRESH` operation token; the upgrade digest must equal the server-canonical complete retained snapshot digest. Freeze the exact columns `singleton_key=NOTIFICATION_FINALIZATION`, `state`, `fresh_provenance_token`, `validated_upgrade_history_digest`, `state_operation_token` and `row_version`. Add the immutable table `notification_fresh_installation_provenance` empty. Freeze these exact retained axes and names rather than a reduced “zero snapshot”: ```text provenance_token PK fresh_initialization_operation_token UNIQUE canonical_signed_payload canonical_signed_payload_digest signature_algorithm = ED25519 detached_signature issuer_identity_digest issuer_key_revision issuer_public_key_spki issuer_public_key_digest trust_snapshot_canonical_payload trust_snapshot_digest acceptance_window_profile_revision allowed_clock_skew_ms acceptance_margin_ms issued_at expires_at server_verified = true server_verified_at server_verifier_revision database_resource_canonical_payload database_resource_identity_digest database_birth_certificate_canonical_payload database_birth_certificate_digest database_system_identifier_digest database_identity_digest schema_identity_digest environment_identity_digest final_artifact_digest canonical_route_set_digest application_workload_inventory_count = 0 application_workload_inventory_digest business_consumer_inventory_count = 0 business_consumer_inventory_digest legacy_node_inventory_count = 0 legacy_node_inventory_digest provider_call_ledger_identity_digest provider_call_ledger_snapshot_digest provider_call_ledger_snapshot_cut_revision provider_call_ledger_snapshot_cut_at provider_call_ledger_entry_count = 0 provider_call_ledger_open_count = 0 provider_call_ledger_indeterminate_count = 0 no_legacy_authority_fence_token no_legacy_authority_fence_revision no_legacy_authority_fence_canonical_payload no_legacy_authority_fence_digest no_legacy_authority_fence_committed_at no_legacy_authority_fence_read_back_at no_legacy_authority_fence_irreversible = true no_legacy_authority_enforcement_revision no_legacy_authority_enforcement_digest no_legacy_authority_enforcement_activated_at no_legacy_authority_enforcement_read_back_at post_enforcement_zero_manifest_canonical_payload post_enforcement_zero_manifest_digest post_enforcement_zero_observation_revision post_enforcement_zero_observed_at legacy_deployment_generation_deny_set_digest legacy_deployment_generation_tombstone_set_digest legacy_database_credential_issuance_disabled = true legacy_database_credential_revocation_set_digest legacy_database_credential_revocation_complete = true legacy_database_session_inventory_digest legacy_database_session_open_count = 0 legacy_database_session_termination_evidence_digest legacy_database_ingress_denied = true legacy_database_ingress_denial_policy_digest legacy_database_ingress_blocks_established_flows = true legacy_provider_credential_issuance_disabled = true legacy_provider_credential_revocation_set_digest legacy_provider_credential_revocation_complete = true legacy_provider_connection_flow_inventory_digest legacy_provider_connection_flow_open_count = 0 legacy_provider_connection_flow_termination_evidence_digest provider_egress_denied = true provider_egress_denial_policy_digest provider_egress_blocks_established_flows = true authorization_digest ``` The external infrastructure issuer must first commit and read back the exact irreversible enforcement revision, then terminate every pre-existing legacy DB session and provider connection/flow, then observe the causally later post-enforcement zero/settled manifest, including ledger entry/open/indeterminate counts 0, then seal-commit and read back the permanent fence, and only then sign the DB-birth authorization. Enforce `no_legacy_authority_enforcement_activated_at <= no_legacy_authority_enforcement_read_back_at <= provider_call_ledger_snapshot_cut_at <= post_enforcement_zero_observed_at <= no_legacy_authority_fence_committed_at <= no_legacy_authority_fence_read_back_at <= issued_at`; every source evidence binds the exact fence token and enforcement revision. Pre-enforcement zero snapshots, sign-before-seal, cross-revision composition, a reopened cached session/flow and any shortened or renamed axis fail with mutation 0. V7 creates only empty structure and V8 never manufactures either authority. Migration/discriminator tests reject every discriminator XOR violation and independently remove, alter or make nonzero/false each exact DB-birth/fence/enforcement/ post-enforcement/ledger/session/connection axis above; no partial provenance row is valid. Operation-child CHECKs encode the closed action/result matrix and forbid `DRAINING/CANONICAL`, caller-selected target owners and any CANONICAL→LEGACY history. - [ ] Establish exactly three database roles in the real-PostgreSQL fixture before GREEN and reject every additional notification-scoped owner/member/grantee: pre-provisioned `notification_migrator` owns Flyway history, notification schema, tables, sequences, trigger/functions and is the only Flyway principal; `notification_runtime` is a non-owner with only the exact runtime DML/SELECT grants needed by the active release and, in the PRE artifact only, the exact transitional function `EXECUTE` grants; `notification_provisioner` has no table/sequence privilege and is reserved for the V8-created two-function fresh-provisioning protocol. The FINAL cleanup migration revokes every PRE transitional `EXECUTE` grant from runtime/PUBLIC. Object ownership and default privileges must make REVOKE effective; running Flyway as the runtime user is a RED failure. Every `SECURITY DEFINER` function is owned by `notification_migrator`, schema-qualified, uses a fixed safe `search_path`, contains no dynamic SQL, revokes PUBLIC EXECUTE and grants only the exact role. Tests cover direct table DML, sequence use, function invocation, role switching, search-path shadowing and forged input. - [ ] Make role creation an explicit external DB-admin/IaC prerequisite, not a migration side effect. `docs/runbooks/notification-database-role-bootstrap.md` freezes the exact principal set, LOGIN/NOINHERIT expectations, external credential references, database/schema ownership handoff and read-only verification queries without embedding credentials. A privileged Testcontainers setup connection may emulate that prerequisite before Flyway, then must close; Flyway starts only afterward as `notification_migrator`. V7/V8 contain no `CREATE ROLE`, credential generation or membership grant: they validate `current_user`, exact owner/member/grantee inventory and object/default privileges, create/alter owned schema objects, and perform the reviewed grants/revokes. Production evidence retains the external bootstrap revision/digest, all three `current_user` probes and the post-migration privilege snapshot. Missing bootstrap evidence or a fourth notification-scoped principal stops rollout before Flyway. - [ ] Verify RED: ```bash cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*NotificationJournalMigrationTest' \ --tests '*NotificationWriterFinalizationDiscriminatorIntegrationTest' \ --tests '*NotificationDatabaseRoleIsolationIntegrationTest' \ --tests '*NotificationRetainedEvidenceNoSaveArchitectureTest' \ --console=plain ``` Expected failure: `V7` and journal schema do not exist. - [ ] Implement additive tables only. Do not rewrite existing outbox/idempotency tables and do not backfill historical events. - [ ] Treat signed headers/trust snapshots, drain inventory, per-node quiescence evidence, fresh provenance and writer-finalization discriminator entities/repositories as retained adapter-internal audit projections. Only the BEGIN adapter may insert a verified inventory header/children, only the attestation adapter may insert a verified quiescence header/per-node evidence in its root transaction, and only the two-function final provisioning protocol may insert provenance and CAS `AWAITING_SIGNED_FRESH_PROVISIONING -> FRESH_PROVISIONED`; V8 alone may establish `AWAITING_SIGNED_FRESH_PROVISIONING` or `UPGRADE_VALIDATED`. From Task 9 onward, provenance, discriminator, signed-header/trust-snapshot and other never-Java-written retained repositories extend only Spring Data's marker `Repository` and expose bounded named reads; they never inherit `CrudRepository`/`JpaRepository` or declare `save`, `saveAll`, `delete` or `flush`. `NotificationRetainedEvidenceNoSaveArchitectureTest` enforces that initial surface. Retain every projection for FINAL startup/evidence reads. - [ ] Store provider leg separately from logical recipient; enforce one open attempt per delivery and one active leg per fallback strategy group in PostgreSQL. - [ ] Keep raw recipient/parameter/provider payload/error out of plaintext columns and indexes. - [ ] Verify GREEN with the same command. - [ ] Acceptance claim: schema invariants on real PostgreSQL, not yet append/claim behavior. **Rollback checkpoint:** deploy schema before code. Do not use destructive down migration; old code must tolerate additive tables. If later ciphertext/state is incompatible with old code, rollback is forward-fix. ### Task 10: Implement same-transaction append, dedupe aliases and frozen plan storage **Owner leaf:** `adapter-outbound-persistence-jpa` (`:adapter:outbound:persistence-jpa`) **Depends on:** Task 9 **Files — create:** - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationStoreAdapter.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationPersistenceMapper.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationPersistenceExceptionTranslator.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationStoreCapabilityDescriptorSource.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationIntentAppendIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationStoreCapabilityDescriptorTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/notification/NotificationSameTransactionAppendContractTest.java` - [ ] RED cases on real PostgreSQL: business write + append commit together; either failure rolls both back; append never uses `REQUIRES_NEW`; same idempotency digest/fingerprint returns existing intent; different fingerprint conflicts; concurrent old/current HMAC alias writers resolve to one semantic owner; frozen legs/template/route/crypto revisions are immutable. - [ ] Verify RED: ```bash cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*NotificationIntentAppendIntegrationTest' \ --tests '*NotificationStoreCapabilityDescriptorTest' \ --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationSameTransactionAppendContractTest' --console=plain ``` - [ ] Encrypt recipient/parameters before persistence and insert all current + retiring HMAC aliases in the same caller transaction. - [ ] Seed representative recipient, parameter, provider payload, error and key-marker values, then scan every notification table/index-visible text representation and captured SQL/log output. The markers may appear only after an explicit decrypt operation in test memory; database plaintext evidence must be zero. - [ ] Map unique conflicts to typed duplicate/mismatch results; never catch-and-ignore arbitrary constraint errors. - [ ] Derive the application-owned store descriptor from the actual migration/schema, crypto profile/key generations and live/retained revision inventory. Expected bootstrap config is not an input to this source. - [ ] Verify GREEN with the same commands. - [ ] Acceptance claim: same-DB durable append is locally verified only for the tested PostgreSQL topology; no provider card R2 is implied. **Rollback checkpoint:** leave canonical binding disabled. Schema/data stay in place if code is rolled forward; never resend persisted rows through legacy code. ### Task 11: Implement PostgreSQL claim, wire authorization, terminal-once result and admission gate **Owner leaf:** `adapter-outbound-persistence-jpa` (`:adapter:outbound:persistence-jpa`) **Depends on:** Task 10 **Files — create:** - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlNotificationClaimRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationAdmissionGateStore.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationReceiptStoreAdapter.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationTechnicalSuppressionStoreAdapter.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/PostgreSqlNotificationCanonicalWriterFenceAdapter.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/PostgreSqlNotificationWriterCutoverAdapter.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/PostgreSqlNotificationWriterQuiescenceAttestationAdapter.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationOperationsSnapshotAdapter.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/PostgreSqlNotificationClaimIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationAdmissionGateIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationReceiptStoreIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationCanonicalWriterFenceIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationWriterCutoverIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationWriterQuiescenceAttestationIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationWriterIrreversibleFenceIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationOperationsSnapshotIntegrationTest.java` - [ ] RED concurrency cases with at least two transaction contexts: `SKIP LOCKED` single owner; bounded ordering/aging; exact token/version predicate; stale owner cannot overwrite; gate generations locked in canonical order; only active generations may commit `WIRE_AUTHORIZED`; one terminal exact result per execution token; late exact result may fill its slot without stale projection overwrite. - [ ] RED park/restart cases: provider/account fault closes shared gate and parks backlog; another node sees it; restart preserves it; resume increments generation and rechecks expiry/cancel/suppression; no hot loop and no initial fallback. - [ ] RED writer-fence/permit cases: atomic route owner/generation/state CAS; all nodes observe one owner; stale legacy and canonical generations both fail closed. Adapter/DB tests enumerate the closed transition matrix: BEGIN only `ACTIVE/LEGACY@g -> DRAINING/LEGACY@g`, terminalize only unchanged `DRAINING/LEGACY@g`, COMPLETE only `DRAINING/LEGACY@g -> ACTIVE/CANONICAL@g+1`, ABORT only `DRAINING/LEGACY@g -> ACTIVE/LEGACY@g+1`; every action from CANONICAL, target-owner input and `DRAINING/CANONICAL` fail with mutation 0. Unique bounded permit acquire/release/expiry uses DB time and exact token/version; `BEGIN_DRAIN` and concurrent acquire serialize so no new permit commits after DRAINING; `COMPLETE_SWITCH` and the operations snapshot require ACTIVE legacy permit 0 for the exact route across every historical fence generation, not merely the current generation; `ABORT_DRAIN` emits a new LEGACY generation without hiding an older-generation live permit. Prove `g` permit active -> abort to `g+1` -> drain again -> complete remains blocked until the `g` permit is terminal. A canonical caller transaction holds a fence share lock through its append commit, so concurrent `BEGIN_DRAIN` cannot commit first and reopen legacy while a stale canonical append later commits; owner switch does not rewrite already accepted canonical rows. - [ ] RED timeout/attestation concurrency cases: a proven hard-bound profile may CAS past-deadline ACTIVE to `EXPIRED_PROVEN`; an unproven profile may only become `TIMED_OUT_UNPROVEN`. For current R0, COMPLETE fails even with permit count 0 until an authenticated attestation committed after BEGIN_DRAIN is supplied. Permit acquire locks the persisted ACTIVE registry row and freezes its proof class/evidence revision; timeout, attestation and COMPLETE reject a permit whose frozen tuple differs or whose profile is unknown. Mixed-profile fixtures prove `QUIESCENCE_REQUIRED + EXPIRED_PROVEN` and `HARD_BOUND_PROVEN + TIMED_OUT_UNPROVEN` are rejected by both DB and runtime. Recording BEGIN first verifies an independently signed exact environment/DB/route/PRE-artifact complete node inventory and freezes its rows/count/digest atomically with the fence CAS. Missing/ duplicate/extra node, unknown issuer, wrong environment/DB/artifact/profile and any historical permit holder omitted by the manifest fail with mutation 0. Attestation recording first requires ACTIVE 0, then locks/snapshots every-generation/multi-profile `(permit token,generation,transport profile,state,rowVersion)` TIMED_OUT_UNPROVEN tuple and distinct holder set and stores the bounded canonical set/count/digests plus the persisted route registry and exact frozen BEGIN inventory digest. Its independently signed quiescence manifest must list that exact node set, bind per-node retired/quiesced facts plus irreversible deployment-generation/legacy-credential/egress fences, consumer inventory/count 0 and provider-call ledger identity/open-count 0; caller digests are ignored/rejected. COMPLETE requires the persisted per-node evidence key set and tombstone/revocation row-set digest to exactly match the signed manifest, attestation summary and BEGIN inventory. Missing/extra/ duplicate node evidence or a digest-only attestation fails. The retained BEGIN inventory header, quiescence/attestation header and evidence trust snapshot persist the canonical signed payload bytes, signature, issuer/key identity, bounded issuer public-key SPKI/digest, verified trust/historical-key snapshot, issued/expires/verified times and pinned skew/acceptance-margin profile, environment/DB/PRE-artifact identity, exact node/consumer inventory identity and provider-ledger identity/snapshot. An exact zero-node inventory is still an issuer-authorized signed statement, never an unsigned empty shortcut. Java verifies Ed25519 and exact identity at evidence admission and again at startup; database constraints enforce only immutable shape, FK, count and digest structure. Expiry rejects new evidence admission but does not make an already committed irreversible tombstone, credential revocation or egress revocation reversible. COMPLETE locks fence, registry, BEGIN inventory, attestation and permit rows in canonical order, recomputes exact equality, appends its attestation token/digest and fence CAS in one root transaction. Cover missing/stale/wrong-route/wrong-drain-generation/wrong-profile-set, unknown, extra or omitted active/retiring profile, catalog/persisted-registry drift, tampered proof class/evidence revision/registry digest, partial multi-profile/node/holder coverage, unsigned/wrong-key/wrong-identity evidence, nonzero facts, changed set after release/timeout, token replay mismatch, concurrent attestation/permit terminal transition and commit failure/result loss. No TTL-only path may reach CANONICAL. - [ ] RED the irreversible COMPLETE proof and stale-node resume race on the exact PostgreSQL 16 profile: COMPLETE requires the exact frozen BEGIN inventory, signed retained quiescence evidence, irreversible deployment-generation tombstones, legacy credential revocations and egress revocations, ACTIVE permit 0 and provider-ledger open-count 0 in one canonically locked snapshot. Pause an old bridge process after it has cached its legacy credential, provider client and connection but before provider I/O. Commit the irreversible evidence and `COMPLETE_SWITCH`, then resume the stale process and prove provider I/O remains 0 because the old deployment generation, credential and egress path are all unusable. Repeat after application/JDBC connection recreation and process restart. Missing/reversible facts, a changed inventory/ledger identity, nonzero permit/ledger state, unsigned or non-reverifiable retained headers, and any stale node omitted from BEGIN are `NOT_QUALIFIED` and prohibit COMPLETE/21C. Commit-success/result-loss replay returns the stored durable result without weakening or refreshing the evidence. - [ ] RED hard-bound pause/resume cases before any profile may use `HARD_BOUND_PROVEN`: acquire root commit freezes `wire_deadline_at + finalize_margin <= expires_at`; commit before provider I/O; the wrapper/client cannot begin network I/O after that absolute deadline and cancellation/connection close completes by it. Pause immediately after acquire commit, let wire deadline and permit expiry pass, terminalize/COMPLETE, then resume: provider call count is 0. Resume just before the wire deadline: any started call ends by the same deadline. Include commit-ack delay, scheduler pause and clock-skew/rollback bounds. Without all evidence the catalog must classify the profile `QUIESCENCE_REQUIRED`. - [ ] RED terminalizer execution cases independently of durable workers: the read-only snapshot never mutates a permit; exact DRAINING fence + DB-time expiry + persisted registry are required; a bounded batch scans all historical generations and CASes each exact token/rowVersion once; concurrent release/terminalize and two terminalizers have one terminal winner per permit. Operation header/route affected set digest and permit terminalization FK commit all-or-none, commit-before-2xx is observable at Task 17, and commit-success/result-loss same-token replay returns the stored affected set even when current selection is empty. Different tokens consume successive bounded batches deterministically. Idempotency lookup/recomputed `request_input_digest` precedes selection; same token with batch bound `10` then `100`, changed actor/reason/route/drain generation or action conflicts with mutation 0. PURE_DISABLED has no terminalizer bean/thread; PRE bridge and CUTOVER_WAIT compose the proxied operation without any scheduler or provider I/O. - [ ] RED initialization/audit-journal cases: two distinct concurrent batch initialization tokens have exactly one winner; a same-token/same-route-set/input replay returns the stored full result; a two-route fixture commits all fences/header/children and the full immutable transport-proof registry snapshot or none; partial/extra/missing route/profile set and absent fences plus a nonempty row in any control/data-plane journal, including an orphan registry row, reject without mutation; initialization commit failure rolls back every fence, operation header/children and registry row; commit-success/result-loss is recoverable from the immutable operation journal; token uniqueness is global and exact route-set/action/input mismatch fails closed. For initialization and `BEGIN_DRAIN|TERMINALIZE_EXPIRED_PERMITS|COMPLETE_SWITCH|ABORT_DRAIN`, append the operation header/route results and applicable fence/permit mutations atomically, update each changed fence's `last_operation_token` to that header, retain the full history after later operations, and replay an old token after newer operations. Verify the fence pointer is only a latest-result integrity pointer and deleting/overwriting an older operation or updating/deleting a registry row is impossible. Replaying initialization returns the stored registry digest and cannot refresh it from a changed catalog. The adapter assigns `operation_sequence`, and attestation its sequence from the same DB sequence, only after the batch-init/global lock or exact route fence lock; committed route history has one causal total order despite concurrent actions, while rollback/global gaps are harmless. Populate operation/terminalization/attestation times only with post-lock `clock_timestamp()`. Start a terminalizer transaction before BEGIN, block it on the fence, then let BEGIN commit: its later sequence and timestamp must both follow BEGIN; a `CURRENT_TIMESTAMP`/transaction-start implementation is a RED failure. Recompute `route_set_digest`/`request_input_digest` from persisted header/children on replay; orphan header/child, empty child set and digest mismatch fail closed. A fake port test is not accepted as evidence for empty-journal checking, concurrency or physical commit semantics. - [ ] RED receipt cases: outer/semantic dedupe, orphan-before-accepted, later attach, conflict quarantine, and atomic persistence of an explicit suppression mutation or explicit no-op supplied by application. Persistence does not classify bounce/complaint policy. - [ ] RED the bounded fresh writer snapshot needed by PRE activation: exact route key set plus owner/state/generation and all-generation blocking permit aggregates plus persisted transport-proof registry digest/profile aggregates are read through `NotificationOperationsSnapshotAdapter`; stale/missing/partial rows are explicit results, not silently cached success. This adapter is implemented in Task 11 so Task 17 can compose only `NotificationOperationsSnapshotUseCase`, never a repository or outbound port. - [ ] Verify RED: ```bash cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*PostgreSqlNotificationClaimIntegrationTest' \ --tests '*NotificationAdmissionGateIntegrationTest' \ --tests '*NotificationReceiptStoreIntegrationTest' \ --tests '*NotificationCanonicalWriterFenceIntegrationTest' \ --tests '*NotificationWriterCutoverIntegrationTest' \ --tests '*NotificationWriterQuiescenceAttestationIntegrationTest' \ --tests '*NotificationWriterIrreversibleFenceIntegrationTest' \ --tests '*NotificationOperationsSnapshotIntegrationTest' \ --console=plain ``` - [ ] Implement vendor SQL only in `.postgresql`; keep JPA entities/repositories adapter-local. - [ ] Use DB time consistently for claim/lease comparisons and bounded batch sizes. - [ ] Keep render/provider calls out of every repository transaction. - [ ] Verify GREEN with the same command, then run `cd src && ./gradlew :adapter:outbound:persistence-jpa:check --console=plain`. - [ ] Acceptance claim: durable local primitives have real-PostgreSQL evidence; dispatcher fault matrix is Task 12 and no provider card R2 is implied. **Rollback checkpoint:** pause new admission/worker first. Preserve all active attempt/gate revisions and inspect accepted/indeterminate inventory before any code rollback. ### Task 12: Prove the deterministic dispatcher/reaper fault matrix across application and PostgreSQL **Owner:** `app-bootstrap` integration harness (`:app-bootstrap`) **Depends on:** Tasks 4, 7, 11 **Files — create:** - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/notification/NotificationDispatcherPostgresContractTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/notification/NotificationDispatcherCrashMatrixTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/notification/NotificationDispatcherContainerSupport.java` - [ ] Build a deterministic fake provider with explicit barriers before/after `WIRE_AUTHORIZED`, wire call, response and finalize; no actual network. - [ ] RED matrix: | crash/fault point | expected restart result | | --- | --- | | before claim commit | eligible, no attempt | | after claim before reserve | lease requeue, no provider call | | after reserve before `WIRE_AUTHORIZED` | safe requeue, no provider call | | `WIRE_AUTHORIZED` transaction fails to commit | authorization absent, provider call 0, safe requeue | | authorization commit succeeds but caller loses commit result | current worker calls provider 0; only reaper acts after deadline/grace | | after `WIRE_AUTHORIZED` before call | wait through deadline/grace, then reconcile or terminal indeterminate | | after possible write before response | no blind retry/fallback | | accepted response before finalize | late exact result or reconcile; duplicate risk explicit | | stale worker finalize after new owner | exact fact may append once; projection CAS rejected | | binding park racing authorization | pre-park authorization completes boundedly; later authorizations blocked | - [ ] Verify RED: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationDispatcherPostgresContractTest' \ --tests '*NotificationDispatcherCrashMatrixTest' \ --console=plain ``` - [ ] Wire real `TransactionPort`, store and application dispatcher manually in the test; do not introduce production scheduler/composition yet. - [ ] For lost commit-result ambiguity, prove the reaper reads the committed authorization only after deadline/grace and chooses provider reconciliation when the exact card supports it, otherwise terminal `INDETERMINATE`; it never treats the occurrence as definitely-not-sent. - [ ] Verify provider invocation occurs outside actual transaction. - [ ] Verify GREEN with the same command. - [ ] Acceptance claim: provider-neutral durable protocol is locally verified; actual process-kill evidence is Task 20 and Slack/SES cards remain unqualified. **Rollback checkpoint:** this is test-only integration. Production remains dark. ### Wave C exit gate - [ ] Run: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*Notification*' \ --console=plain cd src && ./gradlew :application-core:check \ :adapter:outbound:persistence-jpa:check \ --console=plain cd src && ./gradlew verifyCleanArchitectureDependencies \ verifyDependencyLocks --console=plain ``` - [ ] Capture PostgreSQL version, container image digest, test seed and fault matrix results. - [ ] Request durability/concurrency/crypto review before provider work. - [ ] Update the LLM Wiki branch-note with Wave C evidence and an explicit derived-document decision. --- ## Wave D — Slack and SES send providers ### Task 13: Implement Slack Web API `chat.postMessage` protocol **Owner leaf:** `adapter-outbound-notification` (`:adapter:outbound:notification`) **Depends on:** Tasks 5–7 **Files — create:** - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/slack/webapi/SlackWebApiRuntimeProfile.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/slack/webapi/SlackWebApiCredentialHandle.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/slack/webapi/SlackWebApiAttemptClient.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/slack/webapi/SlackWebApiOutcomeMapper.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/slack/webapi/SlackWebApiCapabilityCards.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/slack/webapi/SlackWebApiReadinessProbe.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/slack/webapi/SlackWebApiRateAdmission.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/slack/webapi/SlackWebApiProtocolTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/slack/webapi/SlackWebApiReadinessTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/slack/webapi/SlackWebApiRateAdmissionTest.java` **Files — modify:** - `src/adapter/outbound/notification/build.gradle` - `src/adapter/outbound/notification/gradle.lockfile` - [ ] First run a bounded dependency spike against the official Slack Java SDK. Prove endpoint, TLS/proxy/timeouts, connection lifecycle and retry count are controllable. If not, stop and amend the plan before adding a provider-local HTTP engine. - [ ] RED loopback protocol cases: exact `chat.postMessage` method/payload; one channel target; one physical request per authorization; bearer secret redaction; success `(channel, ts)` -> accepted/conversation reference; explicit `ok=false`; 429/`Retry-After`; auth/scope/account rejection -> park; timeout/connection loss/undecodable success -> indeterminate; payload/Block Kit bounds. - [ ] Verify RED: ```bash cd src && ./gradlew :adapter:outbound:notification:test \ --tests '*SlackWebApiProtocolTest' \ --tests '*SlackWebApiReadinessTest' \ --tests '*SlackWebApiRateAdmissionTest' \ --console=plain ``` - [ ] Add the minimum official SDK dependency, disable SDK retry, update the affected lockfile with the repository lock workflow, and verify actual request count in every test. - [ ] Regenerate and verify the exact notification leaf lock after the SDK declaration: ```bash cd src && ./gradlew :adapter:outbound:notification:resolveAndLockAll \ --write-locks --console=plain cd src && ./gradlew :adapter:outbound:notification:verifyDependencyLocks \ --console=plain ``` - [ ] Implement bounded provider-local admission and a safe control-plane readiness probe using Slack `auth.test`; verify workspace/token identity, scopes/card requirements and rate state without logging token/channel/message content. - [ ] Key Slack admission by exact `(workspaceBindingRevision, channelIdDigest, chat.postMessage)` scope; cap `Retry-After` by the attempt deadline/retry horizon and prove concurrent token-bucket bounds with an injected monotonic clock. - [ ] Acquire/close `SlackWebApiCredentialHandle` per protocol/probe call and test wipe on success, mapped exception, timeout and cancellation. - [ ] Implement both descriptor cards: `slack-web-api-inline-single-local-v1` and `slack-web-api-durable-single-local-v1`; do not add webhook semantics to either. - [ ] Document that response-loss without `ts` has no safe blind retry/native idempotency. - [ ] Verify GREEN with the same command, then run `cd src && ./gradlew :adapter:outbound:notification:check :adapter:outbound:notification:verifyDependencyLocks --console=plain`. - [ ] Acceptance claim: Slack local protocol R1; sandbox Task 19 is required for card R2. **Rollback checkpoint:** remove canonical Slack binding first so client/resources become zero; do not send accepted/indeterminate durable intents through webhook fallback. ### Task 14: Prove Slack inline and durable transaction semantics **Owner:** `app-bootstrap` integration harness (`:app-bootstrap`) **Depends on:** Tasks 2, 12, 13 **Files — create:** - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/notification/SlackInlineTransactionContractTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/notification/SlackDurableDispatchContractTest.java` - [ ] RED inline cases: provider call only after root commit; root rollback/commit failure/ambient transaction rejection -> call 0; returned `InlineCompleted` keeps target outcome; no durable retry claim. - [ ] RED durable cases: append joins business transaction; provider outside transaction; response loss becomes terminal unknown/reconcile unsupported; no blind retry or fallback. - [ ] Verify RED: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*SlackInlineTransactionContractTest' \ --tests '*SlackDurableDispatchContractTest' \ --console=plain ``` - [ ] Use the loopback Slack endpoint/client profile, not live Slack. - [ ] Verify GREEN with the same command. - [ ] Acceptance claim: mode-specific local protocol/config evidence; not sandbox R2. **Rollback checkpoint:** both modes remain unbound by default. No migration data is resent. ### Task 15: Implement Amazon SES v2 one-recipient submission protocol **Owner leaf:** `adapter-outbound-notification` (`:adapter:outbound:notification`) **Depends on:** Tasks 5–8 and Task 13, because both provider tasks modify the same `build.gradle`/`gradle.lockfile` and must serialize those edits **Files — create:** - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/email/ses/SesV2RuntimeProfile.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/email/ses/SesCredentialSourceProfile.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/email/ses/SesV2AttemptClient.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/email/ses/SesV2OutcomeMapper.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/email/ses/SesV2CapabilityCards.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/email/ses/SesV2ReadinessProbe.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/email/ses/SesV2RateAdmission.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/email/ses/SesV2ProtocolTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/email/ses/SesV2ReadinessTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/email/ses/SesV2RateAdmissionTest.java` **Files — modify:** - `src/adapter/outbound/notification/build.gradle` - `src/adapter/outbound/notification/gradle.lockfile` - [ ] RED loopback protocol cases: SES v2 `SendEmail`; exactly one recipient; local-rendered subject/text/HTML; reviewed from-identity/configuration-set; fixed EmailTag `ca_attempt_v1` with opaque pre-send correlation; SDK max physical attempt 1; `MessageId` -> provider accepted only; throttle/auth/account mapping; timeout/connection loss -> indeterminate; no PII/credential in telemetry. - [ ] Verify RED: ```bash cd src && ./gradlew :adapter:outbound:notification:test \ --tests '*SesV2ProtocolTest' \ --tests '*SesV2ReadinessTest' \ --tests '*SesV2RateAdmissionTest' \ --console=plain ``` - [ ] Add only required AWS SDK v2 SES/client modules under the existing BOM version. Disable SDK retry for mutation sends and verify request count. - [ ] Regenerate and verify the exact notification leaf lock after the AWS SDK declaration: ```bash cd src && ./gradlew :adapter:outbound:notification:resolveAndLockAll \ --write-locks --console=plain cd src && ./gradlew :adapter:outbound:notification:verifyDependencyLocks \ --console=plain ``` - [ ] Implement bounded quota/send-rate admission and a safe SES control-plane probe for exact account/region/sandbox/sending-enabled/quota/from-identity/configuration-set facts. Secret credential values remain outside the readiness snapshot. - [ ] Key SES admission by exact account/region/binding revision, intersect local token-bucket limits with current provider quota/send-rate and cap waits by the attempt deadline. The resolved credential source/generation is a readiness fact; only credential secret values are excluded. - [ ] Implement only `aws-ses-v2-durable-single-local-sns-v1`; no multi-recipient, stored-template, SMTP or Gmail aliases. - [ ] Treat the EmailTag as correlation, never provider idempotency. - [ ] Verify GREEN with the same command, then run `cd src && ./gradlew :adapter:outbound:notification:check :adapter:outbound:notification:verifyDependencyLocks --console=plain`. - [ ] Acceptance claim: SES local submission protocol R1; SNS and sandbox evidence still required. **Rollback checkpoint:** remove binding before client/SDK rollback. Preserve correlation/message references and never replay indeterminate sends automatically. ### Wave D exit gate - [ ] Run: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*Slack*' --tests '*Ses*' \ --console=plain cd src && ./gradlew :adapter:outbound:notification:check \ --console=plain cd src && ./gradlew :adapter:outbound:notification:verifyDependencyLocks \ verifyCleanArchitectureDependencies --console=plain ``` - [ ] Record actual loopback request counts and dependency/CVE/license review. - [ ] Request provider protocol review before ingress/composition. - [ ] Update the LLM Wiki branch-note with Wave D evidence and an explicit derived-document decision. --- ## Wave E — SNS receipt, canonical composition and operations ### Task 16: Implement verified SNS HTTPS ingress and normalized SES receipt mapping **Owner leaf:** `adapter-inbound-web` (`:adapter:inbound:web`), with the physical-commit integration owned by `app-bootstrap` (`:app-bootstrap`) **Depends on:** Tasks 4, 11, 15 **Files — create:** - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationReceiptIngressProfile.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationReceiptIngressDescriptor.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationReceiptIngressCapabilityDescriptorSource.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/SnsNotificationController.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/SnsMessageEnvelope.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/SnsSignatureV2Verifier.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/SnsSigningCertificateLoader.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/SesReceiptNormalizer.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/VerifiedNotificationReceiptOperation.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationReceiptIngressException.java` - `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/notification/SnsNotificationControllerTest.java` - `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/notification/SnsSignatureV2VerifierTest.java` - `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/notification/SesReceiptNormalizerTest.java` - `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/notification/NotificationReceiptIngressCapabilityDescriptorTest.java` - `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/notification/VerifiedNotificationReceiptOperationTest.java` **Files — modify:** - `src/.env` - `docs/security/public-paths-snapshot.txt` - [ ] Fix the exact endpoint to `/webhooks/notifications/aws-ses-v1`; append it to the comma-separated `SECURITY_PUBLIC_PATHS` SSOT in `src/.env`, then regenerate the reviewed snapshot with: ```bash cd src && ./gradlew verifyPublicPathSnapshot \ -PapprovePublicPathChange --console=plain ``` `SecurityConfig` already reads `SecuritySettings.publicPaths`; do not hard-code a matcher in Java. The endpoint bypasses JWT only and still requires SNS verification before normalization/use-case invocation. - [ ] RED cases: bounded POST/content-type/body/depth; SignatureVersion 2 canonical string; signature/key rotation; HTTPS allowlisted SNS cert host/path; DNS/IP/redirect/chain/expiry/SSRF rejection; exact TopicArn account/region/name. Freeze the exact card's `maxCallbackAge = SNS HTTP retry horizon + DLQ retention/redrive horizon + clock skew`. `ses-notification-v1` fixes `1h + 7d + 5m = 7d1h5m`, outer tombstone `8d` and inner semantic tombstone `30d`; the ingestion safety margin is `1h`. Arithmetic overflow or topology drift fails composition. A signed delayed retry just inside the bound is accepted/deduped, one just outside is rejected 4xx with receipt/quarantine DB mutation 0. Retention refuses tombstone expiry while the window is open; a forced-corruption fixture with a missing tombstone but retained semantic receipt fact still cannot reapply because of the store unique invariant. An outside-window replay remains age-rejected even after a deliberately expired tombstone. Outer/semantic tombstones outlive max age plus safety margin; supported event mapping and unknown schema/event quarantine remain covered. A signed timestamp beyond the allowed future skew is also rejected without mutation. - [ ] RED controller cases: verified receipt transaction commit before 2xx; transient store/commit failure returns `503`; bounded ingress overload returns `429` only before receipt admission and before any commit; duplicate returns idempotent 2xx; authenticated but unsupported schema/event is durably quarantined and then ACKed 2xx; invalid signature/topology is rejected 4xx without persistence; subscription/unsubscribe confirmation never fetches arbitrary URL; raw body/header/DTO never reaches application/log. Never return success for a receipt whose commit outcome is unknown. - [ ] Verify RED: ```bash cd src && ./gradlew :adapter:inbound:web:test \ --tests '*SnsNotificationControllerTest' \ --tests '*SnsSignatureV2VerifierTest' \ --tests '*SesReceiptNormalizerTest' \ --tests '*NotificationReceiptIngressCapabilityDescriptorTest' \ --tests '*VerifiedNotificationReceiptOperationTest' \ --console=plain ``` - [ ] Implement cert retrieval with a bounded inbound-adapter-local JDK client and strict allowlist; do not add a project edge to outbound httpclient/notification. - [ ] Normalize only after authenticity/topology validation to `NormalizedNotificationReceiptCommand`. - [ ] Derive the application-owned ingress capability descriptor from the actual verifier, endpoint, TopicArn/signature profile, ACK/DLQ contract, retry/DLQ/redrive horizons, max callback age and tombstone retention. Do not rebuild “actual” ingress facts from expected bootstrap settings. - [ ] Define those bounded durations in the checked-in `NotificationReceiptIngressProfile`, not an unrestricted request/env override. Reject invalid arithmetic, overflow, tombstone `<= maxCallbackAge + safety margin` and a topology descriptor that cannot prove the exact horizons. An original envelope older than max age is never directly replayed. A separate authenticated/approved operator procedure republishes the inner SES event through the exact TopicArn to create a new signed outer envelope while preserving the inner semantic fingerprint; it is allowed only while semantic dedupe retention remains. - [ ] Compose `ApplyNotificationReceiptUseCase` manually behind the verified inbound controller so provider signature authentication is not confused with JWT role authentication. The use case still declares the required application capability/permission contract and performs its write through `TransactionPort.inRootWrite`. - [ ] The auto-scanned controller injects only inbound-local `VerifiedNotificationReceiptOperation`, whose method accepts/returns application command/result types. Task 17 supplies a non-advised lambda/implementation bean that captures a distinct manually constructed `ApplyNotificationReceiptUseCase`; the use case itself is not a Spring bean. Controller tests prove raw/unverified requests cannot reach this seam, and composition tests prove it is not a method-security target and cannot be confused with the operator path. - [ ] Add `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/notification/SesSnsReceiptCommitAckContractTest.java` using real PostgreSQL + MockMvc/test server. Prove 2xx is emitted only after physical commit; commit failure/rollback invokes no success ACK, while a committed duplicate returns idempotent 2xx. - [ ] Run the cross-leaf RED/GREEN integration with: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*SesSnsReceiptCommitAckContractTest' --console=plain ``` - [ ] Verify GREEN with the same command, then run `cd src && ./gradlew :adapter:inbound:web:check --console=plain` and: ```bash cd src && ./gradlew verifyPublicPathSnapshot \ verifyCleanArchitectureDependencies --console=plain ``` - [ ] Acceptance claim: offline verified ingress protocol; actual AWS SNS callback remains Task 19. **Rollback checkpoint:** before endpoint removal, pause event destination and inventory SNS retries, DLQ and orphan receipts. Do not drop the inbox while retries are possible. ### Task 17: Add canonical graph settings, an inactive cutover bridge and zero-resource disabled mode **Owner:** `app-bootstrap` plus the thin transitional `adapter-inbound-web` endpoint **Depends on:** Tasks 5, 10–16 **Files — create:** - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationSettings.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationCompositionConfig.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationCompositionValidator.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationSecretMaterialBridge.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationWorkerRuntimeProfile.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationWriterStartupMode.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationWriterActivationGate.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationSameDataSourceTopologyValidator.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationDatabaseRoleSettings.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationDatabaseRoleComposition.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationDatabaseRoleTopologyValidator.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/FencedLegacyNotificationPort.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationZeroResourceTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationSecretMaterialBridgeTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationCanonicalSameTransactionCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationCanonicalWriterFenceSetCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationWriterActivationGateTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationInternalTrustContextCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/FencedLegacyNotificationPortTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationCutoverAuthorizationCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationDatabaseRoleCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationFlywayRoleIsolationTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/notification/NotificationWriterOwnershipCommitAckContractTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/NotificationSecretEnvContractTest.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/cutover/Ed25519NotificationWriterInventoryEvidenceVerifier.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/cutover/NotificationWriterEvidenceTrustCatalog.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/cutover/Ed25519NotificationWriterInventoryEvidenceVerifierTest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipController.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterFenceInitializationRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterFenceInitializationResponse.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterQuiescenceAttestationRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterQuiescenceAttestationResponse.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterPermitTerminalizationRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterPermitTerminalizationResponse.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipResponse.java` - `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipControllerTest.java` **Files — modify:** - `src/app-bootstrap/src/main/resources/application.yml` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/runtime/startup/MigrationStartupConfig.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/runtime/startup/MigrationStartupRunner.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/runtime/startup/RequiredEnvironmentValidator.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/runtime/startup/MigrationStartupRunnerTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/runtime/startup/RequiredEnvironmentValidatorTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/FlywayMigrationCompatibilityContractTest.java` - `src/.env` - `src/build.gradle` - `docs/registries/env-keys.yaml` - `docs/registries/secrets-classification.yaml` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/SecretsClassificationRegistryTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/adapter/outbound/OptionalAdapterBeanGatingTest.java` - `src/app-bootstrap/README.md` - `src/app-bootstrap/CLAUDE.md` - [ ] RED settings/composition cases: Model three disjoint states. `PURE_DISABLED` means canonical `disabled` plus legacy config absent and has every notification runtime resource 0. `PRE_LEGACY_BRIDGE` means canonical `disabled` plus exact legacy-only config in a structurally detected PRE artifact; canonical provider/store/worker resources are 0 while the transitional operator/fence/permit and selected legacy provider exist, with admission/call 0 before initialization. `CANONICAL_CONFIGURED` requires exact binding-ID equality and no legacy key. Unknown/extra/missing binding/property/provider/template/revision fails; expected-mode mismatch fails; exact send/store/ ingress account-region-configuration-set-topic tuple plus retry/DLQ/max-callback-age/tombstone profile; live/retained revision availability; legacy+canonical conflict fails even when values agree. The bounded `APP_NOTIFICATION_EXPECTED_WRITER_GENERATIONS` key set must equal the compiled route catalog; each value is the reviewed canonical target and its legacy predecessor is `target - 1`. A dark legacy bridge may start with no fences but admits 0 until batch initialization creates the complete predecessor set. During route-by-route switching, an exact key set may legally mix `ACTIVE/LEGACY@predecessor` (legacy admits), `DRAINING/LEGACY@predecessor` (neither admits) and `ACTIVE/CANONICAL@target` (canonical admits); each node opens only its owner-matching route. `REQUIRE_CANONICAL` final startup requires every route at the exact canonical target. `CUTOVER_WAIT` is permitted only in a PRE artifact with canonical-only config and the same closed predecessor/target state machine. Each predecessor route has admission, dispatcher claim and provider call 0 and readiness reports bounded `CUTOVER_WAIT`; it activates only after a committed fence read proves `ACTIVE/CANONICAL@target`. Before initialization, absent fences plus an empty persisted proof registry remain dark. Once any initialization row exists, the snapshot's route/profile/admission-role/proof-class/ evidence-revision/digest set must exactly equal `NotificationCutoverRouteCatalog`; partial, extra or drifted registry state closes readiness and every writer. Absent/partial/extra set, unrelated generation/owner/state, mixed legacy+canonical config in one process or caller override fails startup/activation. Runtime drift from the allowed predecessor/target state closes that route and readiness before any new work. `NotificationCompositionValidator` derives `CUTOVER_WAIT` only from the PRE structural stage, canonical-only graph and persisted exact fence state; no request, environment key or generic Spring property may select it. - [ ] RED zero-resource cases: PURE_DISABLED -> provider/store worker/client/executor/scheduler/probe/readiness component/operator/application DML·table scan 0; no secret lookup, cert fetch or provider health call. Run this assertion after release-owned Flyway migrations; V7/V8 DDL validation is not feature-gated application activity. An exact-empty database remains `AWAITING_SIGNED_FRESH_PROVISIONING`, with application admission, workers and provider clients dark until the separate provisioning operation commits. PRE_LEGACY_BRIDGE -> canonical resources 0, only exact transitional + legacy resources present, provider call 0 before audited initialization, then every legacy call requires a committed permit. CANONICAL_CONFIGURED -> legacy/transitional admission path closed as dictated by the ownership state. `NotificationZeroResourceTest`, `NotificationCompositionTest` and `FencedLegacyNotificationPortTest` each cover all applicable states and reject every same-process legacy+canonical overlap. - [ ] RED secret bridge cases: `SecretSource` resolves references only in bootstrap; adapter-owned handles receive versioned material; missing/blank/malformed secret fails without logging it; no raw key or provider credential is retained in settings, application values, entities or adapter-owned immutable records. This is not an end-to-end wipeability claim: the current `SecretSource`/ `EnvironmentSecretSource` API and JDBC credential APIs necessarily expose short-lived Java `String` values that cannot be wiped. Only the adapter-facing mutable copy is wiped on close. Tests prove no logging/exception/settings/entity retention and minimize/shorten source-to-handle copies; they do not claim erasure of an already-created JVM `String`. - [ ] RED same-DB topology cases against the canonical Spring infrastructure: `SpringTransactionPort`, the primary `PlatformTransactionManager`, JPA `EntityManagerFactory`/physical `DataSource` identity and notification store must resolve to one topology; configured durable startup fails closed on any mismatch. Do not require or manufacture a production “representative business repository”: this template intentionally has no production sample aggregate, and `sample-portfolio` must not leak into the runtime graph. Keep the real business-row write plus notification append commit/rollback proof in the Task 10 app-bootstrap integration fixture. Every adopted feature later adds its own composition contract proving its business store joins this primary transaction manager. - [ ] RED the complete but inactive transitional cutover surface before any qualification or deployment. The authenticated batch `POST /api/admin/notifications/writer-ownership/initialize-legacy` accepts only the reviewed initial-generation map keyed by the server-known routes, reason and operation token. The proxied operation loads `NotificationWriterRouteSet` derived from the compiled cutover catalog, derives the canonical ordered set and digest server-side, rejects missing/extra keys, and maps only to `InitializeNotificationWriterFencesOperation`. Neither route revisions outside that catalog nor a caller-provided route-set digest is authoritative. The separate `POST /api/admin/notifications/routes/{routeId}/writer-ownership` accepts only `BEGIN_DRAIN|COMPLETE_SWITCH|ABORT_DRAIN` for one route and maps only to `SwitchNotificationWriterOwnershipOperation`. BEGIN additionally requires a bounded signed deployment inventory manifest; the controller maps only its opaque bytes/key revision, while the application verifier independently authenticates exact environment/DB/route/PRE artifact and complete node inventory and derives every row/count/digest. A third authenticated `POST /api/admin/notifications/routes/{routeId}/writer-quiescence-attestations` maps only to `RecordNotificationWriterQuiescenceAttestationOperation`; request data supplies reviewed drain generation, a bounded signed quiescence manifest and token, never authoritative zero-fact/ old-node/permit digests. The verifier authenticates the exact frozen node inventory, each node's retired/quiesced fact plus irreversible deployment-generation/credential/egress fence, consumer inventory/count 0 and provider-call-ledger identity/open-count 0. The server uses post-lock DB time to validate the signer's bounded issued/expires window and record verified time, snapshots the locked persisted proof registry, permit/holder set and BEGIN inventory, and derives actor. The exact least-privilege mapping is `notification-operator -> notification:cutover,notification:cutover-terminalize, notification:cutover-attest`; default `admin` receives none. Controller tests cover 401/403, validation, actor spoof rejection and DTO/command mapping; composition tests prove distinct interface-based method-security initializer/terminalizer/switch/attestation proxies and internal-delegate separation. The switch request accepts `quiescenceAttestationToken` only for COMPLETE on a `QUIESCENCE_REQUIRED` route; it is required there and forbidden for BEGIN/ABORT/HARD_BOUND_PROVEN. Caller-supplied actor, profile/permit/holder/node-set or zero-fact digests are never authoritative. The fourth authenticated `POST /api/admin/notifications/routes/{routeId}/writer-permits/terminalize-expired` maps only to `TerminalizeExpiredNotificationWriterPermitsOperation`; request fields are exact drain generation, bounded batch, reason and token. Server derives actor, DB time, persisted registry and affected permit set. It requires `notification:cutover-terminalize`, has provider I/O 0 and cannot be invoked through a scheduler or read query. - [ ] RED/GREEN the Ed25519 verifier before controller composition. Freeze domain-separated `writer-inventory-manifest-v1` and `writer-quiescence-manifest-v1` length-prefixed canonical encodings with sorted bounded node/ fact rows. Cover valid current/retiring issuer key revisions, non-canonical order/encoding, duplicate/unknown fields, oversized node set, signature/algorithm/key downgrade, wrong environment/DB/route/artifact/generation/ledger identity, expiry and one-byte mutation. Only opaque digests may represent node/environment/ledger identity; raw hostnames, credentials and human PII are rejected from retained evidence. The checked-in closed trust catalog pins allowed issuer key IDs, public-key digests and current/retiring windows; request/env data cannot introduce a new trust anchor, and resolved public-key material must match the pinned digest. The reviewed signed profile/closed catalog, not environment input, pins `allowedClockSkew` and `acceptanceMargin`; the TTL env value may only tighten the maximum issuance window and can never relax `issuedAt - allowedClockSkew <= serverVerifiedAt <= expiresAt - acceptanceMargin`. Persist the canonical signed payload and signature together with issuer/trust snapshot, issued/expires/verified times and exact environment/DB/artifact/inventory/ledger identities. Cover signed zero-node authority, write-time verification, persisted round-trip and startup re-verification. SQL checks only immutable structure and digests; Java is the cryptographic authority. Production configuration contains issuer public-key references only, never an issuer private signing key. - [ ] RED real-PostgreSQL + MockMvc initialization cases before this surface can be deployed: exact physical commit before 2xx; commit failure/rollback never returns 2xx; concurrent different-token initialization has one winner; same-token result-loss replay returns the committed full route-set result; absent fences plus empty control/data-plane journals succeed, while partial/existing fence sets, any-nonempty-journal and route-set/digest/generation/token mismatch fail without mutation. Include a two-route fixture proving all fences, operation children and every proof-registry row commit atomically, and that replay returns the stored registry digest rather than refreshing from changed input. These cases live in `NotificationWriterOwnershipCommitAckContractTest`. The same test covers physical commit/failure/result-loss replay and stale-generation rejection for `BEGIN_DRAIN|TERMINALIZE_EXPIRED_PERMITS|COMPLETE_SWITCH|ABORT_DRAIN`, plus terminalizer and attestation commit-before-2xx, signed inventory/quiescence issuer and set mismatch, 401/403, idempotent replay and COMPLETE missing/stale/mismatched-token failures. It pauses an old bridge after credential/client/connection acquisition but before provider I/O, commits COMPLETE from an independent transaction, resumes the old bridge and proves provider I/O 0 because its deployment generation, credential and egress route are irreversibly disabled. A fake application port is not sufficient. - [ ] Freeze this minimum env/property grammar before implementation: | env key | property/use | default/classification | required when | | --- | --- | --- | --- | | `APP_NOTIFICATION_EXPECTED_STATE` | `app.notification.expected-state` | `disabled`, public enum | always | | `APP_NOTIFICATION_EXPECTED_BINDING_IDS` | exact binding ID CSV assertion | empty, public | configured | | `APP_NOTIFICATION_EXPECTED_WRITER_GENERATIONS` | exact `route-revision:canonical-target-generation` set assertion | empty, public bounded CSV | legacy bridge or configured canonical | | `APP_NOTIFICATION_CUTOVER_ATTESTATION_TTL` | maximum signed-evidence issuance/acceptance window; never a committed irreversible-proof lease | `5m`, public upper bound only | PRE cutover surface | | `APP_NOTIFICATION_CUTOVER_INVENTORY_ISSUER_KEY_REFS` | trusted Ed25519 inventory/quiescence verifier public-key SPKI refs; never signing keys | empty, public verification-material bounded CSV | PRE cutover surface | | `APP_NOTIFICATION_DB_EXPECTED_RUNTIME_ROLE` | exact runtime database principal | `notification_runtime`, public fixed value | notification schema present | | `APP_NOTIFICATION_DB_EXPECTED_MIGRATOR_ROLE` | exact Flyway owner principal | `notification_migrator`, public fixed value | Flyway enabled | | `APP_NOTIFICATION_DB_RUNTIME_USERNAME_REF` | runtime nonowner username reference | null, sensitive-config | notification schema present | | `APP_NOTIFICATION_DB_RUNTIME_PASSWORD_REF` | runtime nonowner password reference | null, sensitive-config | notification schema present | | `APP_NOTIFICATION_DB_MIGRATOR_USERNAME_REF` | Flyway owner username reference | null, sensitive-config | Flyway enabled | | `APP_NOTIFICATION_DB_MIGRATOR_PASSWORD_REF` | Flyway owner password reference | null, sensitive-config | Flyway enabled | | `APP_NOTIFICATION_DISPATCH_BATCH_SIZE` | bounded worker batch | `20`, public positive `<=100` | durable binding | | `APP_NOTIFICATION_DISPATCH_CONCURRENCY` | bounded worker concurrency | `4`, public positive | durable binding | | `APP_NOTIFICATION_CLAIM_LEASE` | claim lease | `30s`, public bounded duration | durable binding | | `APP_NOTIFICATION_ATTEMPT_TIMEOUT` | absolute provider attempt budget | `10s`, public bounded duration | any binding | | `APP_NOTIFICATION_FINALIZE_GRACE` | post-attempt drain/finalize grace | `30s`, public bounded duration | durable binding | | `APP_NOTIFICATION_RECEIPT_RECONCILE_INTERVAL` | orphan/reconcile cadence | `30s`, public bounded duration | receipt binding | | `APP_NOTIFICATION_RETENTION_INTERVAL` | redaction/purge cadence | `1h`, public bounded duration | durable binding | | `APP_NOTIFICATION_SLACK_WORKSPACE_REF` | exact workspace identity reference | null, sensitive-config | Slack binding | | `APP_NOTIFICATION_SLACK_DESTINATION_REF` | reviewed destination reference | null, sensitive-config | Slack binding | | `APP_NOTIFICATION_SLACK_TOKEN_REF` | bound reference to a `SecretSource` key | null, sensitive-config | Slack binding | | `APP_NOTIFICATION_SLACK_BOT_TOKEN` | env-backed secret-source-only material for baseline ref | null, secret | referenced Slack key | | `APP_NOTIFICATION_SES_REGION` | exact AWS region | null, public enum/region grammar | SES binding | | `APP_NOTIFICATION_SES_EXPECTED_CREDENTIAL_SOURCE` | workload credential mode assertion | null, public enum | SES binding | | `APP_NOTIFICATION_SES_FROM_IDENTITY_REF` | verified identity reference | null, sensitive-config | SES binding | | `APP_NOTIFICATION_SES_CONFIGURATION_SET` | exact event configuration set | null, sensitive-config | SES binding | | `APP_NOTIFICATION_SES_TOPIC_ARN` | exact feedback TopicArn | null, sensitive-config | SES binding | | `APP_NOTIFICATION_SES_DLQ_REF` | infrastructure DLQ identity | null, sensitive-config | SES binding | | `APP_NOTIFICATION_PAYLOAD_AEAD_CURRENT_KEY_REF` | bound current AEAD key reference/version | null, sensitive-config | durable binding | | `APP_NOTIFICATION_PAYLOAD_AEAD_RETIRING_KEY_REFS` | bounded retiring AEAD ref CSV | empty, sensitive-config | retained old ciphertext | | `APP_NOTIFICATION_LOOKUP_HMAC_CURRENT_KEY_REF` | bound current HMAC key reference/version | null, sensitive-config | durable/receipt binding | | `APP_NOTIFICATION_LOOKUP_HMAC_RETIRING_KEY_REFS` | bounded retiring HMAC ref CSV | empty, sensitive-config | rotating aliases | | `APP_NOTIFICATION_PAYLOAD_AEAD_KEY_V1` | secret-source-only AES-256-GCM material | null, secret | selected v1 ref | | `APP_NOTIFICATION_LOOKUP_HMAC_KEY_V1` | secret-source-only HMAC root material | null, secret | selected v1 ref | Binding IDs, kind/mode assertions, route/template/card revisions and ordered provider targets remain checked-in closed YAML/code catalog entries; do not accept an unrestricted env map that can invent them. Adding key version v2 means an additive versioned env/secret registry row, never overwriting v1 while retained rows reference it. - [ ] Add all public/sensitive keys to `env-keys.yaml` and `.env` with safe blank/default examples. Add token/AEAD/HMAC and sensitive identity rows to `secrets-classification.yaml`. Optional notification secrets use an explicit `required_when` condition and are validated only by `NotificationCompositionValidator` when the matching binding is compiled; disabled mode performs no secret lookup. - [ ] Bind only provider/identity/key `*_REF` values into immutable `NotificationSettings`; raw token/AEAD/HMAC bytes must never be an `application.yml` placeholder or settings field. The bridge calls `SecretSource.resolve(ref)` only after the canonical graph selects that provider/key revision. Database runtime/migrator references bind exclusively to `NotificationDatabaseRoleSettings` and the dedicated runtime/Flyway data-source wiring, never to application commands/records or `NotificationSettings`. Provisioner references are absent from the normal application and exist only in the Task 21 provisioning source set. - [ ] Extend `verifyEnvKeys` narrowly: an `.env` key without an `application.yml` placeholder is legal only when `secrets-classification.yaml` registers it as `secret-source-only` with a `required_when` condition. Unknown/orphan public keys still fail. Add the secrets registry as a task input and tests for allowed secret-source-only, misspelled secret, disabled zero-lookup and ordinary orphan rejection. - [ ] Update `SecretsClassificationRegistryTest` so unconditional secret rows still match `SecretSourceValidator.REQUIRED_PROD_SECRETS` 1:1, while `required_when` rows are excluded from that global list and are covered by exact notification composition tests. Do not make optional Notification secrets globally required in prod. - [ ] Verify RED: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationCompositionTest' \ --tests '*NotificationZeroResourceTest' \ --tests '*NotificationSecretMaterialBridgeTest' \ --tests '*NotificationCanonicalSameTransactionCompositionTest' \ --tests '*NotificationCanonicalWriterFenceSetCompositionTest' \ --tests '*NotificationWriterActivationGateTest' \ --tests '*NotificationInternalTrustContextCompositionTest' \ --tests '*FencedLegacyNotificationPortTest' \ --tests '*NotificationCutoverAuthorizationCompositionTest' \ --tests '*NotificationWriterOwnershipCommitAckContractTest' \ --tests '*NotificationSecretEnvContractTest' \ --console=plain cd src && ./gradlew :adapter:inbound:web:test \ --tests '*NotificationWriterOwnershipControllerTest' --console=plain cd src && ./gradlew :adapter:outbound:notification:test \ --tests '*Ed25519NotificationWriterInventoryEvidenceVerifierTest' --console=plain ``` - [ ] Bind canonical settings once, derive minimal outbound/inbound/persistence profiles, and pass application-owned provider/store/ingress capability descriptors to the pure application compatibility validator. - [ ] At startup, have `NotificationSameDataSourceTopologyValidator` inspect the canonical transaction port, primary transaction-manager/JPA resource and notification-store topology descriptor and reject durable mode unless they share the same physical transaction manager/data-source identity. Feature-specific business-store atomicity remains an explicit feature composition test, not a fabricated bootstrap bean. This composition-root invariant must not introduce a new project dependency edge. - [ ] Wire exactly the three non-interchangeable PostgreSQL principals `notification_migrator`, `notification_runtime`, `notification_provisioner` and reject any additional notification-scoped owner/member/grantee. `notification_migrator` owns the notification schema and is used only by the dedicated Flyway data source; `notification_runtime` is a nonowner used by the application transaction manager; the `notification_provisioner` credential is not composed here and is available only to the explicit Task 21 provisioning source set. Resolve migrator/runtime credential references separately, assert `current_user` and ownership/grants at startup, and fail on shared credentials, owner runtime, role inheritance or unexpected membership. Tests prove runtime cannot run DDL, mutate retained evidence/provenance directly, use transitional functions or consume sequences beyond its exact runtime grants, except that the exact PRE artifact grants runtime only its explicitly enumerated transitional function `EXECUTE` surface and FINAL revokes that entire surface. Flyway cannot be reached through an application bean. - [ ] Before resolving either runtime or migrator credentials, require the Task 9 external DB-admin/IaC bootstrap revision and exact role-inventory/ownership probe from `docs/runbooks/notification-database-role-bootstrap.md`. Application startup never creates, alters or grants role membership. Missing/mismatched bootstrap evidence, shared credentials, unexpected membership or an additional notification-scoped principal fails before Flyway or runtime table access. - [ ] Update the existing executable startup seam, not a parallel notification-only migration path. `MigrationStartupConfig` builds Flyway from the dedicated migrator data source; `MigrationStartupRunner` asserts its connection `current_user=notification_migrator` before `migrate()` and never receives the primary runtime data source; `RequiredEnvironmentValidator` requires the runtime credential refs always and migrator refs exactly when startup migration is enabled. Keep the DB URL/schema locations common, but make the credential binding choice explicit: `spring.datasource.*` remains the `notification_runtime` application data source, while `spring.flyway.user` and `spring.flyway.password` are intentionally unbound/forbidden so Boot cannot treat secret refs as credentials or fall back to the runtime principal. The dedicated Flyway data source gets resolved username/password bytes only from `NotificationDatabaseRoleSettings`; tests fail if Flyway is constructed from the primary data source, if either role ref aliases the other, or if migration-on-startup can run without the migrator refs. - [ ] Invoke `NotificationCanonicalWriterFenceGuard` from canonical intent admission inside the same business-write/append transaction; configured bindings require an exact checked-in route/fence generation. A stale generation or non-canonical owner rolls back both business write and intent append and prevents worker activation. Bootstrap never calls the guard or `NotificationCanonicalWriterFencePort` directly. - [ ] Freeze the trust-context wiring matrix: scheduler/worker/health, feature-internal notification orchestration and legacy bridge use manually composed non-bean application delegates; verified SNS ingress uses a distinct manually composed receipt delegate after signature/topology authentication; neither path is subjected to JWT method security or exposed to a normal controller. Only the transitional human initializer and cutover operations are registered as Spring method-security beans behind `InitializeNotificationWriterFencesOperation`, `TerminalizeExpiredNotificationWriterPermitsOperation`, `RecordNotificationWriterQuiescenceAttestationOperation` and `SwitchNotificationWriterOwnershipOperation`. Never reuse any proxied target as an internal delegate, and never publish the internal delegates as Spring use-case beans. Context tests assert bean absence/identity separation so schedulers/SNS do not fail for missing `Authentication` and operator calls cannot bypass AOP. - [ ] Build the legacy fence wrapper and full operator endpoint in this task, but keep them dark: no automatic fence initialization, no provider call while the fence is absent, and no rollout before Wave F qualifies this exact artifact. This ordering is deliberate: Task 19/20 manifests may say `PRE_CUTOVER_BRIDGE` only when the compiled bridge/operator/permit surface being deployed in 21A is already present. Task 21A/21B perform human-controlled state transitions and rerun the same tests; they do not add or alter production code before cleanup. - [ ] `NotificationSettings` is the only `@ConfigurationProperties` binder. Provider and worker `*RuntimeProfile` types are plain immutable derived slices with no binding annotation or independent defaults. - [ ] `NotificationWriterActivationGate` calls only `NotificationOperationsSnapshotUseCase` for bounded committed refreshes. It never injects `NotificationOperationsSnapshotPort`, a repository, entity, `EntityManager` or JDBC type. Snapshot freshness gates readiness/worker admission, while `NotificationCanonicalWriterFenceGuard` inside each business-write/append transaction remains the final authoritative fence. Tests prove stale cache cannot authorize an append and no route opens before a fresh exact committed snapshot. - [ ] Collect actual descriptors only from the three adapter-owned descriptor sources. Keep expected settings as a separate input and add negative tests that mutate each actual source independently; a validator that derives expected and actual from the same settings is tautological and must fail review. - [ ] Default `application.yml` to explicit `disabled`; no blank-is-disabled ambiguity. - [ ] Add every new env key with type/default/secret classification and safe example; no secret value in YAML/docs/tests. - [ ] Verify GREEN with the same command and: ```bash cd src && ./gradlew :adapter:inbound:web:check --console=plain cd src && ./gradlew verifyEnvKeys \ verifyCleanArchitectureDependencies --console=plain ``` - [ ] Acceptance claim: exact local composition and disabled resource safety; real provider state remains unqualified. **Rollback checkpoint:** deploy canonical code/config dark. Never enable canonical and legacy for the same route. Before `COMPLETE_SWITCH`, rollback begins by setting canonical expected-state disabled, pausing workers and using `ABORT_DRAIN` only from DRAINING. After COMPLETE, use the forward-only incident path in Task 21B; do not re-enable legacy. ### Task 18: Add bounded workers, retention, observability and readiness truth **Owner:** `app-bootstrap` composition with adapter-owned operations **Depends on:** Task 17 **Files — create:** - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationDispatcherScheduler.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationReceiptReconcilerScheduler.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationRetentionScheduler.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationMetrics.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/MeteredNotificationIntentAppendPort.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/MeteredInlineNotificationAttemptPort.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/MeteredNotificationProviderAttemptPort.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/MeteredNotificationReceiptStorePort.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationHealthIndicator.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationReadiness.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationLifecycleTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationObservabilityPrivacyTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationMeteredDecoratorTest.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationMaintenanceStoreAdapter.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationRetentionIntegrationTest.java` - [ ] RED lifecycle cases: bounded executor/queue; no unbounded scheduler overlap; graceful stop halts new claims then drains bounded authorized attempts; stale lease recovery; readiness down without liveness restart loop; disabled has zero threads. - [ ] RED observability/privacy cases: bounded tags only; no intent/recipient/provider message/tenant raw IDs; representative PII/secret markers absent from log/span/metric/health/exception/DB plaintext; backlog and oldest age expose only bounded route/card labels. Cover append, inline attempt, provider attempt duration/outcome and receipt/orphan event counters, including synchronous inline and SNS paths. - [ ] RED metering isolation cases: each bootstrap-owned decorator delegates exactly once; timing/tag construction and meter registry failures are swallowed into a bounded diagnostic and never change send/store/use-case result, exception or transaction semantics. No Micrometer/bootstrap type may cross into `application-core`, outbound notification, persistence or inbound web. - [ ] RED retention/rotation cases: payload redaction separated from dedupe tombstone; purge order honors FK; active/backlog/ receipt window blocks key/template removal; old HMAC alias matches then upgrades to current; indefinite suppression re-HMAC before ciphertext removal. SNS outer/semantic tombstones outlive the checked-in max callback age plus ingestion safety margin, and semantic retention covers the approved manual redrive horizon. Purge just before either bound fails; just after all bounds and orphan/backup needs pass may succeed. - [ ] Verify RED: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationLifecycleTest' \ --tests '*NotificationObservabilityPrivacyTest' \ --tests '*NotificationMeteredDecoratorTest' \ --console=plain cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*NotificationRetentionIntegrationTest' \ --tests '*NotificationOperationsSnapshotIntegrationTest' \ --console=plain ``` - [ ] Implement scheduler beans only for compiled durable/receipt bindings; use bounded batch, concurrency, retry and shutdown deadlines from reviewed settings caps. - [ ] Schedulers invoke `NotificationDispatchUseCase`, receipt/reconciliation use cases and `NotificationMaintenanceUseCase`; they never invoke repositories or persistence entities. - [ ] Metrics, health and readiness obtain backlog/oldest-age/card facts only through `NotificationOperationsSnapshotUseCase` and adapter-owned provider readiness probes. Bootstrap never calls `NotificationOperationsSnapshotPort`, repositories or entities directly, and it never calls Slack/AWS SDKs directly. - [ ] Writer-cutover snapshot fields report ACTIVE permit count/max expiry by exact route and legacy owner across every historical fence generation, plus `TIMED_OUT_UNPROVEN` count/profile-set digest and only bounded attestation freshness/status facts—never raw token/evidence. A timestamp-past ACTIVE row remains active until exact token/version CAS; unproven timeout stays mechanically blocking without attestation. Tests include an old-generation/multi-profile permit surviving `ABORT_DRAIN` and blocking the next `COMPLETE_SWITCH`. - [ ] Wrap application outbound ports only at the composition root with bootstrap-owned metered decorators, following the existing `MeteredDistributedLockPort` pattern. Never author or wrap an application inbound use case in bootstrap. Decorators observe only bounded application result enums/card IDs and elapsed time; they neither own policy nor cause provider/store retries. - [ ] Complete the persistence binding matrix: append/delivery store -> `NotificationStoreAdapter`, receipt store -> `NotificationReceiptStoreAdapter`, technical suppression -> `NotificationTechnicalSuppressionStoreAdapter`, maintenance -> `NotificationMaintenanceStoreAdapter`, safe operational projection -> `NotificationOperationsSnapshotAdapter`. - [ ] Health/readiness must report exact card/profile and backlog state; fake/offline evidence cannot make a provider card ready. - [ ] Verify GREEN with the same commands. - [ ] Acceptance claim: local operational safety/privacy evidence; sandbox/load evidence still pending. **Rollback checkpoint:** pause new admission, stop schedulers, inventory in-flight/accepted/ indeterminate rows, preserve all referenced revisions and prefer forward-fix. ### Wave E exit gate - [ ] Run: ```bash cd src && ./gradlew :adapter:inbound:web:check \ :adapter:outbound:persistence-jpa:check \ :app-bootstrap:check \ verifyEnvKeys \ verifyPublicPathSnapshot \ verifyCleanArchitectureDependencies \ --console=plain ``` - [ ] Request inbound-security, configuration, operations and privacy review. - [ ] Update the LLM Wiki branch-note with Wave E evidence and an explicit derived-document decision. --- ## Wave F — Exact provider qualification ### Task 19: Produce opt-in Slack/SES real-provider and SNS topology evidence **Owner:** `app-bootstrap` verification source sets (`:app-bootstrap`) **Depends on:** Tasks 13–18 **Files — create:** - `src/app-bootstrap/src/notificationSlackReadiness/java/dev/caskeleton/bootstrap/notification/SlackNotificationReadinessTest.java` - `src/app-bootstrap/src/notificationSesReadiness/java/dev/caskeleton/bootstrap/notification/SesNotificationReadinessTest.java` - `src/app-bootstrap/src/notificationSesReadiness/java/dev/caskeleton/bootstrap/notification/SesSnsFeedbackReadinessTest.java` - `src/app-bootstrap/src/notificationSesDlqReadiness/java/dev/caskeleton/bootstrap/notification/SesSnsDlqReadinessTest.java` - `src/app-bootstrap/src/notificationReadinessSupport/java/dev/caskeleton/bootstrap/notification/NotificationReleaseStage.java` - `src/app-bootstrap/src/notificationReadinessSupport/java/dev/caskeleton/bootstrap/notification/NotificationReleaseStageDetector.java` - `src/app-bootstrap/src/notificationReadinessSupport/java/dev/caskeleton/bootstrap/notification/NotificationProductionEvidenceIssuerInput.java` - `src/app-bootstrap/src/notificationReadinessSupport/java/dev/caskeleton/bootstrap/notification/NotificationProductionEvidenceIssuerClient.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationReleaseStageDetectorTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationReleaseStageLegacyMarkerAllowlistTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationEvidenceManifestTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationProductionEvidenceIssuerClientTest.java` - `src/app-bootstrap/src/test/resources/notification/evidence/notification-evidence-schema-v1.json` **Files — modify:** - `src/app-bootstrap/build.gradle` - `src/app-bootstrap/gradle.lockfile` - `docs/registries/env-keys.yaml` - `docs/registries/secrets-classification.yaml` - `src/adapter/outbound/notification/README.md` - `src/adapter/outbound/notification/CLAUDE.md` - [ ] Register proposed tasks: `notificationSlackReadiness`, `notificationSesReadiness` and the separately authorized `notificationSesDlqReadiness`, plus a shared `notificationReadinessSupport` source set consumed by every readiness/qualification lane. Do not register the final aggregator here and do not attach any real-provider task to ordinary `test` or `check`. - [ ] Have the build generate an immutable artifact-structure input from the compiled production JAR class/resource inventory, reviewed production dependency locks, source digest and artifact digest. `NotificationReleaseStageDetector` derives `PRE_CUTOVER_BRIDGE|FINAL_CLEANUP` from that input only; no property, environment variable, test argument or caller may override it. Both stages require the additive V7 journal schema; historical table/column/migration names are not executable legacy markers. `PRE_CUTOVER_BRIDGE` requires the executable legacy notifier, `FencedLegacyNotificationPort`, PRE cutover catalog/route set, initializer/switch/permit/ terminalizer/quiescence-attestation classes/beans/controller, all three operator permissions and absence of the final cleanup migration. `FINAL_CLEANUP` requires those executable legacy/bridge/operator/permit/terminalizer/attestation/`CUTOVER_WAIT` classes/beans/config/role mappings absent, the retained canonical route catalog/route set, canonical fence guard/adapter, canonical-only `REQUIRE_CANONICAL` config, retained V7 proof-registry/permit/operation history plus canonical signed BEGIN-inventory header, quiescence/attestation header, evidence trust snapshot, `NotificationWriterFinalizationDiscriminatorEntity`/ `NotificationWriterFinalizationDiscriminatorJpaRepository`, fresh-provenance schema resource and the exact closed discriminator states `AWAITING_SIGNED_FRESH_PROVISIONING|FRESH_PROVISIONED|UPGRADE_VALIDATED`, reviewed V8 validation/awaiting state and cleanup migration present. The structural detector rejects an absent/unknown discriminator, a fresh token plus validated-history digest overlap, and any state/resource combination outside the Task 9 XOR. It also requires the explicit post-migration `notificationFreshProvisioning` task contract and rejects migration-time provisioning. Partial, contradictory, unknown or digest-mismatched inventories fail before a provider side effect. - [ ] Keep the explicit legacy marker names only in `NotificationReleaseStageDetector` and its two reviewed detector tests. Do not obfuscate names with string concatenation. `NotificationReleaseStageLegacyMarkerAllowlistTest` scans the repository, requires the complete expected marker set in the detector, and fails on any occurrence outside the exact path/symbol allowlist. Production consumer-zero hygiene is a separate scan over every registered production leaf's `src/**/src/main` tree plus production config. - [ ] Before any real provider lane, prepare the isolated sandbox by the detector-derived stage; fixture SQL and caller stage override are forbidden: - `PRE_CUTOVER_BRIDGE`: deploy the exact PRE artifact with legacy-only config and admission closed; invoke authenticated batch `INITIALIZE_LEGACY`; start canonical-only instances of the same artifact in `CUTOVER_WAIT` with the reviewed future generation set and prove their admission/worker/provider-call count is 0. For every route, call the independent infrastructure issuer through `NotificationProductionEvidenceIssuerClient`; it signs the complete environment/DB/route/artifact old-node set and that manifest is bound to `BEGIN_DRAIN`. Omitted/extra node or permit holder fails. If expired ACTIVE permits remain, invoke the authenticated bounded terminalizer until the read-only snapshot reports ACTIVE 0. Then select exactly one PRE proof arm. On current R0, `QUIESCENCE` proves every frozen instance has an irreversible deployment-generation tombstone and legacy credential/egress revocation, consumer inventory 0 and provider-ledger open-count 0; the issuer signs that exact manifest and the operation root-commits its attestation before COMPLETE. Reverify the retained signed headers/trust snapshot in Java and prove a paused old process with cached credential/client/connection performs provider I/O 0 after COMPLETE. `HARD_BOUND` also Java-reverifies the retained signed BEGIN inventory header/children before COMPLETE, proves the exact all-hard-bound registry/evidence revision and safe terminal permits, keeps ACTIVE 0, and forbids quiescence attestation and its child facts. Verify latest operation/attestation results and that only exact `ACTIVE/CANONICAL@g_final` routes activate on the waiting instances. Capture commit-before-2xx, wait-to-active evidence and the immutable cutover operation-history digest including BEGIN inventory plus exactly the selected arm: attestation permit/holder/node sets, irreversible fence identities and consumer/provider-ledger zero snapshot for `QUIESCENCE`, or registry/evidence revision and safe terminal permits for `HARD_BOUND`. - `FINAL_CLEANUP`: deploy the exact cleanup artifact either on the already-canonical upgrade sandbox whose V8 validation preserves the route set/history, or on a clean-provisioning V1..V8 sandbox. V8 must leave the latter `AWAITING_SIGNED_FRESH_PROVISIONING`; obtain an independent issuer-signed database-birth authorization through the production issuer client only after its control plane has durably committed and independently observed the permanent no-legacy-authority fence. The authorization and retained row must include the complete Task 9 `notification_fresh_installation_provenance` field set and causal order, not a summarized subset: enforcement revision/digest/activation/read-back precedes cached DB-session and provider-flow termination; the causally later post-enforcement zero manifest includes provider-ledger cut revision/time plus entry/open/indeterminate counts 0; the permanent fence canonical payload/digest is seal-committed, marked irreversible and read back before signing. Require the exact credential issuance/revocation set digests and completion facts, ingress/egress denial policy digests and established-flow blocks, `legacy_database_session_inventory_digest`, `legacy_database_session_open_count=0`, `legacy_database_session_termination_evidence_digest`, `legacy_provider_connection_flow_inventory_digest`, `legacy_provider_connection_flow_open_count=0`, and `legacy_provider_connection_flow_termination_evidence_digest`. The issuer refuses a pre-enforcement zero snapshot, cross-revision composition, sign-before-seal, unsigned, uncommitted or reversible fence. Invoke the opt-in `notificationFreshProvisioning` Gradle/CLI operation. It commits signed provenance, `INITIALIZE_CANONICAL_FRESH` and the exact route fence set in one transaction, after which a restart may enter `REQUIRE_CANONICAL`. Prove an unsigned/expired/wrong environment/DB/artifact/inventory/ledger authorization, a nonempty partial fence set and an existing-history fresh marker all fail. Pause/inject an old deployment generation, cached legacy DB/provider credential, established DB session/provider connection and legacy egress path before provisioning; prove both exact session/connection inventories are open-count 0, both termination evidences are valid and both established-flow block facts are true, and legacy DB I/O and provider I/O are both 0 before and after provisioning, including after resume. Prove canonical catalog/config/fence exact equality and transitional endpoint/class/role 0. Do not reference or invoke deleted initializer, switch, permit, terminalizer or attestation types. Capture the cleanup migration/validated-fence-set digest. Missing/partial fences, mixed-stage markers, direct SQL, a PRE operation in FINAL or a legacy/canonical config overlap makes the sandbox ineligible. - [ ] Make each real lane fail closed when explicitly invoked without required exact credentials, sandbox destination/account/region/workspace/configuration set/topic/DLQ inputs. Do not convert absent configuration to JUnit success/skip. - [ ] Require an already deployed, isolated sandbox topology; a Gradle process on localhost is never considered reachable by SNS: the same release source/artifact revision runs behind public HTTPS `/webhooks/notifications/aws-ses-v1`; an SES configuration-set event destination targets the exact SNS TopicArn; the HTTPS subscription is confirmed and has an explicit redrive policy to the reviewed DLQ; its bounded SNS HTTP `DeliveryPolicy` fixes retry count, min/max delay, backoff function and total retry horizon; the deployed service and readiness runner observe the same sandbox PostgreSQL notification journal/application store. Validate subscription ARN/status, TopicArn, configuration set, endpoint, delivery-policy digest/horizon, DLQ/redrive policy/retention/redrive horizon, derived max callback age, ingress tombstone retention and deployed revision before sending. Actual values must equal the checked-in ingress profile. - [ ] Freeze required readiness inputs, all registered/classified without logging their values: `APP_NOTIFICATION_READINESS_HTTPS_BASE_URL`, `APP_NOTIFICATION_READINESS_DEPLOYED_REVISION`, `APP_NOTIFICATION_READINESS_JOURNAL_DB_REF`, `APP_NOTIFICATION_READINESS_SES_SUBSCRIPTION_ARN`, `APP_NOTIFICATION_READINESS_SES_DELIVERY_POLICY_DIGEST`, `APP_NOTIFICATION_READINESS_SES_DLQ_REF`, `APP_NOTIFICATION_READINESS_MAX_WAIT`, and `APP_NOTIFICATION_READINESS_DLQ_DRILL_ENABLED`, `APP_NOTIFICATION_READINESS_EVIDENCE_ISSUER_ENDPOINT_REF`, `APP_NOTIFICATION_READINESS_EVIDENCE_ISSUER_CLIENT_CREDENTIAL_REF` and `APP_NOTIFICATION_READINESS_DB_BIRTH_AUTHORIZATION_REF`, plus `APP_NOTIFICATION_READINESS_NO_LEGACY_AUTHORITY_FENCE_REF`. `NotificationProductionEvidenceIssuerClient` submits only bounded environment/DB/artifact/route/node/consumer/provider-ledger identities and resolves client-auth references outside logs. No production source, configuration, environment key, test resource or artifact may contain an evidence issuer private signing key. Provider account/workspace/destination and credential refs remain the exact Task 17 settings, not a second defaulting configuration tree. - [ ] Slack lane: send a bounded sandbox probe with both exact mode profiles; capture `(channel, ts)` and actual request count; verify token/workspace/channel scope, hidden retry 0, no sensitive artifact. - [ ] SES lane: send exactly one simulator/verified sandbox recipient with `ca_attempt_v1`; observe `MessageId`; receive an authentic SNS HTTPS callback at the deployed endpoint; poll the same sandbox journal for the committed correlation/receipt projection; verify TopicArn/configuration-set/tag matching and commit-before-ACK without recording recipient/content. Use a unique correlation, bounded wait and cleanup only through `NotificationMaintenanceUseCase`. - [ ] Keep the DLQ drill separate and human-approved. With `APP_NOTIFICATION_READINESS_DLQ_DRILL_ENABLED=true`, use an isolated copy of the same artifact and SNS subscription whose notification database is deliberately unavailable, publish a bounded authentic SNS probe, observe application `503`, SNS retry and eventual movement to the exact DLQ, then restore the sandbox and purge only the correlated probe. Task 16 remains the deterministic proof of commit-failure-to-503; this lane proves the deployed SNS retry/redrive topology. Require `READINESS_MAX_WAIT` to exceed the queried bounded retry horizon plus a reviewed observation margin while remaining below the task-wide safety cap. It must never run against a production subscription. - [ ] Emit a sanitized provider manifest for each lane with the complete comparison axes: card and binding revision; channel/mode/strategy/route; template/render/serialization/escaping revision; submission/correlation/idempotency/reconciliation profile; receipt/projection profile; credential-source generation; account/region/workspace digest; SES configuration set, TopicArn, subscription ARN, exact delivery-policy digest/retry horizon, DLQ/redrive and ingress profile, max callback age and tombstone/manual-redrive retention; persistence schema/crypto profile; writer route-set digest and exact canonical generation-set digest; a closed stage-discriminated ownership evidence union: `PRE.QUIESCENCE` is signed BEGIN inventory + signed quiescence/attestation header and trust snapshot + exact irreversible tombstone/credential/egress facts + ACTIVE permit 0 + consumer inventory 0 + provider-ledger 0 + cutover history; `PRE.HARD_BOUND` is signed BEGIN inventory + the exact all-hard-bound registry/evidence revision + safe terminal permits + ACTIVE permit 0 and forbids a quiescence attestation. `FINAL.FRESH` is V8/cleanup structural digest + discriminator `FRESH_PROVISIONED` with its fresh token and null validated-history digest + signed DB birth certificate + the complete exact Task 9 fresh-provenance field set and enforcement-read-back → cached-session/flow termination → post-enforcement zero manifest (provider-ledger entry/open/indeterminate 0) → irreversible fence seal/read-back → signature causal order + signed fresh provenance/trust snapshot + `INITIALIZE_CANONICAL_FRESH`; `FINAL.UPGRADE` is V8/cleanup structural digest + discriminator `UPGRADE_VALIDATED` with its complete retained-history digest, null fresh token and absent fresh provenance. `AWAITING_SIGNED_FRESH_PROVISIONING` cannot emit readiness evidence. Source/dependency/artifact/deployed revision; actual request count; lane/run timestamp/expiry. Include immutable `release_stage`; the schema rejects unknown/missing/overlapping union arms, evidence forbidden by the selected discriminator, detector/manifest stage mismatch and sensitive raw values. - [ ] Task 19 only produces real-provider/callback/DLQ evidence. It does not aggregate local durability qualification and by itself does not authorize an operational R2 claim. - [ ] RED then GREEN the evidence schema before any live side effect. Cover every required axis, unknown/missing fields, sensitive raw value rejection, lane/card mismatch, expiry, skipped evidence and release-stage mismatch. RED/GREEN the structural detector for exact PRE, exact FINAL, mixed, unknown, caller-override attempts and artifact digest mismatch. Exact FINAL must include retained V7 proof-registry/permit/operation history and signed inventory/quiescence/attestation headers, trust snapshots, the actual finalization discriminator entity/repository with its three exact states/XOR, and a fresh-provenance schema resource whose row is required exactly for `FRESH_PROVISIONED` and forbidden for `AWAITING_SIGNED_FRESH_PROVISIONING|UPGRADE_VALIDATED`, plus V8 awaiting/validation and cleanup migration while all executable permit/terminalizer/cutover classes/beans/config are absent; broad text scanning that classifies historical V7 as PRE must fail. PRE tests remove each required terminalizer or attestation operation, endpoint, permission and schema marker in turn and require mixed/unknown. FINAL tests require executable terminalizer/attestation/role 0 while retained history remains and enumerate only `FRESH_PROVISIONED|UPGRADE_VALIDATED`; PRE tests enumerate only `QUIESCENCE|HARD_BOUND`. The production issuer-client test rejects a response not signed by the pinned public trust snapshot and asserts that no private-key input is bindable: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationEvidenceManifestTest' \ --tests '*NotificationReleaseStageDetectorTest' \ --tests '*NotificationReleaseStageLegacyMarkerAllowlistTest' \ --tests '*NotificationProductionEvidenceIssuerClientTest' \ --console=plain ``` - [ ] After registering the readiness source sets/tasks and all their dependencies, regenerate and verify the exact app-bootstrap lock before resolving or executing any readiness task: ```bash cd src && ./gradlew :app-bootstrap:resolveAndLockAll \ --write-locks --console=plain cd src && ./gradlew :app-bootstrap:verifyDependencyLocks --console=plain ``` - [ ] Run only in an explicitly prepared sandbox: ```bash cd src && ./gradlew :app-bootstrap:notificationSlackReadiness \ --console=plain cd src && ./gradlew :app-bootstrap:notificationSesReadiness \ --console=plain # Separate human-approved destructive sandbox drill only: cd src && ./gradlew :app-bootstrap:notificationSesDlqReadiness \ --console=plain ``` - [ ] If these cannot run, record the exact blocker and keep the affected card `NOT_QUALIFIED`; do not mark this Task complete. - [ ] Acceptance claim: exact provider connectivity and callback/topology smoke evidence only; final operational R2 eligibility is decided by Task 20 aggregation. **Rollback checkpoint:** readiness tests create external sandbox side effects. Use dedicated probe destinations and retention cleanup; never run against arbitrary production recipients. ### Task 20: Produce local qualification evidence and aggregate production readiness **Owner:** cross-leaf verification harness, aggregated by `app-bootstrap` **Implementation depends on:** Task 18 **Final aggregation depends on:** Task 19 provider manifests plus this task's local manifest **Files — create:** - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/NotificationLoadQualificationTest.java` - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/NotificationRotationQualificationTest.java` - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/NotificationRollingRevisionContractTest.java` - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/NotificationProcessCrashRecoveryTest.java` - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/NotificationCrashScenarioMain.java` - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/NotificationAttemptLedgerServer.java` - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/NotificationQualificationOwnershipSetup.java` - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/TestOnlyNotificationWriterEvidenceIssuer.java` - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/TestOnlyNotificationWriterEvidenceIssuerTest.java` - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/NotificationProductionReadinessAggregationTest.java` - `docs/runbooks/notification.md` **Files — modify:** - `src/app-bootstrap/build.gradle` - `src/app-bootstrap/gradle.lockfile` - `src/app-bootstrap/src/test/resources/notification/evidence/notification-evidence-schema-v1.json` - [ ] Define bounded steady/burst/throttle/callback/reconcile profiles and pass/fail thresholds before running them. - [ ] Make local qualification ownership setup stage-discriminated and fail closed: - `PRE_CUTOVER_BRIDGE` reaches canonical ownership only through the same application root-transaction operations as Task 19: exact batch initialization, canonical-only `CUTOVER_WAIT` instances with call/claim 0, route-specific BEGIN with trusted signed complete node inventory, invoking the authenticated bounded terminalizer for expired ACTIVE permits before ACTIVE 0 is claimed, and then exactly one proof arm. `QUIESCENCE` adds signed node-retirement/credential/egress facts, consumer/ledger zero, attestation commit and stale-node provider I/O 0. `HARD_BOUND` Java-reverifies the same retained signed BEGIN inventory, adds only the all-hard-bound registry/evidence revision and safe terminal permits, and forbids attestation. COMPLETE uses the selected exact proof and then proves exact-generation activation. - `FINAL_CLEANUP` uses either an exact validated canonical upgrade fixture or runs V1..V8 to `AWAITING_SIGNED_FRESH_PROVISIONING` and then invokes the explicit `notificationFreshProvisioning` operation with a signed database-birth authorization issued only after the permanent irreversible no-legacy-authority fence is committed. The fresh fixture must finish with discriminator `FRESH_PROVISIONED`, fresh token/provenance present, validated-history digest absent, and retain the birth certificate plus the complete exact Task 9 `notification_fresh_installation_provenance` field set and causal order, including enforcement activation/read-back, cached-session/flow termination, the later post-enforcement zero manifest, provider-ledger entry/open/indeterminate counts 0, and irreversible fence seal/read-back before signature. The upgrade fixture must finish with `UPGRADE_VALIDATED`, the complete retained-history digest present, fresh token and provenance absent. `VerifyRetainedNotificationWriterEvidenceUseCase`, never a bootstrap repository/entity read, verifies the selected arm from the actual `NotificationWriterFinalizationDiscriminatorEntity` and bounded persistence read adapter; a manifest-only enum is not evidence. Unsigned/uncommitted/reversible-fence, provenance-less or ambiguous empty fixtures fail. Its qualification source must compile and run after all initializer/switch/permit/terminalizer/attestation types are deleted and must prove those types/endpoints are absent. Both paths require route-set/generation-set plus the stage-specific ownership evidence digest to match the selected release profile. Direct fence seed SQL outside Flyway, mixed markers, PRE operation calls in FINAL and stage override all fail. - [ ] Keep `TestOnlyNotificationWriterEvidenceIssuer` only in the `notificationQualification` source set. It uses a deterministic local test key and signs the same canonical inventory, quiescence and DB-birth authorization payloads for repeatable tests; its manifests are explicitly `LOCAL_TEST`, have a lower evidence grade and cannot satisfy Task 19 or `notificationProductionReadiness`. Production JAR/config/source sets contain neither this class nor its private key. The final aggregator accepts production ownership evidence only from the Task 19 independent issuer client. - [ ] RED/qualification matrix includes: claim/finalize contention; provider throttle; callback burst; response loss; DB finalize outage; key/credential/template rotation; old/new writer HMAC aliases; rolling worker revisions; park/resume restart; retention/redaction; cancellation/expiry racing wire authorization. - [ ] Add an actual forked-JVM crash harness, distinct from Task 12's deterministic fault injection. The parent owns PostgreSQL plus `NotificationAttemptLedgerServer`; each child reports a durable phase marker and calls `Runtime.halt(91)` at the requested point. A fresh child then runs recovery while the parent asserts journal state and physical request count. Cover claim, reserve, immediately before/after committed `WIRE_AUTHORIZED`, possible provider write, response and finalize; include authorization commit failure and commit-success/result-loss. No in-process exception may be accepted as process-crash evidence. - [ ] At every post-authorization ambiguous point, assert request count is at most the exact card-specific bound, there is no blind retry/fallback, and recovery ends only in an exact terminal fact, provider reconciliation, or explicit `INDETERMINATE`. - [ ] Extend the manifest schema/test for the local qualification and final aggregate rows, then run it GREEN before registering/executing qualification. Every local/provider/DLQ/aggregate row carries the detector-derived immutable `release_stage`, and the aggregator requires all input rows to match its own artifact stage. Model ownership evidence as a closed discriminated union: `PRE.QUIESCENCE` requires only signed BEGIN inventory, signed quiescence/attestation header + trust snapshot, irreversible node/credential/egress facts, ACTIVE 0, consumer/ledger 0 and cutover history; `PRE.HARD_BOUND` requires only signed BEGIN inventory, the all-hard-bound registry/evidence revision, safe terminal permits and ACTIVE 0, and forbids the attestation. `FINAL.FRESH` requires only cleanup/V8 structural evidence, discriminator `FRESH_PROVISIONED` with fresh token and null validated-history digest, signed DB birth certificate, the complete exact Task 9 fresh-provenance field set and enforcement-read-back → cached-session/flow termination → post-enforcement zero manifest (provider-ledger entry/open/indeterminate 0) → irreversible fence seal/read-back → signature causal order, signed fresh provenance/trust snapshot and `INITIALIZE_CANONICAL_FRESH`; `FINAL.UPGRADE` requires only cleanup/V8 structural evidence, discriminator `UPGRADE_VALIDATED` with complete retained-history digest and null fresh token, and forbids fresh provenance. Awaiting, missing, overlapping or cross-stage/arm evidence fails: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationEvidenceManifestTest' \ --tests '*NotificationReleaseStageDetectorTest' \ --console=plain ``` - [ ] First register `notificationQualification` and `notificationProductionReadiness` in `src/app-bootstrap/build.gradle`. Keep long load/rotation/ crash drills out of ordinary `test`/`check`. The first task emits a fresh local qualification manifest; the second is only an aggregator and performs no provider send. - [ ] After source-set/task/dependency registration and before executing either task, regenerate and verify strict lock state: ```bash cd src && ./gradlew :app-bootstrap:resolveAndLockAll \ --write-locks --console=plain cd src && ./gradlew :app-bootstrap:verifyDependencyLocks --console=plain ``` - [ ] Run deterministic local qualification: ```bash cd src && ./gradlew :app-bootstrap:notificationQualification --console=plain ``` - [ ] Run real-provider portions only through Task 19 lanes; never embed live credentials/network in these ordinary tests. - [ ] Aggregate only fresh, schema-valid manifests for the release-selected exact card set: the local qualification manifest plus each required Task 19 provider/callback/DLQ manifest. Compare every frozen axis, source/dependency/artifact/deployed revision and expiry. Missing, stale, extra, mismatched or skipped evidence fails `notificationProductionReadiness` with `NOT_QUALIFIED`; never infer evidence from a passing unit test. - [ ] After both evidence producers have run, execute: ```bash cd src && ./gradlew :app-bootstrap:notificationProductionReadiness \ --console=plain ``` - [ ] Write runbook actions for backlog, indeterminate, bounce/complaint, provider outage, credential/key/template rotation, SNS retry/DLQ and route pause/resume. - [ ] Preserve the explicit no-exactly-once claim and card-specific duplicate risk. - [ ] Require the structural detector to emit `PRE_CUTOVER_BRIDGE` for every Wave F manifest from the exact Task 17 bridge artifact; never hardcode that label in a task or test fixture. Aggregator success makes only that exact bridge artifact eligible for the bounded canary/switch decision; it is not final R2 evidence because Task 21C changes production source, configuration, dependencies, release stage and artifact revision. **Rollback checkpoint:** qualification itself does not authorize production rollout. Rollout remains route-specific and human-controlled. ### Wave F exit gate - [ ] Record the exact local/provider manifest digests, expiry, source/artifact revision and every skipped/not-run lane. Never copy evidence between cards or environments. - [ ] Request independent durability, provider-protocol, ingress-security and operations review. - [ ] Update the LLM Wiki branch-note with Wave F evidence and an explicit derived-document decision. --- ## Wave G — Canonical cutover, legacy removal and final verification ### Task 21: Cut over the canonical graph and remove the R0 legacy path **Owner leaves:** application, notification, persistence-jpa, inbound-web, bootstrap **Depends on:** Tasks 17–20 and route-specific human cutover decision **Delete only after `rg` proves production consumer 0 and canonical tests are GREEN:** - `src/application-core/src/main/java/dev/caskeleton/application/notification/Channel.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/Notification.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationPort.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationPortContractTest.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/NotificationConfig.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/NotificationRoutesSettings.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/core/FailOpenNotificationProvider.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/core/NotificationProvider.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/core/RoutingNotifier.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/email/google/GoogleEmailClient.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/email/google/GoogleEmailNotificationAdapterConfig.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/email/google/GoogleEmailProvider.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/slack/webhook/SlackClient.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/slack/webhook/SlackNotificationAdapterConfig.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/slack/webhook/SlackWebhookProvider.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/core/NotificationAdapterTest.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/core/RoutingNotifierTest.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationCutoverRouteCatalog.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/catalog/NotificationCutoverRouteCatalogTest.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationLegacyWriterPermitCommand.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationLegacyWriterPermitResult.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/TerminalizeExpiredNotificationWriterPermitsCommand.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/TerminalizeExpiredNotificationWriterPermitsResult.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/TerminalizeExpiredNotificationWriterPermitsOperation.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/TerminalizeExpiredNotificationWriterPermitsUseCase.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationWriterCutoverPort.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationWriterRouteSet.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationWriterQuiescenceAttestationPort.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/InitializeNotificationWriterFencesCommand.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/InitializeNotificationWriterFencesResult.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/InitializeNotificationWriterFencesOperation.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/InitializeNotificationWriterFencesUseCase.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationLegacyWriterPermitUseCase.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/RecordNotificationWriterQuiescenceAttestationCommand.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/RecordNotificationWriterQuiescenceAttestationResult.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/RecordNotificationWriterQuiescenceAttestationOperation.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/RecordNotificationWriterQuiescenceAttestationUseCase.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/SwitchNotificationWriterOwnershipCommand.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/SwitchNotificationWriterOwnershipResult.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/SwitchNotificationWriterOwnershipOperation.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/SwitchNotificationWriterOwnershipUseCase.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationLegacyWriterPermitUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/TerminalizeExpiredNotificationWriterPermitsUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/RecordNotificationWriterQuiescenceAttestationUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/InitializeNotificationWriterFencesUseCaseTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/SwitchNotificationWriterOwnershipUseCaseTest.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/PostgreSqlNotificationWriterCutoverAdapter.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/PostgreSqlNotificationWriterQuiescenceAttestationAdapter.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationWriterCutoverIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationWriterQuiescenceAttestationIntegrationTest.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/FencedLegacyNotificationPort.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/FencedLegacyNotificationPortTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationCutoverAuthorizationCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/notification/NotificationWriterOwnershipCommitAckContractTest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipController.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterFenceInitializationRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterFenceInitializationResponse.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterQuiescenceAttestationRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterQuiescenceAttestationResponse.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterPermitTerminalizationRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterPermitTerminalizationResponse.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipResponse.java` - `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipControllerTest.java` **Modify:** - `src/.env` - `src/app-bootstrap/src/test/java/dev/caskeleton/adapter/outbound/OptionalAdapterBeanGatingTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/adapter/outbound/DisabledAdapterSentinelTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/DisabledAdapterArchitectureTest.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationSettings.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationCompositionConfig.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationCompositionValidator.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationWriterStartupMode.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationWriterActivationGate.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationDatabaseRoleSettings.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationDatabaseRoleComposition.java` - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/NotificationDatabaseRoleTopologyValidator.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationZeroResourceTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationInternalTrustContextCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationWriterActivationGateTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationCanonicalWriterFenceSetCompositionTest.java` - `src/app-bootstrap/src/notificationQualification/java/dev/caskeleton/bootstrap/notification/NotificationQualificationOwnershipSetup.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/OptionalAdapterConditionalExecutionContractTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/support/conditional/EnabledIfEmailNotificationConfigured.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/support/conditional/EnabledIfSlackNotificationConfigured.java` - `src/app-bootstrap/src/main/resources/application.yml` - `src/app-bootstrap/build.gradle` - `src/app-bootstrap/gradle.lockfile` - `src/sample-portfolio/src/main/resources/application.yml` - `src/adapter/outbound/support/src/test/java/dev/caskeleton/adapter/outbound/support/FailOpenDependencyLoggerTest.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationOperationsSnapshot.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationPortBoundaryTest.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/NotificationOperationsSnapshotUseCaseTest.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationOperationsSnapshotAdapter.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationOperationsSnapshotIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationRouteWriterPermitJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterOperationJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterOperationRouteJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterTransportProofRegistryJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterQuiescenceAttestationJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterInventoryManifestJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterDrainNodeInventoryJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterQuiescenceManifestJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterQuiescenceNodeEvidenceJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterEvidenceTrustSnapshotJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationFreshInstallationProvenanceJpaRepository.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/repository/NotificationWriterFinalizationDiscriminatorJpaRepository.java` - `docs/registries/env-keys.yaml` - `docs/registries/secrets-classification.yaml` - `src/README.md` - all affected README/CLAUDE files and the deep design implementation-status section **Transitional lifecycle inventory — create in Task 17, qualify in Wave F, then remove after observation:** - `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/notification/FencedLegacyNotificationPort.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/FencedLegacyNotificationPortTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/notification/NotificationWriterOwnershipCommitAckContractTest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipController.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterFenceInitializationRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterFenceInitializationResponse.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterQuiescenceAttestationRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterQuiescenceAttestationResponse.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterPermitTerminalizationRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterPermitTerminalizationResponse.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipRequest.java` - `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipResponse.java` - `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/notification/NotificationWriterOwnershipControllerTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationCutoverAuthorizationCompositionTest.java` **Create for 21C fresh canonical installation:** - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationRetainedWriterEvidenceQuery.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationRetainedWriterEvidenceSnapshot.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationRetainedWriterEvidenceQueryPort.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationRetainedWriterEvidenceVerifierPort.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationRetainedWriterEvidenceVerification.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/VerifyRetainedNotificationWriterEvidenceUseCase.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/VerifyRetainedNotificationWriterEvidenceUseCaseTest.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationFreshProvisioningAuthorization.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/SignedNotificationFreshProvisioningAuthorization.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationFreshProvisioningAuthorizationVerifierPort.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/NotificationFreshProvisioningPort.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/ProvisionFreshNotificationWriterFencesCommand.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/ProvisionFreshNotificationWriterFencesResult.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/ProvisionFreshNotificationWriterFencesOperation.java` - `src/application-core/src/main/java/dev/caskeleton/application/notification/ProvisionFreshNotificationWriterFencesUseCase.java` - `src/application-core/src/test/java/dev/caskeleton/application/notification/ProvisionFreshNotificationWriterFencesUseCaseTest.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provisioning/Ed25519NotificationFreshProvisioningAuthorizationVerifier.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/provisioning/NotificationFreshProvisioningTrustCatalog.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/provisioning/Ed25519NotificationFreshProvisioningAuthorizationVerifierTest.java` - `src/adapter/outbound/notification/src/main/java/dev/caskeleton/adapter/outbound/notification/evidence/Ed25519NotificationRetainedWriterEvidenceVerifier.java` - `src/adapter/outbound/notification/src/test/java/dev/caskeleton/adapter/outbound/notification/evidence/Ed25519NotificationRetainedWriterEvidenceVerifierTest.java` - `src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V8__validate_or_prepare_canonical_notification_writer_fence.sql` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/postgresql/evidence/PostgreSqlNotificationRetainedWriterEvidenceQueryAdapter.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/postgresql/evidence/PostgreSqlNotificationRetainedWriterEvidenceQueryAdapterTest.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/postgresql/provisioning/PostgreSqlNotificationFreshProvisioningAdapter.java` - `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/notification/postgresql/provisioning/PostgreSqlNotificationProvisionerTransactionAdapter.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/postgresql/provisioning/PostgreSqlNotificationFreshProvisioningIntegrationTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/postgresql/provisioning/PostgreSqlNotificationProvisionerTransactionAdapterTest.java` - `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/notification/NotificationCanonicalFenceInitializationMigrationTest.java` - `src/app-bootstrap/src/notificationFreshProvisioning/java/dev/caskeleton/bootstrap/notification/NotificationFreshProvisioningCli.java` - `src/app-bootstrap/src/notificationFreshProvisioning/java/dev/caskeleton/bootstrap/notification/NotificationFreshProvisioningComposition.java` - `src/app-bootstrap/src/notificationFreshProvisioning/java/dev/caskeleton/bootstrap/notification/NotificationFreshProvisioningSettings.java` - `src/app-bootstrap/src/notificationFreshProvisioning/java/dev/caskeleton/bootstrap/notification/NotificationProvisionerDataSourceConfig.java` - `src/app-bootstrap/src/notificationFreshProvisioningTest/java/dev/caskeleton/bootstrap/notification/NotificationFreshProvisioningCliIntegrationTest.java` - `src/app-bootstrap/src/notificationFreshProvisioningTest/java/dev/caskeleton/bootstrap/notification/NotificationFreshProvisioningSettingsTest.java` - `src/app-bootstrap/src/notificationFreshProvisioningTest/java/dev/caskeleton/bootstrap/notification/NotificationProvisionerTransactionCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationRetainedWriterEvidenceCompositionTest.java` - `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/notification/NotificationRetainedWriterEvidenceStartupTest.java` - [ ] Before deletion, run: ```bash rg --hidden -n \ 'NotificationPort|RoutingNotifier|slack-webhook|google-email|app\.notification\.routes|APP_NOTIFICATION_(SLACK|EMAIL)_PROVIDER|APP_NOTIFICATION_SLACK_WEBHOOK_URL' \ src docs/registries -g '!**/build/**' -g '!**/.git/**' ``` #### Task 21A — bridge release, no deletion - [ ] Freeze and test this per-node/per-database truth table. A single process may never contain both legacy and canonical keys; rolling nodes may temporarily use different rows only because the shared database owner/generation makes one side fail closed: | phase/node config | legacy keys | canonical expected state/binding | DB owner | legacy admits | canonical admits | | --- | --- | --- | --- | --- | --- | | 21A before audited initialization | exact legacy-only | `disabled` / none | absent | no | no | | 21A bridge | exact legacy-only | `disabled` / none | `LEGACY@g` | yes, with committed permit | no | | 21B canonical-ready node before switch | absent | `configured` / derived `CUTOVER_WAIT`, exact future set | `LEGACY@g` | no | no | | 21B old bridge node after switch | exact legacy-only | `disabled` / none | route set `CANONICAL@g_final` | no | no | | 21B canonical node after switch | absent | `configured` / exact `g_final` set | route set `CANONICAL@g_final` | no | yes | | 21C cleanup | absent/unknown | canonical-only exact `g_final` set | route set `CANONICAL@g_final` | path absent | yes | | 21C exact-empty after V8 | absent/unknown | `AWAITING_SIGNED_FRESH_PROVISIONING` | absent | path absent | no | | 21C after signed fresh provisioning | absent/unknown | canonical-only reviewed initial set | provenance-bound route set `CANONICAL@initial` | path absent | yes | Rows are evaluated per route. During 21B, the exact route key set may contain a reviewed mix of `LEGACY@target-1`, `DRAINING@target-1` and `CANONICAL@target`; owner-mismatched nodes reject that route without preventing other routes from continuing. Any same-process legacy+canonical combination fails startup. After 21C every legacy key is unknown and fails startup. - [ ] Before bridge admission opens, deploy the exact Wave F-qualified `PRE_CUTOVER_BRIDGE` artifact, which already contains the inactive transitional operator controller and exact least-privilege mapping `notification-operator -> notification:cutover,notification:cutover-terminalize, notification:cutover-attest`; default `admin` inherits none. Its authenticated `INITIALIZE_LEGACY` action derives actor from `AuthenticatedPrincipal` and calls only the method-security-proxied `InitializeNotificationWriterFencesOperation`. A human supplies the reviewed initial predecessor generations (`configured canonical target - 1`) for the server-disclosed exact route set, reason and operation token; the operation derives the ordered route set/digest from the compiled catalog. The root transaction inserts the entire `ACTIVE/LEGACY@predecessor` set only for absent fences + empty control/data-plane journals and atomically freezes the exact current+retiring proof registry, is idempotent for the same token/set/registry, fails on partial/mismatched/nonempty state, and reports success only after physical commit. Direct SQL, sequential per-route or automatic bootstrap initialization is forbidden. - [ ] Re-run the already implemented Task 17 `FencedLegacyNotificationPort` tests without changing the qualified source/artifact. The wrapper calls `NotificationLegacyWriterPermitUseCase` to reject ambient transactions and root-commit a bounded permit before provider I/O, then root-commit release afterward. Acquire commit failure means provider call 0; release failure leaves the lease visible and blocks switch until guarded recovery/expiry. Canonical guard failure rolls back business state and intent append together. - [ ] A permit expiry becomes `EXPIRED_PROVEN` only when the catalog transport profile proves an acquire-committed DB-time absolute wire deadline, network-start refusal after it, connection close/cancellation by it and `wire deadline + finalize margin <= permit expiry` in an integration evidence revision, including acquire-commit→process-pause→expiry→resume call 0. Current R0 is `QUIESCENCE_REQUIRED`, so timeout becomes `TIMED_OUT_UNPROVEN`; stop 21B until a post-BEGIN authenticated attestation verifies a trusted signed manifest over the exact BEGIN-frozen complete old-node inventory, per-node irreversible retirement/credential/egress fencing, consumer inventory/count 0, provider-call-ledger identity/open-count 0 and the server-derived persisted-registry/ TIMED_OUT_UNPROVEN/holder sets. COMPLETE mechanically requires that token even when the set is empty; never infer call completion from TTL or a caller digest/runbook checkbox alone. COMPLETE instead requires the already accepted signed retained header plus exact irreversible deployment-generation/credential/egress facts, ACTIVE 0 and provider-ledger 0. A crashed/release-failed ACTIVE permit is changed only after BEGIN by the authenticated `notification:cutover-terminalize` operation; repeat bounded batches until the read-only snapshot reports ACTIVE 0. Neither snapshot nor COMPLETE performs this mutation. - [ ] Run bridge RED/GREEN: ```bash cd src && ./gradlew :application-core:test \ --tests '*NotificationCanonicalWriterFenceGuardTest' \ --tests '*InitializeNotificationWriterFencesUseCaseTest' \ --tests '*NotificationLegacyWriterPermitUseCaseTest' \ --tests '*TerminalizeExpiredNotificationWriterPermitsUseCaseTest' \ --tests '*RecordNotificationWriterQuiescenceAttestationUseCaseTest' \ --tests '*SwitchNotificationWriterOwnershipUseCaseTest' \ --console=plain cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*NotificationCanonicalWriterFenceIntegrationTest' \ --tests '*NotificationWriterCutoverIntegrationTest' \ --tests '*NotificationWriterQuiescenceAttestationIntegrationTest' \ --tests '*NotificationWriterIrreversibleFenceIntegrationTest' \ --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*FencedLegacyNotificationPortTest' \ --tests '*NotificationCutoverAuthorizationCompositionTest' \ --tests '*NotificationDatabaseRoleCompositionTest' \ --tests '*NotificationFlywayRoleIsolationTest' \ --tests '*MigrationStartupRunnerTest' \ --tests '*RequiredEnvironmentValidatorTest' \ --tests '*FlywayMigrationCompatibilityContractTest' \ --tests '*NotificationWriterOwnershipCommitAckContractTest' \ --console=plain cd src && ./gradlew :adapter:inbound:web:test \ --tests '*NotificationWriterOwnershipControllerTest' --console=plain ``` - [ ] Human gate: after the qualified artifact and operator endpoint are deployed dark, execute the audited `INITIALIZE_LEGACY` action where the fence is absent; then open bridge admission on every old/new node, keep `LEGACY@g`, and prove no pre-bridge node remains before continuing. The initialization cases above must be GREEN before this mutation. Canonical code stays dark; do not delete any legacy code/config in 21A. Any code/config/lock change after Wave F invalidates its manifest and requires all Task 19/20 PRE lanes to rerun before deployment. #### Task 21B — route-specific ownership switch and observation - [ ] Reuse the Task 17/Wave F-qualified authenticated batch initialization, expired-permit terminalization, route ownership and route quiescence-attestation endpoints; do not add or extend production code between PRE qualification and this switch. None is a public path. Existing JWT/method-security enforcement requires `notification:cutover` for initialization and switch, `notification:cutover-terminalize` for terminalization, and `notification:cutover-attest` for attestation. The thin controller maps the reviewed generation map/reason/token only to `InitializeNotificationWriterFencesOperation`; that operation, not request data, supplies the compiled exact route set/digest. The controller validates exact route/action (`BEGIN_DRAIN|COMPLETE_SWITCH|ABORT_DRAIN`), expected generation, reason and operation token before mapping switch actions only to `SwitchNotificationWriterOwnershipOperation`. BEGIN also maps a bounded signed inventory manifest, never a caller-authored node digest; the application verifier derives and freezes the exact server-trusted set. It derives the audited actor from `AuthenticatedPrincipal`, never request data; target/expected owner is not a request field. Direct SQL/repository access and bootstrap handlers are forbidden. COMPLETE on a `QUIESCENCE_REQUIRED` route must carry the exact `quiescenceAttestationToken`; the attestation endpoint maps a signed quiescence manifest, and other actions/profiles reject those fields. - [ ] Reconfirm the exact 21A least-privilege role mapping and the already registered distinct interface-based initializer, terminalizer, attestation and switch targets as method-security proxied Spring beans; the controller has no duplicate permission annotation and cannot obtain the manual internal delegate. `NotificationCutoverAuthorizationCompositionTest` proves proxy creation, authorized initializer/terminalizer/attestation/switch success, admin/missing-role 403, internal no-auth delegates still work, and no final-class/proxy startup failure. Controller tests cover unauthenticated 401, validation, actor spoof rejection and DTO/command mapping only: ```bash cd src && ./gradlew :adapter:inbound:web:test \ --tests '*NotificationWriterOwnershipControllerTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationCutoverAuthorizationCompositionTest' --console=plain ``` - [ ] Re-run the Task 17 app-bootstrap real-PostgreSQL + MockMvc `NotificationWriterOwnershipCommitAckContractTest`. Prove `BEGIN_DRAIN`, `TERMINALIZE_EXPIRED_PERMITS`, `COMPLETE_SWITCH`, `ABORT_DRAIN` and `INITIALIZE_LEGACY` commit failure/rollback never return 2xx; success response is written only after physical root commit; stale generation conflicts; and commit-success/result-loss replay with the same operation token is idempotent. Also prove terminalizer commit/replay/401/403, signed inventory/attestation commit/replay/401/403, omitted/extra node or permit holder and COMPLETE missing/stale/wrong token/digest/profile-set failures before 2xx. Pause an old node with cached credential/client/connection before provider I/O, commit COMPLETE from another transaction, resume the old node and prove provider I/O 0 under revoked deployment generation, credential and egress. The inbound leaf never imports persistence to make this claim: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationWriterOwnershipCommitAckContractTest' --console=plain ``` - [ ] Roll canonical-only config nodes while the shared owner remains `LEGACY@g`; PRE composition derives `CUTOVER_WAIT`. Those nodes are liveness-healthy but readiness reports `CUTOVER_WAIT`, and their admission/claim/provider-call counts remain 0 while bridge nodes may still acquire legacy permits. No node has both config grammars. - [ ] After canonical-ready nodes are liveness-healthy in `CUTOVER_WAIT`, a human first invokes `BEGIN_DRAIN` with an independently issued short-lived manifest of the complete environment/DB/route/artifact old-writer node set. The root transaction verifies its signature, rejects any known permit holder omitted from the inventory, freezes every node row/count/digest, and closes new legacy permit acquisition. Poll active permit count/max expiry through `NotificationOperationsSnapshotUseCase` outside a transaction. The count includes ACTIVE legacy permits for the route across all old/current fence generations and does not ignore a row merely because `expires_at` passed; only an exact terminal CAS removes it from the count. For expired ACTIVE rows, call the authenticated terminalizer in bounded batches. It checks the exact DRAINING generation and persisted registry, records the affected immutable set in the operation journal and commits before 2xx; poll again until ACTIVE 0. For the current `QUIESCENCE_REQUIRED` R0 profile, only after that committed BEGIN and ACTIVE count 0, obtain an independently signed manifest that lists the exact frozen node set, every node's retired/quiesced fact plus deployment-generation tombstone and legacy credential/egress revocation, production consumer inventory/count 0 and provider-call-ledger identity/open-count 0. Call the authenticated quiescence-attestation endpoint with that opaque manifest. Its root transaction verifies the issuer/environment/DB/artifact/generation, derives the catalog-equality-checked persisted profile set and locks/snapshots every-generation `TIMED_OUT_UNPROVEN` `(token,generation,profile,state,rowVersion)` plus distinct permit-holder sets, then requires exact node equality/holder subset and commits the derived evidence. Invoke `COMPLETE_SWITCH` with that exact attestation token; it locks and recomputes the same sets and rejects missing/mismatched evidence, reversible or absent tombstone/credential/egress facts, a changed consumer/provider-ledger identity, nonzero ledger state or any ACTIVE permit. Before either COMPLETE arm, Java reverifies the retained signed BEGIN inventory canonical payload, signature, issuer public verification material, trust snapshot, issued/expires/verified acceptance and header/child semantic equality. QUIESCENCE_REQUIRED additionally reverifies the retained signed attestation bundle. Expiry controls admission of new signed evidence, while accepted irreversible facts remain durable. For a genuinely `HARD_BOUND_PROVEN` profile, every permit must instead be `RELEASED|EXPIRED_PROVEN`, the COMPLETE request forbids an attestation token, and the valid retained signed BEGIN remains mandatory. After the applicable proof, invoke `COMPLETE_SWITCH`; its `inRootWrite` CAS to route-specific `CANONICAL@g_final/ACTIVE` must physically commit before success is reported. Missing durable evidence or a stale-node resume that can reach a provider leaves the route DRAINING and makes the rollout `NOT_QUALIFIED`. `ABORT_DRAIN` is the only rollback operation and emits a new LEGACY generation, so one or more aborts make `g_final` greater than the naïve `g+1`; none of these operations waits or sleeps inside the use case. Any abort makes the prior expected-generation profile and PRE manifests stale. Update `APP_NOTIFICATION_EXPECTED_WRITER_GENERATIONS`, reproduce the resulting generation set through the same audited sandbox operations, and rerun all required Task 19/20 PRE lanes before the next expansion/switch decision. - [ ] On each committed `COMPLETE_SWITCH`, `NotificationWriterActivationGate` opens only the matching route on canonical `CUTOVER_WAIT` nodes after a fresh committed read; its notification readiness becomes ready only when every configured route is at its exact target. Old bridge nodes observe the canonical owner and keep that route closed. No restart/config watcher may infer activation before the database fact. - [ ] Define observation abort thresholds before the switch: any duplicate occurrence, any new unexpected `INDETERMINATE`, oldest-backlog age over the route SLO, receipt lag over the callback SLO, non-zero unplanned DLQ depth, or parked-gate count above the reviewed bound pauses admission and aborts expansion. Do not automatically resend while diagnosing. - [ ] Keep both code paths packaged during the reviewed observation window, but treat a committed `COMPLETE_SWITCH` as forward-only. `ABORT_DRAIN` is valid only while the route is `DRAINING/LEGACY`; there is no CANONICAL→LEGACY CAS or legacy re-enable after COMPLETE. On a post-COMPLETE threshold breach, close canonical admission and workers through the shared gate, inventory accepted/indeterminate/in-flight work, avoid replay and forward-fix. Any future reverse handoff requires a separate canonical-drain/backlog/provider-result protocol, duplicate policy, design approval and provider requalification. #### Task 21C — cleanup release - [ ] Before reserving `V8`, rescan every Flyway location; if occupied, use the next global version and update the plan first. V8 is validation/preparation only and has two closed outcomes: - `UPGRADE_VALIDATED`: V8 establishes the Task 9 singleton discriminator in this exact state with a server-canonical digest of the complete validated retained history, null fresh token and absent signed fresh provenance. V8 inserts no fence or cutover history and preserves the existing canonical fence/history/evidence rows byte-for-byte. The persisted route key set exactly matches the reviewed set; every fence is `ACTIVE/CANONICAL@g_final`, its latest pointer resolves to a matching committed `COMPLETE_SWITCH`, and the complete legacy initialization→switch history passes the structural validation below. A route with no complete history fails even when notification data happens to be empty. - `AWAITING_SIGNED_FRESH_PROVISIONING`: only an exact-empty V1..V8 notification data/control/fence/history/evidence inventory may reach this state. V8 inserts no canonical fence, provenance or initialization operation and writes a discriminator having neither fresh token nor validated-history digest. Any nonempty inventory with a missing, partial or noncanonical fence set fails; an awaiting marker on existing history is tampering. Application admission, claim, worker and provider resources remain dark. Fresh initialization is an explicit post-migration operation, never Flyway lifecycle work. Register the opt-in `:app-bootstrap:notificationFreshProvisioning` Gradle task backed by `NotificationFreshProvisioningCli`; `NotificationFreshProvisioningSettings` binds only the narrow authorization/public-key/provisioner refs listed below, and the CLI composes `ProvisionFreshNotificationWriterFencesOperation` and the PostgreSQL implementation only in the dedicated source set and is not attached to `test`, `check`, application startup or Flyway. Register a separate `notificationFreshProvisioningTest` source set/task for its CLI integration tests; that test task also stays out of ordinary `test`/`check`. The independent infrastructure issuer signs a domain-separated `notification-fresh-provisioning-v1` canonical payload binding authorization nonce/token, environment, a database birth certificate with DB-system/database/schema identity and birth token/revision/digest/`committedAt`, final source/artifact digest, exact canonical route/generation set, and every exact Task 9 `notification_fresh_installation_provenance` field. The issuer control plane must first commit/read back the irreversible enforcement revision and all deployment-generation, credential-issuance/revocation, DB-ingress and provider-egress deny facts; then terminate cached legacy DB sessions and provider connections/flows; then observe the causally later post-enforcement workload/business-consumer/node/session/flow zero manifest and provider ledger settled cut with entry/open/indeterminate counts 0; then seal-commit/read back the permanent irreversible fence; and only then sign. The canonical payload retains the exact enforcement/fence canonical payloads, revisions, digests and activation/commit/read-back times, post-enforcement manifest payload/digest/revision/time, ledger cut revision/time, credential and denial-policy digests/booleans, `legacy_database_session_inventory_digest`, `legacy_database_session_open_count=0`, `legacy_database_session_termination_evidence_digest`, `legacy_provider_connection_flow_inventory_digest`, `legacy_provider_connection_flow_open_count=0`, `legacy_provider_connection_flow_termination_evidence_digest`, and both established-flow-block facts. Every source evidence binds the same fence token and enforcement revision. The independent issuer refuses pre-enforcement zero, cross-revision composition, sign-before-seal, unsigned, uncommitted or reversible evidence. The retained provenance stores those canonical payload bytes, Ed25519 signature, bounded issuer public-key SPKI, key ID/digest, closed trust snapshot and issued/expires/server-verified times. The signed profile pins `allowedClockSkew` and `acceptanceMargin`; Java admits it only when `issuedAt - allowedClockSkew <= serverVerifiedAt <= expiresAt - acceptanceMargin`, verifies the stored payload/signature/SPKI, and requires the historical-key digest to remain allowed and non-revoked in the current closed catalog. Unknown/duplicate fields, noncanonical encoding, algorithm/key downgrade, wrong identity, nonzero authority, missing/reversible fence, stale birth certificate or token mismatch fail. `ProvisionFreshNotificationWriterFencesUseCase` rejects an ambient transaction and owns the orchestration inside `TransactionPort.inRootWrite`. The dedicated provisioning composition creates a provisioner-only `DataSource`, `PlatformTransactionManager` and persistence-owned `PostgreSqlNotificationProvisionerTransactionAdapter` implementing `TransactionPort`; none is a normal runtime/Flyway bean and the adapter proves its connection has `current_user=notification_provisioner`. App-bootstrap owns only the dedicated data-source/ transaction-manager composition and does not implement the transaction adapter. `NotificationFreshProvisioningPort` exposes exactly two methods: `snapshotAndReadLock(...)` and `applyVerifiedProvisioning(...)`. In one physical provisioner connection/transaction the use case (a) calls `snapshotAndReadLock` for the exact awaiting/empty inventory, server DB clock and DB identity, (b) passes that authoritative snapshot and signed bytes to `NotificationFreshProvisioningAuthorizationVerifierPort`, then (c) sends only the verified facts to `applyVerifiedProvisioning`. The two calls may not use a second connection, nested transaction, autocommit or a runtime/migrator transaction manager. `PostgreSqlNotificationFreshProvisioningAdapter`, under `.postgresql.provisioning`, implements only those structural lock/snapshot/CAS/insert operations; it never calls the verifier or owns application policy. V8 exposes exactly two migrator-owned `SECURITY DEFINER` functions, `notification_fresh_provisioning_snapshot_and_lock(...)` and `notification_fresh_provisioning_apply(...)`, one for each port method. The apply function proves the first function ran in this same physical transaction by checking its transaction-local lock/snapshot proof, rechecks that those discriminator/inventory locks are still held, and compares the first-stage DB-computed snapshot digest. It then rechecks `AWAITING_SIGNED_FRESH_PROVISIONING`, store emptiness, current DB identity and the canonical payload semantic digest from authoritative inputs, obtains a fresh `clock_timestamp()` and rechecks the signed issued/expires/skew/acceptance window. A direct apply without the exact first-stage lock ownership and snapshot digest fails before DML. It persists that apply-time DB value as `server_verified_at`; a Java pause that crosses expiry yields mutation 0 even if the first snapshot was valid. The root transaction inserts immutable signed provenance, exactly one `INITIALIZE_CANONICAL_FRESH` header with the complete ordered route children, every `ACTIVE/CANONICAL@initial` fence and CASes the discriminator to `FRESH_PROVISIONED` with its fresh token and null validated-history digest. The operation returns its result only after physical commit. A committed same-token/same-payload replay is a read-only result-recovery branch: both functions lock/recompute the retained discriminator/provenance/init/fence equality, Java reverifies the stored signature/trust/semantic facts and original `server_verified_at` acceptance, and apply returns the persisted result without DML. It does not apply current wall-clock expiry to that already accepted irreversible fact. A missing result, different token/input/identity/digest, partial state or attempted new mutation must take the fresh-time new-mutation branch or fail closed. Thus commit-success/result-loss is recovered without a second initialization. Startup remains dark until this transaction is durably committed, then a fresh startup re-verifies the retained signed provenance in Java and may enter `REQUIRE_CANONICAL`. SQL enforces only structural shape/count/digest/FK/immutability; it is never the Ed25519 authority. Authorization expiry after accepted provisioning does not reverse the committed provenance or fences. Before V8, external DB-admin/IaC has already created the exact three-role set from Task 9. V8 validates that set, ownership and grants and never executes `CREATE ROLE` or creates the principal running itself. `notification_migrator` owns the schema, Flyway history, migration objects and the two narrowly scoped provisioning functions. `notification_runtime` is a nonowner. `notification_provisioner` has no table DML, sequence, ownership, role-membership or DDL privilege and receives only `EXECUTE` on those exact two functions. Both `SECURITY DEFINER` functions are migrator-owned, use `SET search_path = pg_catalog` plus fully qualified objects, contain no dynamic SQL or caller-selected object/action, check the exact caller role and lock their state/token/payload rows; `PUBLIC` and runtime execute are revoked. PRE runtime retains only its separately enumerated transitional `EXECUTE` grants until FINAL, when V8/cleanup revokes them. Tests reject owner substitution, inherited membership, search-path shadowing and direct DML/sequence access. Apply becomes state-closed after success; retained same-token replay is the only allowed result path. On upgrade, replay the full causal journal by the operation and attestation values allocated from the same DB sequence after the global/route fence lock: `INITIALIZE_LEGACY -> (BEGIN -> TERMINALIZE* -> ABORT)* -> BEGIN -> TERMINALIZE* -> COMPLETE`. Each non-init child has the exact drain-BEGIN FK; expected/result owner/state/generation matches the closed matrix, terminalizer is unchanged DRAINING, and COMPLETE is the last mutation. CANONICAL→BEGIN/TERMINALIZE/ABORT/ second-COMPLETE, cross-table sequence collision/reversal, missing predecessor or replay/fence/latest-pointer mismatch fails. `recorded_at`, `terminalized_at` and `observed_at` must be post-lock `clock_timestamp()` values but are only sanity evidence; sequence/FK is the causal SSOT. Reject orphan header/child, empty child set, action mismatch and any server-canonical `route_set_digest`/`request_input_digest` recomputation mismatch, including terminalizer batch bound. Both INITIALIZE actions have the exact reviewed all-route child set; every non-init header has exactly one child. The retained immutable transport-proof registry, not the deleted PRE catalog or caller digest, is upgrade proof authority: exact route/current+retiring profile set, one ACTIVE profile, shared route digest and initialization-child FK; every frozen permit, attestation and COMPLETE profile/proof/evidence/digest matches it. ACTIVE permits fail. All-HARD_BOUND history requires the retained signed BEGIN header, an exact all-hard-bound registry/evidence revision and permits only `RELEASED|EXPIRED_PROVEN`; ACTIVE permit 0 is mandatory and quiescence attestation/children are forbidden. Every terminal permit composite-references its exact unchanged-DRAINING terminalizer child, satisfies `expires_at <= terminalized_at` and `BEGIN.operation_sequence < terminalizer.operation_sequence < first-closing.operation_sequence`, and participates in the recomputed affected count/set digest. QUIESCENCE_REQUIRED history additionally requires: BEGIN's signed complete-node count/set/manifest digest exactly equals every immutable node row; all distinct permit holders are included; attestation references that BEGIN and exact registry/permit/holder/node sets plus independently signed per-node irreversible retirement/credential/egress fencing, consumer inventory/count 0 and provider-call-ledger identity/open-count 0. The retained inventory and quiescence/attestation headers and trust snapshots structurally bind the canonical signed payload/signature, issuer public-key SPKI/key digest, issued/expires/verified fields and exact environment/DB/artifact/inventory/ledger identities. Causal validity is `BEGIN.operation_sequence < attestation.attestation_sequence < COMPLETE.operation_sequence`, not a pre-commit timestamp. Current cleanup time is irrelevant. A superseded/unselected attestation is allowed only with the same BEGIN and sequence before the first closing ABORT/COMPLETE. Missing/extra/forged inventory, holder, attestation, irreversible fact, provider-ledger snapshot or sequence fails. V8 performs structural validation only; before activation Java re-verifies every stored payload/signature/SPKI, applies the historical-key allow/non-revoked decision from the current closed catalog and rejects a malformed acceptance window. Evidence accepted inside its pinned window remains durable after expiry. Preserve fence, registry, permit, node inventory, per-node quiescence evidence, signed headers/trust snapshots, attestation, provenance and operation rows byte-for-byte. - [ ] Implement the retained FINAL read path as a separate bounded application contract. `NotificationRetainedWriterEvidenceQuery` supplies only expected environment/DB/artifact, canonical route/generation set and reviewed per-collection bounds; `NotificationRetainedWriterEvidenceQueryPort` returns one immutable `NotificationRetainedWriterEvidenceSnapshot`. `PostgreSqlNotificationRetainedWriterEvidenceQueryAdapter` performs read-only, deterministic ordered reads and fails on truncation, extra rows, duplicate identities or any bound breach. For `FRESH_PROVISIONED` it reads the discriminator, fresh provenance, exact `INITIALIZE_CANONICAL_FRESH` header/children and exact canonical fence set. For `UPGRADE_VALIDATED` it reads the discriminator plus the complete operation/route-child, transport-proof registry, permit, BEGIN inventory/header/child, quiescence/attestation header/child, trust-snapshot and exact canonical-fence snapshot needed to recompute the stored validated-history digest. Awaiting or an arm overlap is never activation evidence. `VerifyRetainedNotificationWriterEvidenceUseCase` is the only application-facing verifier. It passes each bounded canonical payload/signature/SPKI/trust bundle to `NotificationRetainedWriterEvidenceVerifierPort`; `Ed25519NotificationRetainedWriterEvidenceVerifier` owns canonical decoding, Ed25519 verification, issuer SPKI/digest, pinned trust snapshot and current closed-catalog allow/non-revoked checks, and returns only bounded typed verified facts. The use case owns branch policy: it compares those facts for exact semantic equality with the relational projection, then checks discriminator XOR/token/history digest and exact catalog/generation equality. Neither adapter makes branch/activation decisions, and the inventory-only `NotificationWriterInventoryEvidenceVerifierPort` is not reused as if it covered fresh provenance or the full FINAL snapshot. App-bootstrap startup and readiness inject only this use case; composition/startup tests forbid direct injection/import of a retained repository, JPA entity, `EntityManager` or JDBC type and prove both exact arms activate only after successful Java verification. - [ ] Register the following provisioning-source-set-only inputs; none is a `NotificationSettings` field or normal runtime/Flyway input: | env key | purpose | classification | | --- | --- | --- | | `APP_NOTIFICATION_FRESH_PROVISIONING_AUTHORIZATION_REF` | externally issued signed DB-birth + committed irreversible-fence authorization bytes | sensitive reference | | `APP_NOTIFICATION_FRESH_PROVISIONING_ISSUER_PUBLIC_KEY_REFS` | bounded verifier public-key SPKI refs matching the closed trust catalog | public verification-material refs | | `APP_NOTIFICATION_DB_EXPECTED_PROVISIONER_ROLE` | exact callable principal | fixed `notification_provisioner` | | `APP_NOTIFICATION_DB_PROVISIONER_USERNAME_REF` | dedicated provisioner username reference | sensitive reference | | `APP_NOTIFICATION_DB_PROVISIONER_PASSWORD_REF` | dedicated provisioner password reference | sensitive reference | The CLI never logs or retains credentials in settings/application records/entities and wipes only adapter-facing mutable copies on close. Do not claim end-to-end erasure: current `SecretSource`/`EnvironmentSecretSource` and JDBC username/password APIs necessarily create unavoidable short-lived immutable Java `String` values. Minimize copies and lifetime, run the CLI as a dedicated forked process, assert `current_user`, immediately close the provisioner `DataSource` and process after commit/failure, prohibit heap dumps for that process, rotate short-TTL credentials, and prefer workload identity or certificate authentication where the JDBC/runtime platform supports it. Tests cover no logging/exception/settings/entity retention, minimum bridge copies, immediate close and mutable-copy wipe; residual JVM `String` exposure is explicitly recorded rather than represented as wiped. The CLI carries no signing capability: a production private key is forbidden in source, artifact, environment registry, test resource and material source. Supplying these refs to an upgrade/runtime process, omitting one for explicit fresh provisioning, sharing provisioner/runtime/migrator credentials or mismatching the pinned SPKI digest fails closed. Retained provenance keeps the canonical payload, signature and public verification/trust snapshot needed for future Java re-verification, never a private key or database password. - [ ] After registering `notificationFreshProvisioning` and `notificationFreshProvisioningTest` with their exact application/outbound/persistence classpaths, make the production provisioning configurations inherit only app-bootstrap's already-governed main `implementation`/`runtimeOnly` configurations and its test configurations inherit only the governed test configurations. Add an exact project-edge assertion that scans every provisioning production/test configuration and rejects any direct or inherited project dependency outside app-bootstrap's registry allowlist. Register a non-mutating `notificationFreshProvisioningCheck` aggregate that compiles the production provisioning source set with the repository's Java/Error Prone/static-analysis policy and runs the project-edge and dependency/lock assertions, but does not run `notificationFreshProvisioningTest`, invoke the provisioning CLI or reach a database. The separately invoked `notificationFreshProvisioningTest` owns the Testcontainers/CLI integration cases. Keep both outside ordinary `check` while requiring both explicitly in Task 22. Regenerate and verify the app-bootstrap lock before compiling or invoking either task: ```bash cd src && ./gradlew :app-bootstrap:resolveAndLockAll \ --write-locks --console=plain cd src && ./gradlew :app-bootstrap:verifyDependencyLocks --console=plain cd src && ./gradlew :app-bootstrap:notificationFreshProvisioningCheck --console=plain ``` - [ ] RED/GREEN real-PostgreSQL migration cases: a clean V1..V8 two-route database ends only in `AWAITING_SIGNED_FRESH_PROVISIONING`, with fence/provenance/initialization rows 0 and application provider resources 0. Invoking `notificationFreshProvisioning` with valid authorization then creates the complete reviewed canonical initial set, exact retained provenance and one `INITIALIZE_CANONICAL_FRESH` batch in one transaction and transitions the discriminator to `FRESH_PROVISIONED` with fresh token present/validated-history digest absent. A valid canonical upgrade ends only in `UPGRADE_VALIDATED` with complete-history digest present/fresh token and provenance absent; every XOR/state violation is rejected. V8 rejects every nonempty/partial notification inventory without an exact canonical fence set; the operation rejects manually inserted/spoofed/expired/wrong-DB/wrong-environment/wrong-artifact/wrong-route-set/ nonzero-node, nonzero-consumer or nonzero provider-ledger entry/open/indeterminate-count authorization, missing/wrong DB birth certificate, every missing/mutated Task 9 enforcement, post-enforcement zero-manifest, fence-read-back, credential, denial-policy, session or connection-flow axis, and awaiting-state mutation. Prove the external issuer refuses pre-enforcement zero, cross-revision evidence and signing before the exact enforcement → termination → post-enforcement settled-zero → permanent fence seal/read-back chain completes. Pause/inject an old deployment generation, cached legacy DB/provider credential, established DB session/provider connection and legacy egress path before provisioning; require both signed inventory digests/open counts 0, termination evidences and ingress/egress established-flow block facts, mutate each exact field independently as a RED case, then prove legacy DB I/O and provider I/O are both 0 before and after provisioning and after resume. Prove boundary failures around both allowed clock skew and acceptance margin. Assert both port calls/functions use the same physical connection/root transaction; pause Java verification beyond expiry and require apply mutation 0, invoke the apply function directly after expiry and require mutation 0, and cover rollback after each write stage plus commit-success/result-loss recovery. A failed provisioning transaction leaves awaiting state unchanged; same-token/same-payload retry is exact and mismatch retry fails rather than reclassifying a partial database or blessing an upgrade. A committed same-token replay after current authorization expiry reverifies the stored signature/trust/semantic facts and original `server_verified_at`, returns the stored result with DML 0 and never refreshes acceptance time. Direct apply without the first-stage transaction-local lock/snapshot proof fails with mutation 0. Also invoke both functions directly outside the CLI/use-case seam with forged signature or forged typed facts: regardless of any structurally written row, FINAL startup/readiness Java verification must detect payload/SPKI/trust/semantic inequality and remain dark/ `NOT_QUALIFIED`. A valid signed authorization may reach direct apply only when every apply-time DB-clock/fence/state/identity condition still holds, and provider I/O remains 0 until a subsequent startup successfully re-verifies the retained evidence. A multi-route upgrade with different valid `g_final` values preserves every fence and proof-registry/permit/node-inventory/quiescence-node-evidence/signed-header/trust-snapshot/ attestation/ operation-history row byte-for-byte. Include a valid multi-profile QUIESCENCE_REQUIRED upgrade whose attestation is expired now but was admitted inside its signed acceptance window and whose irreversible facts remain valid. Include successful re-attestation after a permit row-version change, and BEGIN→attestation→ABORT→new BEGIN→new attestation→COMPLETE history; the superseded rows remain byte-identical. BEGIN→ABORT→forged late attestation and BEGIN→COMPLETE→forged late unselected attestation fail without mutation. A pre-BEGIN-expired old-generation ACTIVE permit terminalized after BEGIN then completed passes V8; expiry is not required to follow BEGIN. Add rogue histories that end canonical but contain COMPLETE→BEGIN→ABORT→BEGIN→COMPLETE, terminalizer outside DRAINING, sequence duplicate/cross-table-collision/reversal or a missing drain-BEGIN predecessor; each fails without mutation. Start a terminalizer transaction before BEGIN and release its fence wait after BEGIN; post-lock `clock_timestamp()` and later sequence must make this valid, proving transaction-start time is not used. Add orphan header/child, empty/extra/partial init child set, zero/two-child non-init header, route-set digest, request-input digest and stored batch-bound corruption fixtures. Missing/extra/partial route sets, LEGACY/DRAINING/mixed owners, latest-COMPLETE mismatch, ACTIVE old-generation permit, stale-at-COMPLETE/missing/wrong token or set/profile digest, unknown/omitted/extra registry profile, tampered proof class/evidence revision/registry digest, either invalid proof-class/state pairing, orphan/wrong-action/wrong-route/wrong-set terminalization, terminalized-before-expiry/after-close, omitted/extra inventory node or permit holder, unsigned/wrong-issuer/wrong-identity quiescence manifest, missing/extra/wrong-action/ wrong-attestation/wrong-profile, malformed stored payload/signature/SPKI/trust snapshot, revoked historical key, invalid issued/expires/verified window, nonzero facts, BEGIN-less attestation-only nonempty store and every other nonempty-without-fence fixture all fail without mutation. An app-bootstrap composition test also proves exact equality between the compiled canonical route catalog, `APP_NOTIFICATION_EXPECTED_WRITER_GENERATIONS` and the persisted fence set. The retained-reader tests cover bounded/truncated/extra child snapshots, both valid arms, every semantic payload-to-row mismatch, malformed/revoked SPKI/trust state, discriminator XOR and digest mismatch, and prove bootstrap has no direct repository/entity access: ```bash cd src && ./gradlew :application-core:test \ --tests '*ProvisionFreshNotificationWriterFencesUseCaseTest' \ --tests '*VerifyRetainedNotificationWriterEvidenceUseCaseTest' --console=plain cd src && ./gradlew :adapter:outbound:notification:test \ --tests '*Ed25519NotificationFreshProvisioningAuthorizationVerifierTest' \ --tests '*Ed25519NotificationRetainedWriterEvidenceVerifierTest' \ --console=plain cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*PostgreSqlNotificationFreshProvisioningIntegrationTest' \ --tests '*PostgreSqlNotificationProvisionerTransactionAdapterTest' \ --tests '*PostgreSqlNotificationRetainedWriterEvidenceQueryAdapterTest' \ --tests '*NotificationRetainedEvidenceNoSaveArchitectureTest' \ --tests '*NotificationWriterFinalizationDiscriminatorIntegrationTest' \ --tests '*NotificationCanonicalFenceInitializationMigrationTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationCanonicalWriterFenceSetCompositionTest' \ --tests '*NotificationRetainedWriterEvidenceCompositionTest' \ --tests '*NotificationRetainedWriterEvidenceStartupTest' \ --tests '*NotificationDatabaseRoleCompositionTest' \ --tests '*NotificationFlywayRoleIsolationTest' \ --console=plain cd src && ./gradlew :app-bootstrap:notificationFreshProvisioningTest \ --tests '*NotificationFreshProvisioningCliIntegrationTest' \ --tests '*NotificationFreshProvisioningSettingsTest' \ --tests '*NotificationProvisionerTransactionCompositionTest' --console=plain cd src && ./gradlew :app-bootstrap:notificationFreshProvisioningCheck --console=plain ``` - [ ] Only after the human observation gate passes, delete the listed legacy and bridge files. Remove legacy selector rows from `src/.env`, both application YAML files and `env-keys.yaml`; remove `APP_NOTIFICATION_SLACK_WEBHOOK_URL` from the secret registry only after the legacy webhook code and all references are gone. Update disabled/sentinel/conditional/fail-open tests and root documentation in the same RED/GREEN step. - [ ] Delete the transitional operator controller/DTO/test, proxied switch operation/use case, legacy permit, expired-permit terminalizer and quiescence-attestation write port/operation/use-case/write adapter, bridge and the `notification:cutover`/`notification:cutover-terminalize`/ `notification:cutover-attest` operator mappings in the same cleanup release. Delete `NotificationWriterCutoverPort`, `PostgreSqlNotificationWriterCutoverAdapter`, `PostgreSqlNotificationWriterQuiescenceAttestationAdapter` and their integration tests; remove ACTIVE/TIMED_OUT_UNPROVEN permit, attestation and max-expiry fields/read paths from `NotificationOperationsSnapshot`, its application test and persistence adapter/test. Keep the additive transport-proof-registry/permit/operation/BEGIN-node-inventory/ quiescence-node-evidence/signed inventory/quiescence/attestation headers, trust snapshots, fresh-provenance/finalization-discriminator table history, all corresponding entities and repositories as retained projections, and `PostgreSqlNotificationRetainedWriterEvidenceQueryAdapter`; these are not deletion targets. Keep the Task 9 never-Java-written repositories on their original marker-only contract. After deleting write adapters, narrow every remaining retained transitional Spring Data repository to the marker `Repository` plus only explicitly bounded read methods; none may extend `CrudRepository`/`JpaRepository` or declare `save`, `saveAll`, `delete` or `flush`. `NotificationRetainedEvidenceNoSaveArchitectureTest` scans this contract and the read adapter for a write surface. Keep the separate `NotificationCanonicalWriterFencePort`/adapter/guard, retained inventory and FINAL Ed25519 evidence verifiers and explicit fresh-provisioning operation, but expose no runtime operation capable of selecting `LEGACY` or mutating retained audit rows; configured startup requires the persisted owner to be the exact canonical generation. Remove every bridge/permit/terminalizer/proxied-switch/controller-support bean from `NotificationCompositionConfig` and update composition, zero-resource and trust-context tests so no deleted transitional type remains reachable. FINAL grants revoke generic `INSERT|UPDATE|DELETE` on retained cutover audit/control tables, cutover-sequence use and every PRE transitional function `EXECUTE` from runtime/PUBLIC; runtime retains only exact SELECT and fence read-lock access there. This does not revoke the exact DML/SELECT needed for the active intent/claim/finalize/receipt operational journal. The provisioner retains only the exact two state-closed provisioning-function `EXECUTE` grants, never direct audit-table DML or sequence access. App-bootstrap may reach retained evidence only through `VerifyRetainedNotificationWriterEvidenceUseCase`; repository/entity injection or direct JDBC is a composition-test failure. - [ ] Rewrite `NotificationQualificationOwnershipSetup` in the cleanup source tree as a FINAL-only migration/validated-fence-set setup. It must not import, reflectively load or string-reference any deleted initializer/switch/permit/terminalizer/attestation/controller type. Run the compile/test after deletion so Task 22 can requalify the final artifact without a PRE-only setup path. Its fresh lane must run V8 to `AWAITING_SIGNED_FRESH_PROVISIONING` and invoke only the signed `notificationFreshProvisioning` Gradle/CLI path, ending only in `FRESH_PROVISIONED` with the retained DB birth certificate/no-legacy-authority fence and fresh-token XOR arm; its upgrade lane may only validate an existing complete canonical history and end in `UPGRADE_VALIDATED` with the retained-history-digest XOR arm. - [ ] Delete every `CUTOVER_WAIT` and PRE production branch from `NotificationWriterStartupMode`, settings, composition, activation gate, YAML/env registry and tests. Retain only the closed FINAL modes `AWAITING_SIGNED_FRESH_PROVISIONING|REQUIRE_CANONICAL`. `NotificationWriterActivationGate` keeps admission/claim/call 0 while awaiting; in `REQUIRE_CANONICAL`, absent, predecessor, DRAINING, partial, extra or wrong-generation fences fail startup and admit/claim/call 0. Both fresh and upgrade startup re-read the retained signed headers/provenance through `VerifyRetainedNotificationWriterEvidenceUseCase`, require the matching actual finalization-discriminator arm, and reverify payload/signature/SPKI, semantic equality plus current historical-key status before activation. Awaiting, unknown, discriminator/provenance/history overlap or direct-SQL forged state stays dark. Detector fixtures may retain the explicit marker only under their exact verification allowlist; remove `APP_NOTIFICATION_CUTOVER_ATTESTATION_TTL` and rename the PRE public verifier input to `APP_NOTIFICATION_RETAINED_EVIDENCE_ISSUER_PUBLIC_KEY_REFS`. Retain that read-only runtime trust input and the provisioning-source-set-only authorization/public-key/provisioner references in their narrow registry paths. The compiled runtime configuration contains no CUTOVER_WAIT, terminalizer or attestation-write support, but does retain read-only signed evidence verification. - [ ] Delete PRE-only `NotificationCutoverRouteCatalog` and transitional `NotificationWriterRouteSet` with all legacy alias/transport-proof consumers. Final composition retains only `NotificationCanonicalRouteCatalog -> NotificationCanonicalWriterRouteSet` plus the exact runtime target-generation map; its key set must match V8 and persisted fences. Detector tests require executable legacy alias/proof classes/beans/config 0 while allowing immutable V7 proof-registry history and V8 SQL validation references. - [ ] Remove `spring-web` direct dependency if no new notification code uses it, regenerate only affected locks, and re-run dependency verification. - [ ] Never backfill old generic outbox/log events or automatically resend accepted/indeterminate legacy occurrences. - [ ] Verify the 21C canonical-only artifact: ```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*Notification*' \ --tests '*DisabledAdapterSentinelTest' \ --tests '*OptionalAdapter*' \ --console=plain cd src && ./gradlew :application-core:check \ :adapter:outbound:notification:check \ --console=plain cd src && ./gradlew verifyEnvKeys \ verifyDependencyLocks \ verifyCleanArchitectureDependencies \ --console=plain ``` - [ ] Acceptance: exactly one canonical activation graph remains; legacy names have no production source/config consumer and occur only in the reviewed release-stage detector/test allowlist; optional webhook/Gmail/SMTP can return only as separately designed exact cards. The cleanup artifact remains `NOT_QUALIFIED` until Task 22 final-artifact requalification. **Rollback checkpoint:** after legacy deletion, do not restore it for rows already accepted or indeterminate. Pause admission, retain schema/revisions and forward-fix unless an exact route inventory proves zero duplicate risk. ### Task 22: Synchronize truth, run full gates, independent review and LLM Wiki capture **Owner:** repository-wide verification/documentation **Depends on:** all preceding tasks required by the selected release scope **Files — modify:** - `docs/superpowers/specs/2026-07-28-notification-production-capability-design.md` - `docs/superpowers/plans/2026-07-28-notification-production-capability.md` - `src/application-core/README.md` - `src/application-core/CLAUDE.md` - `src/adapter/outbound/notification/README.md` - `src/adapter/outbound/notification/CLAUDE.md` - `src/adapter/outbound/persistence-jpa/README.md` - `src/adapter/outbound/persistence-jpa/CLAUDE.md` - `src/adapter/inbound/web/README.md` - `src/adapter/inbound/web/CLAUDE.md` - `src/app-bootstrap/README.md` - `src/app-bootstrap/CLAUDE.md` - `docs/runbooks/notification.md` - `docs/runbooks/notification-database-role-bootstrap.md` - LLM Wiki branch-note and only genuinely derived raw documents - [ ] Update implementation status from actual source/test/evidence only. Keep every unexecuted provider/load/rotation lane visibly `NOT_QUALIFIED`. - [ ] Run focused owner gates first: ```bash cd src && ./gradlew :application-core:check \ :adapter:outbound:notification:check \ :adapter:outbound:persistence-jpa:check \ :adapter:inbound:web:check \ :app-bootstrap:check \ --console=plain ``` - [ ] Run the exact database-role/provisioning gates and require exactly three roles: `notification_migrator` owner/Flyway-only, `notification_runtime` nonowner with exact FINAL operational grants and every PRE transitional `EXECUTE` revoked, and `notification_provisioner` execute-only on exactly `notification_fresh_provisioning_snapshot_and_lock` and `notification_fresh_provisioning_apply`. Any additional notification-scoped owner/member/grantee, direct provisioner DML/sequence privilege or PUBLIC/runtime execution of either function fails. Require the external DB-admin/IaC bootstrap revision/digest and prove V7/V8 contain no role creation: ```bash cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*NotificationDatabaseRoleIsolationIntegrationTest' \ --tests '*PostgreSqlNotificationFreshProvisioningIntegrationTest' \ --tests '*PostgreSqlNotificationProvisionerTransactionAdapterTest' \ --tests '*PostgreSqlNotificationRetainedWriterEvidenceQueryAdapterTest' \ --tests '*NotificationRetainedEvidenceNoSaveArchitectureTest' \ --tests '*NotificationWriterFinalizationDiscriminatorIntegrationTest' \ --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationDatabaseRoleCompositionTest' \ --tests '*NotificationFlywayRoleIsolationTest' \ --tests '*NotificationRetainedWriterEvidenceCompositionTest' \ --tests '*NotificationRetainedWriterEvidenceStartupTest' \ --console=plain cd src && ./gradlew :app-bootstrap:notificationFreshProvisioningTest \ --tests '*NotificationFreshProvisioningCliIntegrationTest' \ --tests '*NotificationFreshProvisioningSettingsTest' \ --tests '*NotificationProvisionerTransactionCompositionTest' --console=plain cd src && ./gradlew :app-bootstrap:notificationFreshProvisioningCheck --console=plain ``` - [ ] Run repository gates: ```bash cd src && ./gradlew test --console=plain cd src && ./gradlew check --console=plain cd src && ./gradlew verifyDependencyLocks --console=plain cd src && ./gradlew verifyCleanArchitectureDependencies --console=plain cd src && ./gradlew verifyPublicPathSnapshot --console=plain cd src && ./gradlew verifyEnvKeys --console=plain ``` - [ ] Run documentation/source hygiene: ```bash git diff --check if rg --hidden -n 'slack-webhook|google-email|\bNotificationPort\b|\bRoutingNotifier\b|APP_NOTIFICATION_SLACK_WEBHOOK_URL' \ src docs/registries \ -g '**/src/main/**/*.java' \ -g '**/src/main/**/*.yml' \ -g '**/src/main/**/*.yaml' \ -g '**/src/main/**/*.properties' \ -g '**/build.gradle' -g '.env' -g '*.yml' -g '*.yaml' \ -g '!**/build/**' -g '!**/.git/**'; then echo 'legacy notification production consumer/config references remain' >&2 exit 1 fi cd src && ./gradlew :app-bootstrap:test \ --tests '*NotificationReleaseStageLegacyMarkerAllowlistTest' \ --tests '*NotificationReleaseStageDetectorTest' \ --console=plain ``` - [ ] Perform independent reviews for: Clean Architecture/module boundary; transaction/durability/concurrency; callback/security; provider protocol; privacy/operations. Completion requires blocker 0 and high 0. Any unresolved blocker/high finding keeps this task incomplete and the card `NOT_QUALIFIED`. - [ ] Freeze the final cleanup source tree, docs and dependency locks. Because commits are human-only, a human creates the candidate commit; CI builds the final artifact from that exact commit and records its artifact digest. The agent never stages, commits or pushes. - [ ] Treat every Task 19/20 `PRE_CUTOVER_BRIDGE` manifest as stale for this final artifact. Deploy the exact cleanup artifact to the isolated sandbox. Before any lane, require `NotificationReleaseStageDetector` to derive `FINAL_CLEANUP` from the built artifact; a caller-supplied stage, PRE, mixed or unknown inventory fails. Re-run both FINAL ownership setup variants first. `FRESH` runs V1..V8 to `AWAITING_SIGNED_FRESH_PROVISIONING`, proves every runtime side effect 0, commits the external permanent no-legacy-authority fence, and only then obtains a production issuer-signed database-birth authorization and invokes the explicit task below. It must finish with actual discriminator `FRESH_PROVISIONED`, fresh token present and validated-history digest absent; wrong/absent authorization or an unsigned/uncommitted/reversible fence remains dark. Its retained evidence includes the DB birth certificate and the complete exact Task 9 fresh-provenance field set and causal order: enforcement commit/read-back, cached DB-session/provider-flow termination, causally later post-enforcement zero manifest with provider-ledger entry/open/indeterminate counts 0, then irreversible fence seal/read-back before signature. A paused old generation with cached credential/session/connection proves legacy DB I/O and provider I/O are both 0 before and after provisioning. `UPGRADE` validates complete canonical history byte-for-byte without fresh provenance, never invokes the task and must finish with actual discriminator `UPGRADE_VALIDATED`, complete-history digest present and fresh token absent. Both paths call `VerifyRetainedNotificationWriterEvidenceUseCase` to reverify retained payload/signature/SPKI, semantic equality and closed-catalog historical-key status in Java before activation; bootstrap reads no repository/entity directly: ```bash # FRESH ownership setup only; never run on UPGRADE: cd src && ./gradlew :app-bootstrap:notificationFreshProvisioning --console=plain cd src && ./gradlew :app-bootstrap:notificationQualification --console=plain cd src && ./gradlew :app-bootstrap:notificationSlackReadiness --console=plain cd src && ./gradlew :app-bootstrap:notificationSesReadiness --console=plain # Required only when the release-selected SES card requires a fresh scheduled/manual DLQ drill: cd src && ./gradlew :app-bootstrap:notificationSesDlqReadiness --console=plain cd src && ./gradlew :app-bootstrap:notificationProductionReadiness --console=plain ``` The final aggregator compares the human candidate source commit, production-source/dependency digest, deployed artifact digest, detector-derived `FINAL_CLEANUP` stage and every exact card/topology axis. It must reject all bridge manifests, any hardcoded/caller-overridden stage and any mismatched/expired final lane. Every selected FINAL lane must carry only the cleanup migration/V8 structural digest and exactly one closed arm: `FINAL.FRESH` carries discriminator `FRESH_PROVISIONED` with fresh-token XOR, the signed DB birth certificate, the complete exact Task 9 fresh-provenance field set and enforcement-read-back → cached-session/flow termination → post-enforcement settled-zero manifest → irreversible fence seal/read-back → signature causal order, signed fresh provenance/trust snapshot and `INITIALIZE_CANONICAL_FRESH`; `FINAL.UPGRADE` carries discriminator `UPGRADE_VALIDATED` with complete retained-history-digest XOR and forbids fresh provenance. `AWAITING_SIGNED_FRESH_PROVISIONING`, a PRE ownership arm or reference to a deleted cutover type fails aggregation. If a scheduled destructive DLQ manifest is still fresh under the exact artifact/card policy it may be selected; otherwise the human-approved DLQ lane is rerun. No code/config/docs change is allowed after this build without invalidating the final evidence and repeating this gate. - [ ] Only successful final-artifact aggregation permits the selected exact card to be called an operational R2 candidate. If sandbox credentials/topology or the human candidate commit are not available, record the blocker and keep the card `NOT_QUALIFIED`; do not reuse pre-cutover evidence. - [ ] Before Wiki capture, read the configured vault's `AGENTS.md`, `CLAUDE.md`, and relevant `rules/`, `.agents/`, `.claude/`, `.codex/` instructions. Then update exactly `/home/donghyeon/workspace/ai-tool/llm-wiki-private/raw/branch-notes/.md` with implementation, files, decisions, commands/results/failures, evidence grade and remaining risks. - [ ] Add `raw/errors`, `raw/interviews`, `raw/blog-topics` only when honestly derived; link each child upward and the branch-note cluster back to each child. Otherwise record “없음” explicitly. - [ ] Do not mark this task complete if the canonical Wiki path is unavailable; record the precise capture blocker. - [ ] A repository-wide gate that cannot run because of environment/tooling/authorization is a recorded blocker, not a pass. State the exact command and residual risk; do not claim Task 22 complete. - [ ] Final handoff must list changed files, core behavior, exact verification results, not-run/failed commands, Wiki capture, evidence level and follow-up risks. **Rollback checkpoint:** documentation must describe the deployed/evidenced truth, not the preferred rollback story. Do not rewrite evidence after a failed rollout. --- ## 7. Task dependency graph ```text 1 └─ 2 └─ 3 └─ 4 ├─ 5 ─ 6 ─ 7 ────────────────┐ └─ 8 ─ 9 ─ 10 ─ 11 ─ 12 ────┤ └─ 13 ─┬─ 14 └─ 15 ─ 16 ─ 17 ─ 18 ─ 19 ─ 20 ─ 21 ─ 22 ``` Tasks within one owner leaf may be implemented sequentially by one agent. Parallel work is safe only after the shared application contracts are GREEN: - Task 13 must finish before Task 15 because both change the notification leaf dependency declaration and lock; Task 14 may run in parallel with Task 15 only after Task 13's card/client seam is stable. - Task 16 may start after the normalized receipt contract and SES event profile are frozen. - Persistence Tasks 9–11 must not run concurrently against the same migration/store files. - Composition Task 17 starts only after provider, persistence and ingress descriptors are stable. - Legacy deletion Task 21 is never parallelized with provider/composition work. - Task 20 local harness implementation may begin after Task 18, but its final aggregator cannot run until Task 19 has emitted every selected provider manifest. ## 8. Minimum implementation completion matrix | requirement | proving task | | --- | --- | | canonical binding/expected state, legacy conflict | 5, 17, 21 | | feature-specific semantic pattern | 3–4 | | frozen intent/template/route | 3, 5–6, 10 | | best-effort/durable separation | 2–4, 14 | | same-DB journal | 9–12 | | token/version claim/finalize | 11–12 | | encrypted PII/HMAC/retention | 8–10, 18 | | indeterminate/reconcile/fallback safety | 4, 11–12, 14–16 | | Slack exact cards | 13–14, 19–20 | | SES/SNS exact card | 15–16, 19–20 | | zero-resource disabled | 17–18 | | bounded deadlines/concurrency/amplification | 4–7, 11–15, 18–20 | | health/metrics/traces/runbook | 18, 20 | | independent review | 22 | | LLM Wiki capture | 22 | `minimum implementation R2`는 표의 required task가 실제로 GREEN이고 selected exact card의 no-skip evidence가 fresh할 때만 사용할 수 있다. 일부 task만 끝났다면 해당 evidence row의 좁은 표현만 사용한다. ## 9. Final non-negotiable assertions - provider accepted와 recipient delivered/read는 다르다. - `WIRE_AUTHORIZED` 이후 unknown은 definite-not-sent가 아니다. - Slack/SES send API에는 이 설계가 의존할 exactly-once idempotency가 없다. - shared admission park는 durable state이며 process-local circuit breaker가 아니다. - config는 application mode/admission/business policy를 선택하지 않는다. - callback authenticity와 same-transaction receipt commit 전에는 SNS success ACK를 반환하지 않는다. - disabled와 configured-but-broken은 다른 상태다. - fake/loopback/PostgreSQL evidence만으로 real provider card를 R2라고 부르지 않는다. - agent는 stage/commit/amend/push하지 않는다.