Files
tech-log-backend/docs/superpowers/plans/2026-07-28-messaging-first-r2-polling-producer.md
T

187 KiB
Raw Blame History

Messaging First R2 Polling Producer Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking. Behavior changes also require superpowers:test-driven-development; completion claims require superpowers:verification-before-completion and an independent superpowers:requesting-code-review.

Goal: Build one production-reference Messaging path from a typed integration event through a same-transaction PostgreSQL polling outbox to an acknowledgement-aware Spring Kafka producer, with an authenticated disposition control and exact R2 evidence.

Architecture: application-core owns provider-neutral event/publication/disposition semantics; adapter:outbound:messaging owns deterministic JSON/schema compilation and Kafka; PostgreSQL persistence owns event/delivery/audit rows and token/lease CAS; inbound web owns only operator HTTP mapping; bootstrap composes the exact tuple, readiness and schedulers. The first path is polling-only and keeps consumer, inbox, DLT, replay and CDC disabled.

Tech Stack: Java 21, Spring Boot 4.0.0, Spring Kafka 4.0 through the Boot BOM, Jackson 3, com.networknt:json-schema-validator:3.0.2, PostgreSQL, Flyway, JPA, Gradle, JUnit 5, AssertJ, Testcontainers Kafka/PostgreSQL, Micrometer.


Repository commit policy는 모든 플랫폼에서 human-only다. 이 계획에는 git add, git commit, git amend, git push 단계가 없다. 구현자는 작업 결과와 검증 증거만 전달하고 candidate commit은 사람이 만든다.

1. Exact selected tuple and non-guarantees

첫 구현과 qualification 대상은 다음 tuple 하나다.

messaging-outbox-publish.v1
  + kafka-spring-acknowledged-idempotent.v1
  + postgresql-polling-outbox.v2
  + postgresql-per-record-jit-claim.v1
  + json-schema-envelope.v1
  + external-topic-validated.v1
  + kafka-sasl-ssl-scram-sha-512.v1
  + kafka-compression-none.v1
  + per-key-normal-path-sequence-detectable.v1
  + same-postgresql-transaction-resource.v1
  + authenticated-internal-web-disposition.v1

이 계획이 완료돼도 다음은 주장하지 않는다.

  • broker와 PostgreSQL 사이 exactly-once;
  • consumer effect의 deduplication 또는 inbox 보장;
  • global FIFO, failure/rotation/requeue 뒤 strict FIFO;
  • single-node Kafka test만으로 multi-broker RF/min ISR 내구성;
  • CDC-ready, DLT-ready, replay-ready;
  • local plaintext profile을 production security profile로 승격;
  • ACKNOWLEDGED가 consumer 처리 또는 business effect 완료를 뜻함.

2. Target flow and fixed decisions

feature mapper
  -> IntegrationEventDraft<typed record>
  -> IntegrationEventEncoderPort
  -> ValidatedIntegrationEvent(exact UTF-8 bytes + hashes)
  -> TransactionPort.inWrite(
       business state
       + immutable outbox_event
       + CURRENT/READY outbox_delivery
     )

Outbox relay invocation
  -> acquire one bounded local admission permit
  -> Tx B: one-row JIT claim + token/DB-time lease + ATTEMPT_ADMITTED
  -> no DB transaction: Kafka send + future ACK wait
  -> Tx C: outcome observation + valid-lease/token CAS state transition
  -> release permit

late Kafka callback
  -> bounded payload-free observation source
  -> application drain
  -> DB commit
  -> source ACK

authenticated internal endpoint
  -> inbound DTO/principal mapping
  -> ApplyOutboxDispositionUseCase
  -> permission/policy
  -> PostgreSQL CAS + immutable audit

고정 결정:

  1. src/config/architecture/modules.json이 leaf와 production project edge의 유일한 SSOT다. first R2 production 구현에는 새 leaf나 project edge가 필요 없다.
  2. sample-portfolio -> adapter-outbound-messaging edge는 standalone sample을 실제 ACTIVE producer로 바꾸는 별도 승인 작업 전에는 추가하지 않는다.
  3. application/domain/shared Java API에는 Kafka, Jackson, JSON validator, Spring, JPA 타입을 노출하지 않는다.
  4. physical topic은 application contract가 아니라 outbound destination binding이다.
  5. exact UTF-8 BYTEA가 wire authority다. retry에서 payload를 다시 직렬화하지 않는다.
  6. outbox_event는 immutable event, outbox_delivery는 mutable delivery control이다.
  7. claim/outcome/renew는 opaque token, owner, CURRENT generation, expected version, claim_until > database_now를 모두 확인한다.
  8. local admission을 확보한 뒤 한 record만 JIT claim한다. initial profile의 admitted record upper bound는 1이다.
  9. broker call은 DB transaction 밖에서 수행한다.
  10. ACKNOWLEDGED, ACKNOWLEDGED_MISMATCH, REJECTED, INDETERMINATE는 exhaustive outcome이다.
  11. acceptance certainty와 retry disposition은 독립 축이다.
  12. deadline 뒤 late ACK는 기존 outcome/state를 뒤집지 않고 append-only observation만 제안한다.
  13. operator requeue는 기존 row를 READY로 덮지 않고 이전 authority를 supersede한 뒤 새 delivery generation을 만든다.
  14. requeue generation deadline은 min(generation.created_at + maximumAutomaticPublicationAge, event.created_at + sameEventRequeueHorizon)이다.
  15. P2는 additive schema/control-plane candidate일 뿐이다. LEGACY_POLLING authority는 P3의 fenced cutover까지 유지한다.
  16. live non-empty V3 database는 base template migration이 자동 backfill하지 않는다. 별도 deployment migration design과 승인이 없으면 중단한다.
  17. production ACTIVE는 SASL_SSL + SCRAM-SHA-512, external topic attestation, least-privilege evidence가 없으면 실패한다.
  18. disabled state는 contract/destination/client/AdminClient/thread/scheduler/network/secret refresh가 모두 0이다.

3. Scope boundary and owner leaves

책임 owner leaf Gradle path production edge 변경
typed event, outcome, relay, late drain, disposition policy application-core :application-core 없음
generic envelope schema resource shared-contract :shared-contract 없음
JSON/schema/catalog/Kafka/provider lifecycle adapter-outbound-messaging :adapter:outbound:messaging 외부 dependency만 추가
event/delivery/journal/epoch/CAS adapter-outbound-persistence-jpa :adapter:outbound:persistence-jpa 없음
authenticated operator HTTP mapping adapter-inbound-web :adapter:inbound:web 없음
tuple composition/readiness/schedulers/real-service lane app-bootstrap :app-bootstrap test dependency만 추가
sample payload/schema/contribution fixture sample-portfolio :sample-portfolio messaging edge 없음

금지:

  • controller가 repository, JPA entity 또는 outbound adapter를 직접 사용;
  • persistence mapper/query에 retry, disposition 또는 topic 정책을 넣음;
  • messaging adapter가 sample, persistence 또는 inbound-web를 의존;
  • bootstrap settings/configuration에 business event mapping이나 retry policy를 구현;
  • shared-contract에 WorkLog schema 또는 provider setting을 넣음;
  • 현재 dirty worktree의 Fileserver/Object Storage/Notification 변경을 되돌리거나 덮어씀.

4. Evidence ladder and promotion rule

evidence 허용되는 주장
pure/application unit provider-neutral contract와 state policy가 정의됨
schema/catalog/codec unit/property local deterministic document와 closed catalog가 정의됨
adapter fake gateway outcome mapping과 lifecycle protocol이 정의됨
real PostgreSQL same-store append, constraint, claim/CAS/audit protocol의 local evidence
single-node real Kafka actual ACK metadata와 client/provider behavior evidence
TLS/SASL/ACL lane exact security principal/profile evidence
multi-broker RF/min ISR lane selected topology failure/recovery evidence
fault/capacity/rotation/cutover drill exact tuple의 operational R2 evidence

낮은 row를 높은 row, 다른 broker version, cluster, topic, principal 또는 security profile로 일반화하지 않는다. 모든 selected scenario가 fresh evidence artifact에 PASS일 때만 machine card를 release-eligible로 바꾼다. 그 전에는 최대 implemented-candidate다.

5. Execution rules

  1. 모든 checkbox는 구현 시작 시 [ ]다.
  2. task 시작 전 git status --short, 현재 migration 목록, owner leaf의 가장 가까운 CLAUDE.md, modules.json edge를 다시 확인한다.
  3. behavior task는 RED test 작성 → 같은 focused command에서 예상 원인으로 실패 확인 → 최소 구현 → 같은 command GREEN 순서를 지킨다.
  4. RED가 처음부터 통과하면 기존 coverage인지 잘못된 test인지 조사하고 assertion을 강화한다.
  5. compilation drift, 외부 환경 또는 unrelated dirty change가 RED 원인이면 구현하지 말고 원인을 분리한다.
  6. 한 shared worktree에서 여러 Gradle process를 동시에 실행하지 않는다. 이전에 같은 output directory를 병렬 갱신해 compile collision이 발생했으므로 Gradle command는 한 invocation으로 묶거나 순차 실행한다.
  7. 실제 service가 필요한 release task는 service/credential/image/no-test 문제를 SKIP/PASS로 바꾸지 않는다. local ordinary test와 release qualification task를 분리한다.
  8. migration은 expand-first, forward-only다. 기존 V3__outbox_event.sql은 수정하지 않는다.
  9. 새 provider와 v2 scheduler는 authority cutover 전까지 dark/disabled다.
  10. P2에서 v2 claim/send/authority switch를 활성화하지 않는다.
  11. event/payload/schema/hash/credential/raw header는 log, metric tag, evidence artifact에 넣지 않는다.
  12. 각 Wave exit에서 설계 §0 ledger, card maturity, plan checkbox, LLM Wiki branch-note를 실제 증거에 맞춰 갱신한다.
  13. plan surface 밖의 파일이나 타입이 필요하면 조용히 확장하지 않고 이 문서를 먼저 갱신한다.
  14. 설계와 plan이 충돌하면 구현으로 타협하지 않고 상세 설계를 먼저 수정·재승인한다.

6. Stop conditions

다음 중 하나라도 확인되면 해당 task 또는 Wave를 중단한다.

  • application/domain에 framework, Kafka, JSON, persistence 타입을 넣어야만 진행 가능;
  • module edge가 modules.json에 허용되지 않음;
  • V7이 실행 시점에 이미 다른 migration으로 사용됐거나 다른 승인 계획이 먼저 구현됨. 모든 Flyway location을 다시 스캔해 다음 global version으로 이 계획과 tests를 먼저 갱신한다;
  • V3 legacy row가 live non-empty인데 empty/drained evidence나 별도 live migration 승인이 없음;
  • business repository와 outbox append가 같은 transaction resource임을 증명할 수 없음;
  • Kafka producer retry/timeout/effective setting을 finite하게 고정할 수 없음;
  • adopted JSON Schema validator가 Draft 2020-12, offline registry, format assertion 또는 required adversarial bound를 만족하지 못함;
  • topic/RF/min ISR/ACL을 runtime와 provisioning evidence의 명시된 source로 attest할 수 없음;
  • legacy relay와 v2 relay를 동시에 active하게 해야만 rollout 가능;
  • active writer/relay/producer를 fence하지 않은 채 authority switch가 필요;
  • DB에 INDETERMINATE/HOLD를 기록하지 못한 상태로 producer generation을 강제 전환해야 함;
  • operator endpoint가 active unexpired claim을 무시하거나 raw status update를 해야 함;
  • real Kafka/security/multi-broker evidence 없이 R2/production-ready 표현이 필요.

7. Batch graph and checkpoints

Wave A / P0
  truth + machine registry skeleton
    -> Wave B / P1
       application contract + schema + catalog + codec
         -> Wave C / P2
            additive DB v2 + append + claim/CAS + policy
              -> Wave D / P3
                 Spring Kafka + endpoint + composition + cutover
                   -> Wave E / P4
                      real-service/security/fault/release evidence
Wave exit claim rollback posture
A current R0 truth와 planned cards가 정확함 behavior 변화 없음
B local event contract/codec candidate Kafka/outbox R2 아님
C polling v2 schema/control-plane candidate LEGACY_POLLING 유지, v2 scheduler off
D ACK-aware polling path/cutover candidate pause admission, preserve DB schema/backlog/epoch
E exact evidence가 통과한 tuple만 release-eligible destructive schema downgrade 금지

Wave A — P0 truth freeze and execution scaffolding

Task 1: Freeze current R0 behavior and approved design truth

Owner: documentation + existing application/messaging/persistence/bootstrap tests Depends on: approved detailed design Behavior change: none

Files — modify:

  • docs/superpowers/specs/2026-07-28-messaging-production-capability-design.md
  • src/application-core/src/test/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCaseTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxAppendTransactionalContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxRowLifecycleContractTest.java
  • src/adapter/outbound/messaging/README.md

Files — create:

  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfigTest.java

  • Capture current branch, git status --short, design digest, registry edges, dependency graph, migrations and current test counts in the LLM Wiki branch-note.

  • Add characterization assertions for: broker blank → disabled sentinels; broker selected + missing sender → startup failure; broker ID mismatch → failure; sender normal return → legacy PUBLISHED; sender exception → FAILED/DEAD; ACK-to-mark failure → IN_FLIGHT and possible duplicate; same-transaction append rollback; timestamp FIFO limitation.

  • Keep tests explicitly named legacy or characterization; do not rename current void-return success to broker ACK.

  • Run the baseline sequentially:

    ```bash
    cd src && ./gradlew :application-core:test \
      --tests '*PublishPendingOutboxEventsUseCaseTest' --console=plain
    cd src && ./gradlew :adapter:outbound:messaging:test \
      --tests '*MessagingConfigTest' \
      --tests '*OutboxMessagePublishAdapterTest' --console=plain
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*OutboxAppendTransactionalContractTest' \
      --tests '*OutboxRowLifecycleContractTest' --console=plain
    ```
    
  • Expected GREEN: current behavior is reproducible without source behavior changes.

  • Update §0 to P0=CHARACTERIZED, leaving P1P4 NOT_STARTED.

  • Acceptance: no “Kafka ACK”, “dedupe safe” or “R2” claim is introduced.

Rollback checkpoint: characterization tests and truth documentation are independently reversible; legacy code remains the executable baseline through the P3 cutover window.

Task 2: Add fail-closed Messaging card registries and verification task skeleton

Owner: repository configuration + app-bootstrap contract tests Depends on: Task 1

Files — create:

  • src/config/messaging/readiness-cards.yaml
  • src/config/messaging/profile-compatibility.yaml
  • src/config/messaging/release-profile-assertions.yaml
  • src/config/messaging/evidence/build-evidence-manifest-v1.schema.json
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingCapabilityRegistryContractTest.java

Files — modify:

  • src/build.gradle

  • src/app-bootstrap/build.gradle

  • src/app-bootstrap/README.md

  • Write a RED contract test that requires exactly the P0P4 first tuple rows, closed maturity values not-implemented|implemented-candidate|release-eligible, wildcard-free compatibility, unique IDs, declared evidence tasks/scenarios/runbooks and no consumer/CDC/EOS/schema-registry rows.

  • Seed all first tuple rows with maturity: not-implemented and empty evidence fingerprint; do not predeclare future extension-ledger names.

  • Define one checked-in, payload-free build-evidence schema with required source/artifact digest, producer task, scenario IDs/counts, command/timestamp, profile/catalog/schema/settings hashes, failures, skips and unsupported claims. Every later local manifest validates against this schema before release aggregation; a producer may add a stricter offline schema but may not weaken these common fields.

  • Define these task names in src/build.gradle without making them pass yet:

    ```text
    verifyMessagingContracts
    verifyMessagingJsonSchemaV1
    verifyMessagingPollingOutboxR2
    verifyMessagingKafkaProducerR2
    verifyMessagingSecurityR2
    verifyMessagingReleaseProfile
    verifyMessagingTargetBindingPreflight
    verifyMessagingTargetBinding
    verifyMessagingDeploymentCutover
    verifyMessagingCleanupTargetBinding
    verifyMessagingFinalR2Profile
    ```
    
    Each task must fail on no matching tests. Release aggregation must reject missing, skipped,
    stale, wrong-source or mismatched-profile evidence.
    
  • Verify RED then GREEN for registry structure only:

    ```bash
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*MessagingCapabilityRegistryContractTest' --console=plain
    ```
    
  • Verify the existing dependency boundary remains unchanged:

    ```bash
    cd src && ./gradlew verifyCleanArchitectureDependencies --console=plain
    ```
    
  • Acceptance: registry truth exists, every card is not-implemented, and no verification task can falsely claim R2.

Rollback checkpoint: registry/task scaffolding creates no runtime resources and may be removed without data migration.


Wave B — P1 typed contract, schema, catalog and deterministic bytes

Task 3: Add framework-free integration-event contract and contribution SPI

Owner: application-core (:application-core) Depends on: Task 2

Files — create under src/application-core/src/main/java/dev/caskeleton/application/messaging/:

  • contract/IntegrationPayload.java
  • contract/IntegrationEventContractContribution.java
  • contract/ContractId.java
  • contract/LogicalDestinationId.java
  • contract/SchemaResourceId.java
  • contract/Sha256.java
  • contract/ContractDescriptor.java
  • event/EventId.java
  • event/AggregateIdentity.java
  • event/AggregateOrder.java
  • event/IntegrationEventDraft.java
  • event/ValidatedIntegrationEvent.java
  • event/IntegrationEventEncoderPort.java

Files — create under src/application-core/src/test/java/dev/caskeleton/application/messaging/:

  • contract/IntegrationEventContractContributionTest.java
  • event/IntegrationEventDraftTest.java
  • event/ValidatedIntegrationEventTest.java

Files — modify:

  • src/application-core/README.md

  • src/application-core/CLAUDE.md

  • Write RED value tests for canonical ASCII event ID grammar, closed contract/destination IDs, positive versions, nonblank canonical tenant scope, aggregate sequence/index bounds, immutable/defensively-copied bytes and fixed SHA-256 length.

  • Write RED SPI tests requiring exact final Java record payload type, canonical component order, schema resource/hash and provider-neutral descriptor. Reject Map, raw JSON string/tree, assignable-type discovery and Java class-name routing.

  • Implement one-public-type-per-file framework-free records/interfaces. The boundary shape is:

    ```java
    public interface IntegrationPayload {}
    
    public interface IntegrationEventContractContribution<P extends IntegrationPayload> {
      ContractId contractId();
      int payloadVersion();
      Class<P> exactPayloadRecordType();
      List<String> canonicalRecordComponentOrder();
      SchemaResourceId payloadSchemaResource();
      Sha256 payloadSchemaHash();
      ContractDescriptor descriptor();
    }
    
    public interface IntegrationEventEncoderPort {
      ValidatedIntegrationEvent encode(IntegrationEventDraft<?> draft);
    }
    ```
    
  • Keep physical topic, Kafka record metadata, JSON node, serializer, schema validator and publication epoch out of these types.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :application-core:test \
      --tests 'dev.caskeleton.application.messaging.*' --console=plain
    ```
    
  • Run application purity:

    ```bash
    cd src && ./gradlew verifyApplicationCoreDependencyPurity \
      verifyOneTypePerFile --console=plain
    ```
    
  • Acceptance claim: framework-free semantic contract R1 only; no schema/Kafka/persistence R2.

Rollback checkpoint: these are additive contracts; legacy NewOutboxEvent remains until the validated append path is green.

Task 4: Check in the generic envelope schema and sample payload contract

Owner leaves: shared-contract (:shared-contract), sample-portfolio (:sample-portfolio) Depends on: Task 3

Files — create:

  • src/shared-contract/src/main/resources/contracts/messaging/envelope/v1.schema.json
  • src/shared-contract/src/main/resources/contracts/messaging/envelope/v1.schema.sha256
  • src/shared-contract/src/test/java/dev/caskeleton/shared/contract/messaging/MessagingEnvelopeSchemaResourceTest.java
  • src/sample-portfolio/src/main/resources/contracts/messaging/portfolio.worklog.reserved/v1.schema.json
  • src/sample-portfolio/src/main/resources/contracts/messaging/portfolio.worklog.reserved/v1.schema.sha256
  • src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/WorkLogReservedPayload.java
  • src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/WorkLogReservedContractContribution.java
  • src/sample-portfolio/src/test/resources/contracts/messaging/portfolio.worklog.reserved/v1.valid.json
  • src/sample-portfolio/src/test/resources/contracts/messaging/portfolio.worklog.reserved/v1.invalid-unknown-field.json
  • src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/event/WorkLogReservedContractContributionTest.java

Files — modify:

  • src/shared-contract/README.md

  • src/shared-contract/CLAUDE.md

  • src/sample-portfolio/README.md

  • src/sample-portfolio/CLAUDE.md

  • Write RED resource tests requiring UTF-8, explicit Draft 2020-12 $schema, immutable absolute $id, checked-in lowercase SHA-256, unevaluatedProperties: false, bounded strings/arrays, required/null/missing policy and no HTTP/file remote $ref.

  • Define envelope v1 with the exact fields frozen by design:

    ```json
    {
      "envelopeVersion": 1,
      "eventId": "event-1",
      "contractId": "portfolio.worklog.reserved",
      "payloadVersion": 1,
      "logicalDestination": "portfolio-domain-events",
      "aggregate": {
        "type": "worklog",
        "id": "worklog-42",
        "sequence": 17,
        "eventIndex": 0
      },
      "occurredAt": "2026-07-28T05:10:30.123Z",
      "correlationId": "corr-1",
      "contentType": "application/json",
      "payload": {
        "workLogId": "worklog-42"
      }
    }
    ```
    
  • Keep the envelope business-free and keep the WorkLog payload schema only in sample.

  • Make WorkLogReservedPayload a typed immutable record implementing IntegrationPayload; contribution provides type/order/resource/hash only and no JSON mapper.

  • Do not add sample-portfolio -> adapter-outbound-messaging to modules.json or Gradle.

  • Verify RED then GREEN sequentially:

    ```bash
    cd src && ./gradlew :shared-contract:test \
      --tests '*MessagingEnvelopeSchemaResourceTest' --console=plain
    cd src && ./gradlew :sample-portfolio:test \
      --tests '*WorkLogReservedContractContributionTest' --console=plain
    ```
    
  • Acceptance claim: checked-in generic/sample contract artifacts exist; validator compatibility is still unproven until Task 6.

Rollback checkpoint: resources and sample contribution are additive; no production runtime discovers or publishes them yet.

Task 5: Compile the closed contract, destination and exact capability binding

Owner: adapter:outbound:messaging (:adapter:outbound:messaging) Depends on: Tasks 34

Files — create under src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/:

  • contract/ContractCatalogCompiler.java
  • contract/CompiledIntegrationEventContract.java
  • contract/ContractCatalogDigest.java
  • destination/DestinationBindingSettings.java
  • destination/DestinationBindingCompiler.java
  • destination/CompiledPublicationBinding.java
  • destination/PartitionKeyV1.java
  • config/MessagingCapabilityCardRegistry.java
  • config/CompiledMessagingDescriptor.java

Files — create under src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/:

  • contract/ContractCatalogCompilerTest.java
  • contract/ContractCatalogDigestTest.java
  • destination/DestinationBindingCompilerTest.java
  • destination/PartitionKeyV1Test.java
  • config/MessagingCapabilityCardRegistryTest.java

Files — modify:

  • src/adapter/outbound/messaging/README.md

  • src/adapter/outbound/messaging/CLAUDE.md

  • Write RED tests for duplicate contract/destination/schema IDs; missing binding; unknown card; final-record exact type; component-order mismatch; code/deployment byte-bound intersection; config attempting to relax ordering/schema/security; legacy + canonical conflict; unsupported future card rejection.

  • Add golden partition-key vectors using the design's domain-separated, length-prefixed SHA-256 input. Assert exactly 64 lowercase hex ASCII characters and tenant-enabled/disabled canonical non-null scope.

  • Compile:

    ```text
    contract descriptor
    + destination descriptor
    + producer/serialization/security/card descriptor
    = immutable CompiledPublicationBinding
    ```
    
    Physical topic and bootstrap servers stay only in the compiled deployment binding.
    
  • Compute stable catalog/settings/schema digests using sorted IDs and length-prefixed bytes; never depend on Map iteration order or toString().

  • Empty catalog + DISABLED must compile to a zero-resource descriptor. ACTIVE + empty catalog must fail before any client/thread is created.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :adapter:outbound:messaging:test \
      --tests '*ContractCatalog*Test' \
      --tests '*DestinationBindingCompilerTest' \
      --tests '*PartitionKeyV1Test' \
      --tests '*MessagingCapabilityCardRegistryTest' --console=plain
    ```
    
  • Acceptance claim: closed local binding compiler R1; no wire bytes or Kafka client yet.

Rollback checkpoint: compiler is not wired into MessagingConfig; legacy selection remains authoritative.

Task 6: Implement the deterministic JSON Schema envelope encoder

Owner: adapter:outbound:messaging (:adapter:outbound:messaging) Depends on: Task 5

Files — create:

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/LocalJsonSchemaRegistry.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/DeterministicEnvelopeWriter.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/JsonSchemaIntegrationEventEncoder.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/EnvelopeAdmissionLimits.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/EnvelopeHashV1.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/envelope/LocalJsonSchemaRegistryTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/envelope/JsonSchemaIntegrationEventEncoderTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/envelope/EnvelopeAdversarialCorpusTest.java
  • src/adapter/outbound/messaging/src/test/resources/contracts/messaging/test.event/v1.schema.json
  • src/adapter/outbound/messaging/src/test/resources/contracts/messaging/test.event/v1.valid.json
  • src/adapter/outbound/messaging/src/test/resources/contracts/messaging/test.event/v1.invalid.json

Files — modify:

  • src/adapter/outbound/messaging/build.gradle

  • src/adapter/outbound/messaging/gradle.lockfile

  • src/build.gradle

  • Write RED tests for Draft 2020-12 meta-schema, checksum mismatch, duplicate $id, unknown dialect/vocabulary, remote/unmapped $ref, cycles beyond the supported depth, pathological regex corpus, format assertion, valid/invalid envelope and payload, required/null/missing, unknown property and unsupported payload version.

  • Write RED parser/admission tests for duplicate JSON key, malformed UTF-8, unpaired surrogate, trailing garbage, depth/string/array/object/number bounds, non-finite number, exact UTF-8 value/key/header bytes and deterministic field/scalar order.

  • Add:

    ```groovy
    implementation 'org.springframework.boot:spring-boot-starter-json'
    implementation('com.networknt:json-schema-validator:3.0.2') {
        exclude group: 'com.fasterxml.jackson.dataformat', module: 'jackson-dataformat-yaml'
    }
    ```
    
    Keep Jackson/schema runtime in the messaging leaf. Regenerate only affected dependency locks
    and review the resolved Jackson 3 graph, license and vulnerability report.
    
  • Configure NetworkNT Draft 2020-12 with format assertions enabled and an exact classpath resource map. After startup compilation, network/file schema resolution is impossible.

  • Make the writer consume only exact registered final record types. Disable polymorphic typing, feature-provided serializers, unknown properties and reflective assignable-type search.

  • Compute:

    ```text
    SHA-256(
      UTF8("ca-skeleton.messaging.envelope.v1") || 0x00
      || u32be(len(exactEnvelopeBytes))
      || exactEnvelopeBytes
    )
    ```
    
    and return defensive copies in `ValidatedIntegrationEvent`.
    
  • Add validator compatibility evidence using the adopted JSON Schema Test Suite/Bowtie corpus; custom contract compatibility still requires repository golden vectors.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :adapter:outbound:messaging:test \
      --tests '*LocalJsonSchemaRegistryTest' \
      --tests '*JsonSchemaIntegrationEventEncoderTest' \
      --tests '*EnvelopeAdversarialCorpusTest' --console=plain
    cd src && ./gradlew verifyMessagingJsonSchemaV1 \
      verifyDependencyLocks --console=plain
    ```
    
  • On GREEN, verifyMessagingJsonSchemaV1 validates and writes this exact payload-free candidate manifest:

    ```text
    src/build/messaging-evidence/contracts-schema/manifest.json
    ```
    
    It conforms to `src/config/messaging/evidence/build-evidence-manifest-v1.schema.json` and binds
    the human/CI-supplied source/artifact digest, schema/catalog hashes, dependency-lock digest,
    exact scenario IDs/counts, command/timestamp, failed=0, skipped=0 and unsupported claims.
    Missing digest input fails the manifest-producing lane; an ordinary focused unit test may
    still run without claiming release evidence.
    
  • Update JSON/schema cards to implemented-candidate only after the exact tests and locks pass.

  • Acceptance claim: deterministic local wire contract candidate; Kafka and durable outbox R2 are still unimplemented.

Rollback checkpoint: encoder/catalog stays unwired from production append; removing it does not change legacy rows.

Wave B exit checkpoint

  • Run:

    ```bash
    cd src && ./gradlew :application-core:check \
      :shared-contract:check \
      :adapter:outbound:messaging:check \
      :sample-portfolio:check \
      verifyMessagingContracts \
      verifyCleanArchitectureDependencies \
      --console=plain
    ```
    
  • Confirm no production leaf imports dev.caskeleton.sample.

  • Update the design ledger to P1=IMPLEMENTED_CANDIDATE only if all Wave B evidence is GREEN.

  • Update the LLM Wiki branch-note; record whether a new derived raw document exists or explicitly record “없음”.


Wave C — P2 immutable event, polling delivery and operator policy

Task 7: Add the forward-only PostgreSQL outbox v2 schema

Owner: adapter:outbound:persistence-jpa (:adapter:outbound:persistence-jpa) Depends on: Wave B Activation: schema/control-plane only; LEGACY_POLLING remains ACTIVE

Candidate migration file:

  • src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V7__messaging_outbox_v2.sql

V7 is the current candidate because the sample Flyway location already contains V6__poster.sql. The Notification plan also uses V7/V8 as candidates; plan text is not a simultaneous Flyway reservation. Before implementation, scan every runtime Flyway location and all implemented or actively executing plans. The first implementation claims the next global version; the later plan must reserve the following version and update every path/test before writing SQL. Messaging and Notification persistence migrations must not execute concurrently with unresolved version ownership.

Files — create:

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDeliveryId.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDeliveryEntity.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDeliveryAttemptObservationEntity.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDispositionAuditEntity.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxPublicationEpochEntity.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxAuthorityCutoverEvidenceEntity.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryJpaRepository.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAttemptObservationJpaRepository.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDispositionAuditJpaRepository.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPublicationEpochJpaRepository.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2MigrationContractTest.java

Files — modify:

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxEventEntity.java

  • src/adapter/outbound/persistence-jpa/README.md

  • src/adapter/outbound/persistence-jpa/CLAUDE.md

  • Before editing, classify V3 row count/state/data and assert the base card accepts only a fresh or verified empty/drained legacy table. Any live non-empty database fails this task pending a separate deployment-specific migration plan.

  • Write a RED real-PostgreSQL migration test. Expected failure: v2 columns/tables/constraints do not exist.

  • Keep V3__outbox_event.sql byte-for-byte unchanged. Add immutable metadata columns additively while retaining legacy columns for compatibility.

  • Widen event_id VARCHAR(64) to VARCHAR(96) in the forward migration; this is compatible with old writers' shorter grammar. Keep the other V3 NOT NULL columns through the rollback window and choose one explicit compatibility projection for every canonical insert:

    ```text
    event_type      = contract_id legacy alias
    payload         = exact UTF-8 envelope bytes decoded as text
    status          = PENDING compatibility sentinel
    attempt_count   = 0
    next_attempt_at = occurred_at
    idempotency_key = event_id
    ```
    
    These columns are not authority after `POLLING_V2`. Epoch predicates prevent every legacy
    claim/mutation/reaper from observing canonical rows, and a post-cutover immutable guard
    prevents them from drifting. Do not relax NOT NULL/defaults or leave canonical inserts
    unspecified.
    
  • Create:

    ```text
    outbox_delivery
    outbox_delivery_attempt_observation
    outbox_disposition_audit
    outbox_publication_epoch
    outbox_authority_cutover_evidence
    outbox_write_admission
    outbox_runtime_node_lease
    ```
    
    with event/generation primary keys, one-CURRENT partial unique constraint, delivery FK,
    authority/state CHECKs, DB timestamps, row version, immutable automatic deadline and an
    expiring one-shot cutover evidence identity/digest. A cutover attempt has the closed durable
    state machine `CUTOVER_PENDING -> FINALIZING_V2 -> CONSUMED_V2` or
    `CUTOVER_PENDING -> RECOVERING_LEGACY -> RECOVERED_LEGACY`; the two branches are mutually
    exclusive CAS transitions. `RECOVERING_LEGACY` also stores an opaque recovery operation ID,
    owner/lease deadline and recovery evidence digest. Lease expiry permits recovery-only
    takeover and never resets the attempt to `CUTOVER_PENDING`. Give every attempt the constant
    database authority scope `OUTBOX_PUBLICATION`, ACTIVE legacy epoch ID, frozen fence generation
    and target-binding digest. A partial unique constraint permits exactly one nonterminal
    (`CUTOVER_PENDING`, `FINALIZING_V2`, `RECOVERING_LEGACY`) attempt in that authority scope.
    Attempt creation, finalization and recovery lock the write-admission singleton first and the
    ACTIVE epoch second, then validate the exact frozen generation/target binding before touching
    the attempt. The write admission singleton starts OPEN at generation 1; runtime node leases
    are bounded and bind
    node/source/artifact/fence-protocol identity without payload or credentials.
    
  • Add event ID, partition-key, SHA-256, tenant-scope, order uniqueness and exact BYTEA constraints. Protect immutable event columns with a post-cutover guard that is dormant during compatibility migration and enabled only by the fenced P3 cutover.

  • Seed exactly one ACTIVE LEGACY_POLLING epoch/generation for fresh/empty base template. Do not activate POLLING_V2, create v2 delivery for live legacy rows or claim/send from v2.

  • RED/GREEN cases: fresh V1V7; V3-empty upgrade; non-empty preflight rejection; duplicate current generation; nullable tenant attack; invalid hash/key/event ID; mutable event update after guard; FK/audit retention; publication epoch uniqueness; duplicate nonterminal authority attempt under concurrent insert; illegal finalization/recovery state transition; 6596 character event ID; old-writer short ID; canonical compatibility projection satisfying every retained V3 NOT NULL constraint.

  • Verify:

    ```bash
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*OutboxV2MigrationContractTest' --console=plain
    ```
    
  • Acceptance claim: additive polling v2 schema candidate only; legacy relay authority unchanged.

Rollback checkpoint: rollback disables new code and keeps additive schema/backlog. Never destructively downgrade the database.

Task 8: Append validated event and initial delivery in the business transaction

Owner leaves: application-core, adapter-outbound-persistence-jpa, app-bootstrap test fixture Depends on: Task 7

Files — create:

  • src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxAppendPort.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAppendAdapter.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapter.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAppendAdapterTest.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapterTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2AppendTransactionalContractTest.java

Files — modify:

  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAppendPort.java

  • src/application-core/src/main/java/dev/caskeleton/application/outbox/NewOutboxEvent.java

  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxEvent.java

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapter.java

  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapterTest.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapter.java

  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxContainerTestSupport.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxAppendTransactionalContractTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxPublisherLeaderElectionContractTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxRowLifecycleContractTest.java

  • src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisher.java

  • src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogUseCase.java

  • src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisherTest.java

  • src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogOutboxTest.java

  • src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/WorkLogUseCasesTest.java

  • src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/authz/WorkLogAuthorizationContractTest.java

  • Write RED application tests making OutboxAppendPort.append(ValidatedIntegrationEvent) the only canonical method. Move raw NewOutboxEvent append to a separately named/deprecated LegacyOutboxAppendPort; never overload or silently reinterpret raw payload as v1.

  • Split the current combined store/append implementation: OutboxStoreAdapter remains only the legacy relay store during the observation window, while a separately named LegacyOutboxAppendAdapter implements only LegacyOutboxAppendPort. It has no component annotation and bootstrap may compose it only for the explicit sample/R0 compatibility graph.

  • Write RED real-PostgreSQL tests proving business state + event + current delivery commit or rollback together; encoder/schema failure rolls back business state; exact BYTEA and hash round-trip. Cross-resource ACTIVE startup rejection belongs to Task 19 after both leaf descriptors exist.

  • In the persistence adapter, lock/read the ACTIVE publication epoch inside the caller-owned transaction, attach DB-authoritative created_at, publication_epoch, dispatch_authority, transaction_resource_id, and insert initial delivery only when authority is POLLING_V2.

  • For an initial current delivery, compute and persist in that same DB transaction:

    ```text
    automaticAttemptDeadline =
      min(eventDbCreatedAt + maximumAutomaticPublicationAge,
          eventDbCreatedAt + contract.sameEventRequeueHorizon)
    ```
    
    The value is immutable and profile reload never moves an existing generation's deadline.
    
  • During compatibility LEGACY_POLLING, write both legacy required columns and validated v2 metadata in the same transaction but do not create/send a v2 current delivery.

  • Make the legacy claim read model distinguish true v0 rows from rows carrying canonical v1 metadata without exposing a physical topic in application. For a canonical row, OutboxMessagePublishAdapter resolves the stored logical destination through the closed compiled binding and sends the stored partition-key bytes plus immutable envelope_bytes byte-for-byte. It must not invoke OutboxEnvelopeJson or reinterpret the retained V3 payload projection. Only a true v0 row may use the old wrapper/event-type route.

  • Add golden cases for canonical append under LEGACY_POLLING → legacy claim → exact compiled destination/key/envelope bytes → legacy PUBLISHED. This remains LEGACY_RECORDED_UNVERIFIED at cutover and is never automatically resent by v2. Test v0 and canonical branches independently; mixed/missing metadata fails closed.

  • Add a post-cutover canonical append fixture proving the retained V3 NOT NULL compatibility projection, immutable event + CURRENT/READY delivery and deadline all commit together while the epoch-fenced legacy claim/reaper sees the row count as 0.

  • Use an app-bootstrap test-source typed contribution/draft to prove the canonical append path. Do not inject the messaging encoder into sample-portfolio or add a project edge in this task. Change the sample use case dependency explicitly to LegacyOutboxAppendPort; the existing sample mapper remains a visibly R0, non-active compatibility fixture until the separate standalone-sample activation plan.

  • Remove component auto-discovery from the legacy append/store adapter. Bootstrap may compose it only for an exact LEGACY_POLLING compatibility graph; canonical POLLING_V2 must have LegacyOutboxAppendPort bean count 0. Update every exact existing legacy/sample fixture listed above in the same task so changing OutboxAppendPort cannot leave compile-only hidden users.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :application-core:test \
      --tests '*Outbox*' --console=plain
    cd src && ./gradlew :adapter:outbound:persistence-jpa:test \
      --tests '*OutboxAppendAdapterTest' \
      --tests '*LegacyOutboxAppendAdapterTest' --console=plain
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*OutboxV2AppendTransactionalContractTest' --console=plain
    cd src && ./gradlew :sample-portfolio:test \
      --tests '*CreateWorkLogOutboxTest' --console=plain
    ```
    
  • Acceptance claim: validated same-transaction append candidate; v2 relay remains disabled.

Rollback checkpoint: keep compatibility writes while rolling back the new relay. Do not generate a second event ID or dual-write outside the transaction.

Task 9: Define exhaustive publication outcomes and one-record relay policy

Owner: application-core (:application-core) Depends on: Task 8

Files — create under src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/:

  • PublicationOutcome.java
  • PublicationReceipt.java
  • PublicationFailure.java
  • AcceptanceCertainty.java
  • RetryDisposition.java
  • PublicationFailureStage.java
  • PublicationFailureClass.java
  • PublicationAttemptId.java
  • PublicationAdmission.java
  • PublicationAdmissionPort.java
  • AcknowledgedPublicationPort.java

Files — create under src/application-core/src/main/java/dev/caskeleton/application/outbox/:

  • OutboxDelivery.java
  • OutboxDeliveryState.java
  • DeliveryAuthorityStatus.java
  • ClaimToken.java
  • ClaimedOutboxDelivery.java
  • OutboxDeliveryStorePort.java
  • PublishNextOutboxDeliveryCommand.java
  • PublishNextOutboxDeliveryResult.java
  • PublishNextOutboxDeliveryUseCase.java

Files — create under src/application-core/src/test/java/dev/caskeleton/application/:

  • messaging/publication/PublicationOutcomeTest.java
  • outbox/PublishNextOutboxDeliveryUseCaseTest.java
  • outbox/OutboxDeliveryStateTest.java

Files — modify or retain as legacy until Task 26:

  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxMessagePublishPort.java

  • src/application-core/src/main/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCase.java

  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxBackoffPolicy.java

  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayResult.java

  • Write RED tests for the sealed outcome shape:

    ```java
    public sealed interface PublicationOutcome {
      record Acknowledged(PublicationReceipt receipt) implements PublicationOutcome {}
      record AcknowledgedMismatch(PublicationReceipt receipt) implements PublicationOutcome {}
      record Rejected(PublicationFailure failure) implements PublicationOutcome {}
      record Indeterminate(PublicationFailure failure) implements PublicationOutcome {}
    }
    ```
    
    Receipt/failure contains bounded provider-neutral values only; no Kafka SDK type or raw
    exception/message.
    
  • Test certainty and retry as independent axes. Ambiguous/post-admission/timeout/unknown maps to INDETERMINATE; REJECTED requires definite non-acceptance.

  • Write relay RED tests for this exact sequence:

    ```text
    acquire bounded admission
    -> Tx B claim exactly one row + ATTEMPT_ADMITTED
    -> publish outside DB transaction
    -> Tx C outcome observation + token/valid-lease CAS transition
    -> release admission
    ```
    
  • Cover: no admission → claim 0; no eligible row → release permit; ACK → DELIVERY_RECORDED; mismatch → HOLD; definite transient rejection → RETRY_WAIT; permanent/budget exhaustion → EXHAUSTED; indeterminate → duplicate-aware retry or HOLD according to remaining finite budget; transition failure propagates; diagnostic reporter failure cannot change persisted state.

  • Enforce one command invocation/one record. A scheduler may invoke it again; the use case must not loop and open per-row REQUIRES_NEW transactions.

  • Replace attempt-only backoff with a descriptor that includes maximum attempts, immutable automatic deadline, bounded delay/jitter and same-event horizon. Do not start age at first_attempt_at.

  • Keep legacy void port/use case explicitly deprecated and separately wired until Task 26; canonical code must not adapt exception-only success into Acknowledged.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :application-core:test \
      --tests '*PublicationOutcomeTest' \
      --tests '*PublishNextOutboxDeliveryUseCaseTest' \
      --tests '*OutboxDeliveryStateTest' --console=plain
    ```
    
  • Acceptance claim: application polling/outcome policy candidate; provider and DB CAS remain adapter work.

Rollback checkpoint: canonical use case remains unwired. Legacy relay continues to serve LEGACY_POLLING.

Task 10: Implement per-record JIT claim, valid-lease CAS and attempt journal

Owner: adapter:outbound:persistence-jpa (:adapter:outbound:persistence-jpa) Depends on: Task 9

Files — create:

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryStoreAdapter.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryClaimRepository.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxDeliveryClaimRepository.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryStoreAdapterTest.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxDeliveryClaimRepositoryTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2ClaimCasContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2MultiWorkerContractTest.java

Files — modify:

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlPersistenceConfig.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxContainerTestSupport.java

  • Write unit RED tests that the adapter maps application values without adding retry/topic policy and requires affected-row count exactly 1 for every CAS.

  • Write real-PostgreSQL RED tests for: two workers claim disjoint rows; same aggregate total order uses sequence/index rather than timestamp; different aggregates progress; hot aggregate does not starve all others; expired claim reclaim; same-token renew; stale token/owner/generation/version rejection; expired but not yet reclaimed owner cannot record ACK/failure.

  • The worker-owned mutation predicate must include:

    ```sql
    WHERE event_id = :event_id
      AND delivery_generation = :generation
      AND authority_status = 'CURRENT'
      AND state = 'CLAIMED'
      AND claim_token = :token
      AND claim_owner = :owner
      AND claim_until > CURRENT_TIMESTAMP
      AND row_version = :expected_row_version
    ```
    
  • Claim eligibility is CURRENT READY, due RETRY_WAIT or expired CLAIMED, subject to ordering-head eligibility. EXHAUSTED, HOLD, LEGACY_RECORDED_UNVERIFIED never release the next ordered event.

  • Tx B atomically updates claim count/token/owner/DB-time lease/publication attempt count and inserts ATTEMPT_ADMITTED. Raw claim token is never copied; journal stores a domain-separated digest.

  • Before ATTEMPT_ADMITTED, use DB time to verify both the automatic deadline and a full application-attempt/Tx-C safety window remain. If database_now >= deadline or the full window does not fit, perform a fenced EXHAUSTED transition without admission/send.

  • When reclaiming an expired CLAIMED row whose previous publicationAttemptId has ATTEMPT_ADMITTED but no outcome, append exactly one idempotent OUTCOME_OBSERVED(INDETERMINATE) for that old attempt before replacing the token and admitting the new attempt. Never fabricate a definite rejection or erase the prior attempt.

  • Tx C atomically inserts OUTCOME_OBSERVED and performs ACK/retry/exhaust/hold CAS. Provider metadata is a bounded opaque reference.

  • Use DB time for eligibility, lease, created-at and retry due. Before send admission, ensure remaining lease exceeds the full attempt + DB transition + safety budget.

  • Verify RED then GREEN sequentially:

    ```bash
    cd src && ./gradlew :adapter:outbound:persistence-jpa:test \
      --tests '*OutboxDeliveryStoreAdapterTest' \
      --tests '*PostgreSqlOutboxDeliveryClaimRepositoryTest' --console=plain
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*OutboxV2ClaimCasContractTest' \
      --tests '*OutboxV2MultiWorkerContractTest' --console=plain
    ```
    
  • Acceptance claim: real-PostgreSQL JIT claim/CAS protocol candidate; no Kafka send or authority cutover.

Rollback checkpoint: leave v2 scheduler off and LEGACY_POLLING active. Claimed test rows are disposable; production rollback preserves all event/delivery rows.

Task 11: Persist late publication observations with DB-commit-before-source-ACK

Owner leaves: application-core, adapter-outbound-persistence-jpa Depends on: Task 10

Files — create in application-core:

  • src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/ObservationId.java
  • src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/LatePublicationObservation.java
  • src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/LatePublicationObservationSourcePort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAttemptObservationPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsCommand.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsResult.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsUseCase.java
  • src/application-core/src/test/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsUseCaseTest.java

Files — create in adapter:outbound:persistence-jpa:

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAttemptObservationAdapter.java

  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAttemptObservationAdapterTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLateObservationContractTest.java

  • Write application RED tests for:

    ```text
    poll/lease bounded batch
    -> tx.inNew(idempotent DB append) returns after commit
    -> acknowledgePersisted
    ```
    
    DB append/commit failure calls `releaseForRetry`; source ACK never runs in a transaction
    callback.
    
  • Use observation identity (eventId, deliveryGeneration, publicationAttemptId, LATE_ACK_OBSERVED) and a DB unique constraint/ON CONFLICT DO NOTHING.

  • Test DB commit → process crash before source ACK by redelivering the same observation; exactly one journal fact remains.

  • Test late observation never changes DELIVERY_RECORDED, RETRY_WAIT, EXHAUSTED, HOLD or current generation. It is diagnostic, not correctness authority.

  • Test empty poll, bounded maximum, poison item release, source ACK failure and commit failure.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :application-core:test \
      --tests '*RecordLatePublicationObservationsUseCaseTest' --console=plain
    cd src && ./gradlew :adapter:outbound:persistence-jpa:test \
      --tests '*OutboxAttemptObservationAdapterTest' --console=plain
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*OutboxLateObservationContractTest' --console=plain
    ```
    
  • Acceptance claim: durable idempotent late-observation drain boundary; callback capture is still bounded-loss and no messaging queue exists until Task 16.

Rollback checkpoint: disabling the drain loses only bounded diagnostics, never changes delivery authority. Alert/readiness must expose the degradation.

Task 12: Implement audited application disposition policy and atomic persistence transitions

Owner leaves: application-core, adapter-outbound-persistence-jpa Depends on: Tasks 1011

Files — create in application-core:

  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxDisposition.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionCommand.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionResult.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxDispositionResultCodec.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxDispositionPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionUseCase.java
  • src/application-core/src/test/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionUseCaseTest.java

Files — create in adapter:outbound:persistence-jpa:

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDispositionAdapter.java

  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDispositionAdapterTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxDispositionTransactionalContractTest.java

  • Write application RED tests with @RequiresPermission("outbox:disposition") and @UseCaseCapability(idempotency = Idempotency.KEYED, ...). SKIP_WITH_GAP and COMPENSATE additionally call AuthorizationPort for outbox:disposition:destructive.

  • Command requires:

    ```text
    eventId
    expectedDeliveryGeneration
    expectedRowVersion
    disposition
    bounded reason
    incident/change reference
    IdempotencyContext(scope + request fingerprint + bounded TTL)
    operator principal
    destructive approval reference when required
    compensation event reference for COMPENSATE
    ```
    
  • Inject the existing application-owned IdempotencyExecutor into ApplyOutboxDispositionUseCase. Execute authorization/policy/CAS exactly once under the command's IdempotencyContext, using a framework-free deterministic OutboxDispositionResultCodec. Replay returns the stored application result; same key with a different request fingerprint raises the existing mismatch exception. Controller and persistence adapter must not implement their own idempotency state machine.

  • Application tests cover first execution, completed replay, in-flight conflict, request mismatch, action failure/discard and bounded TTL. Persistence integration reuses the existing IdempotencyStorePort adapter to prove atomic claim/complete; outbox_disposition_audit remains the immutable business/operation audit rather than a second idempotency registry.

  • Validate allowed source state, ordering impact, active-unexpired-claim absence and finite same-event requeue horizon in application policy for early feedback. This precheck is not the concurrency fence.

  • In the persistence transaction, lock the CURRENT delivery row and atomically re-evaluate expected generation/state/row version plus NOT (state='CLAIMED' AND claim_until > database_now) before audit/mutation. Add a race test that inserts a worker claim after application precheck but before the locked mutation; operator CAS must fail without partial audit/handoff.

  • Write real-PostgreSQL RED/GREEN for: stale generation/version; live claim; idempotency replay/mismatch; concurrent requeue; one CURRENT constraint; partial handoff rollback; old generation never claimable again; immutable audit.

  • REQUEUE transaction locks current row, inserts audit, marks old authority SUPERSEDED, sets superseded_by_generation, and inserts generation + 1 CURRENT/READY with:

    ```text
    automaticAttemptDeadline =
      min(newDeliveryDbCreatedAt + maximumAutomaticPublicationAge,
          eventDbCreatedAt + sameEventRequeueHorizon)
    ```
    
  • HOLD/SKIP/COMPENSATE/legacy accept use expected generation/state/row version and do not mimic the worker token predicate. COMPENSATED requires an already-created immutable compensating event reference in the same transaction.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :application-core:test \
      --tests '*ApplyOutboxDispositionUseCaseTest' --console=plain
    cd src && ./gradlew :adapter:outbound:persistence-jpa:test \
      --tests '*OutboxDispositionAdapterTest' --console=plain
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*OutboxDispositionTransactionalContractTest' --console=plain
    ```
    
  • Acceptance claim: provider-neutral authenticated disposition policy and DB protocol candidate; no HTTP surface yet.

Rollback checkpoint: operator surface is not exposed. Data/audit rows are forward-only and must not be rewritten by raw SQL.

Task 13: Prove P2 compatibility fence and no-dual-authority state

Owner: persistence/bootstrap integration Depends on: Tasks 712

Files — create:

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxPublicationEpochContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyV2CompatibilityContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyCanonicalWireCompatibilityContractTest.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxV2RetentionAdapter.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxV2RetentionAdapterTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2RetentionContractTest.java

Files — modify:

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapter.java

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxClaimRepository.java

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaper.java

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxEventJpaRepository.java

  • Write the epoch, canonical-wire compatibility and retention tests first, then run RED:

    ```bash
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*OutboxPublicationEpochContractTest' \
      --tests '*OutboxLegacyV2CompatibilityContractTest' \
      --tests '*OutboxLegacyCanonicalWireCompatibilityContractTest' \
      --tests '*OutboxV2RetentionContractTest' --console=plain
    ```
    
    Expected non-zero: the legacy claim/reaper lacks an epoch fence and v2 retention protocol is
    absent. A compile failure unrelated to those missing contracts is not an accepted RED.
    
  • RED test that the compatibility legacy writer fills v2 immutable metadata in the same transaction while the legacy relay can claim only the exact ACTIVE LEGACY_POLLING epoch/generation.

  • Add a legacy mutation/claim fence predicate tied to the ACTIVE publication epoch. Old pre-fence binaries are explicitly incompatible and must be drained to zero before P3.

  • Prove POLLING_V2 claim is rejected while LEGACY_POLLING is active and legacy claim is rejected after the epoch changes.

  • Prove no row can be claimed/sent by both paths; publication epoch lock and expected generation are mandatory.

  • Prove the compatibility publisher sends a canonical metadata row exactly once through the legacy authority using the compiled destination, stored key and byte-identical v1 envelope; v0 rows still use the old wrapper. Nested envelope, event-type-as-topic for canonical rows, mixed metadata, and automatic resend of legacy PUBLISHED after cutover all fail.

  • Bind the legacy reaper to the exact ACTIVE LEGACY_POLLING epoch and stop/drain it before cutover. Implement v2 retention separately: delete only when the unique CURRENT generation is resolved as DELIVERY_RECORDED or audited SKIPPED/COMPENSATED/ LEGACY_ACCEPTED_UNVERIFIED, with no claim/requeue, unresolved observation, legal/operator hold, audit-retention or replay-horizon obligation.

  • Real PostgreSQL retention cases cover reaper-vs-claim/requeue, delivery/audit FK and no-silent-cascade, superseded generations, EXHAUSTED, HOLD, LEGACY_RECORDED_UNVERIFIED, unresolved late observation and epoch mismatch.

  • Do not perform legacy row reconciliation, v2 delivery creation or authority switch in this task.

  • Re-run the same focused command GREEN; all three exact tests must pass with no skip. Then run the candidate gate:

    ```bash
    cd src && ./gradlew verifyMessagingPollingOutboxR2 --console=plain
    ```
    
    At P2, `verifyMessagingPollingOutboxR2` may report `implemented-candidate`; it must not emit a
    release-eligible claim. On GREEN it validates and writes the exact candidate manifest:
    
    ```text
    src/app-bootstrap/build/messaging-evidence/polling-outbox-r2/manifest.json
    ```
    
    The manifest conforms to
    `src/config/messaging/evidence/build-evidence-manifest-v1.schema.json` and binds the supplied
    source/artifact digest, migration/schema/card/profile hashes, exact PostgreSQL scenario
    IDs/counts, commands/timestamps, failed=0, skipped=0 and unsupported Kafka/security claims.
    
  • Update P2 card rows to implemented-candidate only if real PostgreSQL cases pass.

Rollback checkpoint: keep LEGACY_POLLING ACTIVE and canonical v2 scheduler off. If the fence cannot be deployed to all nodes, do not proceed to Wave D.

Wave C exit checkpoint

  • Run:

    ```bash
    cd src && ./gradlew :application-core:check \
      :adapter:outbound:persistence-jpa:check \
      :app-bootstrap:test \
      verifyCleanArchitectureDependencies \
      --console=plain
    ```
    
  • Confirm broker/client/network resources are still 0 and v2 authority has not switched.

  • Update the design ledger to P2=IMPLEMENTED_CANDIDATE only from actual tests.

  • Update the LLM Wiki branch-note and derived-document decision.


Wave D — P3 ACK-aware Spring Kafka, operator endpoint and reference-path cutover

Task 14: Bind and compile finite canonical Messaging settings

Owner: adapter:outbound:messaging (:adapter:outbound:messaging) Depends on: Wave C

Files — create:

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingExpectedState.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingR2Settings.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingSettingsCompiler.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerSettings.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/CompiledKafkaProducerSettings.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingSettingsCompilerTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingDisabledResourceContractTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerSettingsTest.java

Files — modify:

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingSettings.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaAdapterSettings.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfig.java

  • Write RED binding/compiler tests for exact DISABLED|ACTIVE expected state and the canonical fields in design §21.1. Raw Map<String,Object> Kafka overrides are forbidden.

  • Freeze the first effective profile:

    ```text
    acks=all
    enable.idempotence=true
    retries=MAX/effectively-unbounded under delivery.timeout.ms
    max.in.flight.requests.per.connection<=5
    compression.type=none
    partitioner.ignore.keys=false
    finite request/delivery/max.block/linger/batch/buffer/request bounds
    maximumAdmittedRecords=1
    ```
    
  • Validate:

    ```text
    deliveryTimeout >= requestTimeout + linger
    applicationAttemptBudget >=
      admissionWait + maxBlock + deliveryTimeout + callback/transitionReserve
    claimLease >
      applicationAttemptBudget + dbTransitionReserve + schedulingSafetyMargin
    ```
    
  • Reject ACTIVE + unknown/missing card/provider/bootstrap/destination/security; plaintext in production; literal credentials; contract bytes over any bound; ordering + null key; transaction resource mismatch; legacy + canonical keys; active durable contract + disabled dispatch.

  • DISABLED must instantiate no schema compiler with active contracts, producer factory, template, AdminClient, semaphore, observation queue, scheduler, secret refresh or network connection.

  • Legacy keys are parsed only into an R0 descriptor and conflict with canonical keys. Do not map broker=kafka to kafka-spring or relay-enabled=true to polling v2.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :adapter:outbound:messaging:test \
      --tests '*MessagingSettingsCompilerTest' \
      --tests '*MessagingDisabledResourceContractTest' \
      --tests '*KafkaProducerSettingsTest' --console=plain
    ```
    
  • Acceptance claim: finite static descriptor candidate; no Kafka client created yet.

Rollback checkpoint: canonical activation remains disabled; legacy settings continue only in R0 mode.

Task 15: Add explicit Spring Kafka producer factory and ACK-aware gateway

Owner: adapter:outbound:messaging (:adapter:outbound:messaging) Depends on: Task 14

Files — create:

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerFactoryConfig.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaPublishGateway.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGateway.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaPublicationFailureClassifier.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/publication/AckAwareOutboxPublicationAdapter.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerFactoryConfigTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGatewayTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaPublicationFailureClassifierTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/publication/AckAwareOutboxPublicationAdapterTest.java

Files — modify:

  • src/adapter/outbound/messaging/build.gradle

  • src/adapter/outbound/messaging/gradle.lockfile

  • Add implementation 'org.springframework.kafka:spring-kafka'; accept the Spring Boot 4.0.0 BOM version unless a separately reviewed compatibility override is necessary. Regenerate and review the messaging lockfile.

  • RED test exact producer properties and DefaultKafkaProducerFactory<byte[], byte[]> / KafkaTemplate<byte[], byte[]>. Use byte serializers; no JSON serialization in Kafka callbacks.

  • RED gateway tests for:

    ```text
    future success + metadata + expected topic -> ACKNOWLEDGED
    future success + metadata + wrong topic    -> ACKNOWLEDGED_MISMATCH
    definite pre-admission/local rejection     -> REJECTED
    ambiguous/post-admission/deadline/unknown  -> INDETERMINATE
    ```
    
  • Build ProducerRecord<byte[],byte[]> only from compiled topic, stored partition-key bytes, exact envelope bytes and bounded allowlisted headers.

  • Await the future to a monotonic application deadline and verify non-null metadata. Do not call per-message flush(). cancel() is not delivery cancellation evidence.

  • Map Kafka exception categories to stable stage/class/certainty/disposition without exposing class names or messages. Ambiguity defaults to INDETERMINATE.

  • The adapter returns bounded provider generation, local ACK observation time and safe opaque record reference. Application never routes from it.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :adapter:outbound:messaging:test \
      --tests '*KafkaProducerFactoryConfigTest' \
      --tests '*SpringKafkaPublishGatewayTest' \
      --tests '*KafkaPublicationFailureClassifierTest' \
      --tests '*AckAwareOutboxPublicationAdapterTest' --console=plain
    cd src && ./gradlew verifyDependencyLocks --console=plain
    ```
    
  • Acceptance claim: fake-gateway ACK-aware provider candidate; real Kafka ACK remains Task 21.

Rollback checkpoint: producer beans remain gated/dark and v2 scheduler off.

Task 16: Add bounded admission, late-completion source and producer generation lifecycle

Owner leaves: application-core (:application-core), adapter:outbound:messaging (:adapter:outbound:messaging) Depends on: Task 15

Files — create:

  • src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/PublicationGenerationLifecyclePort.java
  • src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/PublicationGenerationDrainResult.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RotatePublicationGenerationCommand.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RotatePublicationGenerationResult.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RotatePublicationGenerationUseCase.java
  • src/application-core/src/test/java/dev/caskeleton/application/outbox/RotatePublicationGenerationUseCaseTest.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/publication/BoundedPublicationAdmission.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/observation/BoundedLatePublicationObservationSource.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerGeneration.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerGenerationManager.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerLifecycle.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/publication/BoundedPublicationAdmissionTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/observation/BoundedLatePublicationObservationSourceTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerGenerationManagerTest.java

Files — modify:

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGateway.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/publication/AckAwareOutboxPublicationAdapter.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerFactoryConfig.java

  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGatewayTest.java

  • RED test finite permit count/queue wait, deadline inclusion, saturation, interrupt, release-on-all outcomes and zero queued outbox rows after admission exhaustion.

  • Implement one atomic terminal marker per send. If deadline wins, synchronous outcome stays INDETERMINATE; one later successful callback may enqueue one payload-free late observation.

  • Wire the actual Kafka future callback in SpringKafkaPublishGateway to the bounded source, capturing only stable event ID, delivery generation, publication attempt ID, producer generation and binding revision before send. Test before-deadline completion, callback-wins, deadline-wins, duplicate callback, late success, late failure, overflow and generation-close race against the same atomic terminal marker.

  • RED test observation source lease/poll/ACK/release, bounded capacity, duplicate callback, timeout-callback race, queue overflow/drop metric and payload/header absence.

  • Overflow never mutates delivery state. It degrades readiness and alerts; capacity qualification requires zero drop.

  • RED producer generation tests for: stop admission/claim; bounded drain; unresolved attempts durably reported INDETERMINATE/HOLD before swap; bounded old close; secret generation resolve; new create/attest; global generation barrier; no old/new overlap.

  • Keep the messaging implementation provider-only: PublicationGenerationLifecyclePort returns bounded admitted/in-flight resolution facts and performs pause/drain/create/attest/close, but it imports no outbox store, transaction or persistence type and never chooses HOLD/retry policy.

  • RotatePublicationGenerationUseCase owns orchestration. It pauses new admission/claim through provider-neutral ports, asks the provider to drain, persists every unresolved durable attempt as INDETERMINATE and every affected ordered scope as HOLD through OutboxDeliveryStorePort/TransactionPort, then permits close/create/attest/barrier switch. Bootstrap invokes this use case; messaging configuration never calls persistence directly.

  • A DB outage preventing durable INDETERMINATE/HOLD blocks generation switch and keeps admission closed. Best-effort unresolved sends may return INDETERMINATE but are never auto-replayed.

  • Fatal producer state blocks new admission, lowers readiness and recreates a new immutable generation; it does not change acceptance certainty or remove duplicate risk.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :adapter:outbound:messaging:test \
      --tests '*BoundedPublicationAdmissionTest' \
      --tests '*BoundedLatePublicationObservationSourceTest' \
      --tests '*KafkaProducerGenerationManagerTest' --console=plain
    cd src && ./gradlew :application-core:test \
      --tests '*RotatePublicationGenerationUseCaseTest' --console=plain
    ```
    
  • Acceptance claim: bounded local resource/lifecycle protocol candidate; security/topology and real broker evidence remain.

Rollback checkpoint: close the dark producer generation and keep canonical scheduler off.

Task 17: Attest external topic topology and production Kafka security

Owner: adapter:outbound:messaging (:adapter:outbound:messaging) Depends on: Task 16

Files — create:

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecurityProfile.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecuritySettings.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretReference.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretMaterial.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretMaterialResolver.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaTopicTopologyAttestor.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProvisioningEvidence.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaTopicAttestation.java

  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecuritySettingsTest.java

  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaTopicTopologyAttestorTest.java

  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretRedactionTest.java

  • RED test typed allowlist: local-only plaintext, TLS server auth, and production SASL_SSL + SCRAM-SHA-512. First production tuple rejects PLAIN/OAuth/mTLS profiles, plaintext downgrade, trust-all, hostname verification disable and literal JAAS credentials.

  • KafkaSecretMaterial exposes no secret in toString, exception, descriptor or log; it carries bounded generation/expiry and clear/close lifecycle. Configuration carries only secret://messaging/kafka/producer.

  • Freeze the first supported resolver boundary as mounted-secret-files-v1. KafkaSecretMaterialResolver accepts only the exact typed reference and returns SCRAM username/password plus trust material, generation and expiry; no application/shared type contains these provider details. Unknown scheme/path traversal, missing field, wrong permission/format, expired generation and literal credential fail closed.

  • RED topology tests for topic existence, partitions, RF, min ISR, cleanup policy, retention, max bytes, leader/ISR and wrong cluster/binding.

  • Runtime AdminClient uses only bounded Describe and exact-topic DescribeConfigs. It never creates/alters/deletes topics, enumerates all ACLs or requires broker-wide configuration.

  • Assert the same resolved generation is applied to both producer factory and AdminClient: security.protocol=SASL_SSL, sasl.mechanism=SCRAM-SHA-512, hostname verification enabled and no literal JAAS value in settings/descriptor/log. A partial producer-only or AdminClient-only resolution fails startup.

  • Provisioning evidence supplies runtime-inaccessible assertions: broker policy, auto-create/unclean election, exact positive/negative ACL probes, cluster/topic resource identity, config/ACL digest, issuer/provenance, generated/expiry time and release assertion digest.

  • Missing, stale, wrong-cluster, invalid provenance or runtime/provisioning mismatch prevents ACTIVE_READY. Transient broker unavailability yields bounded ACTIVE_NOT_READY; static credential/security/binding errors fail closed.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :adapter:outbound:messaging:test \
      --tests '*KafkaSecuritySettingsTest' \
      --tests '*KafkaTopicTopologyAttestorTest' \
      --tests '*KafkaSecretRedactionTest' --console=plain
    ```
    
  • Acceptance claim: local topology/security validation candidate; actual TLS/SASL/ACL and multi-broker evidence remain Wave E.

Rollback checkpoint: attestation failure keeps relay admission off; it never falls back to topic auto-create, wildcard ACL or plaintext.

Task 18: Expose the authenticated, idempotent disposition endpoint

Owner: adapter:inbound:web (:adapter:inbound:web) Depends on: Task 12

Files — create:

  • src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/controller/MessagingOutboxDispositionController.java
  • src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/dto/request/OutboxDispositionRequest.java
  • src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/dto/response/OutboxDispositionResponse.java
  • src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/mapper/OutboxDispositionWebMapper.java
  • src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/controller/MessagingOutboxDispositionControllerWireTest.java
  • src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/mapper/OutboxDispositionWebMapperTest.java
  • src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/controller/MessagingOutboxDispositionOpenApiContractTest.java
  • src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/ratelimit/MessagingOutboxDispositionRateLimitTest.java
  • src/adapter/inbound/web/src/test/resources/openapi/messaging-outbox-disposition-openapi-snapshot.json

Files — modify:

  • src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/error/GlobalExceptionHandler.java

  • src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/authz/RolePermissionPolicyTest.java

  • src/adapter/inbound/web/README.md

  • src/adapter/inbound/web/CLAUDE.md

  • RED wire tests for:

    ```text
    POST /internal/operations/messaging/outbox/{eventId}/dispositions
    required header: Idempotency-Key
    base permission: outbox:disposition
    destructive permission: outbox:disposition:destructive
    ```
    
  • Cover unauthenticated, insufficient permission, missing/malformed key, invalid DTO, stale generation/version, idempotency replay/mismatch, live claim conflict, horizon exceeded, missing destructive approval/compensation reference and success response.

  • Request contains expected delivery generation, expected row version, closed disposition, bounded reason and incident/change reference. Mapper converts AuthenticatedPrincipal/request/path/header to framework-free command; no web/security type crosses into application.

  • Use IdempotencyKeySupport to build the existing principal/use-case-scoped IdempotencyScope and compute RequestFingerprint from the canonical disposition request fields, including event ID, expected generation/version, disposition, reason, incident, approval and compensation reference. Pass the resulting IdempotencyContext to the use case; never pass only a raw header string.

  • Reuse IdempotencyKeySupport and existing authorization enforcement. Controller calls only ApplyOutboxDispositionUseCase; it imports no repository, entity, outbound adapter or transaction manager.

  • Keep the endpoint authenticated and absent from the public-path allowlist. Add explicit internal network/rate-bound contract and a committed endpoint OpenAPI snapshot. The public path snapshot must remain unchanged; verifyPublicPathSnapshot proves the endpoint was not accidentally allowlisted.

  • Map stale CAS to conflict, invalid policy to safe 4xx, authorization to existing envelope and unknown failures to safe 5xx without event payload/hash leakage.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :adapter:inbound:web:test \
      --tests '*MessagingOutboxDispositionControllerWireTest' \
      --tests '*OutboxDispositionWebMapperTest' \
      --tests '*MessagingOutboxDispositionOpenApiContractTest' \
      --tests '*MessagingOutboxDispositionRateLimitTest' \
      --tests '*RolePermissionPolicyTest' --console=plain
    cd src && ./gradlew verifyPublicPathSnapshot --console=plain
    ```
    
  • Acceptance claim: authenticated transport mapping candidate; persistence/application tests remain authority for policy/CAS.

Rollback checkpoint: disable route exposure through composition/network policy, not by allowing raw SQL mutation.

Task 19: Compose exact tuple, schedulers, readiness and observability

Owner: app-bootstrap (:app-bootstrap) Depends on: Tasks 1418

Files — create:

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfig.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityReadiness.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingRuntimeDescriptor.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MountedKafkaSecretResolverSettings.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MountedKafkaSecretMaterialResolver.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/KafkaSecretRefreshScheduler.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/LatePublicationObservationScheduler.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/transaction/PersistenceTransactionResourceDescriptor.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfigTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityReadinessTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingRuntimeDescriptorTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MountedKafkaSecretMaterialResolverTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/KafkaSecretRefreshSchedulerTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/LatePublicationObservationSchedulerTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingDisabledZeroResourceContractTest.java

Files — modify:

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxSettings.java

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRelayScheduler.java

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxMetrics.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxConfigTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxSettingsTest.java

  • src/app-bootstrap/src/main/resources/application.yml

  • src/app-bootstrap/src/test/resources/application-test.yml

  • src/app-bootstrap/README.md

  • src/app-bootstrap/CLAUDE.md

  • RED composition tests prove settings bind/compile before secret/client, then producer, topic attestation, readiness, v2 relay and late-drain scheduler in that order.

  • Bootstrap aggregates leaf descriptors and same transaction resource identity only. It must not reimplement catalog/schema/retry/disposition/provider rules.

  • Persistence exposes a sanitized transactionResourceId plus resolved DataSource/EntityManagerFactory/PlatformTransactionManager identity descriptor. Bootstrap compares it with TransactionPort, canonical append adapter and the business repository resource before creating ACTIVE clients or schedulers. Add a composition RED/GREEN case where a second DataSource causes startup rejection and Kafka/network resource count remains 0.

  • Implement mounted-secret-files-v1 in bootstrap with an explicit bounded root, exact reference-to-directory mapping, no symlink/path escape, owner/permission checks where the platform exposes them, atomic generation manifest read, expiry validation, redacted failure and prompt clearing of old char/byte material. The refresh scheduler invokes RotatePublicationGenerationUseCase; it never mutates a live producer object.

  • Missing/wrong/expired secret blocks ACTIVE before producer/AdminClient creation. Refresh failure may retain the old generation only until its configured expiry/safety margin, then closes admission/readiness. DISABLED creates resolver/refresh/file-watch resource count 0.

  • Gate the v2 relay and late-drain scheduler on canonical ACTIVE + POLLING_V2 epoch + fresh producer/topic/security readiness. Remove the legacy relay-enabled boolean from canonical mode.

  • Exact LEGACY_POLLING compatibility composition may expose LegacyOutboxAppendPort; canonical POLLING_V2 composition must assert legacy append/store/publish/relay bean count 0.

  • Readiness roles stay separate:

    ```text
    relay = producer + topic/security + DB claim + catalog
    durable write = DB append + backlog capacity
    direct required producer = producer/topic/security
    liveness = process-internal only
    ```
    
  • Add bounded hysteresis/freshness and explicit STARTING|ACTIVE_NOT_READY|ACTIVE_READY. Static configuration/security mismatch fails startup; transient broker outage never starts relay admission.

  • Runtime descriptor exposes only card IDs, versions, catalog/schema/settings digests, destination aliases/revisions, resource ID, epoch/authority, generation, readiness, evidence status, non-guarantees and runbook IDs. Redact servers/topics where policy requires; never expose credentials/payload/hash/raw headers.

  • Replace legacy metrics with bounded dimensions for logical attempt, certainty, failure stage, claim conflict/lease/backlog/order block/generation/late-drop. Reject event/aggregate/tenant/ key/correlation/hash/exception-message tags. One confirmed persisted transition owns the canonical error.

  • DISABLED integration test asserts client/factory/template/AdminClient/semaphore/queue/thread/ scheduler/secret resolver/network count 0.

  • Verify RED then GREEN:

    ```bash
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*MessagingCapability*Test' \
      --tests '*MessagingRuntimeDescriptorTest' \
      --tests '*MountedKafkaSecretMaterialResolverTest' \
      --tests '*KafkaSecretRefreshSchedulerTest' \
      --tests '*LatePublicationObservationSchedulerTest' \
      --tests '*MessagingDisabledZeroResourceContractTest' \
      --tests '*OutboxConfigTest' \
      --tests '*OutboxSettingsTest' --console=plain
    ```
    
  • Acceptance claim: complete dark reference graph candidate; target deployment authority remains legacy until Task 25.

Rollback checkpoint: keep canonical expected-state DISABLED and LEGACY_POLLING ACTIVE. No DB schema downgrade.

Task 20: Implement and rehearse the fenced authority cutover without production switch

Owner leaves: application-core, adapter-outbound-persistence-jpa, app-bootstrap, adapter-outbound-messaging Depends on: Tasks 1319 Base template gate: fresh or verified empty/drained V3 only

Every cutover in this task runs against a disposable rehearsal database and test broker. It proves the code/protocol but does not change a target deployment, start its v2 relay, resume its business writes or delete legacy runtime. Production remains LEGACY_POLLING.

Files — create:

  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxWriteAdmissionControlPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelayControlPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxWriteAdmissionSnapshot.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelaySnapshot.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionEvidence.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverCommand.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverResult.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAuthorityCutoverPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCase.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionCommand.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionResult.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionUseCase.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxPreCommitRecoveryPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityCommand.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityResult.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCase.java
  • src/application-core/src/test/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCaseTest.java
  • src/application-core/src/test/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionUseCaseTest.java
  • src/application-core/src/test/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCaseTest.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxCutoverPreconditionAdapter.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapter.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxWriteAdmissionEntity.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxRuntimeNodeLeaseEntity.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxWriteAdmissionJpaRepository.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxRuntimeNodeLeaseJpaRepository.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/PostgreSqlOutboxWriteAdmissionGuard.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxWriteAdmissionControlAdapter.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxRuntimeNodeLeaseAdapter.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapter.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapterTest.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxWriteAdmissionControlAdapterTest.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxRuntimeNodeLeaseAdapterTest.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapterTest.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFence.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFenceTest.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinator.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinator.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapter.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRuntimeNodeLeaseScheduler.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverJobSettings.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunner.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinatorTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinatorTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapterTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxRuntimeNodeLeaseSchedulerTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunnerTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyToV2CutoverContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxWriteAdmissionMultiNodeContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2SentinelContractTest.java

Files — modify:

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/transaction/SpringTransactionPort.java

  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/transaction/SpringTransactionPortTest.java

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaper.java

  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaperTest.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapter.java

  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRelayScheduler.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxConfigTest.java

  • src/app-bootstrap/build.gradle

  • src/build.gradle

  • RED application tests scope FinalizeOutboxAuthorityCutoverUseCase to the atomic database finalization contract. It accepts an opaque, human-approved cutover evidence ID and never trusts command booleans for writer/relay/producer drain. Before reconciliation it must CAS the exact fresh attempt CUTOVER_PENDING -> FINALIZING_V2 in the same database transaction that commits the epoch, after taking the global write-admission/ACTIVE-epoch locks in the fixed order and proving it is the sole nonterminal OUTBOX_PUBLICATION attempt; RECOVERING_LEGACY, an expired attempt or a different operation owner is a hard rejection. Rollback restores CUTOVER_PENDING, while a successful epoch commit records CONSUMED_V2. This is a one-shot maintenance use case, not a second web endpoint.

  • OutboxLegacyToV2CutoverCoordinator is the deployment/composition owner. Through OutboxWriteAdmissionControlPort, LegacyOutboxRelayControlPort and PublicationGenerationLifecyclePort, it freezes writes, drains writers/relay/futures, closes/fences legacy Write, compiles/attests the canonical tuple and asks OutboxCutoverPreconditionPort to persist a short-lived one-shot evidence record containing exact node/writer/relay/producer generations, zero-active facts, epoch, manifest digest, approver and expiry in CUTOVER_PENDING. Evidence creation uses the global lock order write-admission singleton FOR UPDATE then ACTIVE epoch FOR UPDATE, requires the exact FROZEN generation/target binding and rejects any nonterminal attempt in OUTBOX_PUBLICATION; the partial unique constraint is the final concurrent-insert guard. Bootstrap imports only application ports; it never queries repositories or Kafka adapter internals.

  • Implement the production write fence with the PostgreSQL singleton created in Task 7. SpringTransactionPort.inWrite begins its transaction, acquires FOR KEY SHARE through PostgreSqlOutboxWriteAdmissionGuard, and verifies OPEN + expected fence generation before invoking any business action. The control adapter takes FOR UPDATE, waits for all older share-holding writers to commit/rollback, writes FROZEN generation and then returns a durable zero-active snapshot. New inWrite calls fail and roll back; inNew remains available only for maintenance/outbox/audit and never bypasses a business write.

  • Implement both explicit generation-CAS exits from FROZEN without a raw status update. ResumePollingV2WriteAdmissionUseCase requires expected frozen generation, exact ACTIVE POLLING_V2 epoch, canonical cutover sentinel created through OutboxAppendAdapter and persisted as DELIVERY_RECORDED, fresh target binding/readiness and no legacy runtime. It writes OPEN(generation+1) once; mismatch/replay/failure leaves FROZEN. RecoverLegacyOutboxAuthorityUseCase is legal only before epoch commit. Before any external ACL mutation, its prepare operation takes the same global write-admission/ACTIVE-epoch lock order, proves the exact attempt is the sole nonterminal authority attempt plus FROZEN/LEGACY_POLLING/zero-v2-authority, then CAS-claims CUTOVER_PENDING -> RECOVERING_LEGACY and atomically invalidates that attempt for v2 finalization. Choosing this branch is irreversible for that attempt; only recovery completion or recovery-only lease takeover remains legal. The complete operation follows the separately fenced recovery protocol below.

  • Register bounded outbox_runtime_node_lease heartbeats for every runtime node with node ID, source/artifact digest, write-fence protocol version, epoch and scheduler roles. Precondition evidence requires all deployment-inventory instances to have a matching fresh lease and rejects stale, unknown, pre-fence or missing nodes. A lease table alone does not prove the absence of an unregistered process; target deployment inventory/provenance is also mandatory.

  • LegacyOutboxRelayControlAdapter owns composition of existing runtime controls: pause new OutboxRelayScheduler cycles, wait active cycles/claims to the finite deadline, pause/drain the epoch-fenced OutboxReaper, and close LegacyPublicationWriteFence so OutboxMessagePublishAdapter rejects every later send. Snapshot counts/generations are bounded facts only. Reopen methods require expected component generations plus fresh pre-commit recovery evidence and reject once ACTIVE epoch is not LEGACY_POLLING; no generic boolean setter exists. Application cutover policy sees the port, not concrete scheduler/reaper/messaging types.

  • The disposable security rehearsal and actual target preflight use distinct legacy/canonical principals. Revoke legacy exact-topic Write and require a negative Write probe while canonical Describe/DescribeConfigs/Write stays positive. The in-process fence plus external ACL evidence are both required; neither substitutes for the other.

  • Add the exact non-web one-shot operational entrypoint MessagingAuthorityCutoverApplicationRunner. It activates only for the closed operations legacy-to-polling-v2, recover-legacy-precommit, or resume-polling-v2-writes. It requires opaque operation/approval-evidence IDs plus expected target/source/artifact/epoch/fence generation, invokes only the corresponding application use case/coordinator, emits no payload/secret, and exits non-zero on mismatch/replay/failure. The main cutover exits 0 only after sentinel/readiness proof and write admission OPEN(generation+1); a post-commit resume failure stays FROZEN and requires the separately one-shot resume-polling-v2-writes operation. Consumed DB evidence/operation IDs make retries non-reentrant; no controller endpoint is added.

    ```text
    ca-skeleton.messaging.maintenance.operation
    ca-skeleton.messaging.maintenance.operation-id
    ca-skeleton.messaging.maintenance.approval-evidence-id
    ca-skeleton.messaging.maintenance.expected-target-alias
    ca-skeleton.messaging.maintenance.expected-source-digest
    ca-skeleton.messaging.maintenance.expected-artifact-digest
    ca-skeleton.messaging.maintenance.expected-epoch
    ca-skeleton.messaging.maintenance.expected-fence-generation
    ```
    
    Task 23 registers these exact maintenance-only keys and the runbook's non-web launcher
    contract; none has a default that enables the runner.
    
  • In the rehearsal harness: deploy ACK-aware producer/v2 relay scheduler-disabled; compile the candidate tuple; prove disposition auth/CAS negatives; execute the real PostgreSQL business-write admission freeze; drain active writers/epoch share holders; stop legacy new claims and legacy reaper; drain IN_FLIGHT to the maximum budget; audit remaining indeterminate; close/fence legacy producer Write and DB legacy mutation.

  • Real multi-node PostgreSQL tests hold old inWrite transactions across freeze, start new writers during/after freeze, inject a stale/pre-fence node lease and omit a deployment inventory member. Freeze must wait for old holders, reject new writes without partial business/ outbox state, and refuse evidence until every live instance/fence/relay/reaper/producer fact is exact and zero-active.

  • In one PostgreSQL transaction:

    ```text
    lock OUTBOX_PUBLICATION write-admission singleton FOR UPDATE
    -> lock ACTIVE LEGACY_POLLING epoch FOR UPDATE
    -> assert exact FROZEN generation/target binding and sole nonterminal attempt
    -> lock exact fresh cutover attempt
    -> CAS CUTOVER_PENDING -> FINALIZING_V2
    -> assert writer/legacy mutation fences
    -> capture fixed legacy handoff watermark
    -> final reconcile every row through watermark including final delta
    -> assert exactly one CURRENT delivery per event, active claims 0
    -> assert row count + event ID/hash manifest, unmapped/duplicate count 0
    -> switch ACTIVE epoch LEGACY_POLLING -> POLLING_V2
    -> append v2 cutover sentinel through the canonical append adapter
       with retained V3 projection + CURRENT/READY delivery
    -> mark the same attempt CONSUMED_V2
    -> commit
    ```
    
  • Missing, expired, reused, wrong-epoch/generation, wrong-manifest or non-zero cutover evidence rolls back before reconciliation. RECOVERING_LEGACY, RECOVERED_LEGACY, a foreign recovery owner or any non-CUTOVER_PENDING state also rejects finalization. The final transaction marks the evidence consumed; the coordinator cannot replay it.

  • Migration-only state mapping is exact:

    ```text
    PENDING   -> READY
    FAILED    -> RETRY_WAIT with reviewed DB-time due/budget
    DEAD      -> EXHAUSTED
    PUBLISHED -> LEGACY_RECORDED_UNVERIFIED
    IN_FLIGHT -> HOLD + remaining-indeterminate audit
    ```
    
    Preserve reviewed attempt count/due/deadline and every historical observation; never
    fabricate broker metadata, definite rejection, ACK observed time or `DELIVERY_RECORDED`.
    
  • Any unknown contract/status/hash mismatch, duplicate current row, count/manifest mismatch, active claim, fence failure or sentinel failure rolls the whole transaction back and leaves LEGACY_POLLING authoritative.

  • In the disposable rehearsal only, after commit start v2 relay, require sentinel ACK/DELIVERY_RECORDED, prove the sentinel used the canonical append path and retained V3 NOT NULL projection, verify legacy writer/claim/reaper/send 0, then execute the exact FROZEN→OPEN generation CAS. Only after OPEN, exercise an ordinary TransactionPort.inWrite canonical append; failure immediately freezes a new generation and fails the rehearsal.

  • RED/GREEN fault cases at every numbered point, including crash before transaction, after watermark, during final delta, before epoch switch, before/after sentinel insert and after commit. No case permits dual authority or missing manifest row.

  • Rehearse the two authority zones and both pre-commit choices:

    ```text
    pre-commit:
      keep all fences closed
      -> bounded forward retry while evidence/approval remains fresh
      OR
      prove epoch still LEGACY_POLLING + v2 business send/sentinel authority 0 + exact inventory
      -> DB-CAS exact attempt CUTOVER_PENDING -> RECOVERING_LEGACY
         and atomically invalidate it for every forward finalizer
      -> externally regrant legacy exact-topic Write and verify a fresh positive probe
      -> re-lock attempt + epoch and revalidate RECOVERING_LEGACY owner/lease + LEGACY_POLLING
         (on mismatch immediately revoke legacy Write and prove a fresh negative probe)
      -> append immutable recovery/ACL audit
      -> reopen in-process legacy Write fence, reaper and relay with expected generations
      -> CAS write admission FROZEN -> OPEN(generation+1) and mark RECOVERED_LEGACY last
    post-commit, regardless of whether a business v2 send occurred:
      legacy reactivation/reverse epoch is unsupported
      -> keep write admission FROZEN
      -> after sentinel/readiness/canonical-projection proof, CAS OPEN(generation+1)
      -> otherwise preserve backlog/schema/epoch/audit and forward-fix
    ```
    
    Once `RECOVERING_LEGACY` is claimed, no forward retry or separately created attempt can commit
    in `OUTBOX_PUBLICATION`, including after recovery lease expiry. Inject barrier races in both
    lock orders for same-attempt finalization versus recovery prepare, different-attempt creation/
    finalization versus recovery prepare/completion, plus crash/abort before and after the durable
    recovery claim, external ACL regrant, post-ACL epoch revalidation, each component reopen and
    final admission CAS.
    Wrong epoch/inventory/ACL evidence, stale generation, partial legacy reopen, duplicate
    operation, resume-before-sentinel and DB failure must never open business writes. If
    post-ACL revalidation fails, immediately revoke legacy Write and require a new negative probe;
    if a legacy component was reopened but final admission CAS failed, business writes stay
    FROZEN and the coordinator re-fences or records the exact safe degraded state for idempotent
    recovery-only retry.
    
  • Verify:

    ```bash
    cd src && ./gradlew :application-core:test \
      --tests '*FinalizeOutboxAuthorityCutoverUseCaseTest' \
      --tests '*ResumePollingV2WriteAdmissionUseCaseTest' \
      --tests '*RecoverLegacyOutboxAuthorityUseCaseTest' --console=plain
    cd src && ./gradlew :adapter:outbound:persistence-jpa:test \
      --tests '*OutboxAuthorityCutoverAdapterTest' \
      --tests '*OutboxWriteAdmissionControlAdapterTest' \
      --tests '*OutboxRuntimeNodeLeaseAdapterTest' \
      --tests '*OutboxPreCommitRecoveryAdapterTest' \
      --tests '*SpringTransactionPortTest' --console=plain
    cd src && ./gradlew :adapter:outbound:messaging:test \
      --tests '*LegacyPublicationWriteFenceTest' \
      --tests '*OutboxMessagePublishAdapterTest' --console=plain
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*OutboxLegacyToV2CutoverCoordinatorTest' \
      --tests '*OutboxLegacyPreCommitRecoveryCoordinatorTest' \
      --tests '*LegacyOutboxRelayControlAdapterTest' \
      --tests '*OutboxRuntimeNodeLeaseSchedulerTest' \
      --tests '*MessagingAuthorityCutoverApplicationRunnerTest' --console=plain
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*OutboxLegacyToV2CutoverContractTest' \
      --tests '*OutboxWriteAdmissionMultiNodeContractTest' \
      --tests '*OutboxV2SentinelContractTest' --console=plain
    ```
    
  • After the disposable RED/GREEN matrix passes, schema-validate and write:

    ```text
    src/app-bootstrap/build/messaging-evidence/cutover-rehearsal/manifest.json
    ```
    
    It binds the supplied source/artifact digest, migration/card/profile/catalog/settings hashes,
    disposable database/broker identity, every fault point and rollback-zone scenario, exact row/
    manifest/sentinel assertions, pre-commit recovery and post-commit resume generation-CAS
    scenarios, commands/timestamps, failed=0, skipped=0 and the explicit non-claim
    `targetDeploymentCutOver=false`. It conforms to the common build-evidence schema.
    
  • Acceptance claim: cutover implementation and disposable fault rehearsal candidate only. Target deployments remain LEGACY_POLLING; no legacy code/config is deleted and the tuple remains non-R2 until Wave E evidence.

Rollback checkpoint: discard the rehearsal database/broker. Never apply rehearsal evidence as a target deployment switch or destructively downgrade schema.

Wave D exit checkpoint

  • Run focused owner checks and:

    ```bash
    cd src && ./gradlew verifyMessagingContracts \
      verifyMessagingJsonSchemaV1 \
      verifyMessagingPollingOutboxR2 \
      verifyCleanArchitectureDependencies \
      verifyEnvKeys \
      verifyPublicPathSnapshot \
      --console=plain
    ```
    
  • Keep Kafka/security cards at most implemented-candidate.

  • Update the design ledger to P3=IMPLEMENTED_CANDIDATE only after the dark graph and disposable cutover rehearsal pass; target authority is still legacy.

  • Update the LLM Wiki branch-note and derived-document decision.


Wave E — P4 real-service, security, fault and release qualification

Task 21: Prove real PostgreSQL + real Kafka reference behavior

Owner: app-bootstrap qualification tests Depends on: Wave D

Files — create:

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingKafkaR2ContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingPollingKafkaEndToEndContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingKafkaFaultContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingKafkaContainerSupport.java
  • src/app-bootstrap/src/test/resources/messaging/evidence/messaging-evidence-schema-v1.json

Files — modify:

  • src/app-bootstrap/build.gradle

  • src/app-bootstrap/gradle.lockfile

  • src/build.gradle

  • Add test-only:

    ```groovy
    testImplementation 'org.testcontainers:testcontainers-kafka'
    testImplementation 'org.testcontainers:testcontainers-toxiproxy'
    testImplementation 'org.springframework.kafka:spring-kafka-test'
    ```
    
    Pin container image digest in qualification settings and record broker/client/Spring
    versions.
    
  • Register :app-bootstrap:messagingKafkaProducerR2 with these exact filters and failOnNoMatchingTests=true:

    ```text
    dev.caskeleton.bootstrap.integration.messaging.MessagingKafkaR2ContractTest
    dev.caskeleton.bootstrap.integration.messaging.MessagingPollingKafkaEndToEndContractTest
    dev.caskeleton.bootstrap.integration.messaging.MessagingKafkaFaultContractTest
    ```
    
    Root `verifyMessagingKafkaProducerR2` depends on that Test task and validates its evidence.
    Docker/image pull/test skip is failure, not PASS.
    
  • After registering the task but before implementing the three tests, run RED:

    ```bash
    cd src && ./gradlew :app-bootstrap:messagingKafkaProducerR2 --console=plain
    ```
    
    Expected non-zero: no matching required tests or absent real-service evidence. Any unrelated
    compile failure must be fixed before proceeding.
    
  • Real Kafka RED/GREEN cases: actual topic/partition/offset metadata; expected-topic mismatch; acks=all/idempotence effective config; stable key/partition; header/record oversize; missing topic with auto-create disabled; broker unavailable before send; leader/retriable failure; response loss/deadline/ late ACK; local buffer saturation/max-block; throttle; no per-message flush; graceful/forced close; fatal generation recreation.

  • Combined real PostgreSQL + Kafka cases: event/delivery commit; JIT claim/admission; ACK → delivery CAS; ACK-to-DB crash/reclaim duplicate; stale token after late ACK; outcome commit failure; late DB-commit-before-source-ACK duplicate absorption; backlog outage/recovery; multi-worker disjoint claim/order/fairness.

  • Add the rolling-compatibility golden path against the real broker: canonical append while LEGACY_POLLING is active → legacy claim → broker-observed exact compiled topic, stored key and byte-identical v1 envelope → legacy terminal PUBLISHED → disposable cutover maps LEGACY_RECORDED_UNVERIFIED with no automatic v2 resend. Nested envelope, legacy event-type routing for a canonical row and mixed metadata are negative cases.

  • Fault injection must observe both broker event IDs and DB state at:

    ```text
    after event commit
    after claim commit
    before send
    after request write
    after broker append before ACK receipt
    after ACK before DB transition
    during DB transition commit
    after DB success before scheduler result
    during shutdown
    ```
    
  • Single-node evidence is labelled provider baseline only. It cannot satisfy RF/min ISR, leader-loss or production security rows.

  • Generate a sanitized manifest at the non-versioned exact path:

    ```text
    src/app-bootstrap/build/messaging-evidence/real-kafka-postgresql-r2/manifest.json
    ```
    
    Validate it against
    `src/app-bootstrap/src/test/resources/messaging/evidence/messaging-evidence-schema-v1.json`.
    Also validate the same bytes against
    `src/config/messaging/evidence/build-evidence-manifest-v1.schema.json`; the lane schema may add
    fields but cannot weaken the common source/artifact/scenario/failure/skip contract.
    CI retains the same bytes under
    `ci-artifact://messaging/{sourceDigest}/real-kafka-postgresql-r2/manifest.json`. The manifest
    contains source/artifact digest supplied by human/CI, commands/timestamps,
    test counts, versions/image digests, non-secret effective settings, hashes, scenarios/results,
    skips/failures, unsupported claims and runbook IDs.
    
  • Re-run GREEN:

    ```bash
    cd src && ./gradlew :app-bootstrap:messagingKafkaProducerR2 \
      verifyMessagingKafkaProducerR2 \
      verifyMessagingPollingOutboxR2 --console=plain
    ```
    
    Expected: all listed scenario IDs occur exactly once, failed=0, skipped=0, schema validation
    PASS and source/artifact digests match.
    
  • Acceptance claim: real single-node Kafka + PostgreSQL R2-candidate evidence; production tuple remains NOT_QUALIFIED.

Rollback checkpoint: qualification uses disposable services. Production activation is still blocked by Task 22.

Task 22: Qualify SASL_SSL/SCRAM, least privilege and multi-broker topology

Owner: deployment/security qualification lane + app-bootstrap aggregator Depends on: Task 21

Files — create:

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/qualification/messaging/MessagingSecurityR2QualificationTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/qualification/messaging/MessagingMultiBrokerR2QualificationTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/qualification/messaging/MessagingRotationShutdownQualificationTest.java
  • src/app-bootstrap/src/test/resources/messaging/qualification/docker-compose.kafka-r2.yml
  • src/app-bootstrap/src/test/resources/messaging/qualification/README.md
  • src/config/messaging/evidence/messaging-release-evidence-schema-v1.json

Files — modify:

  • src/app-bootstrap/build.gradle

  • src/build.gradle

  • src/config/messaging/release-profile-assertions.yaml

  • Register three non-ordinary Test tasks with exact filters and failOnNoMatchingTests=true:

    ```text
    :app-bootstrap:messagingSecurityR2
      -> dev.caskeleton.bootstrap.qualification.messaging.MessagingSecurityR2QualificationTest
    :app-bootstrap:messagingMultiBrokerR2
      -> dev.caskeleton.bootstrap.qualification.messaging.MessagingMultiBrokerR2QualificationTest
    :app-bootstrap:messagingRotationShutdownR2
      -> dev.caskeleton.bootstrap.qualification.messaging.MessagingRotationShutdownQualificationTest
    ```
    
    Root `verifyMessagingSecurityR2` depends on all three evidence validators. Missing topology,
    credential fixture, certificate, Docker/image or tests fails the release task.
    
  • After task registration but before the qualification environment/tests are complete, run RED:

    ```bash
    cd src && ./gradlew :app-bootstrap:messagingSecurityR2 \
      :app-bootstrap:messagingMultiBrokerR2 \
      :app-bootstrap:messagingRotationShutdownR2 \
      --console=plain
    ```
    
    Expected non-zero for an exact missing test/topology/security prerequisite. SKIPPED is not an
    accepted RED or GREEN result.
    
  • Use a pinned three-broker topology with RF=3/min ISR=2 and production-like SASL_SSL/SCRAM-SHA-512. Ephemeral test credentials/certificates never enter source/evidence.

  • Security positive/negative cases: trusted TLS; untrusted CA; hostname mismatch; expired/not-yet-valid cert; valid/invalid SCRAM; missing/expired secret; production plaintext rejection; redaction; exact-topic Describe/ DescribeConfigs/Write; denied Create/Delete/Alter/other-topic Write/consumer Read. Use distinct canonical and legacy fixture principals and prove the legacy principal's exact-topic Write can be revoked without removing canonical Describe/DescribeConfigs/Write.

  • Topology cases: expected partitions/RF/min ISR; cleanup/retention/max bytes drift; wrong cluster/topic; auto-create disabled; leader loss with ISR sufficient; below-min-ISR rejection/indeterminate mapping; recovery; provisioning evidence freshness/provenance.

  • Provisioning evidence includes the selected broker's replica.lag.time.max.ms, the producer's effective request.timeout.ms and the approved compatibility relation from design §16.6. Add a mismatch negative case; do not infer the broker value from a client default.

  • Rotation/lifecycle cases: stop admission; bounded old drain; forced unresolved → durable INDETERMINATE/HOLD; old close; new secret/producer/attestation; generation barrier; no old/new overlap; DB-unavailable switch rejection; shutdown under load.

  • Capacity/soak cases: sustained drain, hot aggregate, broker throttle/outage/recovery storm, retry amplification, producer memory/buffer/GC, DB pool/claim query, metric cardinality and late-observation drop 0. Record numbers as selected-environment evidence, not universal repository performance claims.

  • Emit and schema-validate these non-versioned exact files:

    ```text
    src/app-bootstrap/build/messaging-evidence/security-r2/manifest.json
    src/app-bootstrap/build/messaging-evidence/multi-broker-r2/manifest.json
    src/app-bootstrap/build/messaging-evidence/rotation-shutdown-r2/manifest.json
    ```
    
    Validate each byte-identical file against both
    `src/config/messaging/evidence/build-evidence-manifest-v1.schema.json` and the stricter
    `src/config/messaging/evidence/messaging-release-evidence-schema-v1.json`. Add a contract test
    proving the lane schema retains every common required field and rejection rule.
    CI retains byte-identical artifacts under the source-digest-qualified
    `ci-artifact://messaging/` namespace. Each manifest must match exact source/artifact digest,
    `qualificationEnvironmentIdentity` (fixture broker/image/principal provenance), topic/security
    profile, card/settings/catalog/schema hashes, scenario set and freshness window. Any mismatch
    or skip keeps all affected cards `implemented-candidate`. This identity is never reused as a
    target `deploymentBindingIdentity`; only capability/profile, supported broker/client version
    constraints, settings/catalog/schema and scenario-contract revisions are portable.
    
  • Re-run GREEN:

    ```bash
    cd src && ./gradlew :app-bootstrap:messagingSecurityR2 \
      :app-bootstrap:messagingMultiBrokerR2 \
      :app-bootstrap:messagingRotationShutdownR2 \
      verifyMessagingSecurityR2 --console=plain
    ```
    
    Expected: every required scenario ID exactly once, failed=0, skipped=0, all three manifests
    pass `messaging-release-evidence-schema-v1.json`, and all source/artifact/profile digests
    match.
    
  • Acceptance claim: exact production security/topology candidate only after all required scenarios PASS. No consumer/CDC claim.

Rollback checkpoint: failed qualification prevents release promotion; do not weaken RF/min ISR, ACL, TLS or card requirements to make the lane green.

Task 23: Synchronize configuration, registries, runbooks and operational truth

Owner: repository documentation/configuration Depends on: Tasks 1922

Files — modify:

  • src/app-bootstrap/src/main/resources/application.yml
  • src/app-bootstrap/src/test/resources/application-test.yml
  • src/.env
  • docs/registries/env-keys.yaml
  • docs/registries/capabilities.yaml
  • docs/registries/error-codes.yaml
  • docs/registries/metrics.yaml
  • docs/registries/secrets-classification.yaml
  • src/application-core/README.md
  • src/application-core/CLAUDE.md
  • src/shared-contract/README.md
  • src/shared-contract/CLAUDE.md
  • src/adapter/outbound/messaging/README.md
  • src/adapter/outbound/messaging/CLAUDE.md
  • src/adapter/outbound/persistence-jpa/README.md
  • src/adapter/outbound/persistence-jpa/CLAUDE.md
  • src/adapter/inbound/web/README.md
  • src/adapter/inbound/web/CLAUDE.md
  • src/app-bootstrap/README.md
  • src/app-bootstrap/CLAUDE.md
  • docs/runbooks/outbox-publish-failed.md
  • docs/runbooks/outbox-dead-letter.md
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/RunbookCoverageContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/outbox/OutboxStatusRegistryContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/outbox/EventPayloadPiiContractTest.java

Files — create:

  • docs/runbooks/messaging-producer-unavailable-or-unauthorized.md

  • docs/runbooks/messaging-outbox-backlog-and-stale-lease.md

  • docs/runbooks/messaging-delivery-indeterminate-and-duplicate-burst.md

  • docs/runbooks/messaging-schema-poison-or-record-too-large.md

  • docs/runbooks/messaging-terminal-delivery-disposition.md

  • docs/runbooks/messaging-topic-policy-or-partition-change.md

  • docs/runbooks/messaging-shutdown-deploy-and-secret-rotation.md

  • docs/runbooks/messaging-legacy-to-v2-relay-authority-cutover.md

  • Modify the three listed contract tests first, then run RED:

    ```bash
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*RunbookCoverageContractTest' \
      --tests '*OutboxStatusRegistryContractTest' \
      --tests '*EventPayloadPiiContractTest' --console=plain
    ```
    
    Expected non-zero because new status/key/metric/error/runbook/redaction entries are absent.
    Register owner, type/default/allowlist, secret classification, validation, compatibility
    impact and required test for each implemented key.
    
  • Remove canonical reliance on:

    ```text
    APP_MESSAGING_BROKER
    APP_MESSAGING_KAFKA_BROKERS
    ca-skeleton.outbox.relay-enabled
    ```
    
    in the canonical graph. Retain them only as explicit R0 compatibility inputs through the
    deployment observation window; legacy + canonical keys fail with no silent precedence. Final
    removal belongs to Task 26. Do not add `consumer.enabled`, `cdc.enabled` or
    `schemaRegistry.url`.
    
  • Keep base skeleton default DISABLED with active contracts/destinations/resource count 0. Deployment-specific destination/topic/security values are explicit placeholders or secret references, never usable credentials.

  • Replace producer DEAD/dead-letter vocabulary with EXHAUSTED; distinguish it from future consumer DLT. Remove raw SQL status rewrite and fabricated consumer-dedupe claims from old runbooks.

  • Each first-R2 runbook contains detection, blast radius, guarantee degradation, safe first response, evidence, non-destructive mitigation, destructive approval boundary, reconciliation, recovery proof, rollback, audit and related cards/metrics/errors.

  • The authority-cutover runbook contains the exact pre-commit bounded-forward-retry and abort-to-legacy recovery state machine, the irreversible CUTOVER_PENDING -> RECOVERING_LEGACY claim before external mutation, recovery-only lease takeover, the OUTBOX_PUBLICATION sole-nonterminal-attempt constraint and global lock order, same-/cross-attempt finalizer rejection, external legacy ACL regrant/positive probe, post-ACL epoch revalidation and revoke/negative-probe compensation, expected generation ordering, partial-reopen re-fence behavior, and the post-commit resume-polling-v2-writes path. It explicitly forbids any post-commit legacy reactivation or raw admission/epoch SQL.

  • Document exact state/table/class names, token/generation/audit operator API, role readiness, no-dual-authority cutover and non-guarantees. No stub alert/dashboard references count as evidence.

  • Re-run the same three tests GREEN after registries/runbooks are complete, then verify global drift gates:

    ```bash
    cd src && ./gradlew verifyEnvKeys \
      verifyPublicPathSnapshot \
      :app-bootstrap:test \
      --tests '*RunbookCoverageContractTest' \
      --tests '*OutboxStatusRegistryContractTest' \
      --tests '*EventPayloadPiiContractTest' \
      --console=plain
    ```
    
    Expected: all three contract tests PASS, no skip, and env/public-path verification PASS.
    
  • Acceptance claim: documentation/configuration reflects actual implementation and evidence; unexecuted lanes remain NOT_QUALIFIED.

Rollback checkpoint: docs describe deployed/evidenced truth, not preferred state. Never rewrite failed evidence or instruct operators to dual-send/raw-update.

Task 24: Freeze and aggregate the pre-cutover release candidate

Owner: repository-wide verification and documentation Depends on: every selected Task 123 requirement

Files — modify:

  • docs/superpowers/specs/2026-07-28-messaging-production-capability-design.md
  • docs/superpowers/plans/2026-07-28-messaging-first-r2-polling-producer.md
  • src/config/messaging/readiness-cards.yaml
  • src/config/messaging/release-profile-assertions.yaml
  • src/build.gradle

Files — create:

  • src/config/messaging/evidence/deployment-rollout-manifest-v1.schema.json

  • src/config/messaging/evidence/deployment-binding-attestation-v1.schema.json

  • src/config/messaging/evidence/final-r2-profile-v1.schema.json

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingDeploymentRolloutEvidenceContractTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingFinalR2ProfileContractTest.java

  • Freeze the candidate source tree and dependency locks. A human supplies the candidate commit/source digest; CI builds the exact artifact. Agent never stages, commits or pushes.

  • Make verifyMessagingReleaseProfile consume these exact build outputs:

    ```text
    src/build/messaging-evidence/contracts-schema/manifest.json
    src/app-bootstrap/build/messaging-evidence/polling-outbox-r2/manifest.json
    src/app-bootstrap/build/messaging-evidence/cutover-rehearsal/manifest.json
    src/app-bootstrap/build/messaging-evidence/real-kafka-postgresql-r2/manifest.json
    src/app-bootstrap/build/messaging-evidence/security-r2/manifest.json
    src/app-bootstrap/build/messaging-evidence/multi-broker-r2/manifest.json
    src/app-bootstrap/build/messaging-evidence/rotation-shutdown-r2/manifest.json
    ```
    
    Each producer validates its schema before writing. The aggregator verifies all required
    scenario IDs, source/artifact digest, card/profile/catalog/schema/settings hashes, cluster/
    qualification-environment identity, freshness, failed=0 and skipped=0, then writes:
    
    ```text
    src/build/reports/messaging/release-profile/manifest.json
    ```
    
    CI retains the exact bytes under a source-digest-qualified
    `ci-artifact://messaging/` release-profile path.
    
  • Revalidate every one of the seven inputs against the common build-evidence schema and its lane-specific schema. Add contract fixtures proving a lane schema cannot omit or relax common source/artifact/scenario/failure/skip fields.

  • Implement fail-closed deployment/final gate contracts before any target cutover: verifyMessagingTargetBindingPreflight validates a non-mutating target preflight; verifyMessagingTargetBinding consumes the fresh target-specific topology/security/ACL attestation created inside maintenance after the legacy Write fence; verifyMessagingDeploymentCutover consumes that immutable original attestation plus the exact local target rollout manifest; verifyMessagingCleanupTargetBinding consumes a distinct cleanup-artifact attestation; verifyMessagingFinalR2Profile requires both attestations, the qualified cleanup release manifest, original deployment-cutover manifest and cleanup-rollout manifest, validates the final-profile schema and writes the final aggregate. Missing/stale/ wrong-target/wrong-digest/failed/skipped evidence is non-zero.

  • Run the gate contract tests RED then GREEN with checked-in invalid/valid payload-free fixtures:

    ```bash
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*MessagingDeploymentRolloutEvidenceContractTest' \
      --tests '*MessagingFinalR2ProfileContractTest' --console=plain
    ```
    
    RED is an intentionally invalid fixture accepted or a missing required validator; GREEN means
    every invalid fixture is rejected and every exact valid fixture is accepted. This does not
    create target rollout evidence.
    
  • Run focused owner gates sequentially:

    ```bash
    cd src && ./gradlew :application-core:check \
      :shared-contract:check \
      :adapter:outbound:messaging:check \
      :adapter:outbound:persistence-jpa:check \
      :adapter:inbound:web:check \
      :app-bootstrap:check \
      :sample-portfolio:check \
      --console=plain
    ```
    
  • Run Messaging gates:

    ```bash
    cd src && ./gradlew verifyMessagingContracts \
      verifyMessagingJsonSchemaV1 \
      verifyMessagingPollingOutboxR2 \
      verifyMessagingKafkaProducerR2 \
      verifyMessagingSecurityR2 \
      verifyMessagingReleaseProfile \
      --console=plain
    ```
    
    Expected: every exact input manifest exists and validates; no mismatch/stale/skip; aggregate
    manifest PASS. Missing service/credential/image/test is non-zero, never PASS.
    
  • Run repository gates:

    ```bash
    cd src && ./gradlew test --console=plain
    cd src && ./gradlew check --console=plain
    cd src && ./gradlew verifyDependencyLocks --console=plain
    cd src && ./gradlew verifyCleanArchitectureDependencies --console=plain
    cd src && ./gradlew verifyPublicPathSnapshot --console=plain
    cd src && ./gradlew verifyEnvKeys --console=plain
    cd src && ./gradlew verifyOneTypePerFile \
      verifyApplicationCoreDependencyPurity --console=plain
    git diff --check
    ```
    
  • Perform independent reviews for: Clean Architecture/module ownership; schema/contract evolution; transaction/concurrency/CAS; Kafka outcome/lifecycle; security/topology; operator endpoint; migration/cutover/rollback; evidence/no-skip/operations. Candidate qualification requires blocker 0 and high 0.

  • From the aggregate only, promote the exact artifact/card rows to release-eligible. Record §0 as P4=RELEASE_CANDIDATE_QUALIFIED_DEPLOYMENT_NOT_CUT_OVER; target publication authority and production runtime remain legacy. P5/P6 stay DESIGNED_NOT_IMPLEMENTED, P7 stays OPTIONAL_BACKLOG.

  • Update the branch-note with release-candidate evidence and explicitly state target cutover/cleanup are pending. Do not make the final implementation-complete claim.

Rollback checkpoint: if aggregation fails, keep cards implemented-candidate, target LEGACY_POLLING, and preserve schema/backlog/evidence. Never weaken a gate or copy evidence from another artifact.

Task 25: Execute the approved target deployment cutover

Owner: deployment coordinator + application/persistence cutover protocol Depends on: Tasks 2324, human deployment approval, exact fresh/empty-drained migration card Source change: none

  • Fail before maintenance unless the target matches the exact Task 24 source/artifact digest, release profile, schema/catalog/settings hashes, supported broker/client constraints and the checked-in cutover runbook. Do not require the target cluster/principal to equal Task 22's qualification fixture. A live non-empty V3 target stops for a separate approved deployment migration plan.

  • Against the actual target, resolve the exact canonical secret generation and run a non-mutating fresh topology/security preflight. Do not revoke the still-authoritative legacy principal before maintenance. Bind the prepared exact ACL change/provenance and emit identical sanitized bytes:

    ```text
    src/app-bootstrap/build/messaging-evidence/target-binding-preflight/manifest.json
    ci-artifact://messaging/{targetAlias}/{sourceDigest}/target-binding-preflight/manifest.json
    ```
    
    `deploymentBindingIdentity` binds target cluster/topic/canonical and legacy principal
    identities, canonical secret generation, provisioning provenance and prepared mutation. It
    proves canonical Describe/DescribeConfigs/Write and validates the target constraints, but
    explicitly records `legacyWriteRevoked=false`; it is not cutover evidence.
    
  • Run before maintenance:

    ```bash
    cd src && ./gradlew verifyMessagingTargetBindingPreflight --console=plain
    ```
    
    Expected GREEN only for a fresh exact target/source/artifact/release/profile binding with
    failed=0 and skipped=0. Fixture identity cannot satisfy this gate.
    
  • Before target execution, run the fail-closed rollout gate once with no current rollout artifact:

    ```bash
    cd src && ./gradlew verifyMessagingDeploymentCutover --console=plain
    ```
    
    Expected non-zero: the final in-maintenance target attestation and target rollout artifact are
    absent. A stale prior-target artifact must fail for target/source/release-digest mismatch, not
    satisfy this RED.
    
  • Execute the runbook preflight: deploy candidate with v2 relay scheduler-disabled; attest exact tuple; prove disposition auth/CAS; freeze durable business-write admission; drain writers/epoch holders; stop/drain legacy claim and reaper; record remaining IN_FLIGHT as indeterminate/HOLD; fence legacy producer Write and DB mutation. Only after zero-active drain, apply the prepared ACL mutation, require legacy exact-topic Write negative and canonical Describe/DescribeConfigs/Write positive, then write and internally schema-validate:

    ```text
    src/app-bootstrap/build/messaging-evidence/target-binding-attestation/manifest.json
    ci-artifact://messaging/{targetAlias}/{sourceDigest}/target-binding-attestation/manifest.json
    ```
    
    The one-shot precondition evidence binds this attestation digest. The maintenance runner uses
    the same fail-closed validator as `verifyMessagingTargetBinding` before it may call finalization;
    raw server, credential and certificate bytes are excluded.
    
  • Invoke the exact non-web MessagingAuthorityCutoverApplicationRunner operation legacy-to-polling-v2 with opaque operation/approval-evidence IDs and expected target/source/artifact/epoch. It calls the coordinator and FinalizeOutboxAuthorityCutoverUseCase; the single transaction repeats the Task 20 CUTOVER_PENDING -> FINALIZING_V2 CAS, watermark/final-delta/manifest/current-delivery assertions, epoch switch, sentinel insert and CONSUMED_V2 transition. RECOVERING_LEGACY is rejected before reconciliation. Any mismatch exits non-zero and rolls back to legacy authority. Reusing the operation or consumed evidence ID exits non-zero without mutation.

    ```text
    --spring.main.web-application-type=none
    --ca-skeleton.messaging.maintenance.operation=legacy-to-polling-v2
    --ca-skeleton.messaging.maintenance.operation-id={opaqueOperationId}
    --ca-skeleton.messaging.maintenance.approval-evidence-id={opaqueApprovalEvidenceId}
    --ca-skeleton.messaging.maintenance.expected-target-alias={targetAlias}
    --ca-skeleton.messaging.maintenance.expected-source-digest={sourceDigest}
    --ca-skeleton.messaging.maintenance.expected-artifact-digest={artifactDigest}
    --ca-skeleton.messaging.maintenance.expected-epoch={legacyEpoch}
    --ca-skeleton.messaging.maintenance.expected-fence-generation={openFenceGeneration}
    ```
    
  • If finalization fails before epoch commit, keep every fence closed. Either retry forward within the still-fresh bounded evidence/approval window, or execute the approved abort-to-legacy protocol. The latter first proves epoch still LEGACY_POLLING, v2 business send/sentinel authority 0 and exact inventory, then runs the prepare half of recover-legacy-precommit: DB-CAS the exact attempt CUTOVER_PENDING -> RECOVERING_LEGACY, bind the recovery operation/lease/evidence digest and atomically make every forward finalizer reject it. This choice is irreversible for that attempt, including after lease expiry. The prepare transaction holds the global write-admission/ACTIVE-epoch locks in order and proves the partial-unique-protected attempt is the sole nonterminal OUTBOX_PUBLICATION attempt. Only then may external provisioning regrant legacy exact-topic Write and emit a fresh positive probe. The completion half re-locks the global authority, exact attempt and epoch after that external mutation; owner/lease, uniqueness or LEGACY_POLLING mismatch immediately re-revokes legacy Write, proves a fresh negative probe and leaves writes FROZEN. On success it appends recovery/ACL audit, reopens the in-process legacy Write fence/reaper/ relay, then CAS-opens durable writes and marks RECOVERED_LEGACY last. Emit:

    ```text
    src/app-bootstrap/build/messaging-evidence/precommit-legacy-recovery/manifest.json
    ci-artifact://messaging/{targetAlias}/{sourceDigest}/precommit-legacy-recovery/manifest.json
    ```
    
    Validate identical bytes against the common and deployment-rollout schemas with
    `outcome=ABORTED_PRECOMMIT`, exact restored legacy/write-admission generations, recovery
    scenario IDs exactly once, failed=0 and skipped=0.
    Any partial failure leaves writes FROZEN and is retried/re-fenced; recovery lease takeover is
    recovery-only and never restores forward-finalization eligibility. Race tests must run
    recovery-prepare versus same- and different-attempt creation/finalization in both lock orders,
    and crash tests must cover every boundary before/after claim, ACL regrant, epoch revalidation,
    component reopen and final admission CAS. The protocol never uses raw SQL. An aborted attempt
    ends Task 25 without cutover; another attempt is rejected until recovery atomically records
    `RECOVERED_LEGACY` and opens a new fence generation, then requires fresh preflight, approval
    and target attestation.
    
  • After epoch commit, start only v2 relay while writes remain FROZEN. Require the canonical sentinel DELIVERY_RECORDED, retained V3 projection, fresh target binding/readiness and legacy writer/claim/reaper/send 0. Then ResumePollingV2WriteAdmissionUseCase CAS-opens OPEN(generation+1). If the original runner dies or resume fails after commit, resume-polling-v2-writes is the only recovery operation; it rechecks the same facts and cannot reactivate legacy. After OPEN, exercise an ordinary canonical append; failure freezes a new generation and keeps the rollout non-healthy.

  • Apply the rehearsed rollback zones exactly: pre-commit failure retains legacy DB authority but stays in maintenance until bounded forward retry or the full audited recovery above succeeds; after epoch commit, reverse epoch and legacy reactivation are unsupported even before the first business v2 send. Keep admission FROZEN, preserve backlog/schema/epoch/audit and forward-fix or run the guarded v2 resume.

  • Abort and fence admission on any duplicate anomaly, unexpected indeterminate, backlog/SLO breach, late-observation drop, stale legacy mutation, topic/security drift or readiness loss. Never automatically resend while diagnosing.

  • Emit a sanitized target rollout artifact:

    ```text
    src/app-bootstrap/build/messaging-evidence/deployment-cutover/manifest.json
    ci-artifact://messaging/{targetAlias}/{sourceDigest}/deployment-cutover/manifest.json
    ```
    
    It binds target environment/cluster alias, source/artifact/release manifest digest,
    target-binding-attestation digest, precondition evidence digest, watermark/manifest counts,
    epoch/sentinel facts, final OPEN fence generation, ordinary post-resume append probe,
    commands/timestamps, rollback zone, failures/skips and operator approvals. Payload, event hash,
    credential and raw IDs remain excluded. The target runner writes identical bytes to the local
    handoff and retained CI URI; both validate against the common build-evidence schema and
    `deployment-rollout-manifest-v1.schema.json`.
    
  • After commit, sentinel proof and artifact handoff, run verifyMessagingTargetBinding verifyMessagingDeploymentCutover GREEN. Expected: binding and deployment schemas PASS, exact target/source/artifact/release/attestation digest match, approval/epoch/sentinel scenario IDs present exactly once, failed=0 and skipped=0.

  • Acceptance claim: the exact target uses POLLING_V2 and the sentinel/reference path is healthy, and durable write admission is OPEN at the recorded post-cutover generation. Legacy source/runtime cleanup is still pending the observation/rollback window.

Rollback checkpoint: use only the two rehearsed zones. Schema is forward-only; dual relay authority and destructive downgrade are forbidden.

Task 26: Complete observation, remove legacy runtime and requalify the cleanup artifact

Owner: all affected leaves + repository-wide verification/Wiki Depends on: Task 25, reviewed observation window, human cleanup approval

Files — delete:

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/core/MessageBroker.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSender.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaMessageBroker.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaAdapterConfig.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaAdapterSettings.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/DisabledOutboxMessagePublisher.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFence.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxEnvelopeJson.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapter.java
  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/Slf4jOutboxRelayFailureReportAdapter.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfigTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFenceTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java
  • src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/Slf4jOutboxRelayFailureReportAdapterTest.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxMessagePublishPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelayControlPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelaySnapshot.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionEvidence.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverCommand.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverResult.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAuthorityCutoverPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCase.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxPreCommitRecoveryPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityCommand.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityResult.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCase.java
  • src/application-core/src/test/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCaseTest.java
  • src/application-core/src/test/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCaseTest.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxStorePort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxEvent.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxEventStatus.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayFailureReport.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayFailureReportPort.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayResult.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsCommand.java
  • src/application-core/src/main/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCase.java
  • src/application-core/src/test/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCaseTest.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapter.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxCutoverPreconditionAdapter.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapter.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapter.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapterTest.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapterTest.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxClaimRepository.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxClaimRepository.java
  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaper.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapterTest.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaperTest.java
  • src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaperWiringTest.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLeaderElectionToken.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinator.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinator.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapter.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverJobSettings.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunner.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinatorTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinatorTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapterTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunnerTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyToV2CutoverContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxAppendTransactionalContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxPublisherLeaderElectionContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxRowLifecycleContractTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/support/conditional/EnabledIfMessagingBrokerConfigured.java

Files — create:

  • src/config/messaging/legacy-runtime-denylist.txt
  • src/config/messaging/legacy-runtime-allowlist.txt
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingLegacyRuntimeDenylistTest.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingLegacyActivationRunbookContractTest.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingWriteAdmissionRecoverySettings.java
  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingWriteAdmissionRecoveryApplicationRunner.java
  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/MessagingWriteAdmissionRecoveryApplicationRunnerTest.java

Files — modify:

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxEventJpaRepository.java

  • src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlPersistenceConfig.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfig.java

  • src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingSettings.java

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxMetrics.java

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRelayScheduler.java

  • src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfig.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxConfigTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxSettingsTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfigTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2SentinelContractTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/adapter/outbound/OptionalAdapterBeanGatingTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/ArchitectureViolationFixtureTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/violations/application/OutboundWithoutPermissionUseCase.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/OptionalAdapterConditionalExecutionContractTest.java

  • src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/outbox/OutboxStatusRegistryContractTest.java

  • src/app-bootstrap/src/main/resources/application.yml

  • src/app-bootstrap/src/test/resources/application-test.yml

  • src/sample-portfolio/src/main/resources/application.yml

  • src/sample-portfolio/src/test/resources/application-test.yml

  • src/.env

  • src/config/messaging/readiness-cards.yaml

  • src/config/messaging/profile-compatibility.yaml

  • src/config/messaging/release-profile-assertions.yaml

  • docs/registries/env-keys.yaml

  • docs/registries/capabilities.yaml

  • docs/registries/error-codes.yaml

  • docs/registries/metrics.yaml

  • docs/registries/secrets-classification.yaml

  • docs/runbooks/outbox-publish-failed.md

  • docs/runbooks/outbox-dead-letter.md

  • docs/runbooks/messaging-producer-unavailable-or-unauthorized.md

  • docs/runbooks/messaging-outbox-backlog-and-stale-lease.md

  • docs/runbooks/messaging-delivery-indeterminate-and-duplicate-burst.md

  • docs/runbooks/messaging-schema-poison-or-record-too-large.md

  • docs/runbooks/messaging-terminal-delivery-disposition.md

  • docs/runbooks/messaging-topic-policy-or-partition-change.md

  • docs/runbooks/messaging-shutdown-deploy-and-secret-rotation.md

  • docs/runbooks/messaging-legacy-to-v2-relay-authority-cutover.md

  • src/README.md

  • src/application-core/README.md

  • src/application-core/CLAUDE.md

  • src/shared-contract/README.md

  • src/shared-contract/CLAUDE.md

  • src/adapter/outbound/messaging/README.md

  • src/adapter/outbound/messaging/CLAUDE.md

  • src/adapter/outbound/persistence-jpa/README.md

  • src/adapter/outbound/persistence-jpa/CLAUDE.md

  • src/adapter/inbound/web/README.md

  • src/adapter/inbound/web/CLAUDE.md

  • src/app-bootstrap/README.md

  • src/app-bootstrap/CLAUDE.md

  • src/sample-portfolio/README.md

  • src/sample-portfolio/CLAUDE.md

  • src/app-bootstrap/build.gradle

  • src/build.gradle

  • docs/superpowers/specs/2026-07-28-messaging-production-capability-design.md

  • docs/superpowers/plans/2026-07-28-messaging-first-r2-polling-producer.md

  • /home/donghyeon/workspace/ai-tool/llm-wiki-private/raw/branch-notes/main.md and only genuinely derived raw documents

  • Observation exit requires the reviewed duration with duplicate anomaly 0, unexpected indeterminate 0, late-drop 0, stable backlog/readiness, no legacy mutation and successful disposition/rotation/shutdown drills. Durable write admission must remain OPEN at the recorded post-cutover generation except for audited drills with successful guarded resume. Any breach postpones cleanup.

  • Write MessagingLegacyRuntimeDenylistTest first and run RED; it must find every exact production symbol/config key above plus legacy reaper repository methods/beans. The denylist contains exact FQCNs and keys, including:

    ```text
    dev.caskeleton.adapter.outbound.messaging.core.MessageBroker
    dev.caskeleton.adapter.outbound.messaging.kafka.KafkaSender
    dev.caskeleton.adapter.outbound.messaging.kafka.KafkaMessageBroker
    dev.caskeleton.adapter.outbound.messaging.kafka.KafkaAdapterConfig
    dev.caskeleton.adapter.outbound.messaging.kafka.KafkaAdapterSettings
    dev.caskeleton.adapter.outbound.messaging.outbox.DisabledOutboxMessagePublisher
    dev.caskeleton.adapter.outbound.messaging.outbox.LegacyPublicationWriteFence
    dev.caskeleton.adapter.outbound.messaging.outbox.OutboxEnvelopeJson
    dev.caskeleton.adapter.outbound.messaging.outbox.OutboxMessagePublishAdapter
    dev.caskeleton.adapter.outbound.messaging.outbox.Slf4jOutboxRelayFailureReportAdapter
    dev.caskeleton.application.outbox.OutboxMessagePublishPort
    dev.caskeleton.application.outbox.LegacyOutboxRelayControlPort
    dev.caskeleton.application.outbox.LegacyOutboxRelaySnapshot
    dev.caskeleton.application.outbox.OutboxCutoverPreconditionEvidence
    dev.caskeleton.application.outbox.OutboxCutoverPreconditionPort
    dev.caskeleton.application.outbox.FinalizeOutboxAuthorityCutoverCommand
    dev.caskeleton.application.outbox.FinalizeOutboxAuthorityCutoverResult
    dev.caskeleton.application.outbox.OutboxAuthorityCutoverPort
    dev.caskeleton.application.outbox.FinalizeOutboxAuthorityCutoverUseCase
    dev.caskeleton.application.outbox.OutboxPreCommitRecoveryPort
    dev.caskeleton.application.outbox.RecoverLegacyOutboxAuthorityCommand
    dev.caskeleton.application.outbox.RecoverLegacyOutboxAuthorityResult
    dev.caskeleton.application.outbox.RecoverLegacyOutboxAuthorityUseCase
    dev.caskeleton.application.outbox.OutboxStorePort
    dev.caskeleton.application.outbox.OutboxEvent
    dev.caskeleton.application.outbox.OutboxEventStatus
    dev.caskeleton.application.outbox.OutboxRelayFailureReport
    dev.caskeleton.application.outbox.OutboxRelayFailureReportPort
    dev.caskeleton.application.outbox.OutboxRelayResult
    dev.caskeleton.application.outbox.PublishPendingOutboxEventsCommand
    dev.caskeleton.application.outbox.PublishPendingOutboxEventsUseCase
    dev.caskeleton.adapter.outbound.persistence.outbox.OutboxStoreAdapter
    dev.caskeleton.adapter.outbound.persistence.outbox.OutboxCutoverPreconditionAdapter
    dev.caskeleton.adapter.outbound.persistence.outbox.OutboxAuthorityCutoverAdapter
    dev.caskeleton.adapter.outbound.persistence.outbox.OutboxPreCommitRecoveryAdapter
    dev.caskeleton.adapter.outbound.persistence.outbox.OutboxClaimRepository
    dev.caskeleton.adapter.outbound.persistence.postgresql.PostgreSqlOutboxClaimRepository
    dev.caskeleton.adapter.outbound.persistence.outbox.OutboxReaper
    dev.caskeleton.bootstrap.outbox.OutboxLeaderElectionToken
    dev.caskeleton.bootstrap.outbox.OutboxLegacyToV2CutoverCoordinator
    dev.caskeleton.bootstrap.outbox.OutboxLegacyPreCommitRecoveryCoordinator
    dev.caskeleton.bootstrap.outbox.LegacyOutboxRelayControlAdapter
    dev.caskeleton.bootstrap.outbox.MessagingAuthorityCutoverJobSettings
    dev.caskeleton.bootstrap.outbox.MessagingAuthorityCutoverApplicationRunner
    OutboxEventJpaRepository.deletePublishedBefore
    OutboxEventJpaRepository.countGroupedByStatus
    OutboxEventJpaRepository.findOldestUnpublishedOccurredAtByEventType
    OutboxClaimRepository.claimEligible
    outboxLeaderElection
    outboxReaper
    app.messaging.broker
    app.messaging.kafka.brokers
    APP_MESSAGING_BROKER
    APP_MESSAGING_KAFKA_BROKERS
    ca-skeleton.outbox.relay-enabled
    ca-skeleton.messaging.maintenance.operation
    ca-skeleton.messaging.maintenance.operation-id
    ca-skeleton.messaging.maintenance.approval-evidence-id
    ca-skeleton.messaging.maintenance.expected-target-alias
    ca-skeleton.messaging.maintenance.expected-source-digest
    ca-skeleton.messaging.maintenance.expected-artifact-digest
    ca-skeleton.messaging.maintenance.expected-epoch
    ca-skeleton.messaging.maintenance.expected-fence-generation
    ```
    
    This is the complete mandatory production token/key set derived one-to-one from `Files —
    delete` plus the legacy repository methods, bean names and configuration keys. The contract
    test snapshots the exact eight-key Task 20 maintenance settings/registry set and asserts set
    equality with these eight denylist keys before scanning source/config/runbook references. It
    rejects a missing or extra maintenance key, a missing denylist entry, an unclassified deleted
    production class, and any allowlist entry outside the exact sample/R0 list below.
    
    The allowlist contains only these repository-relative paths:
    
    ```text
    src/application-core/src/main/java/dev/caskeleton/application/outbox/NewOutboxEvent.java
    src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxAppendPort.java
    src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapter.java
    src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapterTest.java
    src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisher.java
    src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogUseCase.java
    src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisherTest.java
    src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogOutboxTest.java
    src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/WorkLogUseCasesTest.java
    src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/authz/WorkLogAuthorizationContractTest.java
    ```
    
    No directory wildcard or silently ignored unknown path is allowed.
    
  • Run the denylist RED before deletion:

    ```bash
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*MessagingLegacyRuntimeDenylistTest' --console=plain
    ```
    
    Expected non-zero with every still-present forbidden FQCN/key/bean reported. Missing scan
    roots or a test skip is not an accepted RED.
    
  • Delete the listed runtime/relay/reaper sources and obsolete config keys. Retain NewOutboxEvent, LegacyOutboxAppendPort and a separately named sample-only LegacyOutboxAppendAdapter only as the documented R0 fixture until standalone sample activation. Canonical ACTIVE context must prove legacy append/store/publish/relay/reaper bean count 0. Replace the deleted cutover runner with the disabled-by-default MessagingWriteAdmissionRecoveryApplicationRunner, which accepts only resume-polling-v2-writes, imports no legacy/cutover type, and uses ResumePollingV2WriteAdmissionUseCase with expected FROZEN generation, ACTIVE POLLING_V2 epoch and fresh sentinel/readiness evidence.

    ```text
    ca-skeleton.messaging.write-admission-recovery.operation
    ca-skeleton.messaging.write-admission-recovery.operation-id
    ca-skeleton.messaging.write-admission-recovery.approval-evidence-id
    ca-skeleton.messaging.write-admission-recovery.expected-target-alias
    ca-skeleton.messaging.write-admission-recovery.expected-epoch
    ca-skeleton.messaging.write-admission-recovery.expected-fence-generation
    ```
    
    No operation/default means no runner resource. Unknown or legacy operation names fail before
    mutation; replay/mismatch and resume-before-sentinel remain non-zero.
    
  • Keep retained V3 DB columns and canonical compatibility projection until a later forward schema cleanup. Do not drop columns or rewrite history here.

  • Re-run the runtime denylist GREEN across production Java, build files, YAML/env and registries; only the exact sample/R0 source allowlist may remain. Explicitly exclude design/spec/plan/Wiki history from raw-symbol matching: historical documentation is evidence, not an activation surface. Compile success alone is not bean/resource absence evidence. Run the same focused command GREEN; expect forbidden runtime match 0, canonical legacy bean/resource count 0 and every allowlisted sample/R0 reference classified exactly.

  • Run MessagingLegacyActivationRunbookContractTest RED before cleanup and GREEN after cleanup. It scans operational runbooks semantically for executable legacy activation keys, commands, dual-relay instructions or rollback-to-legacy actions, while allowing clearly labelled historical facts and “must remain disabled/forbidden” statements. It does not raw-match this plan/spec/branch-note.

    ```bash
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*MessagingLegacyActivationRunbookContractTest' --console=plain
    ```
    
  • Because cleanup changes source/artifact digest, freeze a new human candidate and rerun the Task 21 and 22 qualification lanes plus the Task 24 focused/Messaging/repository commands and release-profile aggregator against the cleanup artifact. Do not replay Task 24's pre-cutover status wording: production is already POLLING_V2. Pre-cleanup evidence is stale and cannot qualify the cleanup artifact.

  • Before deploying the cleanup artifact, run:

    ```bash
    cd src && ./gradlew verifyMessagingFinalR2Profile --console=plain
    ```
    
    Expected non-zero because the qualified cleanup release manifest and cleanup rollout artifact
    do not yet form a matching final chain.
    
  • Deploy the requalified cleanup artifact without changing the already-active POLLING_V2 epoch. Before deployment, rerun target topology/security/ACL probes for the cleanup source/ artifact and emit:

    ```text
    src/app-bootstrap/build/messaging-evidence/cleanup-target-binding-attestation/manifest.json
    ci-artifact://messaging/{targetAlias}/{cleanupSourceDigest}/cleanup-target-binding-attestation/manifest.json
    ```
    
    Run `verifyMessagingCleanupTargetBinding` GREEN; it must bind the same target identity and
    current secret generation to the cleanup release digest without overwriting the immutable
    original Task 25 attestation. Then deploy and emit byte-identical sanitized evidence:
    
    ```text
    src/app-bootstrap/build/messaging-evidence/cleanup-rollout/manifest.json
    ci-artifact://messaging/{targetAlias}/{cleanupSourceDigest}/cleanup-rollout/manifest.json
    ```
    
    It binds the target alias, cleanup source/artifact digest, qualified cleanup release-manifest
    digest, cleanup-target-attestation digest, original deployment-cutover manifest digest,
    unchanged epoch, recorded OPEN write-admission generation, sentinel/backlog/readiness/
    legacy-bean-0 facts, approval,
    commands/timestamps, failed=0 and skipped=0. Validate it against the common and
    deployment-rollout schemas.
    
  • Re-run verifyMessagingDeploymentCutover, verifyMessagingCleanupTargetBinding and verifyMessagingFinalR2Profile GREEN:

    ```bash
    cd src && ./gradlew verifyMessagingTargetBinding \
      verifyMessagingDeploymentCutover \
      verifyMessagingCleanupTargetBinding \
      verifyMessagingFinalR2Profile --console=plain
    ```
    
    The final task consumes these exact local inputs:
    
    ```text
    src/build/reports/messaging/release-profile/manifest.json
    src/app-bootstrap/build/messaging-evidence/target-binding-attestation/manifest.json
    src/app-bootstrap/build/messaging-evidence/deployment-cutover/manifest.json
    src/app-bootstrap/build/messaging-evidence/cleanup-target-binding-attestation/manifest.json
    src/app-bootstrap/build/messaging-evidence/cleanup-rollout/manifest.json
    ```
    
    If qualification/build cleanup removed a Task 25 local handoff, restore only the byte-identical
    retained CI artifact to its exact path after verifying its recorded digest/signature and target
    provenance. Never synthesize, edit or substitute a current artifact for the immutable original.
    The final task writes:
    
    ```text
    src/build/reports/messaging/final-r2-profile/manifest.json
    ```
    
    Expected: exact target/cleanup source/artifact/release digest chain, approval/epoch/sentinel/
    write-admission-OPEN/legacy-zero scenario IDs exactly once, failed=0, skipped=0 and schema
    PASS. CI retains the
    final bytes under the target/cleanup-source-qualified namespace.
    
  • Run final focused, Messaging and repository gates exactly as Task 24 plus:

    ```bash
    cd src && ./gradlew :app-bootstrap:test \
      --tests '*MessagingLegacyRuntimeDenylistTest' \
      --tests '*MessagingWriteAdmissionRecoveryApplicationRunnerTest' \
      --tests '*MessagingDisabledZeroResourceContractTest' --console=plain
    ```
    
  • Update §0 and card status from the final R2 aggregate only. First R2 may be marked complete only after verifyMessagingFinalR2Profile passes; P5/P6/P7 remain unchanged.

  • Before final Wiki capture, read the configured vault's AGENTS.md, CLAUDE.md and relevant rules/, .agents/, .claude/, .codex/ instructions. Resolve the branch with git branch --show-current; for this plan's current main branch the canonical target is:

    ```text
    /home/donghyeon/workspace/ai-tool/llm-wiki-private/raw/branch-notes/main.md
    ```
    
    Record implementation, files, decisions, exact commands/results/failures, release and rollout
    evidence, unsupported claims and remaining risks. Add/link derived raw documents only when
    honestly produced; otherwise record “없음”. Canonical vault unavailability is an explicit
    completion blocker.
    
  • Final handoff lists changed files, behavior, exact verification counts/results, failed/not-run lanes, release/rollout fingerprints, Wiki capture and follow-up risks.

Rollback checkpoint: after cleanup, do not restore legacy code for events already sent by v2. Pause admission, preserve schema/backlog/epoch/audit and forward-fix.


8. Task dependency graph

1 -> 2
     |
     3 -> 4 -> 5 -> 6
                    |
                    7 -> 8 -> 9 -> 10 -> 11 -> 12 -> 13
                                                  |
                                                  14 -> 15 -> 16 -> 17
                                                        |           |
                                                        +-------> 18
                                                                    |
                                                                    19 -> 20
                                                                          |
                                                                          21 -> 22 -> 23 -> 24
                                                                                               |
                                                                                               25 -> 26

Parallel work is allowed only at non-overlapping stable boundaries:

  • Task 4 shared/sample resource work may run in parallel after Task 3, but Task 5 starts only after both resource owners are stable.
  • Persistence Tasks 713 are sequential because they share migration/entity/repository/CAS surfaces.
  • Task 18 may start after Task 12 while Tasks 1517 proceed, but Task 19 waits for both.
  • Disposable cutover rehearsal Task 20 is never parallelized with producer, persistence or configuration changes.
  • Real Kafka Task 21 and security topology setup for Task 22 may prepare in parallel only after the final canonical artifact is frozen; their evidence aggregation remains ordered.
  • Target cutover Task 25 is a serialized deployment operation after Task 24 qualification. Task 26 cleanup starts only after the reviewed observation window and requires a newly qualified artifact.
  • Shared-worktree Gradle invocations remain sequential even when source subtasks are delegated.

9. Minimum completion matrix

Requirement Proving task
approved truth/no ACK overclaim 1
closed first-tuple registry/no future switches 2, 24
framework-free typed event/SPI 3
generic envelope + sample-owned payload schema 4
closed catalog/destination/digest/key 5
Draft 2020-12 deterministic bytes/admission 6
forward-only immutable event/delivery/journal/epoch 7
same-transaction validated append 8
exhaustive outcome + one-record relay 9
JIT claim/token/unexpired-lease CAS 10
late observation DB commit before source ACK 11
audited requeue/hold/skip/compensate 12
legacy/v2 mutual exclusion 13
finite typed config/disabled 0 14
actual ACK-aware Spring Kafka gateway 15
admission/late queue/generation lifecycle 16
topic/security attestation 17
authenticated internal disposition endpoint 18
composition/readiness/observability 19
atomic watermark/reconcile/epoch/sentinel cutover implementation + disposable rehearsal 20
real PostgreSQL + real Kafka fault evidence 21
TLS/SASL/ACL + multi-broker RF/min ISR 22
env/registries/runbooks 23
no-skip pre-cutover release artifact + independent review 24
exact-target approved deployment cutover evidence 25
observation exit + legacy cleanup + cleanup-artifact requalification + final Wiki 26

10. Follow-up plans after first R2

다음은 이 계획을 확장하는 checkbox가 아니라 별도 설계 승인과 실행 계획이다.

  1. Inbound Kafka + inbox + DLT/replay (P5)
    • 20번째 adapter:inbound:messaging-kafka leaf registry migration;
    • manual ACK after application commit;
    • inbox/effect identity, bounded retry, DLT ACK, replay/audit.
  2. PostgreSQL Debezium CDC (P6)
    • external Connect/Debezium asset, publication/slot/offset/WAL;
    • insert-only mapping, shadow, authority-exclusive cutover/rollback and retention proof.
  3. Optional cards (P7)
    • Avro/Protobuf registry, Kafka EOS, retry topic, compaction, object-storage claim check, alternate broker, multi-cluster, module split;
    • each requirement gets an independent card/compatibility/evidence plan.
  4. Live non-empty legacy database migration
    • collect actual row volume/state distribution, lock/replication budget, data classification, maintenance window and rollback evidence;
    • separately approve either LIVE_ADDITIVE_BACKFILL_IN_PLACE.v1 or COPY_AND_CUTOVER_WITH_RECONCILIATION.v1.

11. Final non-negotiable assertions

  • KafkaSender.send() return is not broker ACK.
  • Kafka future success plus metadata is ACK observation; it is not consumer processing.
  • acks=all without RF/min ISR/unclean-election evidence is not durable topology evidence.
  • Kafka producer idempotence does not remove ACK-to-DB, restart, late-ACK or operator-requeue duplicates.
  • EXHAUSTED does not mean definitely not delivered.
  • late ACK observation never rewrites authoritative delivery state.
  • timestamp/random event ID is not aggregate total order.
  • outbox_event wire bytes are immutable authority; JSONB/re-serialization is not.
  • polling and CDC may never be simultaneous production dispatch authorities.
  • operator mutation requires auth, expected generation/version, no active claim and immutable audit.
  • disabled and configured-but-not-ready are different states.
  • fake/single-node/local evidence cannot be relabelled as production R2.
  • implementation completion requires exact commands/results and LLM Wiki capture.
  • agent never stages, commits, amends or pushes.