- CI 단계 분리 계획 추가 (docs/superpowers/plans/2026-09-16-ci-stage-separation.md). 빌드·CI 레이어 전수 리뷰 133건의 결론과 Track A/B/C 작업 순서를 담는다. - public-path 보안 기준선을 실제 배포 기본값(/v1/healthcheck)으로 재생성. 이전 값은 gitignore 된 src/.env 에서 유래해 재현이 불가능했다. - 진행 중이던 ADR·리뷰·테스트 전략 문서 반영, 대체된 grpc 계획 문서 제거. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
187 KiB
Messaging First R2 Polling Producer Implementation Plan
For agentic workers: REQUIRED SUB-SKILL: Use
superpowers:subagent-driven-development(recommended) orsuperpowers:executing-plansto implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking. Behavior changes also requiresuperpowers:test-driven-development; completion claims requiresuperpowers:verification-before-completionand an independentsuperpowers:requesting-code-review.
Goal: Build one production-reference Messaging path from a typed integration event through a same-transaction PostgreSQL polling outbox to an acknowledgement-aware Spring Kafka producer, with an authenticated disposition control and exact R2 evidence.
Architecture: application-core owns provider-neutral event/publication/disposition semantics;
adapter:outbound:messaging owns deterministic JSON/schema compilation and Kafka; PostgreSQL
persistence owns event/delivery/audit rows and token/lease CAS; inbound web owns only operator HTTP
mapping; bootstrap composes the exact tuple, readiness and schedulers. The first path is polling-only
and keeps consumer, inbox, DLT, replay and CDC disabled.
Tech Stack: Java 21, Spring Boot 4.0.0, Spring Kafka 4.0 through the Boot BOM, Jackson 3,
com.networknt:json-schema-validator:3.0.2, PostgreSQL, Flyway, JPA, Gradle, JUnit 5, AssertJ,
Testcontainers Kafka/PostgreSQL, Micrometer.
- 작성일: 2026-07-28
- 상태: 실행 계획 작성·독립 검토 완료, 모든 task 미착수
- 설계 정본: Messaging Production Capability Deep Design
- 구현 범위: P0–P4의 first R2 polling producer tuple
- 명시적 비범위: inbound Kafka consumer, inbox, retry topic, DLT/replay, Kafka EOS, Debezium/Kafka Connect CDC, Avro/Protobuf/schema registry, alternate broker, multi-cluster
- 비교한 계획: Redis Foundation, Redis Runtime, HTTP Client Foundation, HTTP Client Total Deadline, Fileserver Foundation, Fileserver Durable Recovery, Notification
Repository commit policy는 모든 플랫폼에서 human-only다. 이 계획에는 git add, git commit,
git amend, git push 단계가 없다. 구현자는 작업 결과와 검증 증거만 전달하고 candidate
commit은 사람이 만든다.
1. Exact selected tuple and non-guarantees
첫 구현과 qualification 대상은 다음 tuple 하나다.
messaging-outbox-publish.v1
+ kafka-spring-acknowledged-idempotent.v1
+ postgresql-polling-outbox.v2
+ postgresql-per-record-jit-claim.v1
+ json-schema-envelope.v1
+ external-topic-validated.v1
+ kafka-sasl-ssl-scram-sha-512.v1
+ kafka-compression-none.v1
+ per-key-normal-path-sequence-detectable.v1
+ same-postgresql-transaction-resource.v1
+ authenticated-internal-web-disposition.v1
이 계획이 완료돼도 다음은 주장하지 않는다.
- broker와 PostgreSQL 사이 exactly-once;
- consumer effect의 deduplication 또는 inbox 보장;
- global FIFO, failure/rotation/requeue 뒤 strict FIFO;
- single-node Kafka test만으로 multi-broker RF/min ISR 내구성;
- CDC-ready, DLT-ready, replay-ready;
- local plaintext profile을 production security profile로 승격;
ACKNOWLEDGED가 consumer 처리 또는 business effect 완료를 뜻함.
2. Target flow and fixed decisions
feature mapper
-> IntegrationEventDraft<typed record>
-> IntegrationEventEncoderPort
-> ValidatedIntegrationEvent(exact UTF-8 bytes + hashes)
-> TransactionPort.inWrite(
business state
+ immutable outbox_event
+ CURRENT/READY outbox_delivery
)
Outbox relay invocation
-> acquire one bounded local admission permit
-> Tx B: one-row JIT claim + token/DB-time lease + ATTEMPT_ADMITTED
-> no DB transaction: Kafka send + future ACK wait
-> Tx C: outcome observation + valid-lease/token CAS state transition
-> release permit
late Kafka callback
-> bounded payload-free observation source
-> application drain
-> DB commit
-> source ACK
authenticated internal endpoint
-> inbound DTO/principal mapping
-> ApplyOutboxDispositionUseCase
-> permission/policy
-> PostgreSQL CAS + immutable audit
고정 결정:
src/config/architecture/modules.json이 leaf와 production project edge의 유일한 SSOT다. first R2 production 구현에는 새 leaf나 project edge가 필요 없다.sample-portfolio -> adapter-outbound-messagingedge는 standalone sample을 실제 ACTIVE producer로 바꾸는 별도 승인 작업 전에는 추가하지 않는다.- application/domain/shared Java API에는 Kafka, Jackson, JSON validator, Spring, JPA 타입을 노출하지 않는다.
- physical topic은 application contract가 아니라 outbound destination binding이다.
- exact UTF-8
BYTEA가 wire authority다. retry에서 payload를 다시 직렬화하지 않는다. outbox_event는 immutable event,outbox_delivery는 mutable delivery control이다.- claim/outcome/renew는 opaque token, owner, CURRENT generation, expected version,
claim_until > database_now를 모두 확인한다. - local admission을 확보한 뒤 한 record만 JIT claim한다. initial profile의 admitted record upper bound는 1이다.
- broker call은 DB transaction 밖에서 수행한다.
ACKNOWLEDGED,ACKNOWLEDGED_MISMATCH,REJECTED,INDETERMINATE는 exhaustive outcome이다.- acceptance certainty와 retry disposition은 독립 축이다.
- deadline 뒤 late ACK는 기존 outcome/state를 뒤집지 않고 append-only observation만 제안한다.
- operator requeue는 기존 row를 READY로 덮지 않고 이전 authority를 supersede한 뒤 새 delivery generation을 만든다.
- requeue generation deadline은
min(generation.created_at + maximumAutomaticPublicationAge, event.created_at + sameEventRequeueHorizon)이다. - P2는 additive schema/control-plane candidate일 뿐이다.
LEGACY_POLLINGauthority는 P3의 fenced cutover까지 유지한다. - live non-empty V3 database는 base template migration이 자동 backfill하지 않는다. 별도 deployment migration design과 승인이 없으면 중단한다.
- production ACTIVE는 SASL_SSL + SCRAM-SHA-512, external topic attestation, least-privilege evidence가 없으면 실패한다.
- disabled state는 contract/destination/client/AdminClient/thread/scheduler/network/secret refresh가 모두 0이다.
3. Scope boundary and owner leaves
| 책임 | owner leaf | Gradle path | production edge 변경 |
|---|---|---|---|
| typed event, outcome, relay, late drain, disposition policy | application-core |
:application-core |
없음 |
| generic envelope schema resource | shared-contract |
:shared-contract |
없음 |
| JSON/schema/catalog/Kafka/provider lifecycle | adapter-outbound-messaging |
:adapter:outbound:messaging |
외부 dependency만 추가 |
| event/delivery/journal/epoch/CAS | adapter-outbound-persistence-jpa |
:adapter:outbound:persistence-jpa |
없음 |
| authenticated operator HTTP mapping | adapter-inbound-web |
:adapter:inbound:web |
없음 |
| tuple composition/readiness/schedulers/real-service lane | app-bootstrap |
:app-bootstrap |
test dependency만 추가 |
| sample payload/schema/contribution fixture | sample-portfolio |
:sample-portfolio |
messaging edge 없음 |
금지:
- controller가 repository, JPA entity 또는 outbound adapter를 직접 사용;
- persistence mapper/query에 retry, disposition 또는 topic 정책을 넣음;
- messaging adapter가 sample, persistence 또는 inbound-web를 의존;
- bootstrap settings/configuration에 business event mapping이나 retry policy를 구현;
shared-contract에 WorkLog schema 또는 provider setting을 넣음;- 현재 dirty worktree의 Fileserver/Object Storage/Notification 변경을 되돌리거나 덮어씀.
4. Evidence ladder and promotion rule
| evidence | 허용되는 주장 |
|---|---|
| pure/application unit | provider-neutral contract와 state policy가 정의됨 |
| schema/catalog/codec unit/property | local deterministic document와 closed catalog가 정의됨 |
| adapter fake gateway | outcome mapping과 lifecycle protocol이 정의됨 |
| real PostgreSQL | same-store append, constraint, claim/CAS/audit protocol의 local evidence |
| single-node real Kafka | actual ACK metadata와 client/provider behavior evidence |
| TLS/SASL/ACL lane | exact security principal/profile evidence |
| multi-broker RF/min ISR lane | selected topology failure/recovery evidence |
| fault/capacity/rotation/cutover drill | exact tuple의 operational R2 evidence |
낮은 row를 높은 row, 다른 broker version, cluster, topic, principal 또는 security profile로
일반화하지 않는다. 모든 selected scenario가 fresh evidence artifact에 PASS일 때만 machine card를
release-eligible로 바꾼다. 그 전에는 최대 implemented-candidate다.
5. Execution rules
- 모든 checkbox는 구현 시작 시
[ ]다. - task 시작 전
git status --short, 현재 migration 목록, owner leaf의 가장 가까운CLAUDE.md,modules.jsonedge를 다시 확인한다. - behavior task는 RED test 작성 → 같은 focused command에서 예상 원인으로 실패 확인 → 최소 구현 → 같은 command GREEN 순서를 지킨다.
- RED가 처음부터 통과하면 기존 coverage인지 잘못된 test인지 조사하고 assertion을 강화한다.
- compilation drift, 외부 환경 또는 unrelated dirty change가 RED 원인이면 구현하지 말고 원인을 분리한다.
- 한 shared worktree에서 여러 Gradle process를 동시에 실행하지 않는다. 이전에 같은 output directory를 병렬 갱신해 compile collision이 발생했으므로 Gradle command는 한 invocation으로 묶거나 순차 실행한다.
- 실제 service가 필요한 release task는 service/credential/image/no-test 문제를 SKIP/PASS로
바꾸지 않는다. local ordinary
test와 release qualification task를 분리한다. - migration은 expand-first, forward-only다. 기존
V3__outbox_event.sql은 수정하지 않는다. - 새 provider와 v2 scheduler는 authority cutover 전까지 dark/disabled다.
- P2에서 v2 claim/send/authority switch를 활성화하지 않는다.
- event/payload/schema/hash/credential/raw header는 log, metric tag, evidence artifact에 넣지 않는다.
- 각 Wave exit에서 설계 §0 ledger, card maturity, plan checkbox, LLM Wiki branch-note를 실제 증거에 맞춰 갱신한다.
- plan surface 밖의 파일이나 타입이 필요하면 조용히 확장하지 않고 이 문서를 먼저 갱신한다.
- 설계와 plan이 충돌하면 구현으로 타협하지 않고 상세 설계를 먼저 수정·재승인한다.
6. Stop conditions
다음 중 하나라도 확인되면 해당 task 또는 Wave를 중단한다.
- application/domain에 framework, Kafka, JSON, persistence 타입을 넣어야만 진행 가능;
- module edge가
modules.json에 허용되지 않음; V7이 실행 시점에 이미 다른 migration으로 사용됐거나 다른 승인 계획이 먼저 구현됨. 모든 Flyway location을 다시 스캔해 다음 global version으로 이 계획과 tests를 먼저 갱신한다;- V3 legacy row가 live non-empty인데 empty/drained evidence나 별도 live migration 승인이 없음;
- business repository와 outbox append가 같은 transaction resource임을 증명할 수 없음;
- Kafka producer retry/timeout/effective setting을 finite하게 고정할 수 없음;
- adopted JSON Schema validator가 Draft 2020-12, offline registry, format assertion 또는 required adversarial bound를 만족하지 못함;
- topic/RF/min ISR/ACL을 runtime와 provisioning evidence의 명시된 source로 attest할 수 없음;
- legacy relay와 v2 relay를 동시에 active하게 해야만 rollout 가능;
- active writer/relay/producer를 fence하지 않은 채 authority switch가 필요;
- DB에 INDETERMINATE/HOLD를 기록하지 못한 상태로 producer generation을 강제 전환해야 함;
- operator endpoint가 active unexpired claim을 무시하거나 raw status update를 해야 함;
- real Kafka/security/multi-broker evidence 없이 R2/production-ready 표현이 필요.
7. Batch graph and checkpoints
Wave A / P0
truth + machine registry skeleton
-> Wave B / P1
application contract + schema + catalog + codec
-> Wave C / P2
additive DB v2 + append + claim/CAS + policy
-> Wave D / P3
Spring Kafka + endpoint + composition + cutover
-> Wave E / P4
real-service/security/fault/release evidence
| Wave | exit claim | rollback posture |
|---|---|---|
| A | current R0 truth와 planned cards가 정확함 | behavior 변화 없음 |
| B | local event contract/codec candidate | Kafka/outbox R2 아님 |
| C | polling v2 schema/control-plane candidate | LEGACY_POLLING 유지, v2 scheduler off |
| D | ACK-aware polling path/cutover candidate | pause admission, preserve DB schema/backlog/epoch |
| E | exact evidence가 통과한 tuple만 release-eligible | destructive schema downgrade 금지 |
Wave A — P0 truth freeze and execution scaffolding
Task 1: Freeze current R0 behavior and approved design truth
Owner: documentation + existing application/messaging/persistence/bootstrap tests Depends on: approved detailed design Behavior change: none
Files — modify:
docs/superpowers/specs/2026-07-28-messaging-production-capability-design.mdsrc/application-core/src/test/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCaseTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxAppendTransactionalContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxRowLifecycleContractTest.javasrc/adapter/outbound/messaging/README.md
Files — create:
-
src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfigTest.java -
Capture current branch,
git status --short, design digest, registry edges, dependency graph, migrations and current test counts in the LLM Wiki branch-note. -
Add characterization assertions for: broker blank → disabled sentinels; broker selected + missing sender → startup failure; broker ID mismatch → failure; sender normal return → legacy
PUBLISHED; sender exception →FAILED/DEAD; ACK-to-mark failure →IN_FLIGHTand possible duplicate; same-transaction append rollback; timestamp FIFO limitation. -
Keep tests explicitly named
legacyorcharacterization; do not rename current void-return success to broker ACK. -
Run the baseline sequentially:
```bash cd src && ./gradlew :application-core:test \ --tests '*PublishPendingOutboxEventsUseCaseTest' --console=plain cd src && ./gradlew :adapter:outbound:messaging:test \ --tests '*MessagingConfigTest' \ --tests '*OutboxMessagePublishAdapterTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*OutboxAppendTransactionalContractTest' \ --tests '*OutboxRowLifecycleContractTest' --console=plain ``` -
Expected GREEN: current behavior is reproducible without source behavior changes.
-
Update §0 to
P0=CHARACTERIZED, leaving P1–P4NOT_STARTED. -
Acceptance: no “Kafka ACK”, “dedupe safe” or “R2” claim is introduced.
Rollback checkpoint: characterization tests and truth documentation are independently reversible; legacy code remains the executable baseline through the P3 cutover window.
Task 2: Add fail-closed Messaging card registries and verification task skeleton
Owner: repository configuration + app-bootstrap contract tests
Depends on: Task 1
Files — create:
src/config/messaging/readiness-cards.yamlsrc/config/messaging/profile-compatibility.yamlsrc/config/messaging/release-profile-assertions.yamlsrc/config/messaging/evidence/build-evidence-manifest-v1.schema.jsonsrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingCapabilityRegistryContractTest.java
Files — modify:
-
src/build.gradle -
src/app-bootstrap/build.gradle -
src/app-bootstrap/README.md -
Write a RED contract test that requires exactly the P0–P4 first tuple rows, closed maturity values
not-implemented|implemented-candidate|release-eligible, wildcard-free compatibility, unique IDs, declared evidence tasks/scenarios/runbooks and no consumer/CDC/EOS/schema-registry rows. -
Seed all first tuple rows with
maturity: not-implementedand empty evidence fingerprint; do not predeclare future extension-ledger names. -
Define one checked-in, payload-free build-evidence schema with required source/artifact digest, producer task, scenario IDs/counts, command/timestamp, profile/catalog/schema/settings hashes, failures, skips and unsupported claims. Every later local manifest validates against this schema before release aggregation; a producer may add a stricter offline schema but may not weaken these common fields.
-
Define these task names in
src/build.gradlewithout making them pass yet:```text verifyMessagingContracts verifyMessagingJsonSchemaV1 verifyMessagingPollingOutboxR2 verifyMessagingKafkaProducerR2 verifyMessagingSecurityR2 verifyMessagingReleaseProfile verifyMessagingTargetBindingPreflight verifyMessagingTargetBinding verifyMessagingDeploymentCutover verifyMessagingCleanupTargetBinding verifyMessagingFinalR2Profile ``` Each task must fail on no matching tests. Release aggregation must reject missing, skipped, stale, wrong-source or mismatched-profile evidence. -
Verify RED then GREEN for registry structure only:
```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*MessagingCapabilityRegistryContractTest' --console=plain ``` -
Verify the existing dependency boundary remains unchanged:
```bash cd src && ./gradlew verifyCleanArchitectureDependencies --console=plain ``` -
Acceptance: registry truth exists, every card is
not-implemented, and no verification task can falsely claim R2.
Rollback checkpoint: registry/task scaffolding creates no runtime resources and may be removed without data migration.
Wave B — P1 typed contract, schema, catalog and deterministic bytes
Task 3: Add framework-free integration-event contract and contribution SPI
Owner: application-core (:application-core)
Depends on: Task 2
Files — create under
src/application-core/src/main/java/dev/caskeleton/application/messaging/:
contract/IntegrationPayload.javacontract/IntegrationEventContractContribution.javacontract/ContractId.javacontract/LogicalDestinationId.javacontract/SchemaResourceId.javacontract/Sha256.javacontract/ContractDescriptor.javaevent/EventId.javaevent/AggregateIdentity.javaevent/AggregateOrder.javaevent/IntegrationEventDraft.javaevent/ValidatedIntegrationEvent.javaevent/IntegrationEventEncoderPort.java
Files — create under
src/application-core/src/test/java/dev/caskeleton/application/messaging/:
contract/IntegrationEventContractContributionTest.javaevent/IntegrationEventDraftTest.javaevent/ValidatedIntegrationEventTest.java
Files — modify:
-
src/application-core/README.md -
src/application-core/CLAUDE.md -
Write RED value tests for canonical ASCII event ID grammar, closed contract/destination IDs, positive versions, nonblank canonical tenant scope, aggregate sequence/index bounds, immutable/defensively-copied bytes and fixed SHA-256 length.
-
Write RED SPI tests requiring exact final Java record payload type, canonical component order, schema resource/hash and provider-neutral descriptor. Reject
Map, raw JSON string/tree, assignable-type discovery and Java class-name routing. -
Implement one-public-type-per-file framework-free records/interfaces. The boundary shape is:
```java public interface IntegrationPayload {} public interface IntegrationEventContractContribution<P extends IntegrationPayload> { ContractId contractId(); int payloadVersion(); Class<P> exactPayloadRecordType(); List<String> canonicalRecordComponentOrder(); SchemaResourceId payloadSchemaResource(); Sha256 payloadSchemaHash(); ContractDescriptor descriptor(); } public interface IntegrationEventEncoderPort { ValidatedIntegrationEvent encode(IntegrationEventDraft<?> draft); } ``` -
Keep physical topic, Kafka record metadata, JSON node, serializer, schema validator and publication epoch out of these types.
-
Verify RED then GREEN:
```bash cd src && ./gradlew :application-core:test \ --tests 'dev.caskeleton.application.messaging.*' --console=plain ``` -
Run application purity:
```bash cd src && ./gradlew verifyApplicationCoreDependencyPurity \ verifyOneTypePerFile --console=plain ``` -
Acceptance claim: framework-free semantic contract R1 only; no schema/Kafka/persistence R2.
Rollback checkpoint: these are additive contracts; legacy NewOutboxEvent remains until the
validated append path is green.
Task 4: Check in the generic envelope schema and sample payload contract
Owner leaves: shared-contract (:shared-contract), sample-portfolio
(:sample-portfolio)
Depends on: Task 3
Files — create:
src/shared-contract/src/main/resources/contracts/messaging/envelope/v1.schema.jsonsrc/shared-contract/src/main/resources/contracts/messaging/envelope/v1.schema.sha256src/shared-contract/src/test/java/dev/caskeleton/shared/contract/messaging/MessagingEnvelopeSchemaResourceTest.javasrc/sample-portfolio/src/main/resources/contracts/messaging/portfolio.worklog.reserved/v1.schema.jsonsrc/sample-portfolio/src/main/resources/contracts/messaging/portfolio.worklog.reserved/v1.schema.sha256src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/WorkLogReservedPayload.javasrc/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/WorkLogReservedContractContribution.javasrc/sample-portfolio/src/test/resources/contracts/messaging/portfolio.worklog.reserved/v1.valid.jsonsrc/sample-portfolio/src/test/resources/contracts/messaging/portfolio.worklog.reserved/v1.invalid-unknown-field.jsonsrc/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/event/WorkLogReservedContractContributionTest.java
Files — modify:
-
src/shared-contract/README.md -
src/shared-contract/CLAUDE.md -
src/sample-portfolio/README.md -
src/sample-portfolio/CLAUDE.md -
Write RED resource tests requiring UTF-8, explicit Draft 2020-12
$schema, immutable absolute$id, checked-in lowercase SHA-256,unevaluatedProperties: false, bounded strings/arrays, required/null/missing policy and no HTTP/file remote$ref. -
Define envelope v1 with the exact fields frozen by design:
```json { "envelopeVersion": 1, "eventId": "event-1", "contractId": "portfolio.worklog.reserved", "payloadVersion": 1, "logicalDestination": "portfolio-domain-events", "aggregate": { "type": "worklog", "id": "worklog-42", "sequence": 17, "eventIndex": 0 }, "occurredAt": "2026-07-28T05:10:30.123Z", "correlationId": "corr-1", "contentType": "application/json", "payload": { "workLogId": "worklog-42" } } ``` -
Keep the envelope business-free and keep the WorkLog payload schema only in sample.
-
Make
WorkLogReservedPayloada typed immutable record implementingIntegrationPayload; contribution provides type/order/resource/hash only and no JSON mapper. -
Do not add
sample-portfolio -> adapter-outbound-messagingtomodules.jsonor Gradle. -
Verify RED then GREEN sequentially:
```bash cd src && ./gradlew :shared-contract:test \ --tests '*MessagingEnvelopeSchemaResourceTest' --console=plain cd src && ./gradlew :sample-portfolio:test \ --tests '*WorkLogReservedContractContributionTest' --console=plain ``` -
Acceptance claim: checked-in generic/sample contract artifacts exist; validator compatibility is still unproven until Task 6.
Rollback checkpoint: resources and sample contribution are additive; no production runtime discovers or publishes them yet.
Task 5: Compile the closed contract, destination and exact capability binding
Owner: adapter:outbound:messaging (:adapter:outbound:messaging)
Depends on: Tasks 3–4
Files — create under
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/:
contract/ContractCatalogCompiler.javacontract/CompiledIntegrationEventContract.javacontract/ContractCatalogDigest.javadestination/DestinationBindingSettings.javadestination/DestinationBindingCompiler.javadestination/CompiledPublicationBinding.javadestination/PartitionKeyV1.javaconfig/MessagingCapabilityCardRegistry.javaconfig/CompiledMessagingDescriptor.java
Files — create under
src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/:
contract/ContractCatalogCompilerTest.javacontract/ContractCatalogDigestTest.javadestination/DestinationBindingCompilerTest.javadestination/PartitionKeyV1Test.javaconfig/MessagingCapabilityCardRegistryTest.java
Files — modify:
-
src/adapter/outbound/messaging/README.md -
src/adapter/outbound/messaging/CLAUDE.md -
Write RED tests for duplicate contract/destination/schema IDs; missing binding; unknown card; final-record exact type; component-order mismatch; code/deployment byte-bound intersection; config attempting to relax ordering/schema/security; legacy + canonical conflict; unsupported future card rejection.
-
Add golden partition-key vectors using the design's domain-separated, length-prefixed SHA-256 input. Assert exactly 64 lowercase hex ASCII characters and tenant-enabled/disabled canonical non-null scope.
-
Compile:
```text contract descriptor + destination descriptor + producer/serialization/security/card descriptor = immutable CompiledPublicationBinding ``` Physical topic and bootstrap servers stay only in the compiled deployment binding. -
Compute stable catalog/settings/schema digests using sorted IDs and length-prefixed bytes; never depend on
Mapiteration order ortoString(). -
Empty catalog + DISABLED must compile to a zero-resource descriptor. ACTIVE + empty catalog must fail before any client/thread is created.
-
Verify RED then GREEN:
```bash cd src && ./gradlew :adapter:outbound:messaging:test \ --tests '*ContractCatalog*Test' \ --tests '*DestinationBindingCompilerTest' \ --tests '*PartitionKeyV1Test' \ --tests '*MessagingCapabilityCardRegistryTest' --console=plain ``` -
Acceptance claim: closed local binding compiler R1; no wire bytes or Kafka client yet.
Rollback checkpoint: compiler is not wired into MessagingConfig; legacy selection remains
authoritative.
Task 6: Implement the deterministic JSON Schema envelope encoder
Owner: adapter:outbound:messaging (:adapter:outbound:messaging)
Depends on: Task 5
Files — create:
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/LocalJsonSchemaRegistry.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/DeterministicEnvelopeWriter.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/JsonSchemaIntegrationEventEncoder.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/EnvelopeAdmissionLimits.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/EnvelopeHashV1.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/envelope/LocalJsonSchemaRegistryTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/envelope/JsonSchemaIntegrationEventEncoderTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/envelope/EnvelopeAdversarialCorpusTest.javasrc/adapter/outbound/messaging/src/test/resources/contracts/messaging/test.event/v1.schema.jsonsrc/adapter/outbound/messaging/src/test/resources/contracts/messaging/test.event/v1.valid.jsonsrc/adapter/outbound/messaging/src/test/resources/contracts/messaging/test.event/v1.invalid.json
Files — modify:
-
src/adapter/outbound/messaging/build.gradle -
src/adapter/outbound/messaging/gradle.lockfile -
src/build.gradle -
Write RED tests for Draft 2020-12 meta-schema, checksum mismatch, duplicate
$id, unknown dialect/vocabulary, remote/unmapped$ref, cycles beyond the supported depth, pathological regex corpus, format assertion, valid/invalid envelope and payload, required/null/missing, unknown property and unsupported payload version. -
Write RED parser/admission tests for duplicate JSON key, malformed UTF-8, unpaired surrogate, trailing garbage, depth/string/array/object/number bounds, non-finite number, exact UTF-8 value/key/header bytes and deterministic field/scalar order.
-
Add:
```groovy implementation 'org.springframework.boot:spring-boot-starter-json' implementation('com.networknt:json-schema-validator:3.0.2') { exclude group: 'com.fasterxml.jackson.dataformat', module: 'jackson-dataformat-yaml' } ``` Keep Jackson/schema runtime in the messaging leaf. Regenerate only affected dependency locks and review the resolved Jackson 3 graph, license and vulnerability report. -
Configure NetworkNT Draft 2020-12 with format assertions enabled and an exact classpath resource map. After startup compilation, network/file schema resolution is impossible.
-
Make the writer consume only exact registered final record types. Disable polymorphic typing, feature-provided serializers, unknown properties and reflective assignable-type search.
-
Compute:
```text SHA-256( UTF8("ca-skeleton.messaging.envelope.v1") || 0x00 || u32be(len(exactEnvelopeBytes)) || exactEnvelopeBytes ) ``` and return defensive copies in `ValidatedIntegrationEvent`. -
Add validator compatibility evidence using the adopted JSON Schema Test Suite/Bowtie corpus; custom contract compatibility still requires repository golden vectors.
-
Verify RED then GREEN:
```bash cd src && ./gradlew :adapter:outbound:messaging:test \ --tests '*LocalJsonSchemaRegistryTest' \ --tests '*JsonSchemaIntegrationEventEncoderTest' \ --tests '*EnvelopeAdversarialCorpusTest' --console=plain cd src && ./gradlew verifyMessagingJsonSchemaV1 \ verifyDependencyLocks --console=plain ``` -
On GREEN,
verifyMessagingJsonSchemaV1validates and writes this exact payload-free candidate manifest:```text src/build/messaging-evidence/contracts-schema/manifest.json ``` It conforms to `src/config/messaging/evidence/build-evidence-manifest-v1.schema.json` and binds the human/CI-supplied source/artifact digest, schema/catalog hashes, dependency-lock digest, exact scenario IDs/counts, command/timestamp, failed=0, skipped=0 and unsupported claims. Missing digest input fails the manifest-producing lane; an ordinary focused unit test may still run without claiming release evidence. -
Update JSON/schema cards to
implemented-candidateonly after the exact tests and locks pass. -
Acceptance claim: deterministic local wire contract candidate; Kafka and durable outbox R2 are still unimplemented.
Rollback checkpoint: encoder/catalog stays unwired from production append; removing it does not change legacy rows.
Wave B exit checkpoint
-
Run:
```bash cd src && ./gradlew :application-core:check \ :shared-contract:check \ :adapter:outbound:messaging:check \ :sample-portfolio:check \ verifyMessagingContracts \ verifyCleanArchitectureDependencies \ --console=plain ``` -
Confirm no production leaf imports
dev.caskeleton.sample. -
Update the design ledger to
P1=IMPLEMENTED_CANDIDATEonly if all Wave B evidence is GREEN. -
Update the LLM Wiki branch-note; record whether a new derived raw document exists or explicitly record “없음”.
Wave C — P2 immutable event, polling delivery and operator policy
Task 7: Add the forward-only PostgreSQL outbox v2 schema
Owner: adapter:outbound:persistence-jpa (:adapter:outbound:persistence-jpa)
Depends on: Wave B
Activation: schema/control-plane only; LEGACY_POLLING remains ACTIVE
Candidate migration file:
src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V7__messaging_outbox_v2.sql
V7 is the current candidate because the sample Flyway location already contains
V6__poster.sql. The Notification plan also uses V7/V8 as candidates; plan text is not a
simultaneous Flyway reservation. Before implementation, scan every runtime Flyway location and all
implemented or actively executing plans. The first implementation claims the next global version;
the later plan must reserve the following version and update every path/test before writing SQL.
Messaging and Notification persistence migrations must not execute concurrently with unresolved
version ownership.
Files — create:
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDeliveryId.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDeliveryEntity.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDeliveryAttemptObservationEntity.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDispositionAuditEntity.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxPublicationEpochEntity.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxAuthorityCutoverEvidenceEntity.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryJpaRepository.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAttemptObservationJpaRepository.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDispositionAuditJpaRepository.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPublicationEpochJpaRepository.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2MigrationContractTest.java
Files — modify:
-
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxEventEntity.java -
src/adapter/outbound/persistence-jpa/README.md -
src/adapter/outbound/persistence-jpa/CLAUDE.md -
Before editing, classify V3 row count/state/data and assert the base card accepts only a fresh or verified empty/drained legacy table. Any live non-empty database fails this task pending a separate deployment-specific migration plan.
-
Write a RED real-PostgreSQL migration test. Expected failure: v2 columns/tables/constraints do not exist.
-
Keep
V3__outbox_event.sqlbyte-for-byte unchanged. Add immutable metadata columns additively while retaining legacy columns for compatibility. -
Widen
event_id VARCHAR(64)toVARCHAR(96)in the forward migration; this is compatible with old writers' shorter grammar. Keep the other V3 NOT NULL columns through the rollback window and choose one explicit compatibility projection for every canonical insert:```text event_type = contract_id legacy alias payload = exact UTF-8 envelope bytes decoded as text status = PENDING compatibility sentinel attempt_count = 0 next_attempt_at = occurred_at idempotency_key = event_id ``` These columns are not authority after `POLLING_V2`. Epoch predicates prevent every legacy claim/mutation/reaper from observing canonical rows, and a post-cutover immutable guard prevents them from drifting. Do not relax NOT NULL/defaults or leave canonical inserts unspecified. -
Create:
```text outbox_delivery outbox_delivery_attempt_observation outbox_disposition_audit outbox_publication_epoch outbox_authority_cutover_evidence outbox_write_admission outbox_runtime_node_lease ``` with event/generation primary keys, one-CURRENT partial unique constraint, delivery FK, authority/state CHECKs, DB timestamps, row version, immutable automatic deadline and an expiring one-shot cutover evidence identity/digest. A cutover attempt has the closed durable state machine `CUTOVER_PENDING -> FINALIZING_V2 -> CONSUMED_V2` or `CUTOVER_PENDING -> RECOVERING_LEGACY -> RECOVERED_LEGACY`; the two branches are mutually exclusive CAS transitions. `RECOVERING_LEGACY` also stores an opaque recovery operation ID, owner/lease deadline and recovery evidence digest. Lease expiry permits recovery-only takeover and never resets the attempt to `CUTOVER_PENDING`. Give every attempt the constant database authority scope `OUTBOX_PUBLICATION`, ACTIVE legacy epoch ID, frozen fence generation and target-binding digest. A partial unique constraint permits exactly one nonterminal (`CUTOVER_PENDING`, `FINALIZING_V2`, `RECOVERING_LEGACY`) attempt in that authority scope. Attempt creation, finalization and recovery lock the write-admission singleton first and the ACTIVE epoch second, then validate the exact frozen generation/target binding before touching the attempt. The write admission singleton starts OPEN at generation 1; runtime node leases are bounded and bind node/source/artifact/fence-protocol identity without payload or credentials. -
Add event ID, partition-key, SHA-256, tenant-scope, order uniqueness and exact
BYTEAconstraints. Protect immutable event columns with a post-cutover guard that is dormant during compatibility migration and enabled only by the fenced P3 cutover. -
Seed exactly one ACTIVE
LEGACY_POLLINGepoch/generation for fresh/empty base template. Do not activatePOLLING_V2, create v2 delivery for live legacy rows or claim/send from v2. -
RED/GREEN cases: fresh V1–V7; V3-empty upgrade; non-empty preflight rejection; duplicate current generation; nullable tenant attack; invalid hash/key/event ID; mutable event update after guard; FK/audit retention; publication epoch uniqueness; duplicate nonterminal authority attempt under concurrent insert; illegal finalization/recovery state transition; 65–96 character event ID; old-writer short ID; canonical compatibility projection satisfying every retained V3 NOT NULL constraint.
-
Verify:
```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*OutboxV2MigrationContractTest' --console=plain ``` -
Acceptance claim: additive polling v2 schema candidate only; legacy relay authority unchanged.
Rollback checkpoint: rollback disables new code and keeps additive schema/backlog. Never destructively downgrade the database.
Task 8: Append validated event and initial delivery in the business transaction
Owner leaves: application-core, adapter-outbound-persistence-jpa, app-bootstrap test
fixture
Depends on: Task 7
Files — create:
src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxAppendPort.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAppendAdapter.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapter.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAppendAdapterTest.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapterTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2AppendTransactionalContractTest.java
Files — modify:
-
src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAppendPort.java -
src/application-core/src/main/java/dev/caskeleton/application/outbox/NewOutboxEvent.java -
src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxEvent.java -
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapter.java -
src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapterTest.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapter.java -
src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java -
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxContainerTestSupport.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxAppendTransactionalContractTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxPublisherLeaderElectionContractTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxRowLifecycleContractTest.java -
src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisher.java -
src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogUseCase.java -
src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisherTest.java -
src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogOutboxTest.java -
src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/WorkLogUseCasesTest.java -
src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/authz/WorkLogAuthorizationContractTest.java -
Write RED application tests making
OutboxAppendPort.append(ValidatedIntegrationEvent)the only canonical method. Move rawNewOutboxEventappend to a separately named/deprecatedLegacyOutboxAppendPort; never overload or silently reinterpret raw payload as v1. -
Split the current combined store/append implementation:
OutboxStoreAdapterremains only the legacy relay store during the observation window, while a separately namedLegacyOutboxAppendAdapterimplements onlyLegacyOutboxAppendPort. It has no component annotation and bootstrap may compose it only for the explicit sample/R0 compatibility graph. -
Write RED real-PostgreSQL tests proving business state + event + current delivery commit or rollback together; encoder/schema failure rolls back business state; exact
BYTEAand hash round-trip. Cross-resource ACTIVE startup rejection belongs to Task 19 after both leaf descriptors exist. -
In the persistence adapter, lock/read the ACTIVE publication epoch inside the caller-owned transaction, attach DB-authoritative
created_at,publication_epoch,dispatch_authority,transaction_resource_id, and insert initial delivery only when authority isPOLLING_V2. -
For an initial current delivery, compute and persist in that same DB transaction:
```text automaticAttemptDeadline = min(eventDbCreatedAt + maximumAutomaticPublicationAge, eventDbCreatedAt + contract.sameEventRequeueHorizon) ``` The value is immutable and profile reload never moves an existing generation's deadline. -
During compatibility
LEGACY_POLLING, write both legacy required columns and validated v2 metadata in the same transaction but do not create/send a v2 current delivery. -
Make the legacy claim read model distinguish true v0 rows from rows carrying canonical v1 metadata without exposing a physical topic in application. For a canonical row,
OutboxMessagePublishAdapterresolves the stored logical destination through the closed compiled binding and sends the stored partition-key bytes plus immutableenvelope_bytesbyte-for-byte. It must not invokeOutboxEnvelopeJsonor reinterpret the retained V3payloadprojection. Only a true v0 row may use the old wrapper/event-type route. -
Add golden cases for canonical append under
LEGACY_POLLING→ legacy claim → exact compiled destination/key/envelope bytes → legacyPUBLISHED. This remainsLEGACY_RECORDED_UNVERIFIEDat cutover and is never automatically resent by v2. Test v0 and canonical branches independently; mixed/missing metadata fails closed. -
Add a post-cutover canonical append fixture proving the retained V3 NOT NULL compatibility projection, immutable event + CURRENT/READY delivery and deadline all commit together while the epoch-fenced legacy claim/reaper sees the row count as 0.
-
Use an app-bootstrap test-source typed contribution/draft to prove the canonical append path. Do not inject the messaging encoder into
sample-portfolioor add a project edge in this task. Change the sample use case dependency explicitly toLegacyOutboxAppendPort; the existing sample mapper remains a visibly R0, non-active compatibility fixture until the separate standalone-sample activation plan. -
Remove component auto-discovery from the legacy append/store adapter. Bootstrap may compose it only for an exact
LEGACY_POLLINGcompatibility graph; canonicalPOLLING_V2must haveLegacyOutboxAppendPortbean count 0. Update every exact existing legacy/sample fixture listed above in the same task so changingOutboxAppendPortcannot leave compile-only hidden users. -
Verify RED then GREEN:
```bash cd src && ./gradlew :application-core:test \ --tests '*Outbox*' --console=plain cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*OutboxAppendAdapterTest' \ --tests '*LegacyOutboxAppendAdapterTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*OutboxV2AppendTransactionalContractTest' --console=plain cd src && ./gradlew :sample-portfolio:test \ --tests '*CreateWorkLogOutboxTest' --console=plain ``` -
Acceptance claim: validated same-transaction append candidate; v2 relay remains disabled.
Rollback checkpoint: keep compatibility writes while rolling back the new relay. Do not generate a second event ID or dual-write outside the transaction.
Task 9: Define exhaustive publication outcomes and one-record relay policy
Owner: application-core (:application-core)
Depends on: Task 8
Files — create under
src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/:
PublicationOutcome.javaPublicationReceipt.javaPublicationFailure.javaAcceptanceCertainty.javaRetryDisposition.javaPublicationFailureStage.javaPublicationFailureClass.javaPublicationAttemptId.javaPublicationAdmission.javaPublicationAdmissionPort.javaAcknowledgedPublicationPort.java
Files — create under
src/application-core/src/main/java/dev/caskeleton/application/outbox/:
OutboxDelivery.javaOutboxDeliveryState.javaDeliveryAuthorityStatus.javaClaimToken.javaClaimedOutboxDelivery.javaOutboxDeliveryStorePort.javaPublishNextOutboxDeliveryCommand.javaPublishNextOutboxDeliveryResult.javaPublishNextOutboxDeliveryUseCase.java
Files — create under
src/application-core/src/test/java/dev/caskeleton/application/:
messaging/publication/PublicationOutcomeTest.javaoutbox/PublishNextOutboxDeliveryUseCaseTest.javaoutbox/OutboxDeliveryStateTest.java
Files — modify or retain as legacy until Task 26:
-
src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxMessagePublishPort.java -
src/application-core/src/main/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCase.java -
src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxBackoffPolicy.java -
src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayResult.java -
Write RED tests for the sealed outcome shape:
```java public sealed interface PublicationOutcome { record Acknowledged(PublicationReceipt receipt) implements PublicationOutcome {} record AcknowledgedMismatch(PublicationReceipt receipt) implements PublicationOutcome {} record Rejected(PublicationFailure failure) implements PublicationOutcome {} record Indeterminate(PublicationFailure failure) implements PublicationOutcome {} } ``` Receipt/failure contains bounded provider-neutral values only; no Kafka SDK type or raw exception/message. -
Test certainty and retry as independent axes. Ambiguous/post-admission/timeout/unknown maps to
INDETERMINATE;REJECTEDrequires definite non-acceptance. -
Write relay RED tests for this exact sequence:
```text acquire bounded admission -> Tx B claim exactly one row + ATTEMPT_ADMITTED -> publish outside DB transaction -> Tx C outcome observation + token/valid-lease CAS transition -> release admission ``` -
Cover: no admission → claim 0; no eligible row → release permit; ACK →
DELIVERY_RECORDED; mismatch →HOLD; definite transient rejection →RETRY_WAIT; permanent/budget exhaustion →EXHAUSTED; indeterminate → duplicate-aware retry or HOLD according to remaining finite budget; transition failure propagates; diagnostic reporter failure cannot change persisted state. -
Enforce one command invocation/one record. A scheduler may invoke it again; the use case must not loop and open per-row
REQUIRES_NEWtransactions. -
Replace attempt-only backoff with a descriptor that includes maximum attempts, immutable automatic deadline, bounded delay/jitter and same-event horizon. Do not start age at
first_attempt_at. -
Keep legacy void port/use case explicitly deprecated and separately wired until Task 26; canonical code must not adapt exception-only success into
Acknowledged. -
Verify RED then GREEN:
```bash cd src && ./gradlew :application-core:test \ --tests '*PublicationOutcomeTest' \ --tests '*PublishNextOutboxDeliveryUseCaseTest' \ --tests '*OutboxDeliveryStateTest' --console=plain ``` -
Acceptance claim: application polling/outcome policy candidate; provider and DB CAS remain adapter work.
Rollback checkpoint: canonical use case remains unwired. Legacy relay continues to serve
LEGACY_POLLING.
Task 10: Implement per-record JIT claim, valid-lease CAS and attempt journal
Owner: adapter:outbound:persistence-jpa (:adapter:outbound:persistence-jpa)
Depends on: Task 9
Files — create:
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryStoreAdapter.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryClaimRepository.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxDeliveryClaimRepository.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryStoreAdapterTest.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxDeliveryClaimRepositoryTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2ClaimCasContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2MultiWorkerContractTest.java
Files — modify:
-
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlPersistenceConfig.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxContainerTestSupport.java -
Write unit RED tests that the adapter maps application values without adding retry/topic policy and requires affected-row count exactly 1 for every CAS.
-
Write real-PostgreSQL RED tests for: two workers claim disjoint rows; same aggregate total order uses sequence/index rather than timestamp; different aggregates progress; hot aggregate does not starve all others; expired claim reclaim; same-token renew; stale token/owner/generation/version rejection; expired but not yet reclaimed owner cannot record ACK/failure.
-
The worker-owned mutation predicate must include:
```sql WHERE event_id = :event_id AND delivery_generation = :generation AND authority_status = 'CURRENT' AND state = 'CLAIMED' AND claim_token = :token AND claim_owner = :owner AND claim_until > CURRENT_TIMESTAMP AND row_version = :expected_row_version ``` -
Claim eligibility is CURRENT
READY, dueRETRY_WAITor expiredCLAIMED, subject to ordering-head eligibility.EXHAUSTED,HOLD,LEGACY_RECORDED_UNVERIFIEDnever release the next ordered event. -
Tx B atomically updates claim count/token/owner/DB-time lease/publication attempt count and inserts
ATTEMPT_ADMITTED. Raw claim token is never copied; journal stores a domain-separated digest. -
Before
ATTEMPT_ADMITTED, use DB time to verify both the automatic deadline and a full application-attempt/Tx-C safety window remain. Ifdatabase_now >= deadlineor the full window does not fit, perform a fencedEXHAUSTEDtransition without admission/send. -
When reclaiming an expired
CLAIMEDrow whose previouspublicationAttemptIdhasATTEMPT_ADMITTEDbut no outcome, append exactly one idempotentOUTCOME_OBSERVED(INDETERMINATE)for that old attempt before replacing the token and admitting the new attempt. Never fabricate a definite rejection or erase the prior attempt. -
Tx C atomically inserts
OUTCOME_OBSERVEDand performs ACK/retry/exhaust/hold CAS. Provider metadata is a bounded opaque reference. -
Use DB time for eligibility, lease, created-at and retry due. Before send admission, ensure remaining lease exceeds the full attempt + DB transition + safety budget.
-
Verify RED then GREEN sequentially:
```bash cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*OutboxDeliveryStoreAdapterTest' \ --tests '*PostgreSqlOutboxDeliveryClaimRepositoryTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*OutboxV2ClaimCasContractTest' \ --tests '*OutboxV2MultiWorkerContractTest' --console=plain ``` -
Acceptance claim: real-PostgreSQL JIT claim/CAS protocol candidate; no Kafka send or authority cutover.
Rollback checkpoint: leave v2 scheduler off and LEGACY_POLLING active. Claimed test rows are
disposable; production rollback preserves all event/delivery rows.
Task 11: Persist late publication observations with DB-commit-before-source-ACK
Owner leaves: application-core, adapter-outbound-persistence-jpa
Depends on: Task 10
Files — create in application-core:
src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/ObservationId.javasrc/application-core/src/main/java/dev/caskeleton/application/messaging/publication/LatePublicationObservation.javasrc/application-core/src/main/java/dev/caskeleton/application/messaging/publication/LatePublicationObservationSourcePort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAttemptObservationPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsCommand.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsResult.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsUseCase.javasrc/application-core/src/test/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsUseCaseTest.java
Files — create in adapter:outbound:persistence-jpa:
-
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAttemptObservationAdapter.java -
src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAttemptObservationAdapterTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLateObservationContractTest.java -
Write application RED tests for:
```text poll/lease bounded batch -> tx.inNew(idempotent DB append) returns after commit -> acknowledgePersisted ``` DB append/commit failure calls `releaseForRetry`; source ACK never runs in a transaction callback. -
Use observation identity
(eventId, deliveryGeneration, publicationAttemptId, LATE_ACK_OBSERVED)and a DB unique constraint/ON CONFLICT DO NOTHING. -
Test DB commit → process crash before source ACK by redelivering the same observation; exactly one journal fact remains.
-
Test late observation never changes
DELIVERY_RECORDED,RETRY_WAIT,EXHAUSTED,HOLDor current generation. It is diagnostic, not correctness authority. -
Test empty poll, bounded maximum, poison item release, source ACK failure and commit failure.
-
Verify RED then GREEN:
```bash cd src && ./gradlew :application-core:test \ --tests '*RecordLatePublicationObservationsUseCaseTest' --console=plain cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*OutboxAttemptObservationAdapterTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*OutboxLateObservationContractTest' --console=plain ``` -
Acceptance claim: durable idempotent late-observation drain boundary; callback capture is still bounded-loss and no messaging queue exists until Task 16.
Rollback checkpoint: disabling the drain loses only bounded diagnostics, never changes delivery authority. Alert/readiness must expose the degradation.
Task 12: Implement audited application disposition policy and atomic persistence transitions
Owner leaves: application-core, adapter-outbound-persistence-jpa
Depends on: Tasks 10–11
Files — create in application-core:
src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxDisposition.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionCommand.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionResult.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxDispositionResultCodec.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxDispositionPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionUseCase.javasrc/application-core/src/test/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionUseCaseTest.java
Files — create in adapter:outbound:persistence-jpa:
-
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDispositionAdapter.java -
src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDispositionAdapterTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxDispositionTransactionalContractTest.java -
Write application RED tests with
@RequiresPermission("outbox:disposition")and@UseCaseCapability(idempotency = Idempotency.KEYED, ...).SKIP_WITH_GAPandCOMPENSATEadditionally callAuthorizationPortforoutbox:disposition:destructive. -
Command requires:
```text eventId expectedDeliveryGeneration expectedRowVersion disposition bounded reason incident/change reference IdempotencyContext(scope + request fingerprint + bounded TTL) operator principal destructive approval reference when required compensation event reference for COMPENSATE ``` -
Inject the existing application-owned
IdempotencyExecutorintoApplyOutboxDispositionUseCase. Execute authorization/policy/CAS exactly once under the command'sIdempotencyContext, using a framework-free deterministicOutboxDispositionResultCodec. Replay returns the stored application result; same key with a different request fingerprint raises the existing mismatch exception. Controller and persistence adapter must not implement their own idempotency state machine. -
Application tests cover first execution, completed replay, in-flight conflict, request mismatch, action failure/discard and bounded TTL. Persistence integration reuses the existing
IdempotencyStorePortadapter to prove atomic claim/complete;outbox_disposition_auditremains the immutable business/operation audit rather than a second idempotency registry. -
Validate allowed source state, ordering impact, active-unexpired-claim absence and finite same-event requeue horizon in application policy for early feedback. This precheck is not the concurrency fence.
-
In the persistence transaction, lock the CURRENT delivery row and atomically re-evaluate expected generation/state/row version plus
NOT (state='CLAIMED' AND claim_until > database_now)before audit/mutation. Add a race test that inserts a worker claim after application precheck but before the locked mutation; operator CAS must fail without partial audit/handoff. -
Write real-PostgreSQL RED/GREEN for: stale generation/version; live claim; idempotency replay/mismatch; concurrent requeue; one CURRENT constraint; partial handoff rollback; old generation never claimable again; immutable audit.
-
REQUEUE transaction locks current row, inserts audit, marks old authority
SUPERSEDED, setssuperseded_by_generation, and inserts generation + 1CURRENT/READYwith:```text automaticAttemptDeadline = min(newDeliveryDbCreatedAt + maximumAutomaticPublicationAge, eventDbCreatedAt + sameEventRequeueHorizon) ``` -
HOLD/SKIP/COMPENSATE/legacy accept use expected generation/state/row version and do not mimic the worker token predicate.
COMPENSATEDrequires an already-created immutable compensating event reference in the same transaction. -
Verify RED then GREEN:
```bash cd src && ./gradlew :application-core:test \ --tests '*ApplyOutboxDispositionUseCaseTest' --console=plain cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*OutboxDispositionAdapterTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*OutboxDispositionTransactionalContractTest' --console=plain ``` -
Acceptance claim: provider-neutral authenticated disposition policy and DB protocol candidate; no HTTP surface yet.
Rollback checkpoint: operator surface is not exposed. Data/audit rows are forward-only and must not be rewritten by raw SQL.
Task 13: Prove P2 compatibility fence and no-dual-authority state
Owner: persistence/bootstrap integration Depends on: Tasks 7–12
Files — create:
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxPublicationEpochContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyV2CompatibilityContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyCanonicalWireCompatibilityContractTest.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxV2RetentionAdapter.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxV2RetentionAdapterTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2RetentionContractTest.java
Files — modify:
-
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapter.java -
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxClaimRepository.java -
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaper.java -
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxEventJpaRepository.java -
Write the epoch, canonical-wire compatibility and retention tests first, then run RED:
```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*OutboxPublicationEpochContractTest' \ --tests '*OutboxLegacyV2CompatibilityContractTest' \ --tests '*OutboxLegacyCanonicalWireCompatibilityContractTest' \ --tests '*OutboxV2RetentionContractTest' --console=plain ``` Expected non-zero: the legacy claim/reaper lacks an epoch fence and v2 retention protocol is absent. A compile failure unrelated to those missing contracts is not an accepted RED. -
RED test that the compatibility legacy writer fills v2 immutable metadata in the same transaction while the legacy relay can claim only the exact ACTIVE
LEGACY_POLLINGepoch/generation. -
Add a legacy mutation/claim fence predicate tied to the ACTIVE publication epoch. Old pre-fence binaries are explicitly incompatible and must be drained to zero before P3.
-
Prove
POLLING_V2claim is rejected whileLEGACY_POLLINGis active and legacy claim is rejected after the epoch changes. -
Prove no row can be claimed/sent by both paths; publication epoch lock and expected generation are mandatory.
-
Prove the compatibility publisher sends a canonical metadata row exactly once through the legacy authority using the compiled destination, stored key and byte-identical v1 envelope; v0 rows still use the old wrapper. Nested envelope, event-type-as-topic for canonical rows, mixed metadata, and automatic resend of legacy
PUBLISHEDafter cutover all fail. -
Bind the legacy reaper to the exact ACTIVE
LEGACY_POLLINGepoch and stop/drain it before cutover. Implement v2 retention separately: delete only when the unique CURRENT generation is resolved asDELIVERY_RECORDEDor auditedSKIPPED/COMPENSATED/ LEGACY_ACCEPTED_UNVERIFIED, with no claim/requeue, unresolved observation, legal/operator hold, audit-retention or replay-horizon obligation. -
Real PostgreSQL retention cases cover reaper-vs-claim/requeue, delivery/audit FK and no-silent-cascade, superseded generations,
EXHAUSTED,HOLD,LEGACY_RECORDED_UNVERIFIED, unresolved late observation and epoch mismatch. -
Do not perform legacy row reconciliation, v2 delivery creation or authority switch in this task.
-
Re-run the same focused command GREEN; all three exact tests must pass with no skip. Then run the candidate gate:
```bash cd src && ./gradlew verifyMessagingPollingOutboxR2 --console=plain ``` At P2, `verifyMessagingPollingOutboxR2` may report `implemented-candidate`; it must not emit a release-eligible claim. On GREEN it validates and writes the exact candidate manifest: ```text src/app-bootstrap/build/messaging-evidence/polling-outbox-r2/manifest.json ``` The manifest conforms to `src/config/messaging/evidence/build-evidence-manifest-v1.schema.json` and binds the supplied source/artifact digest, migration/schema/card/profile hashes, exact PostgreSQL scenario IDs/counts, commands/timestamps, failed=0, skipped=0 and unsupported Kafka/security claims. -
Update P2 card rows to
implemented-candidateonly if real PostgreSQL cases pass.
Rollback checkpoint: keep LEGACY_POLLING ACTIVE and canonical v2 scheduler off. If the fence
cannot be deployed to all nodes, do not proceed to Wave D.
Wave C exit checkpoint
-
Run:
```bash cd src && ./gradlew :application-core:check \ :adapter:outbound:persistence-jpa:check \ :app-bootstrap:test \ verifyCleanArchitectureDependencies \ --console=plain ``` -
Confirm broker/client/network resources are still 0 and v2 authority has not switched.
-
Update the design ledger to
P2=IMPLEMENTED_CANDIDATEonly from actual tests. -
Update the LLM Wiki branch-note and derived-document decision.
Wave D — P3 ACK-aware Spring Kafka, operator endpoint and reference-path cutover
Task 14: Bind and compile finite canonical Messaging settings
Owner: adapter:outbound:messaging (:adapter:outbound:messaging)
Depends on: Wave C
Files — create:
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingExpectedState.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingR2Settings.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingSettingsCompiler.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerSettings.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/CompiledKafkaProducerSettings.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingSettingsCompilerTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingDisabledResourceContractTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerSettingsTest.java
Files — modify:
-
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingSettings.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaAdapterSettings.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfig.java -
Write RED binding/compiler tests for exact
DISABLED|ACTIVEexpected state and the canonical fields in design §21.1. RawMap<String,Object>Kafka overrides are forbidden. -
Freeze the first effective profile:
```text acks=all enable.idempotence=true retries=MAX/effectively-unbounded under delivery.timeout.ms max.in.flight.requests.per.connection<=5 compression.type=none partitioner.ignore.keys=false finite request/delivery/max.block/linger/batch/buffer/request bounds maximumAdmittedRecords=1 ``` -
Validate:
```text deliveryTimeout >= requestTimeout + linger applicationAttemptBudget >= admissionWait + maxBlock + deliveryTimeout + callback/transitionReserve claimLease > applicationAttemptBudget + dbTransitionReserve + schedulingSafetyMargin ``` -
Reject ACTIVE + unknown/missing card/provider/bootstrap/destination/security; plaintext in production; literal credentials; contract bytes over any bound; ordering + null key; transaction resource mismatch; legacy + canonical keys; active durable contract + disabled dispatch.
-
DISABLED must instantiate no schema compiler with active contracts, producer factory, template, AdminClient, semaphore, observation queue, scheduler, secret refresh or network connection.
-
Legacy keys are parsed only into an R0 descriptor and conflict with canonical keys. Do not map
broker=kafkatokafka-springorrelay-enabled=trueto polling v2. -
Verify RED then GREEN:
```bash cd src && ./gradlew :adapter:outbound:messaging:test \ --tests '*MessagingSettingsCompilerTest' \ --tests '*MessagingDisabledResourceContractTest' \ --tests '*KafkaProducerSettingsTest' --console=plain ``` -
Acceptance claim: finite static descriptor candidate; no Kafka client created yet.
Rollback checkpoint: canonical activation remains disabled; legacy settings continue only in R0 mode.
Task 15: Add explicit Spring Kafka producer factory and ACK-aware gateway
Owner: adapter:outbound:messaging (:adapter:outbound:messaging)
Depends on: Task 14
Files — create:
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerFactoryConfig.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaPublishGateway.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGateway.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaPublicationFailureClassifier.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/publication/AckAwareOutboxPublicationAdapter.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerFactoryConfigTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGatewayTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaPublicationFailureClassifierTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/publication/AckAwareOutboxPublicationAdapterTest.java
Files — modify:
-
src/adapter/outbound/messaging/build.gradle -
src/adapter/outbound/messaging/gradle.lockfile -
Add
implementation 'org.springframework.kafka:spring-kafka'; accept the Spring Boot 4.0.0 BOM version unless a separately reviewed compatibility override is necessary. Regenerate and review the messaging lockfile. -
RED test exact producer properties and
DefaultKafkaProducerFactory<byte[], byte[]>/KafkaTemplate<byte[], byte[]>. Use byte serializers; no JSON serialization in Kafka callbacks. -
RED gateway tests for:
```text future success + metadata + expected topic -> ACKNOWLEDGED future success + metadata + wrong topic -> ACKNOWLEDGED_MISMATCH definite pre-admission/local rejection -> REJECTED ambiguous/post-admission/deadline/unknown -> INDETERMINATE ``` -
Build
ProducerRecord<byte[],byte[]>only from compiled topic, stored partition-key bytes, exact envelope bytes and bounded allowlisted headers. -
Await the future to a monotonic application deadline and verify non-null metadata. Do not call per-message
flush().cancel()is not delivery cancellation evidence. -
Map Kafka exception categories to stable stage/class/certainty/disposition without exposing class names or messages. Ambiguity defaults to INDETERMINATE.
-
The adapter returns bounded provider generation, local ACK observation time and safe opaque record reference. Application never routes from it.
-
Verify RED then GREEN:
```bash cd src && ./gradlew :adapter:outbound:messaging:test \ --tests '*KafkaProducerFactoryConfigTest' \ --tests '*SpringKafkaPublishGatewayTest' \ --tests '*KafkaPublicationFailureClassifierTest' \ --tests '*AckAwareOutboxPublicationAdapterTest' --console=plain cd src && ./gradlew verifyDependencyLocks --console=plain ``` -
Acceptance claim: fake-gateway ACK-aware provider candidate; real Kafka ACK remains Task 21.
Rollback checkpoint: producer beans remain gated/dark and v2 scheduler off.
Task 16: Add bounded admission, late-completion source and producer generation lifecycle
Owner leaves: application-core (:application-core),
adapter:outbound:messaging (:adapter:outbound:messaging)
Depends on: Task 15
Files — create:
src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/PublicationGenerationLifecyclePort.javasrc/application-core/src/main/java/dev/caskeleton/application/messaging/publication/PublicationGenerationDrainResult.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RotatePublicationGenerationCommand.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RotatePublicationGenerationResult.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RotatePublicationGenerationUseCase.javasrc/application-core/src/test/java/dev/caskeleton/application/outbox/RotatePublicationGenerationUseCaseTest.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/publication/BoundedPublicationAdmission.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/observation/BoundedLatePublicationObservationSource.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerGeneration.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerGenerationManager.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerLifecycle.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/publication/BoundedPublicationAdmissionTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/observation/BoundedLatePublicationObservationSourceTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerGenerationManagerTest.java
Files — modify:
-
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGateway.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/publication/AckAwareOutboxPublicationAdapter.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerFactoryConfig.java -
src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGatewayTest.java -
RED test finite permit count/queue wait, deadline inclusion, saturation, interrupt, release-on-all outcomes and zero queued outbox rows after admission exhaustion.
-
Implement one atomic terminal marker per send. If deadline wins, synchronous outcome stays INDETERMINATE; one later successful callback may enqueue one payload-free late observation.
-
Wire the actual Kafka future callback in
SpringKafkaPublishGatewayto the bounded source, capturing only stable event ID, delivery generation, publication attempt ID, producer generation and binding revision before send. Test before-deadline completion, callback-wins, deadline-wins, duplicate callback, late success, late failure, overflow and generation-close race against the same atomic terminal marker. -
RED test observation source lease/poll/ACK/release, bounded capacity, duplicate callback, timeout-callback race, queue overflow/drop metric and payload/header absence.
-
Overflow never mutates delivery state. It degrades readiness and alerts; capacity qualification requires zero drop.
-
RED producer generation tests for: stop admission/claim; bounded drain; unresolved attempts durably reported INDETERMINATE/HOLD before swap; bounded old close; secret generation resolve; new create/attest; global generation barrier; no old/new overlap.
-
Keep the messaging implementation provider-only:
PublicationGenerationLifecyclePortreturns bounded admitted/in-flight resolution facts and performs pause/drain/create/attest/close, but it imports no outbox store, transaction or persistence type and never chooses HOLD/retry policy. -
RotatePublicationGenerationUseCaseowns orchestration. It pauses new admission/claim through provider-neutral ports, asks the provider to drain, persists every unresolved durable attempt as INDETERMINATE and every affected ordered scope as HOLD throughOutboxDeliveryStorePort/TransactionPort, then permits close/create/attest/barrier switch. Bootstrap invokes this use case; messaging configuration never calls persistence directly. -
A DB outage preventing durable INDETERMINATE/HOLD blocks generation switch and keeps admission closed. Best-effort unresolved sends may return INDETERMINATE but are never auto-replayed.
-
Fatal producer state blocks new admission, lowers readiness and recreates a new immutable generation; it does not change acceptance certainty or remove duplicate risk.
-
Verify RED then GREEN:
```bash cd src && ./gradlew :adapter:outbound:messaging:test \ --tests '*BoundedPublicationAdmissionTest' \ --tests '*BoundedLatePublicationObservationSourceTest' \ --tests '*KafkaProducerGenerationManagerTest' --console=plain cd src && ./gradlew :application-core:test \ --tests '*RotatePublicationGenerationUseCaseTest' --console=plain ``` -
Acceptance claim: bounded local resource/lifecycle protocol candidate; security/topology and real broker evidence remain.
Rollback checkpoint: close the dark producer generation and keep canonical scheduler off.
Task 17: Attest external topic topology and production Kafka security
Owner: adapter:outbound:messaging (:adapter:outbound:messaging)
Depends on: Task 16
Files — create:
-
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecurityProfile.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecuritySettings.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretReference.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretMaterial.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretMaterialResolver.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaTopicTopologyAttestor.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProvisioningEvidence.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaTopicAttestation.java -
src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecuritySettingsTest.java -
src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaTopicTopologyAttestorTest.java -
src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretRedactionTest.java -
RED test typed allowlist: local-only plaintext, TLS server auth, and production
SASL_SSL + SCRAM-SHA-512. First production tuple rejects PLAIN/OAuth/mTLS profiles, plaintext downgrade, trust-all, hostname verification disable and literal JAAS credentials. -
KafkaSecretMaterialexposes no secret intoString, exception, descriptor or log; it carries bounded generation/expiry and clear/close lifecycle. Configuration carries onlysecret://messaging/kafka/producer. -
Freeze the first supported resolver boundary as
mounted-secret-files-v1.KafkaSecretMaterialResolveraccepts only the exact typed reference and returns SCRAM username/password plus trust material, generation and expiry; no application/shared type contains these provider details. Unknown scheme/path traversal, missing field, wrong permission/format, expired generation and literal credential fail closed. -
RED topology tests for topic existence, partitions, RF, min ISR, cleanup policy, retention, max bytes, leader/ISR and wrong cluster/binding.
-
Runtime AdminClient uses only bounded
Describeand exact-topicDescribeConfigs. It never creates/alters/deletes topics, enumerates all ACLs or requires broker-wide configuration. -
Assert the same resolved generation is applied to both producer factory and AdminClient:
security.protocol=SASL_SSL,sasl.mechanism=SCRAM-SHA-512, hostname verification enabled and no literal JAAS value in settings/descriptor/log. A partial producer-only or AdminClient-only resolution fails startup. -
Provisioning evidence supplies runtime-inaccessible assertions: broker policy, auto-create/unclean election, exact positive/negative ACL probes, cluster/topic resource identity, config/ACL digest, issuer/provenance, generated/expiry time and release assertion digest.
-
Missing, stale, wrong-cluster, invalid provenance or runtime/provisioning mismatch prevents
ACTIVE_READY. Transient broker unavailability yields boundedACTIVE_NOT_READY; static credential/security/binding errors fail closed. -
Verify RED then GREEN:
```bash cd src && ./gradlew :adapter:outbound:messaging:test \ --tests '*KafkaSecuritySettingsTest' \ --tests '*KafkaTopicTopologyAttestorTest' \ --tests '*KafkaSecretRedactionTest' --console=plain ``` -
Acceptance claim: local topology/security validation candidate; actual TLS/SASL/ACL and multi-broker evidence remain Wave E.
Rollback checkpoint: attestation failure keeps relay admission off; it never falls back to topic auto-create, wildcard ACL or plaintext.
Task 18: Expose the authenticated, idempotent disposition endpoint
Owner: adapter:inbound:web (:adapter:inbound:web)
Depends on: Task 12
Files — create:
src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/controller/MessagingOutboxDispositionController.javasrc/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/dto/request/OutboxDispositionRequest.javasrc/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/dto/response/OutboxDispositionResponse.javasrc/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/mapper/OutboxDispositionWebMapper.javasrc/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/controller/MessagingOutboxDispositionControllerWireTest.javasrc/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/mapper/OutboxDispositionWebMapperTest.javasrc/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/controller/MessagingOutboxDispositionOpenApiContractTest.javasrc/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/ratelimit/MessagingOutboxDispositionRateLimitTest.javasrc/adapter/inbound/web/src/test/resources/openapi/messaging-outbox-disposition-openapi-snapshot.json
Files — modify:
-
src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/error/GlobalExceptionHandler.java -
src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/authz/RolePermissionPolicyTest.java -
src/adapter/inbound/web/README.md -
src/adapter/inbound/web/CLAUDE.md -
RED wire tests for:
```text POST /internal/operations/messaging/outbox/{eventId}/dispositions required header: Idempotency-Key base permission: outbox:disposition destructive permission: outbox:disposition:destructive ``` -
Cover unauthenticated, insufficient permission, missing/malformed key, invalid DTO, stale generation/version, idempotency replay/mismatch, live claim conflict, horizon exceeded, missing destructive approval/compensation reference and success response.
-
Request contains expected delivery generation, expected row version, closed disposition, bounded reason and incident/change reference. Mapper converts
AuthenticatedPrincipal/request/path/header to framework-free command; no web/security type crosses into application. -
Use
IdempotencyKeySupportto build the existing principal/use-case-scopedIdempotencyScopeand computeRequestFingerprintfrom the canonical disposition request fields, including event ID, expected generation/version, disposition, reason, incident, approval and compensation reference. Pass the resultingIdempotencyContextto the use case; never pass only a raw header string. -
Reuse
IdempotencyKeySupportand existing authorization enforcement. Controller calls onlyApplyOutboxDispositionUseCase; it imports no repository, entity, outbound adapter or transaction manager. -
Keep the endpoint authenticated and absent from the public-path allowlist. Add explicit internal network/rate-bound contract and a committed endpoint OpenAPI snapshot. The public path snapshot must remain unchanged;
verifyPublicPathSnapshotproves the endpoint was not accidentally allowlisted. -
Map stale CAS to conflict, invalid policy to safe 4xx, authorization to existing envelope and unknown failures to safe 5xx without event payload/hash leakage.
-
Verify RED then GREEN:
```bash cd src && ./gradlew :adapter:inbound:web:test \ --tests '*MessagingOutboxDispositionControllerWireTest' \ --tests '*OutboxDispositionWebMapperTest' \ --tests '*MessagingOutboxDispositionOpenApiContractTest' \ --tests '*MessagingOutboxDispositionRateLimitTest' \ --tests '*RolePermissionPolicyTest' --console=plain cd src && ./gradlew verifyPublicPathSnapshot --console=plain ``` -
Acceptance claim: authenticated transport mapping candidate; persistence/application tests remain authority for policy/CAS.
Rollback checkpoint: disable route exposure through composition/network policy, not by allowing raw SQL mutation.
Task 19: Compose exact tuple, schedulers, readiness and observability
Owner: app-bootstrap (:app-bootstrap)
Depends on: Tasks 14–18
Files — create:
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfig.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityReadiness.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingRuntimeDescriptor.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MountedKafkaSecretResolverSettings.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MountedKafkaSecretMaterialResolver.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/KafkaSecretRefreshScheduler.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/LatePublicationObservationScheduler.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/transaction/PersistenceTransactionResourceDescriptor.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfigTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityReadinessTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingRuntimeDescriptorTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MountedKafkaSecretMaterialResolverTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/KafkaSecretRefreshSchedulerTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/LatePublicationObservationSchedulerTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingDisabledZeroResourceContractTest.java
Files — modify:
-
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java -
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxSettings.java -
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRelayScheduler.java -
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxMetrics.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxConfigTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxSettingsTest.java -
src/app-bootstrap/src/main/resources/application.yml -
src/app-bootstrap/src/test/resources/application-test.yml -
src/app-bootstrap/README.md -
src/app-bootstrap/CLAUDE.md -
RED composition tests prove settings bind/compile before secret/client, then producer, topic attestation, readiness, v2 relay and late-drain scheduler in that order.
-
Bootstrap aggregates leaf descriptors and same transaction resource identity only. It must not reimplement catalog/schema/retry/disposition/provider rules.
-
Persistence exposes a sanitized
transactionResourceIdplus resolved DataSource/EntityManagerFactory/PlatformTransactionManager identity descriptor. Bootstrap compares it withTransactionPort, canonical append adapter and the business repository resource before creating ACTIVE clients or schedulers. Add a composition RED/GREEN case where a second DataSource causes startup rejection and Kafka/network resource count remains 0. -
Implement
mounted-secret-files-v1in bootstrap with an explicit bounded root, exact reference-to-directory mapping, no symlink/path escape, owner/permission checks where the platform exposes them, atomic generation manifest read, expiry validation, redacted failure and prompt clearing of old char/byte material. The refresh scheduler invokesRotatePublicationGenerationUseCase; it never mutates a live producer object. -
Missing/wrong/expired secret blocks ACTIVE before producer/AdminClient creation. Refresh failure may retain the old generation only until its configured expiry/safety margin, then closes admission/readiness. DISABLED creates resolver/refresh/file-watch resource count 0.
-
Gate the v2 relay and late-drain scheduler on canonical ACTIVE +
POLLING_V2epoch + fresh producer/topic/security readiness. Remove the legacyrelay-enabledboolean from canonical mode. -
Exact
LEGACY_POLLINGcompatibility composition may exposeLegacyOutboxAppendPort; canonicalPOLLING_V2composition must assert legacy append/store/publish/relay bean count 0. -
Readiness roles stay separate:
```text relay = producer + topic/security + DB claim + catalog durable write = DB append + backlog capacity direct required producer = producer/topic/security liveness = process-internal only ``` -
Add bounded hysteresis/freshness and explicit
STARTING|ACTIVE_NOT_READY|ACTIVE_READY. Static configuration/security mismatch fails startup; transient broker outage never starts relay admission. -
Runtime descriptor exposes only card IDs, versions, catalog/schema/settings digests, destination aliases/revisions, resource ID, epoch/authority, generation, readiness, evidence status, non-guarantees and runbook IDs. Redact servers/topics where policy requires; never expose credentials/payload/hash/raw headers.
-
Replace legacy metrics with bounded dimensions for logical attempt, certainty, failure stage, claim conflict/lease/backlog/order block/generation/late-drop. Reject event/aggregate/tenant/ key/correlation/hash/exception-message tags. One confirmed persisted transition owns the canonical error.
-
DISABLED integration test asserts client/factory/template/AdminClient/semaphore/queue/thread/ scheduler/secret resolver/network count 0.
-
Verify RED then GREEN:
```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*MessagingCapability*Test' \ --tests '*MessagingRuntimeDescriptorTest' \ --tests '*MountedKafkaSecretMaterialResolverTest' \ --tests '*KafkaSecretRefreshSchedulerTest' \ --tests '*LatePublicationObservationSchedulerTest' \ --tests '*MessagingDisabledZeroResourceContractTest' \ --tests '*OutboxConfigTest' \ --tests '*OutboxSettingsTest' --console=plain ``` -
Acceptance claim: complete dark reference graph candidate; target deployment authority remains legacy until Task 25.
Rollback checkpoint: keep canonical expected-state DISABLED and LEGACY_POLLING ACTIVE. No DB
schema downgrade.
Task 20: Implement and rehearse the fenced authority cutover without production switch
Owner leaves: application-core, adapter-outbound-persistence-jpa, app-bootstrap,
adapter-outbound-messaging
Depends on: Tasks 13–19
Base template gate: fresh or verified empty/drained V3 only
Every cutover in this task runs against a disposable rehearsal database and test broker. It proves
the code/protocol but does not change a target deployment, start its v2 relay, resume its business
writes or delete legacy runtime. Production remains LEGACY_POLLING.
Files — create:
src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxWriteAdmissionControlPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelayControlPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxWriteAdmissionSnapshot.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelaySnapshot.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionEvidence.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverCommand.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverResult.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAuthorityCutoverPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCase.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionCommand.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionResult.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionUseCase.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxPreCommitRecoveryPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityCommand.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityResult.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCase.javasrc/application-core/src/test/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCaseTest.javasrc/application-core/src/test/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionUseCaseTest.javasrc/application-core/src/test/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCaseTest.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxCutoverPreconditionAdapter.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapter.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxWriteAdmissionEntity.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxRuntimeNodeLeaseEntity.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxWriteAdmissionJpaRepository.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxRuntimeNodeLeaseJpaRepository.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/PostgreSqlOutboxWriteAdmissionGuard.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxWriteAdmissionControlAdapter.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxRuntimeNodeLeaseAdapter.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapter.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapterTest.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxWriteAdmissionControlAdapterTest.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxRuntimeNodeLeaseAdapterTest.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapterTest.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFence.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFenceTest.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinator.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinator.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapter.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRuntimeNodeLeaseScheduler.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverJobSettings.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunner.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinatorTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinatorTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapterTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxRuntimeNodeLeaseSchedulerTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunnerTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyToV2CutoverContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxWriteAdmissionMultiNodeContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2SentinelContractTest.java
Files — modify:
-
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/transaction/SpringTransactionPort.java -
src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/transaction/SpringTransactionPortTest.java -
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaper.java -
src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaperTest.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapter.java -
src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java -
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java -
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRelayScheduler.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxConfigTest.java -
src/app-bootstrap/build.gradle -
src/build.gradle -
RED application tests scope
FinalizeOutboxAuthorityCutoverUseCaseto the atomic database finalization contract. It accepts an opaque, human-approved cutover evidence ID and never trusts command booleans for writer/relay/producer drain. Before reconciliation it must CAS the exact fresh attemptCUTOVER_PENDING -> FINALIZING_V2in the same database transaction that commits the epoch, after taking the global write-admission/ACTIVE-epoch locks in the fixed order and proving it is the sole nonterminalOUTBOX_PUBLICATIONattempt;RECOVERING_LEGACY, an expired attempt or a different operation owner is a hard rejection. Rollback restoresCUTOVER_PENDING, while a successful epoch commit recordsCONSUMED_V2. This is a one-shot maintenance use case, not a second web endpoint. -
OutboxLegacyToV2CutoverCoordinatoris the deployment/composition owner. ThroughOutboxWriteAdmissionControlPort,LegacyOutboxRelayControlPortandPublicationGenerationLifecyclePort, it freezes writes, drains writers/relay/futures, closes/fences legacy Write, compiles/attests the canonical tuple and asksOutboxCutoverPreconditionPortto persist a short-lived one-shot evidence record containing exact node/writer/relay/producer generations, zero-active facts, epoch, manifest digest, approver and expiry inCUTOVER_PENDING. Evidence creation uses the global lock order write-admission singletonFOR UPDATEthen ACTIVE epochFOR UPDATE, requires the exact FROZEN generation/target binding and rejects any nonterminal attempt inOUTBOX_PUBLICATION; the partial unique constraint is the final concurrent-insert guard. Bootstrap imports only application ports; it never queries repositories or Kafka adapter internals. -
Implement the production write fence with the PostgreSQL singleton created in Task 7.
SpringTransactionPort.inWritebegins its transaction, acquiresFOR KEY SHAREthroughPostgreSqlOutboxWriteAdmissionGuard, and verifies OPEN + expected fence generation before invoking any business action. The control adapter takesFOR UPDATE, waits for all older share-holding writers to commit/rollback, writes FROZEN generation and then returns a durable zero-active snapshot. NewinWritecalls fail and roll back;inNewremains available only for maintenance/outbox/audit and never bypasses a business write. -
Implement both explicit generation-CAS exits from FROZEN without a raw status update.
ResumePollingV2WriteAdmissionUseCaserequires expected frozen generation, exact ACTIVEPOLLING_V2epoch, canonical cutover sentinel created throughOutboxAppendAdapterand persisted asDELIVERY_RECORDED, fresh target binding/readiness and no legacy runtime. It writesOPEN(generation+1)once; mismatch/replay/failure leaves FROZEN.RecoverLegacyOutboxAuthorityUseCaseis legal only before epoch commit. Before any external ACL mutation, its prepare operation takes the same global write-admission/ACTIVE-epoch lock order, proves the exact attempt is the sole nonterminal authority attempt plus FROZEN/LEGACY_POLLING/zero-v2-authority, then CAS-claimsCUTOVER_PENDING -> RECOVERING_LEGACYand atomically invalidates that attempt for v2 finalization. Choosing this branch is irreversible for that attempt; only recovery completion or recovery-only lease takeover remains legal. The complete operation follows the separately fenced recovery protocol below. -
Register bounded
outbox_runtime_node_leaseheartbeats for every runtime node with node ID, source/artifact digest, write-fence protocol version, epoch and scheduler roles. Precondition evidence requires all deployment-inventory instances to have a matching fresh lease and rejects stale, unknown, pre-fence or missing nodes. A lease table alone does not prove the absence of an unregistered process; target deployment inventory/provenance is also mandatory. -
LegacyOutboxRelayControlAdapterowns composition of existing runtime controls: pause newOutboxRelaySchedulercycles, wait active cycles/claims to the finite deadline, pause/drain the epoch-fencedOutboxReaper, and closeLegacyPublicationWriteFencesoOutboxMessagePublishAdapterrejects every later send. Snapshot counts/generations are bounded facts only. Reopen methods require expected component generations plus fresh pre-commit recovery evidence and reject once ACTIVE epoch is notLEGACY_POLLING; no generic boolean setter exists. Application cutover policy sees the port, not concrete scheduler/reaper/messaging types. -
The disposable security rehearsal and actual target preflight use distinct legacy/canonical principals. Revoke legacy exact-topic Write and require a negative Write probe while canonical Describe/DescribeConfigs/Write stays positive. The in-process fence plus external ACL evidence are both required; neither substitutes for the other.
-
Add the exact non-web one-shot operational entrypoint
MessagingAuthorityCutoverApplicationRunner. It activates only for the closed operationslegacy-to-polling-v2,recover-legacy-precommit, orresume-polling-v2-writes. It requires opaque operation/approval-evidence IDs plus expected target/source/artifact/epoch/fence generation, invokes only the corresponding application use case/coordinator, emits no payload/secret, and exits non-zero on mismatch/replay/failure. The main cutover exits 0 only after sentinel/readiness proof and write admissionOPEN(generation+1); a post-commit resume failure stays FROZEN and requires the separately one-shotresume-polling-v2-writesoperation. Consumed DB evidence/operation IDs make retries non-reentrant; no controller endpoint is added.```text ca-skeleton.messaging.maintenance.operation ca-skeleton.messaging.maintenance.operation-id ca-skeleton.messaging.maintenance.approval-evidence-id ca-skeleton.messaging.maintenance.expected-target-alias ca-skeleton.messaging.maintenance.expected-source-digest ca-skeleton.messaging.maintenance.expected-artifact-digest ca-skeleton.messaging.maintenance.expected-epoch ca-skeleton.messaging.maintenance.expected-fence-generation ``` Task 23 registers these exact maintenance-only keys and the runbook's non-web launcher contract; none has a default that enables the runner. -
In the rehearsal harness: deploy ACK-aware producer/v2 relay scheduler-disabled; compile the candidate tuple; prove disposition auth/CAS negatives; execute the real PostgreSQL business-write admission freeze; drain active writers/epoch share holders; stop legacy new claims and legacy reaper; drain
IN_FLIGHTto the maximum budget; audit remaining indeterminate; close/fence legacy producer Write and DB legacy mutation. -
Real multi-node PostgreSQL tests hold old
inWritetransactions across freeze, start new writers during/after freeze, inject a stale/pre-fence node lease and omit a deployment inventory member. Freeze must wait for old holders, reject new writes without partial business/ outbox state, and refuse evidence until every live instance/fence/relay/reaper/producer fact is exact and zero-active. -
In one PostgreSQL transaction:
```text lock OUTBOX_PUBLICATION write-admission singleton FOR UPDATE -> lock ACTIVE LEGACY_POLLING epoch FOR UPDATE -> assert exact FROZEN generation/target binding and sole nonterminal attempt -> lock exact fresh cutover attempt -> CAS CUTOVER_PENDING -> FINALIZING_V2 -> assert writer/legacy mutation fences -> capture fixed legacy handoff watermark -> final reconcile every row through watermark including final delta -> assert exactly one CURRENT delivery per event, active claims 0 -> assert row count + event ID/hash manifest, unmapped/duplicate count 0 -> switch ACTIVE epoch LEGACY_POLLING -> POLLING_V2 -> append v2 cutover sentinel through the canonical append adapter with retained V3 projection + CURRENT/READY delivery -> mark the same attempt CONSUMED_V2 -> commit ``` -
Missing, expired, reused, wrong-epoch/generation, wrong-manifest or non-zero cutover evidence rolls back before reconciliation.
RECOVERING_LEGACY,RECOVERED_LEGACY, a foreign recovery owner or any non-CUTOVER_PENDINGstate also rejects finalization. The final transaction marks the evidence consumed; the coordinator cannot replay it. -
Migration-only state mapping is exact:
```text PENDING -> READY FAILED -> RETRY_WAIT with reviewed DB-time due/budget DEAD -> EXHAUSTED PUBLISHED -> LEGACY_RECORDED_UNVERIFIED IN_FLIGHT -> HOLD + remaining-indeterminate audit ``` Preserve reviewed attempt count/due/deadline and every historical observation; never fabricate broker metadata, definite rejection, ACK observed time or `DELIVERY_RECORDED`. -
Any unknown contract/status/hash mismatch, duplicate current row, count/manifest mismatch, active claim, fence failure or sentinel failure rolls the whole transaction back and leaves
LEGACY_POLLINGauthoritative. -
In the disposable rehearsal only, after commit start v2 relay, require sentinel ACK/
DELIVERY_RECORDED, prove the sentinel used the canonical append path and retained V3 NOT NULL projection, verify legacy writer/claim/reaper/send 0, then execute the exact FROZEN→OPEN generation CAS. Only after OPEN, exercise an ordinaryTransactionPort.inWritecanonical append; failure immediately freezes a new generation and fails the rehearsal. -
RED/GREEN fault cases at every numbered point, including crash before transaction, after watermark, during final delta, before epoch switch, before/after sentinel insert and after commit. No case permits dual authority or missing manifest row.
-
Rehearse the two authority zones and both pre-commit choices:
```text pre-commit: keep all fences closed -> bounded forward retry while evidence/approval remains fresh OR prove epoch still LEGACY_POLLING + v2 business send/sentinel authority 0 + exact inventory -> DB-CAS exact attempt CUTOVER_PENDING -> RECOVERING_LEGACY and atomically invalidate it for every forward finalizer -> externally regrant legacy exact-topic Write and verify a fresh positive probe -> re-lock attempt + epoch and revalidate RECOVERING_LEGACY owner/lease + LEGACY_POLLING (on mismatch immediately revoke legacy Write and prove a fresh negative probe) -> append immutable recovery/ACL audit -> reopen in-process legacy Write fence, reaper and relay with expected generations -> CAS write admission FROZEN -> OPEN(generation+1) and mark RECOVERED_LEGACY last post-commit, regardless of whether a business v2 send occurred: legacy reactivation/reverse epoch is unsupported -> keep write admission FROZEN -> after sentinel/readiness/canonical-projection proof, CAS OPEN(generation+1) -> otherwise preserve backlog/schema/epoch/audit and forward-fix ``` Once `RECOVERING_LEGACY` is claimed, no forward retry or separately created attempt can commit in `OUTBOX_PUBLICATION`, including after recovery lease expiry. Inject barrier races in both lock orders for same-attempt finalization versus recovery prepare, different-attempt creation/ finalization versus recovery prepare/completion, plus crash/abort before and after the durable recovery claim, external ACL regrant, post-ACL epoch revalidation, each component reopen and final admission CAS. Wrong epoch/inventory/ACL evidence, stale generation, partial legacy reopen, duplicate operation, resume-before-sentinel and DB failure must never open business writes. If post-ACL revalidation fails, immediately revoke legacy Write and require a new negative probe; if a legacy component was reopened but final admission CAS failed, business writes stay FROZEN and the coordinator re-fences or records the exact safe degraded state for idempotent recovery-only retry. -
Verify:
```bash cd src && ./gradlew :application-core:test \ --tests '*FinalizeOutboxAuthorityCutoverUseCaseTest' \ --tests '*ResumePollingV2WriteAdmissionUseCaseTest' \ --tests '*RecoverLegacyOutboxAuthorityUseCaseTest' --console=plain cd src && ./gradlew :adapter:outbound:persistence-jpa:test \ --tests '*OutboxAuthorityCutoverAdapterTest' \ --tests '*OutboxWriteAdmissionControlAdapterTest' \ --tests '*OutboxRuntimeNodeLeaseAdapterTest' \ --tests '*OutboxPreCommitRecoveryAdapterTest' \ --tests '*SpringTransactionPortTest' --console=plain cd src && ./gradlew :adapter:outbound:messaging:test \ --tests '*LegacyPublicationWriteFenceTest' \ --tests '*OutboxMessagePublishAdapterTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*OutboxLegacyToV2CutoverCoordinatorTest' \ --tests '*OutboxLegacyPreCommitRecoveryCoordinatorTest' \ --tests '*LegacyOutboxRelayControlAdapterTest' \ --tests '*OutboxRuntimeNodeLeaseSchedulerTest' \ --tests '*MessagingAuthorityCutoverApplicationRunnerTest' --console=plain cd src && ./gradlew :app-bootstrap:test \ --tests '*OutboxLegacyToV2CutoverContractTest' \ --tests '*OutboxWriteAdmissionMultiNodeContractTest' \ --tests '*OutboxV2SentinelContractTest' --console=plain ``` -
After the disposable RED/GREEN matrix passes, schema-validate and write:
```text src/app-bootstrap/build/messaging-evidence/cutover-rehearsal/manifest.json ``` It binds the supplied source/artifact digest, migration/card/profile/catalog/settings hashes, disposable database/broker identity, every fault point and rollback-zone scenario, exact row/ manifest/sentinel assertions, pre-commit recovery and post-commit resume generation-CAS scenarios, commands/timestamps, failed=0, skipped=0 and the explicit non-claim `targetDeploymentCutOver=false`. It conforms to the common build-evidence schema. -
Acceptance claim: cutover implementation and disposable fault rehearsal candidate only. Target deployments remain
LEGACY_POLLING; no legacy code/config is deleted and the tuple remains non-R2 until Wave E evidence.
Rollback checkpoint: discard the rehearsal database/broker. Never apply rehearsal evidence as a target deployment switch or destructively downgrade schema.
Wave D exit checkpoint
-
Run focused owner checks and:
```bash cd src && ./gradlew verifyMessagingContracts \ verifyMessagingJsonSchemaV1 \ verifyMessagingPollingOutboxR2 \ verifyCleanArchitectureDependencies \ verifyEnvKeys \ verifyPublicPathSnapshot \ --console=plain ``` -
Keep Kafka/security cards at most
implemented-candidate. -
Update the design ledger to
P3=IMPLEMENTED_CANDIDATEonly after the dark graph and disposable cutover rehearsal pass; target authority is still legacy. -
Update the LLM Wiki branch-note and derived-document decision.
Wave E — P4 real-service, security, fault and release qualification
Task 21: Prove real PostgreSQL + real Kafka reference behavior
Owner: app-bootstrap qualification tests
Depends on: Wave D
Files — create:
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingKafkaR2ContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingPollingKafkaEndToEndContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingKafkaFaultContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingKafkaContainerSupport.javasrc/app-bootstrap/src/test/resources/messaging/evidence/messaging-evidence-schema-v1.json
Files — modify:
-
src/app-bootstrap/build.gradle -
src/app-bootstrap/gradle.lockfile -
src/build.gradle -
Add test-only:
```groovy testImplementation 'org.testcontainers:testcontainers-kafka' testImplementation 'org.testcontainers:testcontainers-toxiproxy' testImplementation 'org.springframework.kafka:spring-kafka-test' ``` Pin container image digest in qualification settings and record broker/client/Spring versions. -
Register
:app-bootstrap:messagingKafkaProducerR2with these exact filters andfailOnNoMatchingTests=true:```text dev.caskeleton.bootstrap.integration.messaging.MessagingKafkaR2ContractTest dev.caskeleton.bootstrap.integration.messaging.MessagingPollingKafkaEndToEndContractTest dev.caskeleton.bootstrap.integration.messaging.MessagingKafkaFaultContractTest ``` Root `verifyMessagingKafkaProducerR2` depends on that Test task and validates its evidence. Docker/image pull/test skip is failure, not PASS. -
After registering the task but before implementing the three tests, run RED:
```bash cd src && ./gradlew :app-bootstrap:messagingKafkaProducerR2 --console=plain ``` Expected non-zero: no matching required tests or absent real-service evidence. Any unrelated compile failure must be fixed before proceeding. -
Real Kafka RED/GREEN cases: actual topic/partition/offset metadata; expected-topic mismatch;
acks=all/idempotence effective config; stable key/partition; header/record oversize; missing topic with auto-create disabled; broker unavailable before send; leader/retriable failure; response loss/deadline/ late ACK; local buffer saturation/max-block; throttle; no per-message flush; graceful/forced close; fatal generation recreation. -
Combined real PostgreSQL + Kafka cases: event/delivery commit; JIT claim/admission; ACK → delivery CAS; ACK-to-DB crash/reclaim duplicate; stale token after late ACK; outcome commit failure; late DB-commit-before-source-ACK duplicate absorption; backlog outage/recovery; multi-worker disjoint claim/order/fairness.
-
Add the rolling-compatibility golden path against the real broker: canonical append while
LEGACY_POLLINGis active → legacy claim → broker-observed exact compiled topic, stored key and byte-identical v1 envelope → legacy terminalPUBLISHED→ disposable cutover mapsLEGACY_RECORDED_UNVERIFIEDwith no automatic v2 resend. Nested envelope, legacy event-type routing for a canonical row and mixed metadata are negative cases. -
Fault injection must observe both broker event IDs and DB state at:
```text after event commit after claim commit before send after request write after broker append before ACK receipt after ACK before DB transition during DB transition commit after DB success before scheduler result during shutdown ``` -
Single-node evidence is labelled provider baseline only. It cannot satisfy RF/min ISR, leader-loss or production security rows.
-
Generate a sanitized manifest at the non-versioned exact path:
```text src/app-bootstrap/build/messaging-evidence/real-kafka-postgresql-r2/manifest.json ``` Validate it against `src/app-bootstrap/src/test/resources/messaging/evidence/messaging-evidence-schema-v1.json`. Also validate the same bytes against `src/config/messaging/evidence/build-evidence-manifest-v1.schema.json`; the lane schema may add fields but cannot weaken the common source/artifact/scenario/failure/skip contract. CI retains the same bytes under `ci-artifact://messaging/{sourceDigest}/real-kafka-postgresql-r2/manifest.json`. The manifest contains source/artifact digest supplied by human/CI, commands/timestamps, test counts, versions/image digests, non-secret effective settings, hashes, scenarios/results, skips/failures, unsupported claims and runbook IDs. -
Re-run GREEN:
```bash cd src && ./gradlew :app-bootstrap:messagingKafkaProducerR2 \ verifyMessagingKafkaProducerR2 \ verifyMessagingPollingOutboxR2 --console=plain ``` Expected: all listed scenario IDs occur exactly once, failed=0, skipped=0, schema validation PASS and source/artifact digests match. -
Acceptance claim: real single-node Kafka + PostgreSQL R2-candidate evidence; production tuple remains NOT_QUALIFIED.
Rollback checkpoint: qualification uses disposable services. Production activation is still blocked by Task 22.
Task 22: Qualify SASL_SSL/SCRAM, least privilege and multi-broker topology
Owner: deployment/security qualification lane + app-bootstrap aggregator
Depends on: Task 21
Files — create:
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/qualification/messaging/MessagingSecurityR2QualificationTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/qualification/messaging/MessagingMultiBrokerR2QualificationTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/qualification/messaging/MessagingRotationShutdownQualificationTest.javasrc/app-bootstrap/src/test/resources/messaging/qualification/docker-compose.kafka-r2.ymlsrc/app-bootstrap/src/test/resources/messaging/qualification/README.mdsrc/config/messaging/evidence/messaging-release-evidence-schema-v1.json
Files — modify:
-
src/app-bootstrap/build.gradle -
src/build.gradle -
src/config/messaging/release-profile-assertions.yaml -
Register three non-ordinary Test tasks with exact filters and
failOnNoMatchingTests=true:```text :app-bootstrap:messagingSecurityR2 -> dev.caskeleton.bootstrap.qualification.messaging.MessagingSecurityR2QualificationTest :app-bootstrap:messagingMultiBrokerR2 -> dev.caskeleton.bootstrap.qualification.messaging.MessagingMultiBrokerR2QualificationTest :app-bootstrap:messagingRotationShutdownR2 -> dev.caskeleton.bootstrap.qualification.messaging.MessagingRotationShutdownQualificationTest ``` Root `verifyMessagingSecurityR2` depends on all three evidence validators. Missing topology, credential fixture, certificate, Docker/image or tests fails the release task. -
After task registration but before the qualification environment/tests are complete, run RED:
```bash cd src && ./gradlew :app-bootstrap:messagingSecurityR2 \ :app-bootstrap:messagingMultiBrokerR2 \ :app-bootstrap:messagingRotationShutdownR2 \ --console=plain ``` Expected non-zero for an exact missing test/topology/security prerequisite. SKIPPED is not an accepted RED or GREEN result. -
Use a pinned three-broker topology with RF=3/min ISR=2 and production-like SASL_SSL/SCRAM-SHA-512. Ephemeral test credentials/certificates never enter source/evidence.
-
Security positive/negative cases: trusted TLS; untrusted CA; hostname mismatch; expired/not-yet-valid cert; valid/invalid SCRAM; missing/expired secret; production plaintext rejection; redaction; exact-topic Describe/ DescribeConfigs/Write; denied Create/Delete/Alter/other-topic Write/consumer Read. Use distinct canonical and legacy fixture principals and prove the legacy principal's exact-topic Write can be revoked without removing canonical Describe/DescribeConfigs/Write.
-
Topology cases: expected partitions/RF/min ISR; cleanup/retention/max bytes drift; wrong cluster/topic; auto-create disabled; leader loss with ISR sufficient; below-min-ISR rejection/indeterminate mapping; recovery; provisioning evidence freshness/provenance.
-
Provisioning evidence includes the selected broker's
replica.lag.time.max.ms, the producer's effectiverequest.timeout.msand the approved compatibility relation from design §16.6. Add a mismatch negative case; do not infer the broker value from a client default. -
Rotation/lifecycle cases: stop admission; bounded old drain; forced unresolved → durable INDETERMINATE/HOLD; old close; new secret/producer/attestation; generation barrier; no old/new overlap; DB-unavailable switch rejection; shutdown under load.
-
Capacity/soak cases: sustained drain, hot aggregate, broker throttle/outage/recovery storm, retry amplification, producer memory/buffer/GC, DB pool/claim query, metric cardinality and late-observation drop 0. Record numbers as selected-environment evidence, not universal repository performance claims.
-
Emit and schema-validate these non-versioned exact files:
```text src/app-bootstrap/build/messaging-evidence/security-r2/manifest.json src/app-bootstrap/build/messaging-evidence/multi-broker-r2/manifest.json src/app-bootstrap/build/messaging-evidence/rotation-shutdown-r2/manifest.json ``` Validate each byte-identical file against both `src/config/messaging/evidence/build-evidence-manifest-v1.schema.json` and the stricter `src/config/messaging/evidence/messaging-release-evidence-schema-v1.json`. Add a contract test proving the lane schema retains every common required field and rejection rule. CI retains byte-identical artifacts under the source-digest-qualified `ci-artifact://messaging/` namespace. Each manifest must match exact source/artifact digest, `qualificationEnvironmentIdentity` (fixture broker/image/principal provenance), topic/security profile, card/settings/catalog/schema hashes, scenario set and freshness window. Any mismatch or skip keeps all affected cards `implemented-candidate`. This identity is never reused as a target `deploymentBindingIdentity`; only capability/profile, supported broker/client version constraints, settings/catalog/schema and scenario-contract revisions are portable. -
Re-run GREEN:
```bash cd src && ./gradlew :app-bootstrap:messagingSecurityR2 \ :app-bootstrap:messagingMultiBrokerR2 \ :app-bootstrap:messagingRotationShutdownR2 \ verifyMessagingSecurityR2 --console=plain ``` Expected: every required scenario ID exactly once, failed=0, skipped=0, all three manifests pass `messaging-release-evidence-schema-v1.json`, and all source/artifact/profile digests match. -
Acceptance claim: exact production security/topology candidate only after all required scenarios PASS. No consumer/CDC claim.
Rollback checkpoint: failed qualification prevents release promotion; do not weaken RF/min ISR, ACL, TLS or card requirements to make the lane green.
Task 23: Synchronize configuration, registries, runbooks and operational truth
Owner: repository documentation/configuration Depends on: Tasks 19–22
Files — modify:
src/app-bootstrap/src/main/resources/application.ymlsrc/app-bootstrap/src/test/resources/application-test.ymlsrc/.envdocs/registries/env-keys.yamldocs/registries/capabilities.yamldocs/registries/error-codes.yamldocs/registries/metrics.yamldocs/registries/secrets-classification.yamlsrc/application-core/README.mdsrc/application-core/CLAUDE.mdsrc/shared-contract/README.mdsrc/shared-contract/CLAUDE.mdsrc/adapter/outbound/messaging/README.mdsrc/adapter/outbound/messaging/CLAUDE.mdsrc/adapter/outbound/persistence-jpa/README.mdsrc/adapter/outbound/persistence-jpa/CLAUDE.mdsrc/adapter/inbound/web/README.mdsrc/adapter/inbound/web/CLAUDE.mdsrc/app-bootstrap/README.mdsrc/app-bootstrap/CLAUDE.mddocs/runbooks/outbox-publish-failed.mddocs/runbooks/outbox-dead-letter.mdsrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/RunbookCoverageContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/outbox/OutboxStatusRegistryContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/outbox/EventPayloadPiiContractTest.java
Files — create:
-
docs/runbooks/messaging-producer-unavailable-or-unauthorized.md -
docs/runbooks/messaging-outbox-backlog-and-stale-lease.md -
docs/runbooks/messaging-delivery-indeterminate-and-duplicate-burst.md -
docs/runbooks/messaging-schema-poison-or-record-too-large.md -
docs/runbooks/messaging-terminal-delivery-disposition.md -
docs/runbooks/messaging-topic-policy-or-partition-change.md -
docs/runbooks/messaging-shutdown-deploy-and-secret-rotation.md -
docs/runbooks/messaging-legacy-to-v2-relay-authority-cutover.md -
Modify the three listed contract tests first, then run RED:
```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*RunbookCoverageContractTest' \ --tests '*OutboxStatusRegistryContractTest' \ --tests '*EventPayloadPiiContractTest' --console=plain ``` Expected non-zero because new status/key/metric/error/runbook/redaction entries are absent. Register owner, type/default/allowlist, secret classification, validation, compatibility impact and required test for each implemented key. -
Remove canonical reliance on:
```text APP_MESSAGING_BROKER APP_MESSAGING_KAFKA_BROKERS ca-skeleton.outbox.relay-enabled ``` in the canonical graph. Retain them only as explicit R0 compatibility inputs through the deployment observation window; legacy + canonical keys fail with no silent precedence. Final removal belongs to Task 26. Do not add `consumer.enabled`, `cdc.enabled` or `schemaRegistry.url`. -
Keep base skeleton default DISABLED with active contracts/destinations/resource count 0. Deployment-specific destination/topic/security values are explicit placeholders or secret references, never usable credentials.
-
Replace producer
DEAD/dead-lettervocabulary withEXHAUSTED; distinguish it from future consumer DLT. Remove raw SQL status rewrite and fabricated consumer-dedupe claims from old runbooks. -
Each first-R2 runbook contains detection, blast radius, guarantee degradation, safe first response, evidence, non-destructive mitigation, destructive approval boundary, reconciliation, recovery proof, rollback, audit and related cards/metrics/errors.
-
The authority-cutover runbook contains the exact pre-commit bounded-forward-retry and abort-to-legacy recovery state machine, the irreversible
CUTOVER_PENDING -> RECOVERING_LEGACYclaim before external mutation, recovery-only lease takeover, theOUTBOX_PUBLICATIONsole-nonterminal-attempt constraint and global lock order, same-/cross-attempt finalizer rejection, external legacy ACL regrant/positive probe, post-ACL epoch revalidation and revoke/negative-probe compensation, expected generation ordering, partial-reopen re-fence behavior, and the post-commitresume-polling-v2-writespath. It explicitly forbids any post-commit legacy reactivation or raw admission/epoch SQL. -
Document exact state/table/class names, token/generation/audit operator API, role readiness, no-dual-authority cutover and non-guarantees. No stub alert/dashboard references count as evidence.
-
Re-run the same three tests GREEN after registries/runbooks are complete, then verify global drift gates:
```bash cd src && ./gradlew verifyEnvKeys \ verifyPublicPathSnapshot \ :app-bootstrap:test \ --tests '*RunbookCoverageContractTest' \ --tests '*OutboxStatusRegistryContractTest' \ --tests '*EventPayloadPiiContractTest' \ --console=plain ``` Expected: all three contract tests PASS, no skip, and env/public-path verification PASS. -
Acceptance claim: documentation/configuration reflects actual implementation and evidence; unexecuted lanes remain NOT_QUALIFIED.
Rollback checkpoint: docs describe deployed/evidenced truth, not preferred state. Never rewrite failed evidence or instruct operators to dual-send/raw-update.
Task 24: Freeze and aggregate the pre-cutover release candidate
Owner: repository-wide verification and documentation Depends on: every selected Task 1–23 requirement
Files — modify:
docs/superpowers/specs/2026-07-28-messaging-production-capability-design.mddocs/superpowers/plans/2026-07-28-messaging-first-r2-polling-producer.mdsrc/config/messaging/readiness-cards.yamlsrc/config/messaging/release-profile-assertions.yamlsrc/build.gradle
Files — create:
-
src/config/messaging/evidence/deployment-rollout-manifest-v1.schema.json -
src/config/messaging/evidence/deployment-binding-attestation-v1.schema.json -
src/config/messaging/evidence/final-r2-profile-v1.schema.json -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingDeploymentRolloutEvidenceContractTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingFinalR2ProfileContractTest.java -
Freeze the candidate source tree and dependency locks. A human supplies the candidate commit/source digest; CI builds the exact artifact. Agent never stages, commits or pushes.
-
Make
verifyMessagingReleaseProfileconsume these exact build outputs:```text src/build/messaging-evidence/contracts-schema/manifest.json src/app-bootstrap/build/messaging-evidence/polling-outbox-r2/manifest.json src/app-bootstrap/build/messaging-evidence/cutover-rehearsal/manifest.json src/app-bootstrap/build/messaging-evidence/real-kafka-postgresql-r2/manifest.json src/app-bootstrap/build/messaging-evidence/security-r2/manifest.json src/app-bootstrap/build/messaging-evidence/multi-broker-r2/manifest.json src/app-bootstrap/build/messaging-evidence/rotation-shutdown-r2/manifest.json ``` Each producer validates its schema before writing. The aggregator verifies all required scenario IDs, source/artifact digest, card/profile/catalog/schema/settings hashes, cluster/ qualification-environment identity, freshness, failed=0 and skipped=0, then writes: ```text src/build/reports/messaging/release-profile/manifest.json ``` CI retains the exact bytes under a source-digest-qualified `ci-artifact://messaging/` release-profile path. -
Revalidate every one of the seven inputs against the common build-evidence schema and its lane-specific schema. Add contract fixtures proving a lane schema cannot omit or relax common source/artifact/scenario/failure/skip fields.
-
Implement fail-closed deployment/final gate contracts before any target cutover:
verifyMessagingTargetBindingPreflightvalidates a non-mutating target preflight;verifyMessagingTargetBindingconsumes the fresh target-specific topology/security/ACL attestation created inside maintenance after the legacy Write fence;verifyMessagingDeploymentCutoverconsumes that immutable original attestation plus the exact local target rollout manifest;verifyMessagingCleanupTargetBindingconsumes a distinct cleanup-artifact attestation;verifyMessagingFinalR2Profilerequires both attestations, the qualified cleanup release manifest, original deployment-cutover manifest and cleanup-rollout manifest, validates the final-profile schema and writes the final aggregate. Missing/stale/ wrong-target/wrong-digest/failed/skipped evidence is non-zero. -
Run the gate contract tests RED then GREEN with checked-in invalid/valid payload-free fixtures:
```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*MessagingDeploymentRolloutEvidenceContractTest' \ --tests '*MessagingFinalR2ProfileContractTest' --console=plain ``` RED is an intentionally invalid fixture accepted or a missing required validator; GREEN means every invalid fixture is rejected and every exact valid fixture is accepted. This does not create target rollout evidence. -
Run focused owner gates sequentially:
```bash cd src && ./gradlew :application-core:check \ :shared-contract:check \ :adapter:outbound:messaging:check \ :adapter:outbound:persistence-jpa:check \ :adapter:inbound:web:check \ :app-bootstrap:check \ :sample-portfolio:check \ --console=plain ``` -
Run Messaging gates:
```bash cd src && ./gradlew verifyMessagingContracts \ verifyMessagingJsonSchemaV1 \ verifyMessagingPollingOutboxR2 \ verifyMessagingKafkaProducerR2 \ verifyMessagingSecurityR2 \ verifyMessagingReleaseProfile \ --console=plain ``` Expected: every exact input manifest exists and validates; no mismatch/stale/skip; aggregate manifest PASS. Missing service/credential/image/test is non-zero, never PASS. -
Run repository gates:
```bash cd src && ./gradlew test --console=plain cd src && ./gradlew check --console=plain cd src && ./gradlew verifyDependencyLocks --console=plain cd src && ./gradlew verifyCleanArchitectureDependencies --console=plain cd src && ./gradlew verifyPublicPathSnapshot --console=plain cd src && ./gradlew verifyEnvKeys --console=plain cd src && ./gradlew verifyOneTypePerFile \ verifyApplicationCoreDependencyPurity --console=plain git diff --check ``` -
Perform independent reviews for: Clean Architecture/module ownership; schema/contract evolution; transaction/concurrency/CAS; Kafka outcome/lifecycle; security/topology; operator endpoint; migration/cutover/rollback; evidence/no-skip/operations. Candidate qualification requires blocker 0 and high 0.
-
From the aggregate only, promote the exact artifact/card rows to
release-eligible. Record §0 asP4=RELEASE_CANDIDATE_QUALIFIED_DEPLOYMENT_NOT_CUT_OVER; target publication authority and production runtime remain legacy. P5/P6 stayDESIGNED_NOT_IMPLEMENTED, P7 staysOPTIONAL_BACKLOG. -
Update the branch-note with release-candidate evidence and explicitly state target cutover/cleanup are pending. Do not make the final implementation-complete claim.
Rollback checkpoint: if aggregation fails, keep cards implemented-candidate, target
LEGACY_POLLING, and preserve schema/backlog/evidence. Never weaken a gate or copy evidence from
another artifact.
Task 25: Execute the approved target deployment cutover
Owner: deployment coordinator + application/persistence cutover protocol Depends on: Tasks 23–24, human deployment approval, exact fresh/empty-drained migration card Source change: none
-
Fail before maintenance unless the target matches the exact Task 24 source/artifact digest, release profile, schema/catalog/settings hashes, supported broker/client constraints and the checked-in cutover runbook. Do not require the target cluster/principal to equal Task 22's qualification fixture. A live non-empty V3 target stops for a separate approved deployment migration plan.
-
Against the actual target, resolve the exact canonical secret generation and run a non-mutating fresh topology/security preflight. Do not revoke the still-authoritative legacy principal before maintenance. Bind the prepared exact ACL change/provenance and emit identical sanitized bytes:
```text src/app-bootstrap/build/messaging-evidence/target-binding-preflight/manifest.json ci-artifact://messaging/{targetAlias}/{sourceDigest}/target-binding-preflight/manifest.json ``` `deploymentBindingIdentity` binds target cluster/topic/canonical and legacy principal identities, canonical secret generation, provisioning provenance and prepared mutation. It proves canonical Describe/DescribeConfigs/Write and validates the target constraints, but explicitly records `legacyWriteRevoked=false`; it is not cutover evidence. -
Run before maintenance:
```bash cd src && ./gradlew verifyMessagingTargetBindingPreflight --console=plain ``` Expected GREEN only for a fresh exact target/source/artifact/release/profile binding with failed=0 and skipped=0. Fixture identity cannot satisfy this gate. -
Before target execution, run the fail-closed rollout gate once with no current rollout artifact:
```bash cd src && ./gradlew verifyMessagingDeploymentCutover --console=plain ``` Expected non-zero: the final in-maintenance target attestation and target rollout artifact are absent. A stale prior-target artifact must fail for target/source/release-digest mismatch, not satisfy this RED. -
Execute the runbook preflight: deploy candidate with v2 relay scheduler-disabled; attest exact tuple; prove disposition auth/CAS; freeze durable business-write admission; drain writers/epoch holders; stop/drain legacy claim and reaper; record remaining IN_FLIGHT as indeterminate/HOLD; fence legacy producer Write and DB mutation. Only after zero-active drain, apply the prepared ACL mutation, require legacy exact-topic Write negative and canonical Describe/DescribeConfigs/Write positive, then write and internally schema-validate:
```text src/app-bootstrap/build/messaging-evidence/target-binding-attestation/manifest.json ci-artifact://messaging/{targetAlias}/{sourceDigest}/target-binding-attestation/manifest.json ``` The one-shot precondition evidence binds this attestation digest. The maintenance runner uses the same fail-closed validator as `verifyMessagingTargetBinding` before it may call finalization; raw server, credential and certificate bytes are excluded. -
Invoke the exact non-web
MessagingAuthorityCutoverApplicationRunneroperationlegacy-to-polling-v2with opaque operation/approval-evidence IDs and expected target/source/artifact/epoch. It calls the coordinator andFinalizeOutboxAuthorityCutoverUseCase; the single transaction repeats the Task 20CUTOVER_PENDING -> FINALIZING_V2CAS, watermark/final-delta/manifest/current-delivery assertions, epoch switch, sentinel insert andCONSUMED_V2transition.RECOVERING_LEGACYis rejected before reconciliation. Any mismatch exits non-zero and rolls back to legacy authority. Reusing the operation or consumed evidence ID exits non-zero without mutation.```text --spring.main.web-application-type=none --ca-skeleton.messaging.maintenance.operation=legacy-to-polling-v2 --ca-skeleton.messaging.maintenance.operation-id={opaqueOperationId} --ca-skeleton.messaging.maintenance.approval-evidence-id={opaqueApprovalEvidenceId} --ca-skeleton.messaging.maintenance.expected-target-alias={targetAlias} --ca-skeleton.messaging.maintenance.expected-source-digest={sourceDigest} --ca-skeleton.messaging.maintenance.expected-artifact-digest={artifactDigest} --ca-skeleton.messaging.maintenance.expected-epoch={legacyEpoch} --ca-skeleton.messaging.maintenance.expected-fence-generation={openFenceGeneration} ``` -
If finalization fails before epoch commit, keep every fence closed. Either retry forward within the still-fresh bounded evidence/approval window, or execute the approved abort-to-legacy protocol. The latter first proves epoch still
LEGACY_POLLING, v2 business send/sentinel authority 0 and exact inventory, then runs the prepare half ofrecover-legacy-precommit: DB-CAS the exact attemptCUTOVER_PENDING -> RECOVERING_LEGACY, bind the recovery operation/lease/evidence digest and atomically make every forward finalizer reject it. This choice is irreversible for that attempt, including after lease expiry. The prepare transaction holds the global write-admission/ACTIVE-epoch locks in order and proves the partial-unique-protected attempt is the sole nonterminalOUTBOX_PUBLICATIONattempt. Only then may external provisioning regrant legacy exact-topic Write and emit a fresh positive probe. The completion half re-locks the global authority, exact attempt and epoch after that external mutation; owner/lease, uniqueness orLEGACY_POLLINGmismatch immediately re-revokes legacy Write, proves a fresh negative probe and leaves writes FROZEN. On success it appends recovery/ACL audit, reopens the in-process legacy Write fence/reaper/ relay, then CAS-opens durable writes and marksRECOVERED_LEGACYlast. Emit:```text src/app-bootstrap/build/messaging-evidence/precommit-legacy-recovery/manifest.json ci-artifact://messaging/{targetAlias}/{sourceDigest}/precommit-legacy-recovery/manifest.json ``` Validate identical bytes against the common and deployment-rollout schemas with `outcome=ABORTED_PRECOMMIT`, exact restored legacy/write-admission generations, recovery scenario IDs exactly once, failed=0 and skipped=0. Any partial failure leaves writes FROZEN and is retried/re-fenced; recovery lease takeover is recovery-only and never restores forward-finalization eligibility. Race tests must run recovery-prepare versus same- and different-attempt creation/finalization in both lock orders, and crash tests must cover every boundary before/after claim, ACL regrant, epoch revalidation, component reopen and final admission CAS. The protocol never uses raw SQL. An aborted attempt ends Task 25 without cutover; another attempt is rejected until recovery atomically records `RECOVERED_LEGACY` and opens a new fence generation, then requires fresh preflight, approval and target attestation. -
After epoch commit, start only v2 relay while writes remain FROZEN. Require the canonical sentinel
DELIVERY_RECORDED, retained V3 projection, fresh target binding/readiness and legacy writer/claim/reaper/send 0. ThenResumePollingV2WriteAdmissionUseCaseCAS-opensOPEN(generation+1). If the original runner dies or resume fails after commit,resume-polling-v2-writesis the only recovery operation; it rechecks the same facts and cannot reactivate legacy. After OPEN, exercise an ordinary canonical append; failure freezes a new generation and keeps the rollout non-healthy. -
Apply the rehearsed rollback zones exactly: pre-commit failure retains legacy DB authority but stays in maintenance until bounded forward retry or the full audited recovery above succeeds; after epoch commit, reverse epoch and legacy reactivation are unsupported even before the first business v2 send. Keep admission FROZEN, preserve backlog/schema/epoch/audit and forward-fix or run the guarded v2 resume.
-
Abort and fence admission on any duplicate anomaly, unexpected indeterminate, backlog/SLO breach, late-observation drop, stale legacy mutation, topic/security drift or readiness loss. Never automatically resend while diagnosing.
-
Emit a sanitized target rollout artifact:
```text src/app-bootstrap/build/messaging-evidence/deployment-cutover/manifest.json ci-artifact://messaging/{targetAlias}/{sourceDigest}/deployment-cutover/manifest.json ``` It binds target environment/cluster alias, source/artifact/release manifest digest, target-binding-attestation digest, precondition evidence digest, watermark/manifest counts, epoch/sentinel facts, final OPEN fence generation, ordinary post-resume append probe, commands/timestamps, rollback zone, failures/skips and operator approvals. Payload, event hash, credential and raw IDs remain excluded. The target runner writes identical bytes to the local handoff and retained CI URI; both validate against the common build-evidence schema and `deployment-rollout-manifest-v1.schema.json`. -
After commit, sentinel proof and artifact handoff, run
verifyMessagingTargetBinding verifyMessagingDeploymentCutoverGREEN. Expected: binding and deployment schemas PASS, exact target/source/artifact/release/attestation digest match, approval/epoch/sentinel scenario IDs present exactly once, failed=0 and skipped=0. -
Acceptance claim: the exact target uses
POLLING_V2and the sentinel/reference path is healthy, and durable write admission is OPEN at the recorded post-cutover generation. Legacy source/runtime cleanup is still pending the observation/rollback window.
Rollback checkpoint: use only the two rehearsed zones. Schema is forward-only; dual relay authority and destructive downgrade are forbidden.
Task 26: Complete observation, remove legacy runtime and requalify the cleanup artifact
Owner: all affected leaves + repository-wide verification/Wiki Depends on: Task 25, reviewed observation window, human cleanup approval
Files — delete:
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/core/MessageBroker.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSender.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaMessageBroker.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaAdapterConfig.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaAdapterSettings.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/DisabledOutboxMessagePublisher.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFence.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxEnvelopeJson.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapter.javasrc/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/Slf4jOutboxRelayFailureReportAdapter.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfigTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFenceTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.javasrc/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/Slf4jOutboxRelayFailureReportAdapterTest.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxMessagePublishPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelayControlPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelaySnapshot.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionEvidence.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverCommand.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverResult.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAuthorityCutoverPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCase.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxPreCommitRecoveryPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityCommand.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityResult.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCase.javasrc/application-core/src/test/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCaseTest.javasrc/application-core/src/test/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCaseTest.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxStorePort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxEvent.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxEventStatus.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayFailureReport.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayFailureReportPort.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayResult.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsCommand.javasrc/application-core/src/main/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCase.javasrc/application-core/src/test/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCaseTest.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapter.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxCutoverPreconditionAdapter.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapter.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapter.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapterTest.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapterTest.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxClaimRepository.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxClaimRepository.javasrc/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaper.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapterTest.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaperTest.javasrc/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaperWiringTest.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLeaderElectionToken.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinator.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinator.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapter.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverJobSettings.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunner.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinatorTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinatorTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapterTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunnerTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyToV2CutoverContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxAppendTransactionalContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxPublisherLeaderElectionContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxRowLifecycleContractTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/support/conditional/EnabledIfMessagingBrokerConfigured.java
Files — create:
src/config/messaging/legacy-runtime-denylist.txtsrc/config/messaging/legacy-runtime-allowlist.txtsrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingLegacyRuntimeDenylistTest.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingLegacyActivationRunbookContractTest.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingWriteAdmissionRecoverySettings.javasrc/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingWriteAdmissionRecoveryApplicationRunner.javasrc/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/MessagingWriteAdmissionRecoveryApplicationRunnerTest.java
Files — modify:
-
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxEventJpaRepository.java -
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlPersistenceConfig.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfig.java -
src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingSettings.java -
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java -
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxMetrics.java -
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRelayScheduler.java -
src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfig.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxConfigTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxSettingsTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfigTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2SentinelContractTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/adapter/outbound/OptionalAdapterBeanGatingTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/ArchitectureViolationFixtureTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/violations/application/OutboundWithoutPermissionUseCase.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/OptionalAdapterConditionalExecutionContractTest.java -
src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/outbox/OutboxStatusRegistryContractTest.java -
src/app-bootstrap/src/main/resources/application.yml -
src/app-bootstrap/src/test/resources/application-test.yml -
src/sample-portfolio/src/main/resources/application.yml -
src/sample-portfolio/src/test/resources/application-test.yml -
src/.env -
src/config/messaging/readiness-cards.yaml -
src/config/messaging/profile-compatibility.yaml -
src/config/messaging/release-profile-assertions.yaml -
docs/registries/env-keys.yaml -
docs/registries/capabilities.yaml -
docs/registries/error-codes.yaml -
docs/registries/metrics.yaml -
docs/registries/secrets-classification.yaml -
docs/runbooks/outbox-publish-failed.md -
docs/runbooks/outbox-dead-letter.md -
docs/runbooks/messaging-producer-unavailable-or-unauthorized.md -
docs/runbooks/messaging-outbox-backlog-and-stale-lease.md -
docs/runbooks/messaging-delivery-indeterminate-and-duplicate-burst.md -
docs/runbooks/messaging-schema-poison-or-record-too-large.md -
docs/runbooks/messaging-terminal-delivery-disposition.md -
docs/runbooks/messaging-topic-policy-or-partition-change.md -
docs/runbooks/messaging-shutdown-deploy-and-secret-rotation.md -
docs/runbooks/messaging-legacy-to-v2-relay-authority-cutover.md -
src/README.md -
src/application-core/README.md -
src/application-core/CLAUDE.md -
src/shared-contract/README.md -
src/shared-contract/CLAUDE.md -
src/adapter/outbound/messaging/README.md -
src/adapter/outbound/messaging/CLAUDE.md -
src/adapter/outbound/persistence-jpa/README.md -
src/adapter/outbound/persistence-jpa/CLAUDE.md -
src/adapter/inbound/web/README.md -
src/adapter/inbound/web/CLAUDE.md -
src/app-bootstrap/README.md -
src/app-bootstrap/CLAUDE.md -
src/sample-portfolio/README.md -
src/sample-portfolio/CLAUDE.md -
src/app-bootstrap/build.gradle -
src/build.gradle -
docs/superpowers/specs/2026-07-28-messaging-production-capability-design.md -
docs/superpowers/plans/2026-07-28-messaging-first-r2-polling-producer.md -
/home/donghyeon/workspace/ai-tool/llm-wiki-private/raw/branch-notes/main.mdand only genuinely derived raw documents -
Observation exit requires the reviewed duration with duplicate anomaly 0, unexpected indeterminate 0, late-drop 0, stable backlog/readiness, no legacy mutation and successful disposition/rotation/shutdown drills. Durable write admission must remain OPEN at the recorded post-cutover generation except for audited drills with successful guarded resume. Any breach postpones cleanup.
-
Write
MessagingLegacyRuntimeDenylistTestfirst and run RED; it must find every exact production symbol/config key above plus legacy reaper repository methods/beans. The denylist contains exact FQCNs and keys, including:```text dev.caskeleton.adapter.outbound.messaging.core.MessageBroker dev.caskeleton.adapter.outbound.messaging.kafka.KafkaSender dev.caskeleton.adapter.outbound.messaging.kafka.KafkaMessageBroker dev.caskeleton.adapter.outbound.messaging.kafka.KafkaAdapterConfig dev.caskeleton.adapter.outbound.messaging.kafka.KafkaAdapterSettings dev.caskeleton.adapter.outbound.messaging.outbox.DisabledOutboxMessagePublisher dev.caskeleton.adapter.outbound.messaging.outbox.LegacyPublicationWriteFence dev.caskeleton.adapter.outbound.messaging.outbox.OutboxEnvelopeJson dev.caskeleton.adapter.outbound.messaging.outbox.OutboxMessagePublishAdapter dev.caskeleton.adapter.outbound.messaging.outbox.Slf4jOutboxRelayFailureReportAdapter dev.caskeleton.application.outbox.OutboxMessagePublishPort dev.caskeleton.application.outbox.LegacyOutboxRelayControlPort dev.caskeleton.application.outbox.LegacyOutboxRelaySnapshot dev.caskeleton.application.outbox.OutboxCutoverPreconditionEvidence dev.caskeleton.application.outbox.OutboxCutoverPreconditionPort dev.caskeleton.application.outbox.FinalizeOutboxAuthorityCutoverCommand dev.caskeleton.application.outbox.FinalizeOutboxAuthorityCutoverResult dev.caskeleton.application.outbox.OutboxAuthorityCutoverPort dev.caskeleton.application.outbox.FinalizeOutboxAuthorityCutoverUseCase dev.caskeleton.application.outbox.OutboxPreCommitRecoveryPort dev.caskeleton.application.outbox.RecoverLegacyOutboxAuthorityCommand dev.caskeleton.application.outbox.RecoverLegacyOutboxAuthorityResult dev.caskeleton.application.outbox.RecoverLegacyOutboxAuthorityUseCase dev.caskeleton.application.outbox.OutboxStorePort dev.caskeleton.application.outbox.OutboxEvent dev.caskeleton.application.outbox.OutboxEventStatus dev.caskeleton.application.outbox.OutboxRelayFailureReport dev.caskeleton.application.outbox.OutboxRelayFailureReportPort dev.caskeleton.application.outbox.OutboxRelayResult dev.caskeleton.application.outbox.PublishPendingOutboxEventsCommand dev.caskeleton.application.outbox.PublishPendingOutboxEventsUseCase dev.caskeleton.adapter.outbound.persistence.outbox.OutboxStoreAdapter dev.caskeleton.adapter.outbound.persistence.outbox.OutboxCutoverPreconditionAdapter dev.caskeleton.adapter.outbound.persistence.outbox.OutboxAuthorityCutoverAdapter dev.caskeleton.adapter.outbound.persistence.outbox.OutboxPreCommitRecoveryAdapter dev.caskeleton.adapter.outbound.persistence.outbox.OutboxClaimRepository dev.caskeleton.adapter.outbound.persistence.postgresql.PostgreSqlOutboxClaimRepository dev.caskeleton.adapter.outbound.persistence.outbox.OutboxReaper dev.caskeleton.bootstrap.autoconfigure.outbox.OutboxLeaderElectionToken dev.caskeleton.bootstrap.autoconfigure.outbox.OutboxLegacyToV2CutoverCoordinator dev.caskeleton.bootstrap.autoconfigure.outbox.OutboxLegacyPreCommitRecoveryCoordinator dev.caskeleton.bootstrap.autoconfigure.outbox.LegacyOutboxRelayControlAdapter dev.caskeleton.bootstrap.autoconfigure.outbox.MessagingAuthorityCutoverJobSettings dev.caskeleton.bootstrap.autoconfigure.outbox.MessagingAuthorityCutoverApplicationRunner OutboxEventJpaRepository.deletePublishedBefore OutboxEventJpaRepository.countGroupedByStatus OutboxEventJpaRepository.findOldestUnpublishedOccurredAtByEventType OutboxClaimRepository.claimEligible outboxLeaderElection outboxReaper app.messaging.broker app.messaging.kafka.brokers APP_MESSAGING_BROKER APP_MESSAGING_KAFKA_BROKERS ca-skeleton.outbox.relay-enabled ca-skeleton.messaging.maintenance.operation ca-skeleton.messaging.maintenance.operation-id ca-skeleton.messaging.maintenance.approval-evidence-id ca-skeleton.messaging.maintenance.expected-target-alias ca-skeleton.messaging.maintenance.expected-source-digest ca-skeleton.messaging.maintenance.expected-artifact-digest ca-skeleton.messaging.maintenance.expected-epoch ca-skeleton.messaging.maintenance.expected-fence-generation ``` This is the complete mandatory production token/key set derived one-to-one from `Files — delete` plus the legacy repository methods, bean names and configuration keys. The contract test snapshots the exact eight-key Task 20 maintenance settings/registry set and asserts set equality with these eight denylist keys before scanning source/config/runbook references. It rejects a missing or extra maintenance key, a missing denylist entry, an unclassified deleted production class, and any allowlist entry outside the exact sample/R0 list below. The allowlist contains only these repository-relative paths: ```text src/application-core/src/main/java/dev/caskeleton/application/outbox/NewOutboxEvent.java src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxAppendPort.java src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapter.java src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapterTest.java src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisher.java src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogUseCase.java src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisherTest.java src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogOutboxTest.java src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/WorkLogUseCasesTest.java src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/authz/WorkLogAuthorizationContractTest.java ``` No directory wildcard or silently ignored unknown path is allowed. -
Run the denylist RED before deletion:
```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*MessagingLegacyRuntimeDenylistTest' --console=plain ``` Expected non-zero with every still-present forbidden FQCN/key/bean reported. Missing scan roots or a test skip is not an accepted RED. -
Delete the listed runtime/relay/reaper sources and obsolete config keys. Retain
NewOutboxEvent,LegacyOutboxAppendPortand a separately named sample-onlyLegacyOutboxAppendAdapteronly as the documented R0 fixture until standalone sample activation. Canonical ACTIVE context must prove legacy append/store/publish/relay/reaper bean count 0. Replace the deleted cutover runner with the disabled-by-defaultMessagingWriteAdmissionRecoveryApplicationRunner, which accepts onlyresume-polling-v2-writes, imports no legacy/cutover type, and usesResumePollingV2WriteAdmissionUseCasewith expected FROZEN generation, ACTIVEPOLLING_V2epoch and fresh sentinel/readiness evidence.```text ca-skeleton.messaging.write-admission-recovery.operation ca-skeleton.messaging.write-admission-recovery.operation-id ca-skeleton.messaging.write-admission-recovery.approval-evidence-id ca-skeleton.messaging.write-admission-recovery.expected-target-alias ca-skeleton.messaging.write-admission-recovery.expected-epoch ca-skeleton.messaging.write-admission-recovery.expected-fence-generation ``` No operation/default means no runner resource. Unknown or legacy operation names fail before mutation; replay/mismatch and resume-before-sentinel remain non-zero. -
Keep retained V3 DB columns and canonical compatibility projection until a later forward schema cleanup. Do not drop columns or rewrite history here.
-
Re-run the runtime denylist GREEN across production Java, build files, YAML/env and registries; only the exact sample/R0 source allowlist may remain. Explicitly exclude design/spec/plan/Wiki history from raw-symbol matching: historical documentation is evidence, not an activation surface. Compile success alone is not bean/resource absence evidence. Run the same focused command GREEN; expect forbidden runtime match 0, canonical legacy bean/resource count 0 and every allowlisted sample/R0 reference classified exactly.
-
Run
MessagingLegacyActivationRunbookContractTestRED before cleanup and GREEN after cleanup. It scans operational runbooks semantically for executable legacy activation keys, commands, dual-relay instructions or rollback-to-legacy actions, while allowing clearly labelled historical facts and “must remain disabled/forbidden” statements. It does not raw-match this plan/spec/branch-note.```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*MessagingLegacyActivationRunbookContractTest' --console=plain ``` -
Because cleanup changes source/artifact digest, freeze a new human candidate and rerun the Task 21 and 22 qualification lanes plus the Task 24 focused/Messaging/repository commands and release-profile aggregator against the cleanup artifact. Do not replay Task 24's pre-cutover status wording: production is already
POLLING_V2. Pre-cleanup evidence is stale and cannot qualify the cleanup artifact. -
Before deploying the cleanup artifact, run:
```bash cd src && ./gradlew verifyMessagingFinalR2Profile --console=plain ``` Expected non-zero because the qualified cleanup release manifest and cleanup rollout artifact do not yet form a matching final chain. -
Deploy the requalified cleanup artifact without changing the already-active
POLLING_V2epoch. Before deployment, rerun target topology/security/ACL probes for the cleanup source/ artifact and emit:```text src/app-bootstrap/build/messaging-evidence/cleanup-target-binding-attestation/manifest.json ci-artifact://messaging/{targetAlias}/{cleanupSourceDigest}/cleanup-target-binding-attestation/manifest.json ``` Run `verifyMessagingCleanupTargetBinding` GREEN; it must bind the same target identity and current secret generation to the cleanup release digest without overwriting the immutable original Task 25 attestation. Then deploy and emit byte-identical sanitized evidence: ```text src/app-bootstrap/build/messaging-evidence/cleanup-rollout/manifest.json ci-artifact://messaging/{targetAlias}/{cleanupSourceDigest}/cleanup-rollout/manifest.json ``` It binds the target alias, cleanup source/artifact digest, qualified cleanup release-manifest digest, cleanup-target-attestation digest, original deployment-cutover manifest digest, unchanged epoch, recorded OPEN write-admission generation, sentinel/backlog/readiness/ legacy-bean-0 facts, approval, commands/timestamps, failed=0 and skipped=0. Validate it against the common and deployment-rollout schemas. -
Re-run
verifyMessagingDeploymentCutover,verifyMessagingCleanupTargetBindingandverifyMessagingFinalR2ProfileGREEN:```bash cd src && ./gradlew verifyMessagingTargetBinding \ verifyMessagingDeploymentCutover \ verifyMessagingCleanupTargetBinding \ verifyMessagingFinalR2Profile --console=plain ``` The final task consumes these exact local inputs: ```text src/build/reports/messaging/release-profile/manifest.json src/app-bootstrap/build/messaging-evidence/target-binding-attestation/manifest.json src/app-bootstrap/build/messaging-evidence/deployment-cutover/manifest.json src/app-bootstrap/build/messaging-evidence/cleanup-target-binding-attestation/manifest.json src/app-bootstrap/build/messaging-evidence/cleanup-rollout/manifest.json ``` If qualification/build cleanup removed a Task 25 local handoff, restore only the byte-identical retained CI artifact to its exact path after verifying its recorded digest/signature and target provenance. Never synthesize, edit or substitute a current artifact for the immutable original. The final task writes: ```text src/build/reports/messaging/final-r2-profile/manifest.json ``` Expected: exact target/cleanup source/artifact/release digest chain, approval/epoch/sentinel/ write-admission-OPEN/legacy-zero scenario IDs exactly once, failed=0, skipped=0 and schema PASS. CI retains the final bytes under the target/cleanup-source-qualified namespace. -
Run final focused, Messaging and repository gates exactly as Task 24 plus:
```bash cd src && ./gradlew :app-bootstrap:test \ --tests '*MessagingLegacyRuntimeDenylistTest' \ --tests '*MessagingWriteAdmissionRecoveryApplicationRunnerTest' \ --tests '*MessagingDisabledZeroResourceContractTest' --console=plain ``` -
Update §0 and card status from the final R2 aggregate only. First R2 may be marked complete only after
verifyMessagingFinalR2Profilepasses; P5/P6/P7 remain unchanged. -
Before final Wiki capture, read the configured vault's
AGENTS.md,CLAUDE.mdand relevantrules/,.agents/,.claude/,.codex/instructions. Resolve the branch withgit branch --show-current; for this plan's currentmainbranch the canonical target is:```text /home/donghyeon/workspace/ai-tool/llm-wiki-private/raw/branch-notes/main.md ``` Record implementation, files, decisions, exact commands/results/failures, release and rollout evidence, unsupported claims and remaining risks. Add/link derived raw documents only when honestly produced; otherwise record “없음”. Canonical vault unavailability is an explicit completion blocker. -
Final handoff lists changed files, behavior, exact verification counts/results, failed/not-run lanes, release/rollout fingerprints, Wiki capture and follow-up risks.
Rollback checkpoint: after cleanup, do not restore legacy code for events already sent by v2. Pause admission, preserve schema/backlog/epoch/audit and forward-fix.
8. Task dependency graph
1 -> 2
|
3 -> 4 -> 5 -> 6
|
7 -> 8 -> 9 -> 10 -> 11 -> 12 -> 13
|
14 -> 15 -> 16 -> 17
| |
+-------> 18
|
19 -> 20
|
21 -> 22 -> 23 -> 24
|
25 -> 26
Parallel work is allowed only at non-overlapping stable boundaries:
- Task 4 shared/sample resource work may run in parallel after Task 3, but Task 5 starts only after both resource owners are stable.
- Persistence Tasks 7–13 are sequential because they share migration/entity/repository/CAS surfaces.
- Task 18 may start after Task 12 while Tasks 15–17 proceed, but Task 19 waits for both.
- Disposable cutover rehearsal Task 20 is never parallelized with producer, persistence or configuration changes.
- Real Kafka Task 21 and security topology setup for Task 22 may prepare in parallel only after the final canonical artifact is frozen; their evidence aggregation remains ordered.
- Target cutover Task 25 is a serialized deployment operation after Task 24 qualification. Task 26 cleanup starts only after the reviewed observation window and requires a newly qualified artifact.
- Shared-worktree Gradle invocations remain sequential even when source subtasks are delegated.
9. Minimum completion matrix
| Requirement | Proving task |
|---|---|
| approved truth/no ACK overclaim | 1 |
| closed first-tuple registry/no future switches | 2, 24 |
| framework-free typed event/SPI | 3 |
| generic envelope + sample-owned payload schema | 4 |
| closed catalog/destination/digest/key | 5 |
| Draft 2020-12 deterministic bytes/admission | 6 |
| forward-only immutable event/delivery/journal/epoch | 7 |
| same-transaction validated append | 8 |
| exhaustive outcome + one-record relay | 9 |
| JIT claim/token/unexpired-lease CAS | 10 |
| late observation DB commit before source ACK | 11 |
| audited requeue/hold/skip/compensate | 12 |
| legacy/v2 mutual exclusion | 13 |
| finite typed config/disabled 0 | 14 |
| actual ACK-aware Spring Kafka gateway | 15 |
| admission/late queue/generation lifecycle | 16 |
| topic/security attestation | 17 |
| authenticated internal disposition endpoint | 18 |
| composition/readiness/observability | 19 |
| atomic watermark/reconcile/epoch/sentinel cutover implementation + disposable rehearsal | 20 |
| real PostgreSQL + real Kafka fault evidence | 21 |
| TLS/SASL/ACL + multi-broker RF/min ISR | 22 |
| env/registries/runbooks | 23 |
| no-skip pre-cutover release artifact + independent review | 24 |
| exact-target approved deployment cutover evidence | 25 |
| observation exit + legacy cleanup + cleanup-artifact requalification + final Wiki | 26 |
10. Follow-up plans after first R2
다음은 이 계획을 확장하는 checkbox가 아니라 별도 설계 승인과 실행 계획이다.
- Inbound Kafka + inbox + DLT/replay (P5)
- 20번째
adapter:inbound:messaging-kafkaleaf registry migration; - manual ACK after application commit;
- inbox/effect identity, bounded retry, DLT ACK, replay/audit.
- 20번째
- PostgreSQL Debezium CDC (P6)
- external Connect/Debezium asset, publication/slot/offset/WAL;
- insert-only mapping, shadow, authority-exclusive cutover/rollback and retention proof.
- Optional cards (P7)
- Avro/Protobuf registry, Kafka EOS, retry topic, compaction, object-storage claim check, alternate broker, multi-cluster, module split;
- each requirement gets an independent card/compatibility/evidence plan.
- Live non-empty legacy database migration
- collect actual row volume/state distribution, lock/replication budget, data classification, maintenance window and rollback evidence;
- separately approve either
LIVE_ADDITIVE_BACKFILL_IN_PLACE.v1orCOPY_AND_CUTOVER_WITH_RECONCILIATION.v1.
11. Final non-negotiable assertions
KafkaSender.send()return is not broker ACK.- Kafka future success plus metadata is ACK observation; it is not consumer processing.
acks=allwithout RF/min ISR/unclean-election evidence is not durable topology evidence.- Kafka producer idempotence does not remove ACK-to-DB, restart, late-ACK or operator-requeue duplicates.
EXHAUSTEDdoes not mean definitely not delivered.- late ACK observation never rewrites authoritative delivery state.
- timestamp/random event ID is not aggregate total order.
outbox_eventwire bytes are immutable authority; JSONB/re-serialization is not.- polling and CDC may never be simultaneous production dispatch authorities.
- operator mutation requires auth, expected generation/version, no active claim and immutable audit.
- disabled and configured-but-not-ready are different states.
- fake/single-node/local evidence cannot be relabelled as production R2.
- implementation completion requires exact commands/results and LLM Wiki capture.
- agent never stages, commits, amends or pushes.