3204 lines
187 KiB
Markdown
3204 lines
187 KiB
Markdown
# Messaging First R2 Polling Producer Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
||
> `superpowers:subagent-driven-development` (recommended) or
|
||
> `superpowers:executing-plans` to implement this plan task-by-task. Steps use checkbox (`- [ ]`)
|
||
> syntax for tracking. Behavior changes also require `superpowers:test-driven-development`;
|
||
> completion claims require `superpowers:verification-before-completion` and an independent
|
||
> `superpowers:requesting-code-review`.
|
||
|
||
**Goal:** Build one production-reference Messaging path from a typed integration event through a
|
||
same-transaction PostgreSQL polling outbox to an acknowledgement-aware Spring Kafka producer, with
|
||
an authenticated disposition control and exact R2 evidence.
|
||
|
||
**Architecture:** `application-core` owns provider-neutral event/publication/disposition semantics;
|
||
`adapter:outbound:messaging` owns deterministic JSON/schema compilation and Kafka; PostgreSQL
|
||
persistence owns event/delivery/audit rows and token/lease CAS; inbound web owns only operator HTTP
|
||
mapping; bootstrap composes the exact tuple, readiness and schedulers. The first path is polling-only
|
||
and keeps consumer, inbox, DLT, replay and CDC disabled.
|
||
|
||
**Tech Stack:** Java 21, Spring Boot 4.0.0, Spring Kafka 4.0 through the Boot BOM, Jackson 3,
|
||
`com.networknt:json-schema-validator:3.0.2`, PostgreSQL, Flyway, JPA, Gradle, JUnit 5, AssertJ,
|
||
Testcontainers Kafka/PostgreSQL, Micrometer.
|
||
|
||
---
|
||
|
||
- 작성일: 2026-07-28
|
||
- 상태: 실행 계획 작성·독립 검토 완료, 모든 task 미착수
|
||
- 설계 정본:
|
||
[Messaging Production Capability Deep Design](../specs/2026-07-28-messaging-production-capability-design.md)
|
||
- 구현 범위: P0–P4의 first R2 polling producer tuple
|
||
- 명시적 비범위: inbound Kafka consumer, inbox, retry topic, DLT/replay, Kafka EOS,
|
||
Debezium/Kafka Connect CDC, Avro/Protobuf/schema registry, alternate broker, multi-cluster
|
||
- 비교한 계획:
|
||
[Redis Foundation](2026-07-28-redis-production-capability-foundation.md),
|
||
[Redis Runtime](2026-07-28-redis-runtime-cache.md),
|
||
[HTTP Client Foundation](2026-07-28-httpclient-production-capability-foundation.md),
|
||
[HTTP Client Total Deadline](2026-07-28-httpclient-total-deadline.md),
|
||
[Fileserver Foundation](2026-07-28-fileserver-production-capability-foundation.md),
|
||
[Fileserver Durable Recovery](2026-07-28-fileserver-durable-recovery.md),
|
||
[Notification](2026-07-28-notification-production-capability.md)
|
||
|
||
Repository commit policy는 모든 플랫폼에서 `human-only`다. 이 계획에는 `git add`, `git commit`,
|
||
`git amend`, `git push` 단계가 없다. 구현자는 작업 결과와 검증 증거만 전달하고 candidate
|
||
commit은 사람이 만든다.
|
||
|
||
## 1. Exact selected tuple and non-guarantees
|
||
|
||
첫 구현과 qualification 대상은 다음 tuple 하나다.
|
||
|
||
```text
|
||
messaging-outbox-publish.v1
|
||
+ kafka-spring-acknowledged-idempotent.v1
|
||
+ postgresql-polling-outbox.v2
|
||
+ postgresql-per-record-jit-claim.v1
|
||
+ json-schema-envelope.v1
|
||
+ external-topic-validated.v1
|
||
+ kafka-sasl-ssl-scram-sha-512.v1
|
||
+ kafka-compression-none.v1
|
||
+ per-key-normal-path-sequence-detectable.v1
|
||
+ same-postgresql-transaction-resource.v1
|
||
+ authenticated-internal-web-disposition.v1
|
||
```
|
||
|
||
이 계획이 완료돼도 다음은 주장하지 않는다.
|
||
|
||
- broker와 PostgreSQL 사이 exactly-once;
|
||
- consumer effect의 deduplication 또는 inbox 보장;
|
||
- global FIFO, failure/rotation/requeue 뒤 strict FIFO;
|
||
- single-node Kafka test만으로 multi-broker RF/min ISR 내구성;
|
||
- CDC-ready, DLT-ready, replay-ready;
|
||
- local plaintext profile을 production security profile로 승격;
|
||
- `ACKNOWLEDGED`가 consumer 처리 또는 business effect 완료를 뜻함.
|
||
|
||
## 2. Target flow and fixed decisions
|
||
|
||
```text
|
||
feature mapper
|
||
-> IntegrationEventDraft<typed record>
|
||
-> IntegrationEventEncoderPort
|
||
-> ValidatedIntegrationEvent(exact UTF-8 bytes + hashes)
|
||
-> TransactionPort.inWrite(
|
||
business state
|
||
+ immutable outbox_event
|
||
+ CURRENT/READY outbox_delivery
|
||
)
|
||
|
||
Outbox relay invocation
|
||
-> acquire one bounded local admission permit
|
||
-> Tx B: one-row JIT claim + token/DB-time lease + ATTEMPT_ADMITTED
|
||
-> no DB transaction: Kafka send + future ACK wait
|
||
-> Tx C: outcome observation + valid-lease/token CAS state transition
|
||
-> release permit
|
||
|
||
late Kafka callback
|
||
-> bounded payload-free observation source
|
||
-> application drain
|
||
-> DB commit
|
||
-> source ACK
|
||
|
||
authenticated internal endpoint
|
||
-> inbound DTO/principal mapping
|
||
-> ApplyOutboxDispositionUseCase
|
||
-> permission/policy
|
||
-> PostgreSQL CAS + immutable audit
|
||
```
|
||
|
||
고정 결정:
|
||
|
||
1. `src/config/architecture/modules.json`이 leaf와 production project edge의 유일한 SSOT다.
|
||
first R2 production 구현에는 새 leaf나 project edge가 필요 없다.
|
||
2. `sample-portfolio -> adapter-outbound-messaging` edge는 standalone sample을 실제 ACTIVE
|
||
producer로 바꾸는 별도 승인 작업 전에는 추가하지 않는다.
|
||
3. application/domain/shared Java API에는 Kafka, Jackson, JSON validator, Spring, JPA 타입을
|
||
노출하지 않는다.
|
||
4. physical topic은 application contract가 아니라 outbound destination binding이다.
|
||
5. exact UTF-8 `BYTEA`가 wire authority다. retry에서 payload를 다시 직렬화하지 않는다.
|
||
6. `outbox_event`는 immutable event, `outbox_delivery`는 mutable delivery control이다.
|
||
7. claim/outcome/renew는 opaque token, owner, CURRENT generation, expected version,
|
||
`claim_until > database_now`를 모두 확인한다.
|
||
8. local admission을 확보한 뒤 한 record만 JIT claim한다. initial profile의 admitted record
|
||
upper bound는 1이다.
|
||
9. broker call은 DB transaction 밖에서 수행한다.
|
||
10. `ACKNOWLEDGED`, `ACKNOWLEDGED_MISMATCH`, `REJECTED`, `INDETERMINATE`는 exhaustive outcome이다.
|
||
11. acceptance certainty와 retry disposition은 독립 축이다.
|
||
12. deadline 뒤 late ACK는 기존 outcome/state를 뒤집지 않고 append-only observation만 제안한다.
|
||
13. operator requeue는 기존 row를 READY로 덮지 않고 이전 authority를 supersede한 뒤 새
|
||
delivery generation을 만든다.
|
||
14. requeue generation deadline은
|
||
`min(generation.created_at + maximumAutomaticPublicationAge,
|
||
event.created_at + sameEventRequeueHorizon)`이다.
|
||
15. P2는 additive schema/control-plane candidate일 뿐이다. `LEGACY_POLLING` authority는 P3의
|
||
fenced cutover까지 유지한다.
|
||
16. live non-empty V3 database는 base template migration이 자동 backfill하지 않는다. 별도
|
||
deployment migration design과 승인이 없으면 중단한다.
|
||
17. production ACTIVE는 SASL_SSL + SCRAM-SHA-512, external topic attestation, least-privilege
|
||
evidence가 없으면 실패한다.
|
||
18. disabled state는 contract/destination/client/AdminClient/thread/scheduler/network/secret
|
||
refresh가 모두 0이다.
|
||
|
||
## 3. Scope boundary and owner leaves
|
||
|
||
| 책임 | owner leaf | Gradle path | production edge 변경 |
|
||
| --- | --- | --- | --- |
|
||
| typed event, outcome, relay, late drain, disposition policy | `application-core` | `:application-core` | 없음 |
|
||
| generic envelope schema resource | `shared-contract` | `:shared-contract` | 없음 |
|
||
| JSON/schema/catalog/Kafka/provider lifecycle | `adapter-outbound-messaging` | `:adapter:outbound:messaging` | 외부 dependency만 추가 |
|
||
| event/delivery/journal/epoch/CAS | `adapter-outbound-persistence-jpa` | `:adapter:outbound:persistence-jpa` | 없음 |
|
||
| authenticated operator HTTP mapping | `adapter-inbound-web` | `:adapter:inbound:web` | 없음 |
|
||
| tuple composition/readiness/schedulers/real-service lane | `app-bootstrap` | `:app-bootstrap` | test dependency만 추가 |
|
||
| sample payload/schema/contribution fixture | `sample-portfolio` | `:sample-portfolio` | messaging edge 없음 |
|
||
|
||
금지:
|
||
|
||
- controller가 repository, JPA entity 또는 outbound adapter를 직접 사용;
|
||
- persistence mapper/query에 retry, disposition 또는 topic 정책을 넣음;
|
||
- messaging adapter가 sample, persistence 또는 inbound-web를 의존;
|
||
- bootstrap settings/configuration에 business event mapping이나 retry policy를 구현;
|
||
- `shared-contract`에 WorkLog schema 또는 provider setting을 넣음;
|
||
- 현재 dirty worktree의 Fileserver/Object Storage/Notification 변경을 되돌리거나 덮어씀.
|
||
|
||
## 4. Evidence ladder and promotion rule
|
||
|
||
| evidence | 허용되는 주장 |
|
||
| --- | --- |
|
||
| pure/application unit | provider-neutral contract와 state policy가 정의됨 |
|
||
| schema/catalog/codec unit/property | local deterministic document와 closed catalog가 정의됨 |
|
||
| adapter fake gateway | outcome mapping과 lifecycle protocol이 정의됨 |
|
||
| real PostgreSQL | same-store append, constraint, claim/CAS/audit protocol의 local evidence |
|
||
| single-node real Kafka | actual ACK metadata와 client/provider behavior evidence |
|
||
| TLS/SASL/ACL lane | exact security principal/profile evidence |
|
||
| multi-broker RF/min ISR lane | selected topology failure/recovery evidence |
|
||
| fault/capacity/rotation/cutover drill | exact tuple의 operational R2 evidence |
|
||
|
||
낮은 row를 높은 row, 다른 broker version, cluster, topic, principal 또는 security profile로
|
||
일반화하지 않는다. 모든 selected scenario가 fresh evidence artifact에 PASS일 때만 machine card를
|
||
`release-eligible`로 바꾼다. 그 전에는 최대 `implemented-candidate`다.
|
||
|
||
## 5. Execution rules
|
||
|
||
1. 모든 checkbox는 구현 시작 시 `[ ]`다.
|
||
2. task 시작 전 `git status --short`, 현재 migration 목록, owner leaf의 가장 가까운
|
||
`CLAUDE.md`, `modules.json` edge를 다시 확인한다.
|
||
3. behavior task는 RED test 작성 → 같은 focused command에서 예상 원인으로 실패 확인 → 최소
|
||
구현 → 같은 command GREEN 순서를 지킨다.
|
||
4. RED가 처음부터 통과하면 기존 coverage인지 잘못된 test인지 조사하고 assertion을 강화한다.
|
||
5. compilation drift, 외부 환경 또는 unrelated dirty change가 RED 원인이면 구현하지 말고
|
||
원인을 분리한다.
|
||
6. 한 shared worktree에서 여러 Gradle process를 동시에 실행하지 않는다. 이전에 같은 output
|
||
directory를 병렬 갱신해 compile collision이 발생했으므로 Gradle command는 한 invocation으로
|
||
묶거나 순차 실행한다.
|
||
7. 실제 service가 필요한 release task는 service/credential/image/no-test 문제를 SKIP/PASS로
|
||
바꾸지 않는다. local ordinary `test`와 release qualification task를 분리한다.
|
||
8. migration은 expand-first, forward-only다. 기존 `V3__outbox_event.sql`은 수정하지 않는다.
|
||
9. 새 provider와 v2 scheduler는 authority cutover 전까지 dark/disabled다.
|
||
10. P2에서 v2 claim/send/authority switch를 활성화하지 않는다.
|
||
11. event/payload/schema/hash/credential/raw header는 log, metric tag, evidence artifact에 넣지
|
||
않는다.
|
||
12. 각 Wave exit에서 설계 §0 ledger, card maturity, plan checkbox, LLM Wiki branch-note를 실제
|
||
증거에 맞춰 갱신한다.
|
||
13. plan surface 밖의 파일이나 타입이 필요하면 조용히 확장하지 않고 이 문서를 먼저 갱신한다.
|
||
14. 설계와 plan이 충돌하면 구현으로 타협하지 않고 상세 설계를 먼저 수정·재승인한다.
|
||
|
||
## 6. Stop conditions
|
||
|
||
다음 중 하나라도 확인되면 해당 task 또는 Wave를 중단한다.
|
||
|
||
- application/domain에 framework, Kafka, JSON, persistence 타입을 넣어야만 진행 가능;
|
||
- module edge가 `modules.json`에 허용되지 않음;
|
||
- `V7`이 실행 시점에 이미 다른 migration으로 사용됐거나 다른 승인 계획이 먼저 구현됨.
|
||
모든 Flyway location을 다시 스캔해 다음 global version으로 이 계획과 tests를 먼저 갱신한다;
|
||
- V3 legacy row가 live non-empty인데 empty/drained evidence나 별도 live migration 승인이 없음;
|
||
- business repository와 outbox append가 같은 transaction resource임을 증명할 수 없음;
|
||
- Kafka producer retry/timeout/effective setting을 finite하게 고정할 수 없음;
|
||
- adopted JSON Schema validator가 Draft 2020-12, offline registry, format assertion 또는 required
|
||
adversarial bound를 만족하지 못함;
|
||
- topic/RF/min ISR/ACL을 runtime와 provisioning evidence의 명시된 source로 attest할 수 없음;
|
||
- legacy relay와 v2 relay를 동시에 active하게 해야만 rollout 가능;
|
||
- active writer/relay/producer를 fence하지 않은 채 authority switch가 필요;
|
||
- DB에 INDETERMINATE/HOLD를 기록하지 못한 상태로 producer generation을 강제 전환해야 함;
|
||
- operator endpoint가 active unexpired claim을 무시하거나 raw status update를 해야 함;
|
||
- real Kafka/security/multi-broker evidence 없이 R2/production-ready 표현이 필요.
|
||
|
||
## 7. Batch graph and checkpoints
|
||
|
||
```text
|
||
Wave A / P0
|
||
truth + machine registry skeleton
|
||
-> Wave B / P1
|
||
application contract + schema + catalog + codec
|
||
-> Wave C / P2
|
||
additive DB v2 + append + claim/CAS + policy
|
||
-> Wave D / P3
|
||
Spring Kafka + endpoint + composition + cutover
|
||
-> Wave E / P4
|
||
real-service/security/fault/release evidence
|
||
```
|
||
|
||
| Wave | exit claim | rollback posture |
|
||
| --- | --- | --- |
|
||
| A | current R0 truth와 planned cards가 정확함 | behavior 변화 없음 |
|
||
| B | local event contract/codec candidate | Kafka/outbox R2 아님 |
|
||
| C | polling v2 schema/control-plane candidate | LEGACY_POLLING 유지, v2 scheduler off |
|
||
| D | ACK-aware polling path/cutover candidate | pause admission, preserve DB schema/backlog/epoch |
|
||
| E | exact evidence가 통과한 tuple만 release-eligible | destructive schema downgrade 금지 |
|
||
|
||
---
|
||
|
||
## Wave A — P0 truth freeze and execution scaffolding
|
||
|
||
### Task 1: Freeze current R0 behavior and approved design truth
|
||
|
||
**Owner:** documentation + existing application/messaging/persistence/bootstrap tests
|
||
**Depends on:** approved detailed design
|
||
**Behavior change:** none
|
||
|
||
**Files — modify:**
|
||
|
||
- `docs/superpowers/specs/2026-07-28-messaging-production-capability-design.md`
|
||
- `src/application-core/src/test/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCaseTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxAppendTransactionalContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxRowLifecycleContractTest.java`
|
||
- `src/adapter/outbound/messaging/README.md`
|
||
|
||
**Files — create:**
|
||
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfigTest.java`
|
||
|
||
- [ ] Capture current branch, `git status --short`, design digest, registry edges, dependency graph,
|
||
migrations and current test counts in the LLM Wiki branch-note.
|
||
- [ ] Add characterization assertions for:
|
||
broker blank → disabled sentinels; broker selected + missing sender → startup failure; broker ID
|
||
mismatch → failure; sender normal return → legacy `PUBLISHED`; sender exception →
|
||
`FAILED/DEAD`; ACK-to-mark failure → `IN_FLIGHT` and possible duplicate; same-transaction
|
||
append rollback; timestamp FIFO limitation.
|
||
- [ ] Keep tests explicitly named `legacy` or `characterization`; do not rename current void-return
|
||
success to broker ACK.
|
||
- [ ] Run the baseline sequentially:
|
||
|
||
```bash
|
||
cd src && ./gradlew :application-core:test \
|
||
--tests '*PublishPendingOutboxEventsUseCaseTest' --console=plain
|
||
cd src && ./gradlew :adapter:outbound:messaging:test \
|
||
--tests '*MessagingConfigTest' \
|
||
--tests '*OutboxMessagePublishAdapterTest' --console=plain
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*OutboxAppendTransactionalContractTest' \
|
||
--tests '*OutboxRowLifecycleContractTest' --console=plain
|
||
```
|
||
|
||
- [ ] Expected GREEN: current behavior is reproducible without source behavior changes.
|
||
- [ ] Update §0 to `P0=CHARACTERIZED`, leaving P1–P4 `NOT_STARTED`.
|
||
- [ ] Acceptance: no “Kafka ACK”, “dedupe safe” or “R2” claim is introduced.
|
||
|
||
**Rollback checkpoint:** characterization tests and truth documentation are independently reversible;
|
||
legacy code remains the executable baseline through the P3 cutover window.
|
||
|
||
### Task 2: Add fail-closed Messaging card registries and verification task skeleton
|
||
|
||
**Owner:** repository configuration + `app-bootstrap` contract tests
|
||
**Depends on:** Task 1
|
||
|
||
**Files — create:**
|
||
|
||
- `src/config/messaging/readiness-cards.yaml`
|
||
- `src/config/messaging/profile-compatibility.yaml`
|
||
- `src/config/messaging/release-profile-assertions.yaml`
|
||
- `src/config/messaging/evidence/build-evidence-manifest-v1.schema.json`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingCapabilityRegistryContractTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/build.gradle`
|
||
- `src/app-bootstrap/build.gradle`
|
||
- `src/app-bootstrap/README.md`
|
||
|
||
- [ ] Write a RED contract test that requires exactly the P0–P4 first tuple rows, closed maturity
|
||
values `not-implemented|implemented-candidate|release-eligible`, wildcard-free compatibility,
|
||
unique IDs, declared evidence tasks/scenarios/runbooks and no consumer/CDC/EOS/schema-registry
|
||
rows.
|
||
- [ ] Seed all first tuple rows with `maturity: not-implemented` and empty evidence fingerprint; do
|
||
not predeclare future extension-ledger names.
|
||
- [ ] Define one checked-in, payload-free build-evidence schema with required source/artifact digest,
|
||
producer task, scenario IDs/counts, command/timestamp, profile/catalog/schema/settings hashes,
|
||
failures, skips and unsupported claims. Every later local manifest validates against this
|
||
schema before release aggregation; a producer may add a stricter offline schema but may not
|
||
weaken these common fields.
|
||
- [ ] Define these task names in `src/build.gradle` without making them pass yet:
|
||
|
||
```text
|
||
verifyMessagingContracts
|
||
verifyMessagingJsonSchemaV1
|
||
verifyMessagingPollingOutboxR2
|
||
verifyMessagingKafkaProducerR2
|
||
verifyMessagingSecurityR2
|
||
verifyMessagingReleaseProfile
|
||
verifyMessagingTargetBindingPreflight
|
||
verifyMessagingTargetBinding
|
||
verifyMessagingDeploymentCutover
|
||
verifyMessagingCleanupTargetBinding
|
||
verifyMessagingFinalR2Profile
|
||
```
|
||
|
||
Each task must fail on no matching tests. Release aggregation must reject missing, skipped,
|
||
stale, wrong-source or mismatched-profile evidence.
|
||
- [ ] Verify RED then GREEN for registry structure only:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*MessagingCapabilityRegistryContractTest' --console=plain
|
||
```
|
||
|
||
- [ ] Verify the existing dependency boundary remains unchanged:
|
||
|
||
```bash
|
||
cd src && ./gradlew verifyCleanArchitectureDependencies --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance: registry truth exists, every card is `not-implemented`, and no verification task
|
||
can falsely claim R2.
|
||
|
||
**Rollback checkpoint:** registry/task scaffolding creates no runtime resources and may be removed
|
||
without data migration.
|
||
|
||
---
|
||
|
||
## Wave B — P1 typed contract, schema, catalog and deterministic bytes
|
||
|
||
### Task 3: Add framework-free integration-event contract and contribution SPI
|
||
|
||
**Owner:** `application-core` (`:application-core`)
|
||
**Depends on:** Task 2
|
||
|
||
**Files — create under
|
||
`src/application-core/src/main/java/dev/caskeleton/application/messaging/`:**
|
||
|
||
- `contract/IntegrationPayload.java`
|
||
- `contract/IntegrationEventContractContribution.java`
|
||
- `contract/ContractId.java`
|
||
- `contract/LogicalDestinationId.java`
|
||
- `contract/SchemaResourceId.java`
|
||
- `contract/Sha256.java`
|
||
- `contract/ContractDescriptor.java`
|
||
- `event/EventId.java`
|
||
- `event/AggregateIdentity.java`
|
||
- `event/AggregateOrder.java`
|
||
- `event/IntegrationEventDraft.java`
|
||
- `event/ValidatedIntegrationEvent.java`
|
||
- `event/IntegrationEventEncoderPort.java`
|
||
|
||
**Files — create under
|
||
`src/application-core/src/test/java/dev/caskeleton/application/messaging/`:**
|
||
|
||
- `contract/IntegrationEventContractContributionTest.java`
|
||
- `event/IntegrationEventDraftTest.java`
|
||
- `event/ValidatedIntegrationEventTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/application-core/README.md`
|
||
- `src/application-core/CLAUDE.md`
|
||
|
||
- [ ] Write RED value tests for canonical ASCII event ID grammar, closed contract/destination IDs,
|
||
positive versions, nonblank canonical tenant scope, aggregate sequence/index bounds,
|
||
immutable/defensively-copied bytes and fixed SHA-256 length.
|
||
- [ ] Write RED SPI tests requiring exact final Java record payload type, canonical component order,
|
||
schema resource/hash and provider-neutral descriptor. Reject `Map`, raw JSON string/tree,
|
||
assignable-type discovery and Java class-name routing.
|
||
- [ ] Implement one-public-type-per-file framework-free records/interfaces. The boundary shape is:
|
||
|
||
```java
|
||
public interface IntegrationPayload {}
|
||
|
||
public interface IntegrationEventContractContribution<P extends IntegrationPayload> {
|
||
ContractId contractId();
|
||
int payloadVersion();
|
||
Class<P> exactPayloadRecordType();
|
||
List<String> canonicalRecordComponentOrder();
|
||
SchemaResourceId payloadSchemaResource();
|
||
Sha256 payloadSchemaHash();
|
||
ContractDescriptor descriptor();
|
||
}
|
||
|
||
public interface IntegrationEventEncoderPort {
|
||
ValidatedIntegrationEvent encode(IntegrationEventDraft<?> draft);
|
||
}
|
||
```
|
||
|
||
- [ ] Keep physical topic, Kafka record metadata, JSON node, serializer, schema validator and
|
||
publication epoch out of these types.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :application-core:test \
|
||
--tests 'dev.caskeleton.application.messaging.*' --console=plain
|
||
```
|
||
|
||
- [ ] Run application purity:
|
||
|
||
```bash
|
||
cd src && ./gradlew verifyApplicationCoreDependencyPurity \
|
||
verifyOneTypePerFile --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: framework-free semantic contract R1 only; no schema/Kafka/persistence R2.
|
||
|
||
**Rollback checkpoint:** these are additive contracts; legacy `NewOutboxEvent` remains until the
|
||
validated append path is green.
|
||
|
||
### Task 4: Check in the generic envelope schema and sample payload contract
|
||
|
||
**Owner leaves:** `shared-contract` (`:shared-contract`), `sample-portfolio`
|
||
(`:sample-portfolio`)
|
||
**Depends on:** Task 3
|
||
|
||
**Files — create:**
|
||
|
||
- `src/shared-contract/src/main/resources/contracts/messaging/envelope/v1.schema.json`
|
||
- `src/shared-contract/src/main/resources/contracts/messaging/envelope/v1.schema.sha256`
|
||
- `src/shared-contract/src/test/java/dev/caskeleton/shared/contract/messaging/MessagingEnvelopeSchemaResourceTest.java`
|
||
- `src/sample-portfolio/src/main/resources/contracts/messaging/portfolio.worklog.reserved/v1.schema.json`
|
||
- `src/sample-portfolio/src/main/resources/contracts/messaging/portfolio.worklog.reserved/v1.schema.sha256`
|
||
- `src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/WorkLogReservedPayload.java`
|
||
- `src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/WorkLogReservedContractContribution.java`
|
||
- `src/sample-portfolio/src/test/resources/contracts/messaging/portfolio.worklog.reserved/v1.valid.json`
|
||
- `src/sample-portfolio/src/test/resources/contracts/messaging/portfolio.worklog.reserved/v1.invalid-unknown-field.json`
|
||
- `src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/event/WorkLogReservedContractContributionTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/shared-contract/README.md`
|
||
- `src/shared-contract/CLAUDE.md`
|
||
- `src/sample-portfolio/README.md`
|
||
- `src/sample-portfolio/CLAUDE.md`
|
||
|
||
- [ ] Write RED resource tests requiring UTF-8, explicit Draft 2020-12 `$schema`, immutable absolute
|
||
`$id`, checked-in lowercase SHA-256, `unevaluatedProperties: false`, bounded strings/arrays,
|
||
required/null/missing policy and no HTTP/file remote `$ref`.
|
||
- [ ] Define envelope v1 with the exact fields frozen by design:
|
||
|
||
```json
|
||
{
|
||
"envelopeVersion": 1,
|
||
"eventId": "event-1",
|
||
"contractId": "portfolio.worklog.reserved",
|
||
"payloadVersion": 1,
|
||
"logicalDestination": "portfolio-domain-events",
|
||
"aggregate": {
|
||
"type": "worklog",
|
||
"id": "worklog-42",
|
||
"sequence": 17,
|
||
"eventIndex": 0
|
||
},
|
||
"occurredAt": "2026-07-28T05:10:30.123Z",
|
||
"correlationId": "corr-1",
|
||
"contentType": "application/json",
|
||
"payload": {
|
||
"workLogId": "worklog-42"
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] Keep the envelope business-free and keep the WorkLog payload schema only in sample.
|
||
- [ ] Make `WorkLogReservedPayload` a typed immutable record implementing `IntegrationPayload`;
|
||
contribution provides type/order/resource/hash only and no JSON mapper.
|
||
- [ ] Do not add `sample-portfolio -> adapter-outbound-messaging` to `modules.json` or Gradle.
|
||
- [ ] Verify RED then GREEN sequentially:
|
||
|
||
```bash
|
||
cd src && ./gradlew :shared-contract:test \
|
||
--tests '*MessagingEnvelopeSchemaResourceTest' --console=plain
|
||
cd src && ./gradlew :sample-portfolio:test \
|
||
--tests '*WorkLogReservedContractContributionTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: checked-in generic/sample contract artifacts exist; validator compatibility
|
||
is still unproven until Task 6.
|
||
|
||
**Rollback checkpoint:** resources and sample contribution are additive; no production runtime
|
||
discovers or publishes them yet.
|
||
|
||
### Task 5: Compile the closed contract, destination and exact capability binding
|
||
|
||
**Owner:** `adapter:outbound:messaging` (`:adapter:outbound:messaging`)
|
||
**Depends on:** Tasks 3–4
|
||
|
||
**Files — create under
|
||
`src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/`:**
|
||
|
||
- `contract/ContractCatalogCompiler.java`
|
||
- `contract/CompiledIntegrationEventContract.java`
|
||
- `contract/ContractCatalogDigest.java`
|
||
- `destination/DestinationBindingSettings.java`
|
||
- `destination/DestinationBindingCompiler.java`
|
||
- `destination/CompiledPublicationBinding.java`
|
||
- `destination/PartitionKeyV1.java`
|
||
- `config/MessagingCapabilityCardRegistry.java`
|
||
- `config/CompiledMessagingDescriptor.java`
|
||
|
||
**Files — create under
|
||
`src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/`:**
|
||
|
||
- `contract/ContractCatalogCompilerTest.java`
|
||
- `contract/ContractCatalogDigestTest.java`
|
||
- `destination/DestinationBindingCompilerTest.java`
|
||
- `destination/PartitionKeyV1Test.java`
|
||
- `config/MessagingCapabilityCardRegistryTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/outbound/messaging/README.md`
|
||
- `src/adapter/outbound/messaging/CLAUDE.md`
|
||
|
||
- [ ] Write RED tests for duplicate contract/destination/schema IDs; missing binding; unknown card;
|
||
final-record exact type; component-order mismatch; code/deployment byte-bound intersection;
|
||
config attempting to relax ordering/schema/security; legacy + canonical conflict; unsupported
|
||
future card rejection.
|
||
- [ ] Add golden partition-key vectors using the design's domain-separated, length-prefixed SHA-256
|
||
input. Assert exactly 64 lowercase hex ASCII characters and tenant-enabled/disabled canonical
|
||
non-null scope.
|
||
- [ ] Compile:
|
||
|
||
```text
|
||
contract descriptor
|
||
+ destination descriptor
|
||
+ producer/serialization/security/card descriptor
|
||
= immutable CompiledPublicationBinding
|
||
```
|
||
|
||
Physical topic and bootstrap servers stay only in the compiled deployment binding.
|
||
- [ ] Compute stable catalog/settings/schema digests using sorted IDs and length-prefixed bytes;
|
||
never depend on `Map` iteration order or `toString()`.
|
||
- [ ] Empty catalog + DISABLED must compile to a zero-resource descriptor. ACTIVE + empty catalog
|
||
must fail before any client/thread is created.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :adapter:outbound:messaging:test \
|
||
--tests '*ContractCatalog*Test' \
|
||
--tests '*DestinationBindingCompilerTest' \
|
||
--tests '*PartitionKeyV1Test' \
|
||
--tests '*MessagingCapabilityCardRegistryTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: closed local binding compiler R1; no wire bytes or Kafka client yet.
|
||
|
||
**Rollback checkpoint:** compiler is not wired into `MessagingConfig`; legacy selection remains
|
||
authoritative.
|
||
|
||
### Task 6: Implement the deterministic JSON Schema envelope encoder
|
||
|
||
**Owner:** `adapter:outbound:messaging` (`:adapter:outbound:messaging`)
|
||
**Depends on:** Task 5
|
||
|
||
**Files — create:**
|
||
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/LocalJsonSchemaRegistry.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/DeterministicEnvelopeWriter.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/JsonSchemaIntegrationEventEncoder.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/EnvelopeAdmissionLimits.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/envelope/EnvelopeHashV1.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/envelope/LocalJsonSchemaRegistryTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/envelope/JsonSchemaIntegrationEventEncoderTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/envelope/EnvelopeAdversarialCorpusTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/resources/contracts/messaging/test.event/v1.schema.json`
|
||
- `src/adapter/outbound/messaging/src/test/resources/contracts/messaging/test.event/v1.valid.json`
|
||
- `src/adapter/outbound/messaging/src/test/resources/contracts/messaging/test.event/v1.invalid.json`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/outbound/messaging/build.gradle`
|
||
- `src/adapter/outbound/messaging/gradle.lockfile`
|
||
- `src/build.gradle`
|
||
|
||
- [ ] Write RED tests for Draft 2020-12 meta-schema, checksum mismatch, duplicate `$id`, unknown
|
||
dialect/vocabulary, remote/unmapped `$ref`, cycles beyond the supported depth, pathological
|
||
regex corpus, format assertion, valid/invalid envelope and payload, required/null/missing,
|
||
unknown property and unsupported payload version.
|
||
- [ ] Write RED parser/admission tests for duplicate JSON key, malformed UTF-8, unpaired surrogate,
|
||
trailing garbage, depth/string/array/object/number bounds, non-finite number, exact UTF-8
|
||
value/key/header bytes and deterministic field/scalar order.
|
||
- [ ] Add:
|
||
|
||
```groovy
|
||
implementation 'org.springframework.boot:spring-boot-starter-json'
|
||
implementation('com.networknt:json-schema-validator:3.0.2') {
|
||
exclude group: 'com.fasterxml.jackson.dataformat', module: 'jackson-dataformat-yaml'
|
||
}
|
||
```
|
||
|
||
Keep Jackson/schema runtime in the messaging leaf. Regenerate only affected dependency locks
|
||
and review the resolved Jackson 3 graph, license and vulnerability report.
|
||
- [ ] Configure NetworkNT Draft 2020-12 with format assertions enabled and an exact classpath
|
||
resource map. After startup compilation, network/file schema resolution is impossible.
|
||
- [ ] Make the writer consume only exact registered final record types. Disable polymorphic typing,
|
||
feature-provided serializers, unknown properties and reflective assignable-type search.
|
||
- [ ] Compute:
|
||
|
||
```text
|
||
SHA-256(
|
||
UTF8("ca-skeleton.messaging.envelope.v1") || 0x00
|
||
|| u32be(len(exactEnvelopeBytes))
|
||
|| exactEnvelopeBytes
|
||
)
|
||
```
|
||
|
||
and return defensive copies in `ValidatedIntegrationEvent`.
|
||
- [ ] Add validator compatibility evidence using the adopted JSON Schema Test Suite/Bowtie corpus;
|
||
custom contract compatibility still requires repository golden vectors.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :adapter:outbound:messaging:test \
|
||
--tests '*LocalJsonSchemaRegistryTest' \
|
||
--tests '*JsonSchemaIntegrationEventEncoderTest' \
|
||
--tests '*EnvelopeAdversarialCorpusTest' --console=plain
|
||
cd src && ./gradlew verifyMessagingJsonSchemaV1 \
|
||
verifyDependencyLocks --console=plain
|
||
```
|
||
|
||
- [ ] On GREEN, `verifyMessagingJsonSchemaV1` validates and writes this exact payload-free candidate
|
||
manifest:
|
||
|
||
```text
|
||
src/build/messaging-evidence/contracts-schema/manifest.json
|
||
```
|
||
|
||
It conforms to `src/config/messaging/evidence/build-evidence-manifest-v1.schema.json` and binds
|
||
the human/CI-supplied source/artifact digest, schema/catalog hashes, dependency-lock digest,
|
||
exact scenario IDs/counts, command/timestamp, failed=0, skipped=0 and unsupported claims.
|
||
Missing digest input fails the manifest-producing lane; an ordinary focused unit test may
|
||
still run without claiming release evidence.
|
||
- [ ] Update JSON/schema cards to `implemented-candidate` only after the exact tests and locks pass.
|
||
- [ ] Acceptance claim: deterministic local wire contract candidate; Kafka and durable outbox R2 are
|
||
still unimplemented.
|
||
|
||
**Rollback checkpoint:** encoder/catalog stays unwired from production append; removing it does not
|
||
change legacy rows.
|
||
|
||
### Wave B exit checkpoint
|
||
|
||
- [ ] Run:
|
||
|
||
```bash
|
||
cd src && ./gradlew :application-core:check \
|
||
:shared-contract:check \
|
||
:adapter:outbound:messaging:check \
|
||
:sample-portfolio:check \
|
||
verifyMessagingContracts \
|
||
verifyCleanArchitectureDependencies \
|
||
--console=plain
|
||
```
|
||
|
||
- [ ] Confirm no production leaf imports `dev.caskeleton.sample`.
|
||
- [ ] Update the design ledger to `P1=IMPLEMENTED_CANDIDATE` only if all Wave B evidence is GREEN.
|
||
- [ ] Update the LLM Wiki branch-note; record whether a new derived raw document exists or explicitly
|
||
record “없음”.
|
||
|
||
---
|
||
|
||
## Wave C — P2 immutable event, polling delivery and operator policy
|
||
|
||
### Task 7: Add the forward-only PostgreSQL outbox v2 schema
|
||
|
||
**Owner:** `adapter:outbound:persistence-jpa` (`:adapter:outbound:persistence-jpa`)
|
||
**Depends on:** Wave B
|
||
**Activation:** schema/control-plane only; `LEGACY_POLLING` remains ACTIVE
|
||
|
||
**Candidate migration file:**
|
||
|
||
- `src/adapter/outbound/persistence-jpa/src/main/resources/db/migration/postgresql/V7__messaging_outbox_v2.sql`
|
||
|
||
`V7` is the current candidate because the sample Flyway location already contains
|
||
`V6__poster.sql`. The Notification plan also uses `V7`/`V8` as candidates; plan text is not a
|
||
simultaneous Flyway reservation. Before implementation, scan every runtime Flyway location and all
|
||
implemented or actively executing plans. The first implementation claims the next global version;
|
||
the later plan must reserve the following version and update every path/test before writing SQL.
|
||
Messaging and Notification persistence migrations must not execute concurrently with unresolved
|
||
version ownership.
|
||
|
||
**Files — create:**
|
||
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDeliveryId.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDeliveryEntity.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDeliveryAttemptObservationEntity.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxDispositionAuditEntity.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxPublicationEpochEntity.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxAuthorityCutoverEvidenceEntity.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryJpaRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAttemptObservationJpaRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDispositionAuditJpaRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPublicationEpochJpaRepository.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2MigrationContractTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxEventEntity.java`
|
||
- `src/adapter/outbound/persistence-jpa/README.md`
|
||
- `src/adapter/outbound/persistence-jpa/CLAUDE.md`
|
||
|
||
- [ ] Before editing, classify V3 row count/state/data and assert the base card accepts only a fresh
|
||
or verified empty/drained legacy table. Any live non-empty database fails this task pending a
|
||
separate deployment-specific migration plan.
|
||
- [ ] Write a RED real-PostgreSQL migration test. Expected failure: v2 columns/tables/constraints do
|
||
not exist.
|
||
- [ ] Keep `V3__outbox_event.sql` byte-for-byte unchanged. Add immutable metadata columns
|
||
additively while retaining legacy columns for compatibility.
|
||
- [ ] Widen `event_id VARCHAR(64)` to `VARCHAR(96)` in the forward migration; this is compatible
|
||
with old writers' shorter grammar. Keep the other V3 NOT NULL columns through the rollback
|
||
window and choose one explicit compatibility projection for every canonical insert:
|
||
|
||
```text
|
||
event_type = contract_id legacy alias
|
||
payload = exact UTF-8 envelope bytes decoded as text
|
||
status = PENDING compatibility sentinel
|
||
attempt_count = 0
|
||
next_attempt_at = occurred_at
|
||
idempotency_key = event_id
|
||
```
|
||
|
||
These columns are not authority after `POLLING_V2`. Epoch predicates prevent every legacy
|
||
claim/mutation/reaper from observing canonical rows, and a post-cutover immutable guard
|
||
prevents them from drifting. Do not relax NOT NULL/defaults or leave canonical inserts
|
||
unspecified.
|
||
- [ ] Create:
|
||
|
||
```text
|
||
outbox_delivery
|
||
outbox_delivery_attempt_observation
|
||
outbox_disposition_audit
|
||
outbox_publication_epoch
|
||
outbox_authority_cutover_evidence
|
||
outbox_write_admission
|
||
outbox_runtime_node_lease
|
||
```
|
||
|
||
with event/generation primary keys, one-CURRENT partial unique constraint, delivery FK,
|
||
authority/state CHECKs, DB timestamps, row version, immutable automatic deadline and an
|
||
expiring one-shot cutover evidence identity/digest. A cutover attempt has the closed durable
|
||
state machine `CUTOVER_PENDING -> FINALIZING_V2 -> CONSUMED_V2` or
|
||
`CUTOVER_PENDING -> RECOVERING_LEGACY -> RECOVERED_LEGACY`; the two branches are mutually
|
||
exclusive CAS transitions. `RECOVERING_LEGACY` also stores an opaque recovery operation ID,
|
||
owner/lease deadline and recovery evidence digest. Lease expiry permits recovery-only
|
||
takeover and never resets the attempt to `CUTOVER_PENDING`. Give every attempt the constant
|
||
database authority scope `OUTBOX_PUBLICATION`, ACTIVE legacy epoch ID, frozen fence generation
|
||
and target-binding digest. A partial unique constraint permits exactly one nonterminal
|
||
(`CUTOVER_PENDING`, `FINALIZING_V2`, `RECOVERING_LEGACY`) attempt in that authority scope.
|
||
Attempt creation, finalization and recovery lock the write-admission singleton first and the
|
||
ACTIVE epoch second, then validate the exact frozen generation/target binding before touching
|
||
the attempt. The write admission singleton starts OPEN at generation 1; runtime node leases
|
||
are bounded and bind
|
||
node/source/artifact/fence-protocol identity without payload or credentials.
|
||
- [ ] Add event ID, partition-key, SHA-256, tenant-scope, order uniqueness and exact `BYTEA`
|
||
constraints. Protect immutable event columns with a post-cutover guard that is dormant during
|
||
compatibility migration and enabled only by the fenced P3 cutover.
|
||
- [ ] Seed exactly one ACTIVE `LEGACY_POLLING` epoch/generation for fresh/empty base template. Do not
|
||
activate `POLLING_V2`, create v2 delivery for live legacy rows or claim/send from v2.
|
||
- [ ] RED/GREEN cases:
|
||
fresh V1–V7; V3-empty upgrade; non-empty preflight rejection; duplicate current generation;
|
||
nullable tenant attack; invalid hash/key/event ID; mutable event update after guard; FK/audit
|
||
retention; publication epoch uniqueness; duplicate nonterminal authority attempt under
|
||
concurrent insert; illegal finalization/recovery state transition; 65–96 character event ID;
|
||
old-writer short ID; canonical compatibility projection satisfying every retained V3 NOT NULL
|
||
constraint.
|
||
- [ ] Verify:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*OutboxV2MigrationContractTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: additive polling v2 schema candidate only; legacy relay authority unchanged.
|
||
|
||
**Rollback checkpoint:** rollback disables new code and keeps additive schema/backlog. Never
|
||
destructively downgrade the database.
|
||
|
||
### Task 8: Append validated event and initial delivery in the business transaction
|
||
|
||
**Owner leaves:** `application-core`, `adapter-outbound-persistence-jpa`, `app-bootstrap` test
|
||
fixture
|
||
**Depends on:** Task 7
|
||
|
||
**Files — create:**
|
||
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxAppendPort.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAppendAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAppendAdapterTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapterTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2AppendTransactionalContractTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAppendPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/NewOutboxEvent.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxEvent.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapterTest.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapter.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxContainerTestSupport.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxAppendTransactionalContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxPublisherLeaderElectionContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxRowLifecycleContractTest.java`
|
||
- `src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisher.java`
|
||
- `src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogUseCase.java`
|
||
- `src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisherTest.java`
|
||
- `src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogOutboxTest.java`
|
||
- `src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/WorkLogUseCasesTest.java`
|
||
- `src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/authz/WorkLogAuthorizationContractTest.java`
|
||
|
||
- [ ] Write RED application tests making `OutboxAppendPort.append(ValidatedIntegrationEvent)` the
|
||
only canonical method. Move raw `NewOutboxEvent` append to a separately named/deprecated
|
||
`LegacyOutboxAppendPort`; never overload or silently reinterpret raw payload as v1.
|
||
- [ ] Split the current combined store/append implementation: `OutboxStoreAdapter` remains only the
|
||
legacy relay store during the observation window, while a separately named
|
||
`LegacyOutboxAppendAdapter` implements only `LegacyOutboxAppendPort`. It has no component
|
||
annotation and bootstrap may compose it only for the explicit sample/R0 compatibility graph.
|
||
- [ ] Write RED real-PostgreSQL tests proving business state + event + current delivery commit or
|
||
rollback together; encoder/schema failure rolls back business state; exact `BYTEA` and hash
|
||
round-trip. Cross-resource ACTIVE startup rejection belongs to Task 19 after both leaf
|
||
descriptors exist.
|
||
- [ ] In the persistence adapter, lock/read the ACTIVE publication epoch inside the caller-owned
|
||
transaction, attach DB-authoritative `created_at`, `publication_epoch`,
|
||
`dispatch_authority`, `transaction_resource_id`, and insert initial delivery only when
|
||
authority is `POLLING_V2`.
|
||
- [ ] For an initial current delivery, compute and persist in that same DB transaction:
|
||
|
||
```text
|
||
automaticAttemptDeadline =
|
||
min(eventDbCreatedAt + maximumAutomaticPublicationAge,
|
||
eventDbCreatedAt + contract.sameEventRequeueHorizon)
|
||
```
|
||
|
||
The value is immutable and profile reload never moves an existing generation's deadline.
|
||
- [ ] During compatibility `LEGACY_POLLING`, write both legacy required columns and validated v2
|
||
metadata in the same transaction but do not create/send a v2 current delivery.
|
||
- [ ] Make the legacy claim read model distinguish true v0 rows from rows carrying canonical v1
|
||
metadata without exposing a physical topic in application. For a canonical row,
|
||
`OutboxMessagePublishAdapter` resolves the stored logical destination through the closed
|
||
compiled binding and sends the stored partition-key bytes plus immutable `envelope_bytes`
|
||
byte-for-byte. It must not invoke `OutboxEnvelopeJson` or reinterpret the retained V3
|
||
`payload` projection. Only a true v0 row may use the old wrapper/event-type route.
|
||
- [ ] Add golden cases for canonical append under `LEGACY_POLLING` → legacy claim → exact compiled
|
||
destination/key/envelope bytes → legacy `PUBLISHED`. This remains
|
||
`LEGACY_RECORDED_UNVERIFIED` at cutover and is never automatically resent by v2. Test v0 and
|
||
canonical branches independently; mixed/missing metadata fails closed.
|
||
- [ ] Add a post-cutover canonical append fixture proving the retained V3 NOT NULL compatibility
|
||
projection, immutable event + CURRENT/READY delivery and deadline all commit together while
|
||
the epoch-fenced legacy claim/reaper sees the row count as 0.
|
||
- [ ] Use an app-bootstrap test-source typed contribution/draft to prove the canonical append path.
|
||
Do not inject the messaging encoder into `sample-portfolio` or add a project edge in this
|
||
task. Change the sample use case dependency explicitly to `LegacyOutboxAppendPort`; the
|
||
existing sample mapper remains a visibly R0, non-active compatibility fixture until the
|
||
separate standalone-sample activation plan.
|
||
- [ ] Remove component auto-discovery from the legacy append/store adapter. Bootstrap may compose it
|
||
only for an exact `LEGACY_POLLING` compatibility graph; canonical `POLLING_V2` must have
|
||
`LegacyOutboxAppendPort` bean count 0. Update every exact existing legacy/sample fixture listed
|
||
above in the same task so changing `OutboxAppendPort` cannot leave compile-only hidden users.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :application-core:test \
|
||
--tests '*Outbox*' --console=plain
|
||
cd src && ./gradlew :adapter:outbound:persistence-jpa:test \
|
||
--tests '*OutboxAppendAdapterTest' \
|
||
--tests '*LegacyOutboxAppendAdapterTest' --console=plain
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*OutboxV2AppendTransactionalContractTest' --console=plain
|
||
cd src && ./gradlew :sample-portfolio:test \
|
||
--tests '*CreateWorkLogOutboxTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: validated same-transaction append candidate; v2 relay remains disabled.
|
||
|
||
**Rollback checkpoint:** keep compatibility writes while rolling back the new relay. Do not generate
|
||
a second event ID or dual-write outside the transaction.
|
||
|
||
### Task 9: Define exhaustive publication outcomes and one-record relay policy
|
||
|
||
**Owner:** `application-core` (`:application-core`)
|
||
**Depends on:** Task 8
|
||
|
||
**Files — create under
|
||
`src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/`:**
|
||
|
||
- `PublicationOutcome.java`
|
||
- `PublicationReceipt.java`
|
||
- `PublicationFailure.java`
|
||
- `AcceptanceCertainty.java`
|
||
- `RetryDisposition.java`
|
||
- `PublicationFailureStage.java`
|
||
- `PublicationFailureClass.java`
|
||
- `PublicationAttemptId.java`
|
||
- `PublicationAdmission.java`
|
||
- `PublicationAdmissionPort.java`
|
||
- `AcknowledgedPublicationPort.java`
|
||
|
||
**Files — create under
|
||
`src/application-core/src/main/java/dev/caskeleton/application/outbox/`:**
|
||
|
||
- `OutboxDelivery.java`
|
||
- `OutboxDeliveryState.java`
|
||
- `DeliveryAuthorityStatus.java`
|
||
- `ClaimToken.java`
|
||
- `ClaimedOutboxDelivery.java`
|
||
- `OutboxDeliveryStorePort.java`
|
||
- `PublishNextOutboxDeliveryCommand.java`
|
||
- `PublishNextOutboxDeliveryResult.java`
|
||
- `PublishNextOutboxDeliveryUseCase.java`
|
||
|
||
**Files — create under
|
||
`src/application-core/src/test/java/dev/caskeleton/application/`:**
|
||
|
||
- `messaging/publication/PublicationOutcomeTest.java`
|
||
- `outbox/PublishNextOutboxDeliveryUseCaseTest.java`
|
||
- `outbox/OutboxDeliveryStateTest.java`
|
||
|
||
**Files — modify or retain as legacy until Task 26:**
|
||
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxMessagePublishPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCase.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxBackoffPolicy.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayResult.java`
|
||
|
||
- [ ] Write RED tests for the sealed outcome shape:
|
||
|
||
```java
|
||
public sealed interface PublicationOutcome {
|
||
record Acknowledged(PublicationReceipt receipt) implements PublicationOutcome {}
|
||
record AcknowledgedMismatch(PublicationReceipt receipt) implements PublicationOutcome {}
|
||
record Rejected(PublicationFailure failure) implements PublicationOutcome {}
|
||
record Indeterminate(PublicationFailure failure) implements PublicationOutcome {}
|
||
}
|
||
```
|
||
|
||
Receipt/failure contains bounded provider-neutral values only; no Kafka SDK type or raw
|
||
exception/message.
|
||
- [ ] Test certainty and retry as independent axes. Ambiguous/post-admission/timeout/unknown maps to
|
||
`INDETERMINATE`; `REJECTED` requires definite non-acceptance.
|
||
- [ ] Write relay RED tests for this exact sequence:
|
||
|
||
```text
|
||
acquire bounded admission
|
||
-> Tx B claim exactly one row + ATTEMPT_ADMITTED
|
||
-> publish outside DB transaction
|
||
-> Tx C outcome observation + token/valid-lease CAS transition
|
||
-> release admission
|
||
```
|
||
|
||
- [ ] Cover:
|
||
no admission → claim 0; no eligible row → release permit; ACK → `DELIVERY_RECORDED`; mismatch
|
||
→ `HOLD`; definite transient rejection → `RETRY_WAIT`; permanent/budget exhaustion →
|
||
`EXHAUSTED`; indeterminate → duplicate-aware retry or HOLD according to remaining finite
|
||
budget; transition failure propagates; diagnostic reporter failure cannot change persisted
|
||
state.
|
||
- [ ] Enforce one command invocation/one record. A scheduler may invoke it again; the use case must
|
||
not loop and open per-row `REQUIRES_NEW` transactions.
|
||
- [ ] Replace attempt-only backoff with a descriptor that includes maximum attempts, immutable
|
||
automatic deadline, bounded delay/jitter and same-event horizon. Do not start age at
|
||
`first_attempt_at`.
|
||
- [ ] Keep legacy void port/use case explicitly deprecated and separately wired until Task 26;
|
||
canonical code must not adapt exception-only success into `Acknowledged`.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :application-core:test \
|
||
--tests '*PublicationOutcomeTest' \
|
||
--tests '*PublishNextOutboxDeliveryUseCaseTest' \
|
||
--tests '*OutboxDeliveryStateTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: application polling/outcome policy candidate; provider and DB CAS remain
|
||
adapter work.
|
||
|
||
**Rollback checkpoint:** canonical use case remains unwired. Legacy relay continues to serve
|
||
`LEGACY_POLLING`.
|
||
|
||
### Task 10: Implement per-record JIT claim, valid-lease CAS and attempt journal
|
||
|
||
**Owner:** `adapter:outbound:persistence-jpa` (`:adapter:outbound:persistence-jpa`)
|
||
**Depends on:** Task 9
|
||
|
||
**Files — create:**
|
||
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryStoreAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryClaimRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxDeliveryClaimRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDeliveryStoreAdapterTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxDeliveryClaimRepositoryTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2ClaimCasContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2MultiWorkerContractTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlPersistenceConfig.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxContainerTestSupport.java`
|
||
|
||
- [ ] Write unit RED tests that the adapter maps application values without adding retry/topic
|
||
policy and requires affected-row count exactly 1 for every CAS.
|
||
- [ ] Write real-PostgreSQL RED tests for:
|
||
two workers claim disjoint rows; same aggregate total order uses sequence/index rather than
|
||
timestamp; different aggregates progress; hot aggregate does not starve all others; expired
|
||
claim reclaim; same-token renew; stale token/owner/generation/version rejection; expired but
|
||
not yet reclaimed owner cannot record ACK/failure.
|
||
- [ ] The worker-owned mutation predicate must include:
|
||
|
||
```sql
|
||
WHERE event_id = :event_id
|
||
AND delivery_generation = :generation
|
||
AND authority_status = 'CURRENT'
|
||
AND state = 'CLAIMED'
|
||
AND claim_token = :token
|
||
AND claim_owner = :owner
|
||
AND claim_until > CURRENT_TIMESTAMP
|
||
AND row_version = :expected_row_version
|
||
```
|
||
|
||
- [ ] Claim eligibility is CURRENT `READY`, due `RETRY_WAIT` or expired `CLAIMED`, subject to
|
||
ordering-head eligibility. `EXHAUSTED`, `HOLD`, `LEGACY_RECORDED_UNVERIFIED` never release the
|
||
next ordered event.
|
||
- [ ] Tx B atomically updates claim count/token/owner/DB-time lease/publication attempt count and
|
||
inserts `ATTEMPT_ADMITTED`. Raw claim token is never copied; journal stores a domain-separated
|
||
digest.
|
||
- [ ] Before `ATTEMPT_ADMITTED`, use DB time to verify both the automatic deadline and a full
|
||
application-attempt/Tx-C safety window remain. If `database_now >= deadline` or the full
|
||
window does not fit, perform a fenced `EXHAUSTED` transition without admission/send.
|
||
- [ ] When reclaiming an expired `CLAIMED` row whose previous `publicationAttemptId` has
|
||
`ATTEMPT_ADMITTED` but no outcome, append exactly one idempotent
|
||
`OUTCOME_OBSERVED(INDETERMINATE)` for that old attempt before replacing the token and admitting
|
||
the new attempt. Never fabricate a definite rejection or erase the prior attempt.
|
||
- [ ] Tx C atomically inserts `OUTCOME_OBSERVED` and performs ACK/retry/exhaust/hold CAS. Provider
|
||
metadata is a bounded opaque reference.
|
||
- [ ] Use DB time for eligibility, lease, created-at and retry due. Before send admission, ensure
|
||
remaining lease exceeds the full attempt + DB transition + safety budget.
|
||
- [ ] Verify RED then GREEN sequentially:
|
||
|
||
```bash
|
||
cd src && ./gradlew :adapter:outbound:persistence-jpa:test \
|
||
--tests '*OutboxDeliveryStoreAdapterTest' \
|
||
--tests '*PostgreSqlOutboxDeliveryClaimRepositoryTest' --console=plain
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*OutboxV2ClaimCasContractTest' \
|
||
--tests '*OutboxV2MultiWorkerContractTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: real-PostgreSQL JIT claim/CAS protocol candidate; no Kafka send or authority
|
||
cutover.
|
||
|
||
**Rollback checkpoint:** leave v2 scheduler off and `LEGACY_POLLING` active. Claimed test rows are
|
||
disposable; production rollback preserves all event/delivery rows.
|
||
|
||
### Task 11: Persist late publication observations with DB-commit-before-source-ACK
|
||
|
||
**Owner leaves:** `application-core`, `adapter-outbound-persistence-jpa`
|
||
**Depends on:** Task 10
|
||
|
||
**Files — create in `application-core`:**
|
||
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/ObservationId.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/LatePublicationObservation.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/LatePublicationObservationSourcePort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAttemptObservationPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsCommand.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsResult.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsUseCase.java`
|
||
- `src/application-core/src/test/java/dev/caskeleton/application/outbox/RecordLatePublicationObservationsUseCaseTest.java`
|
||
|
||
**Files — create in `adapter:outbound:persistence-jpa`:**
|
||
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAttemptObservationAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAttemptObservationAdapterTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLateObservationContractTest.java`
|
||
|
||
- [ ] Write application RED tests for:
|
||
|
||
```text
|
||
poll/lease bounded batch
|
||
-> tx.inNew(idempotent DB append) returns after commit
|
||
-> acknowledgePersisted
|
||
```
|
||
|
||
DB append/commit failure calls `releaseForRetry`; source ACK never runs in a transaction
|
||
callback.
|
||
- [ ] Use observation identity
|
||
`(eventId, deliveryGeneration, publicationAttemptId, LATE_ACK_OBSERVED)` and a DB unique
|
||
constraint/`ON CONFLICT DO NOTHING`.
|
||
- [ ] Test DB commit → process crash before source ACK by redelivering the same observation; exactly
|
||
one journal fact remains.
|
||
- [ ] Test late observation never changes `DELIVERY_RECORDED`, `RETRY_WAIT`, `EXHAUSTED`, `HOLD` or
|
||
current generation. It is diagnostic, not correctness authority.
|
||
- [ ] Test empty poll, bounded maximum, poison item release, source ACK failure and commit failure.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :application-core:test \
|
||
--tests '*RecordLatePublicationObservationsUseCaseTest' --console=plain
|
||
cd src && ./gradlew :adapter:outbound:persistence-jpa:test \
|
||
--tests '*OutboxAttemptObservationAdapterTest' --console=plain
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*OutboxLateObservationContractTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: durable idempotent late-observation drain boundary; callback capture is
|
||
still bounded-loss and no messaging queue exists until Task 16.
|
||
|
||
**Rollback checkpoint:** disabling the drain loses only bounded diagnostics, never changes delivery
|
||
authority. Alert/readiness must expose the degradation.
|
||
|
||
### Task 12: Implement audited application disposition policy and atomic persistence transitions
|
||
|
||
**Owner leaves:** `application-core`, `adapter-outbound-persistence-jpa`
|
||
**Depends on:** Tasks 10–11
|
||
|
||
**Files — create in `application-core`:**
|
||
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxDisposition.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionCommand.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionResult.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxDispositionResultCodec.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxDispositionPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionUseCase.java`
|
||
- `src/application-core/src/test/java/dev/caskeleton/application/outbox/ApplyOutboxDispositionUseCaseTest.java`
|
||
|
||
**Files — create in `adapter:outbound:persistence-jpa`:**
|
||
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDispositionAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxDispositionAdapterTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxDispositionTransactionalContractTest.java`
|
||
|
||
- [ ] Write application RED tests with `@RequiresPermission("outbox:disposition")` and
|
||
`@UseCaseCapability(idempotency = Idempotency.KEYED, ...)`.
|
||
`SKIP_WITH_GAP` and `COMPENSATE` additionally call `AuthorizationPort` for
|
||
`outbox:disposition:destructive`.
|
||
- [ ] Command requires:
|
||
|
||
```text
|
||
eventId
|
||
expectedDeliveryGeneration
|
||
expectedRowVersion
|
||
disposition
|
||
bounded reason
|
||
incident/change reference
|
||
IdempotencyContext(scope + request fingerprint + bounded TTL)
|
||
operator principal
|
||
destructive approval reference when required
|
||
compensation event reference for COMPENSATE
|
||
```
|
||
|
||
- [ ] Inject the existing application-owned `IdempotencyExecutor` into
|
||
`ApplyOutboxDispositionUseCase`. Execute authorization/policy/CAS exactly once under the
|
||
command's `IdempotencyContext`, using a framework-free deterministic
|
||
`OutboxDispositionResultCodec`. Replay returns the stored application result; same key with a
|
||
different request fingerprint raises the existing mismatch exception. Controller and
|
||
persistence adapter must not implement their own idempotency state machine.
|
||
- [ ] Application tests cover first execution, completed replay, in-flight conflict, request
|
||
mismatch, action failure/discard and bounded TTL. Persistence integration reuses the existing
|
||
`IdempotencyStorePort` adapter to prove atomic claim/complete; `outbox_disposition_audit`
|
||
remains the immutable business/operation audit rather than a second idempotency registry.
|
||
- [ ] Validate allowed source state, ordering impact, active-unexpired-claim absence and finite
|
||
same-event requeue horizon in application policy for early feedback. This precheck is not the
|
||
concurrency fence.
|
||
- [ ] In the persistence transaction, lock the CURRENT delivery row and atomically re-evaluate
|
||
expected generation/state/row version plus `NOT (state='CLAIMED' AND
|
||
claim_until > database_now)` before audit/mutation. Add a race test that inserts a worker
|
||
claim after application precheck but before the locked mutation; operator CAS must fail
|
||
without partial audit/handoff.
|
||
- [ ] Write real-PostgreSQL RED/GREEN for:
|
||
stale generation/version; live claim; idempotency replay/mismatch; concurrent requeue; one
|
||
CURRENT constraint; partial handoff rollback; old generation never claimable again; immutable
|
||
audit.
|
||
- [ ] REQUEUE transaction locks current row, inserts audit, marks old authority `SUPERSEDED`, sets
|
||
`superseded_by_generation`, and inserts generation + 1 `CURRENT/READY` with:
|
||
|
||
```text
|
||
automaticAttemptDeadline =
|
||
min(newDeliveryDbCreatedAt + maximumAutomaticPublicationAge,
|
||
eventDbCreatedAt + sameEventRequeueHorizon)
|
||
```
|
||
|
||
- [ ] HOLD/SKIP/COMPENSATE/legacy accept use expected generation/state/row version and do not mimic
|
||
the worker token predicate. `COMPENSATED` requires an already-created immutable compensating
|
||
event reference in the same transaction.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :application-core:test \
|
||
--tests '*ApplyOutboxDispositionUseCaseTest' --console=plain
|
||
cd src && ./gradlew :adapter:outbound:persistence-jpa:test \
|
||
--tests '*OutboxDispositionAdapterTest' --console=plain
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*OutboxDispositionTransactionalContractTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: provider-neutral authenticated disposition policy and DB protocol candidate;
|
||
no HTTP surface yet.
|
||
|
||
**Rollback checkpoint:** operator surface is not exposed. Data/audit rows are forward-only and must
|
||
not be rewritten by raw SQL.
|
||
|
||
### Task 13: Prove P2 compatibility fence and no-dual-authority state
|
||
|
||
**Owner:** persistence/bootstrap integration
|
||
**Depends on:** Tasks 7–12
|
||
|
||
**Files — create:**
|
||
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxPublicationEpochContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyV2CompatibilityContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyCanonicalWireCompatibilityContractTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxV2RetentionAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxV2RetentionAdapterTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2RetentionContractTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxClaimRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaper.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxEventJpaRepository.java`
|
||
|
||
- [ ] Write the epoch, canonical-wire compatibility and retention tests first, then run RED:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*OutboxPublicationEpochContractTest' \
|
||
--tests '*OutboxLegacyV2CompatibilityContractTest' \
|
||
--tests '*OutboxLegacyCanonicalWireCompatibilityContractTest' \
|
||
--tests '*OutboxV2RetentionContractTest' --console=plain
|
||
```
|
||
|
||
Expected non-zero: the legacy claim/reaper lacks an epoch fence and v2 retention protocol is
|
||
absent. A compile failure unrelated to those missing contracts is not an accepted RED.
|
||
- [ ] RED test that the compatibility legacy writer fills v2 immutable metadata in the same
|
||
transaction while the legacy relay can claim only the exact ACTIVE `LEGACY_POLLING`
|
||
epoch/generation.
|
||
- [ ] Add a legacy mutation/claim fence predicate tied to the ACTIVE publication epoch. Old
|
||
pre-fence binaries are explicitly incompatible and must be drained to zero before P3.
|
||
- [ ] Prove `POLLING_V2` claim is rejected while `LEGACY_POLLING` is active and legacy claim is
|
||
rejected after the epoch changes.
|
||
- [ ] Prove no row can be claimed/sent by both paths; publication epoch lock and expected generation
|
||
are mandatory.
|
||
- [ ] Prove the compatibility publisher sends a canonical metadata row exactly once through the
|
||
legacy authority using the compiled destination, stored key and byte-identical v1 envelope;
|
||
v0 rows still use the old wrapper. Nested envelope, event-type-as-topic for canonical rows,
|
||
mixed metadata, and automatic resend of legacy `PUBLISHED` after cutover all fail.
|
||
- [ ] Bind the legacy reaper to the exact ACTIVE `LEGACY_POLLING` epoch and stop/drain it before
|
||
cutover. Implement v2 retention separately: delete only when the unique CURRENT generation is
|
||
resolved as `DELIVERY_RECORDED` or audited `SKIPPED/COMPENSATED/
|
||
LEGACY_ACCEPTED_UNVERIFIED`, with no claim/requeue, unresolved observation, legal/operator
|
||
hold, audit-retention or replay-horizon obligation.
|
||
- [ ] Real PostgreSQL retention cases cover reaper-vs-claim/requeue, delivery/audit FK and
|
||
no-silent-cascade, superseded generations, `EXHAUSTED`, `HOLD`,
|
||
`LEGACY_RECORDED_UNVERIFIED`, unresolved late observation and epoch mismatch.
|
||
- [ ] Do not perform legacy row reconciliation, v2 delivery creation or authority switch in this
|
||
task.
|
||
- [ ] Re-run the same focused command GREEN; all three exact tests must pass with no skip. Then run
|
||
the candidate gate:
|
||
|
||
```bash
|
||
cd src && ./gradlew verifyMessagingPollingOutboxR2 --console=plain
|
||
```
|
||
|
||
At P2, `verifyMessagingPollingOutboxR2` may report `implemented-candidate`; it must not emit a
|
||
release-eligible claim. On GREEN it validates and writes the exact candidate manifest:
|
||
|
||
```text
|
||
src/app-bootstrap/build/messaging-evidence/polling-outbox-r2/manifest.json
|
||
```
|
||
|
||
The manifest conforms to
|
||
`src/config/messaging/evidence/build-evidence-manifest-v1.schema.json` and binds the supplied
|
||
source/artifact digest, migration/schema/card/profile hashes, exact PostgreSQL scenario
|
||
IDs/counts, commands/timestamps, failed=0, skipped=0 and unsupported Kafka/security claims.
|
||
- [ ] Update P2 card rows to `implemented-candidate` only if real PostgreSQL cases pass.
|
||
|
||
**Rollback checkpoint:** keep `LEGACY_POLLING` ACTIVE and canonical v2 scheduler off. If the fence
|
||
cannot be deployed to all nodes, do not proceed to Wave D.
|
||
|
||
### Wave C exit checkpoint
|
||
|
||
- [ ] Run:
|
||
|
||
```bash
|
||
cd src && ./gradlew :application-core:check \
|
||
:adapter:outbound:persistence-jpa:check \
|
||
:app-bootstrap:test \
|
||
verifyCleanArchitectureDependencies \
|
||
--console=plain
|
||
```
|
||
|
||
- [ ] Confirm broker/client/network resources are still 0 and v2 authority has not switched.
|
||
- [ ] Update the design ledger to `P2=IMPLEMENTED_CANDIDATE` only from actual tests.
|
||
- [ ] Update the LLM Wiki branch-note and derived-document decision.
|
||
|
||
---
|
||
|
||
## Wave D — P3 ACK-aware Spring Kafka, operator endpoint and reference-path cutover
|
||
|
||
### Task 14: Bind and compile finite canonical Messaging settings
|
||
|
||
**Owner:** `adapter:outbound:messaging` (`:adapter:outbound:messaging`)
|
||
**Depends on:** Wave C
|
||
|
||
**Files — create:**
|
||
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingExpectedState.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingR2Settings.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingSettingsCompiler.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerSettings.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/CompiledKafkaProducerSettings.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingSettingsCompilerTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/config/MessagingDisabledResourceContractTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerSettingsTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingSettings.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaAdapterSettings.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfig.java`
|
||
|
||
- [ ] Write RED binding/compiler tests for exact `DISABLED|ACTIVE` expected state and the canonical
|
||
fields in design §21.1. Raw `Map<String,Object>` Kafka overrides are forbidden.
|
||
- [ ] Freeze the first effective profile:
|
||
|
||
```text
|
||
acks=all
|
||
enable.idempotence=true
|
||
retries=MAX/effectively-unbounded under delivery.timeout.ms
|
||
max.in.flight.requests.per.connection<=5
|
||
compression.type=none
|
||
partitioner.ignore.keys=false
|
||
finite request/delivery/max.block/linger/batch/buffer/request bounds
|
||
maximumAdmittedRecords=1
|
||
```
|
||
|
||
- [ ] Validate:
|
||
|
||
```text
|
||
deliveryTimeout >= requestTimeout + linger
|
||
applicationAttemptBudget >=
|
||
admissionWait + maxBlock + deliveryTimeout + callback/transitionReserve
|
||
claimLease >
|
||
applicationAttemptBudget + dbTransitionReserve + schedulingSafetyMargin
|
||
```
|
||
|
||
- [ ] Reject ACTIVE + unknown/missing card/provider/bootstrap/destination/security; plaintext in
|
||
production; literal credentials; contract bytes over any bound; ordering + null key;
|
||
transaction resource mismatch; legacy + canonical keys; active durable contract + disabled
|
||
dispatch.
|
||
- [ ] DISABLED must instantiate no schema compiler with active contracts, producer factory,
|
||
template, AdminClient, semaphore, observation queue, scheduler, secret refresh or network
|
||
connection.
|
||
- [ ] Legacy keys are parsed only into an R0 descriptor and conflict with canonical keys. Do not map
|
||
`broker=kafka` to `kafka-spring` or `relay-enabled=true` to polling v2.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :adapter:outbound:messaging:test \
|
||
--tests '*MessagingSettingsCompilerTest' \
|
||
--tests '*MessagingDisabledResourceContractTest' \
|
||
--tests '*KafkaProducerSettingsTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: finite static descriptor candidate; no Kafka client created yet.
|
||
|
||
**Rollback checkpoint:** canonical activation remains disabled; legacy settings continue only in R0
|
||
mode.
|
||
|
||
### Task 15: Add explicit Spring Kafka producer factory and ACK-aware gateway
|
||
|
||
**Owner:** `adapter:outbound:messaging` (`:adapter:outbound:messaging`)
|
||
**Depends on:** Task 14
|
||
|
||
**Files — create:**
|
||
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerFactoryConfig.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaPublishGateway.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGateway.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaPublicationFailureClassifier.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/publication/AckAwareOutboxPublicationAdapter.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerFactoryConfigTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGatewayTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaPublicationFailureClassifierTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/publication/AckAwareOutboxPublicationAdapterTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/outbound/messaging/build.gradle`
|
||
- `src/adapter/outbound/messaging/gradle.lockfile`
|
||
|
||
- [ ] Add `implementation 'org.springframework.kafka:spring-kafka'`; accept the Spring Boot 4.0.0 BOM
|
||
version unless a separately reviewed compatibility override is necessary. Regenerate and
|
||
review the messaging lockfile.
|
||
- [ ] RED test exact producer properties and `DefaultKafkaProducerFactory<byte[], byte[]>` /
|
||
`KafkaTemplate<byte[], byte[]>`. Use byte serializers; no JSON serialization in Kafka
|
||
callbacks.
|
||
- [ ] RED gateway tests for:
|
||
|
||
```text
|
||
future success + metadata + expected topic -> ACKNOWLEDGED
|
||
future success + metadata + wrong topic -> ACKNOWLEDGED_MISMATCH
|
||
definite pre-admission/local rejection -> REJECTED
|
||
ambiguous/post-admission/deadline/unknown -> INDETERMINATE
|
||
```
|
||
|
||
- [ ] Build `ProducerRecord<byte[],byte[]>` only from compiled topic, stored partition-key bytes,
|
||
exact envelope bytes and bounded allowlisted headers.
|
||
- [ ] Await the future to a monotonic application deadline and verify non-null metadata. Do not call
|
||
per-message `flush()`. `cancel()` is not delivery cancellation evidence.
|
||
- [ ] Map Kafka exception categories to stable stage/class/certainty/disposition without exposing
|
||
class names or messages. Ambiguity defaults to INDETERMINATE.
|
||
- [ ] The adapter returns bounded provider generation, local ACK observation time and safe opaque
|
||
record reference. Application never routes from it.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :adapter:outbound:messaging:test \
|
||
--tests '*KafkaProducerFactoryConfigTest' \
|
||
--tests '*SpringKafkaPublishGatewayTest' \
|
||
--tests '*KafkaPublicationFailureClassifierTest' \
|
||
--tests '*AckAwareOutboxPublicationAdapterTest' --console=plain
|
||
cd src && ./gradlew verifyDependencyLocks --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: fake-gateway ACK-aware provider candidate; real Kafka ACK remains Task 21.
|
||
|
||
**Rollback checkpoint:** producer beans remain gated/dark and v2 scheduler off.
|
||
|
||
### Task 16: Add bounded admission, late-completion source and producer generation lifecycle
|
||
|
||
**Owner leaves:** `application-core` (`:application-core`),
|
||
`adapter:outbound:messaging` (`:adapter:outbound:messaging`)
|
||
**Depends on:** Task 15
|
||
|
||
**Files — create:**
|
||
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/PublicationGenerationLifecyclePort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/messaging/publication/PublicationGenerationDrainResult.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RotatePublicationGenerationCommand.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RotatePublicationGenerationResult.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RotatePublicationGenerationUseCase.java`
|
||
- `src/application-core/src/test/java/dev/caskeleton/application/outbox/RotatePublicationGenerationUseCaseTest.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/publication/BoundedPublicationAdmission.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/observation/BoundedLatePublicationObservationSource.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerGeneration.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerGenerationManager.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerLifecycle.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/publication/BoundedPublicationAdmissionTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/observation/BoundedLatePublicationObservationSourceTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/lifecycle/KafkaProducerGenerationManagerTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGateway.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/publication/AckAwareOutboxPublicationAdapter.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProducerFactoryConfig.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/SpringKafkaPublishGatewayTest.java`
|
||
|
||
- [ ] RED test finite permit count/queue wait, deadline inclusion, saturation, interrupt, release-on-all
|
||
outcomes and zero queued outbox rows after admission exhaustion.
|
||
- [ ] Implement one atomic terminal marker per send. If deadline wins, synchronous outcome stays
|
||
INDETERMINATE; one later successful callback may enqueue one payload-free late observation.
|
||
- [ ] Wire the actual Kafka future callback in `SpringKafkaPublishGateway` to the bounded source,
|
||
capturing only stable event ID, delivery generation, publication attempt ID, producer
|
||
generation and binding revision before send. Test before-deadline completion, callback-wins,
|
||
deadline-wins, duplicate callback, late success, late failure, overflow and generation-close
|
||
race against the same atomic terminal marker.
|
||
- [ ] RED test observation source lease/poll/ACK/release, bounded capacity, duplicate callback,
|
||
timeout-callback race, queue overflow/drop metric and payload/header absence.
|
||
- [ ] Overflow never mutates delivery state. It degrades readiness and alerts; capacity
|
||
qualification requires zero drop.
|
||
- [ ] RED producer generation tests for:
|
||
stop admission/claim; bounded drain; unresolved attempts durably reported
|
||
INDETERMINATE/HOLD before swap; bounded old close; secret generation resolve; new
|
||
create/attest; global generation barrier; no old/new overlap.
|
||
- [ ] Keep the messaging implementation provider-only:
|
||
`PublicationGenerationLifecyclePort` returns bounded admitted/in-flight resolution facts and
|
||
performs pause/drain/create/attest/close, but it imports no outbox store, transaction or
|
||
persistence type and never chooses HOLD/retry policy.
|
||
- [ ] `RotatePublicationGenerationUseCase` owns orchestration. It pauses new admission/claim through
|
||
provider-neutral ports, asks the provider to drain, persists every unresolved durable attempt
|
||
as INDETERMINATE and every affected ordered scope as HOLD through
|
||
`OutboxDeliveryStorePort`/`TransactionPort`, then permits close/create/attest/barrier switch.
|
||
Bootstrap invokes this use case; messaging configuration never calls persistence directly.
|
||
- [ ] A DB outage preventing durable INDETERMINATE/HOLD blocks generation switch and keeps
|
||
admission closed. Best-effort unresolved sends may return INDETERMINATE but are never
|
||
auto-replayed.
|
||
- [ ] Fatal producer state blocks new admission, lowers readiness and recreates a new immutable
|
||
generation; it does not change acceptance certainty or remove duplicate risk.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :adapter:outbound:messaging:test \
|
||
--tests '*BoundedPublicationAdmissionTest' \
|
||
--tests '*BoundedLatePublicationObservationSourceTest' \
|
||
--tests '*KafkaProducerGenerationManagerTest' --console=plain
|
||
cd src && ./gradlew :application-core:test \
|
||
--tests '*RotatePublicationGenerationUseCaseTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: bounded local resource/lifecycle protocol candidate; security/topology and
|
||
real broker evidence remain.
|
||
|
||
**Rollback checkpoint:** close the dark producer generation and keep canonical scheduler off.
|
||
|
||
### Task 17: Attest external topic topology and production Kafka security
|
||
|
||
**Owner:** `adapter:outbound:messaging` (`:adapter:outbound:messaging`)
|
||
**Depends on:** Task 16
|
||
|
||
**Files — create:**
|
||
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecurityProfile.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecuritySettings.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretReference.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretMaterial.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretMaterialResolver.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaTopicTopologyAttestor.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaProvisioningEvidence.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaTopicAttestation.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecuritySettingsTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaTopicTopologyAttestorTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSecretRedactionTest.java`
|
||
|
||
- [ ] RED test typed allowlist:
|
||
local-only plaintext, TLS server auth, and production
|
||
`SASL_SSL + SCRAM-SHA-512`. First production tuple rejects PLAIN/OAuth/mTLS profiles,
|
||
plaintext downgrade, trust-all, hostname verification disable and literal JAAS credentials.
|
||
- [ ] `KafkaSecretMaterial` exposes no secret in `toString`, exception, descriptor or log; it carries
|
||
bounded generation/expiry and clear/close lifecycle. Configuration carries only
|
||
`secret://messaging/kafka/producer`.
|
||
- [ ] Freeze the first supported resolver boundary as `mounted-secret-files-v1`.
|
||
`KafkaSecretMaterialResolver` accepts only the exact typed reference and returns SCRAM
|
||
username/password plus trust material, generation and expiry; no application/shared type
|
||
contains these provider details. Unknown scheme/path traversal, missing field, wrong
|
||
permission/format, expired generation and literal credential fail closed.
|
||
- [ ] RED topology tests for topic existence, partitions, RF, min ISR, cleanup policy, retention,
|
||
max bytes, leader/ISR and wrong cluster/binding.
|
||
- [ ] Runtime AdminClient uses only bounded `Describe` and exact-topic `DescribeConfigs`. It never
|
||
creates/alters/deletes topics, enumerates all ACLs or requires broker-wide configuration.
|
||
- [ ] Assert the same resolved generation is applied to both producer factory and AdminClient:
|
||
`security.protocol=SASL_SSL`, `sasl.mechanism=SCRAM-SHA-512`, hostname verification enabled
|
||
and no literal JAAS value in settings/descriptor/log. A partial producer-only or
|
||
AdminClient-only resolution fails startup.
|
||
- [ ] Provisioning evidence supplies runtime-inaccessible assertions:
|
||
broker policy, auto-create/unclean election, exact positive/negative ACL probes, cluster/topic
|
||
resource identity, config/ACL digest, issuer/provenance, generated/expiry time and release
|
||
assertion digest.
|
||
- [ ] Missing, stale, wrong-cluster, invalid provenance or runtime/provisioning mismatch prevents
|
||
`ACTIVE_READY`. Transient broker unavailability yields bounded `ACTIVE_NOT_READY`; static
|
||
credential/security/binding errors fail closed.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :adapter:outbound:messaging:test \
|
||
--tests '*KafkaSecuritySettingsTest' \
|
||
--tests '*KafkaTopicTopologyAttestorTest' \
|
||
--tests '*KafkaSecretRedactionTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: local topology/security validation candidate; actual TLS/SASL/ACL and
|
||
multi-broker evidence remain Wave E.
|
||
|
||
**Rollback checkpoint:** attestation failure keeps relay admission off; it never falls back to topic
|
||
auto-create, wildcard ACL or plaintext.
|
||
|
||
### Task 18: Expose the authenticated, idempotent disposition endpoint
|
||
|
||
**Owner:** `adapter:inbound:web` (`:adapter:inbound:web`)
|
||
**Depends on:** Task 12
|
||
|
||
**Files — create:**
|
||
|
||
- `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/controller/MessagingOutboxDispositionController.java`
|
||
- `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/dto/request/OutboxDispositionRequest.java`
|
||
- `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/dto/response/OutboxDispositionResponse.java`
|
||
- `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/mapper/OutboxDispositionWebMapper.java`
|
||
- `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/controller/MessagingOutboxDispositionControllerWireTest.java`
|
||
- `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/mapper/OutboxDispositionWebMapperTest.java`
|
||
- `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/controller/MessagingOutboxDispositionOpenApiContractTest.java`
|
||
- `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/ratelimit/MessagingOutboxDispositionRateLimitTest.java`
|
||
- `src/adapter/inbound/web/src/test/resources/openapi/messaging-outbox-disposition-openapi-snapshot.json`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/inbound/web/src/main/java/dev/caskeleton/adapter/inbound/web/error/GlobalExceptionHandler.java`
|
||
- `src/adapter/inbound/web/src/test/java/dev/caskeleton/adapter/inbound/web/authz/RolePermissionPolicyTest.java`
|
||
- `src/adapter/inbound/web/README.md`
|
||
- `src/adapter/inbound/web/CLAUDE.md`
|
||
|
||
- [ ] RED wire tests for:
|
||
|
||
```text
|
||
POST /internal/operations/messaging/outbox/{eventId}/dispositions
|
||
required header: Idempotency-Key
|
||
base permission: outbox:disposition
|
||
destructive permission: outbox:disposition:destructive
|
||
```
|
||
|
||
- [ ] Cover unauthenticated, insufficient permission, missing/malformed key, invalid DTO, stale
|
||
generation/version, idempotency replay/mismatch, live claim conflict, horizon exceeded,
|
||
missing destructive approval/compensation reference and success response.
|
||
- [ ] Request contains expected delivery generation, expected row version, closed disposition,
|
||
bounded reason and incident/change reference. Mapper converts
|
||
`AuthenticatedPrincipal`/request/path/header to framework-free command; no web/security type
|
||
crosses into application.
|
||
- [ ] Use `IdempotencyKeySupport` to build the existing principal/use-case-scoped
|
||
`IdempotencyScope` and compute `RequestFingerprint` from the canonical disposition request
|
||
fields, including event ID, expected generation/version, disposition, reason, incident,
|
||
approval and compensation reference. Pass the resulting `IdempotencyContext` to the use case;
|
||
never pass only a raw header string.
|
||
- [ ] Reuse `IdempotencyKeySupport` and existing authorization enforcement. Controller calls only
|
||
`ApplyOutboxDispositionUseCase`; it imports no repository, entity, outbound adapter or
|
||
transaction manager.
|
||
- [ ] Keep the endpoint authenticated and absent from the public-path allowlist. Add explicit
|
||
internal network/rate-bound contract and a committed endpoint OpenAPI snapshot. The public
|
||
path snapshot must remain unchanged; `verifyPublicPathSnapshot` proves the endpoint was not
|
||
accidentally allowlisted.
|
||
- [ ] Map stale CAS to conflict, invalid policy to safe 4xx, authorization to existing envelope and
|
||
unknown failures to safe 5xx without event payload/hash leakage.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :adapter:inbound:web:test \
|
||
--tests '*MessagingOutboxDispositionControllerWireTest' \
|
||
--tests '*OutboxDispositionWebMapperTest' \
|
||
--tests '*MessagingOutboxDispositionOpenApiContractTest' \
|
||
--tests '*MessagingOutboxDispositionRateLimitTest' \
|
||
--tests '*RolePermissionPolicyTest' --console=plain
|
||
cd src && ./gradlew verifyPublicPathSnapshot --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: authenticated transport mapping candidate; persistence/application tests
|
||
remain authority for policy/CAS.
|
||
|
||
**Rollback checkpoint:** disable route exposure through composition/network policy, not by allowing
|
||
raw SQL mutation.
|
||
|
||
### Task 19: Compose exact tuple, schedulers, readiness and observability
|
||
|
||
**Owner:** `app-bootstrap` (`:app-bootstrap`)
|
||
**Depends on:** Tasks 14–18
|
||
|
||
**Files — create:**
|
||
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfig.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityReadiness.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingRuntimeDescriptor.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MountedKafkaSecretResolverSettings.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MountedKafkaSecretMaterialResolver.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/KafkaSecretRefreshScheduler.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/LatePublicationObservationScheduler.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/transaction/PersistenceTransactionResourceDescriptor.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfigTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityReadinessTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingRuntimeDescriptorTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MountedKafkaSecretMaterialResolverTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/KafkaSecretRefreshSchedulerTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/LatePublicationObservationSchedulerTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingDisabledZeroResourceContractTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxSettings.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRelayScheduler.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxMetrics.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxConfigTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxSettingsTest.java`
|
||
- `src/app-bootstrap/src/main/resources/application.yml`
|
||
- `src/app-bootstrap/src/test/resources/application-test.yml`
|
||
- `src/app-bootstrap/README.md`
|
||
- `src/app-bootstrap/CLAUDE.md`
|
||
|
||
- [ ] RED composition tests prove settings bind/compile before secret/client, then producer, topic
|
||
attestation, readiness, v2 relay and late-drain scheduler in that order.
|
||
- [ ] Bootstrap aggregates leaf descriptors and same transaction resource identity only. It must not
|
||
reimplement catalog/schema/retry/disposition/provider rules.
|
||
- [ ] Persistence exposes a sanitized `transactionResourceId` plus resolved
|
||
DataSource/EntityManagerFactory/PlatformTransactionManager identity descriptor. Bootstrap
|
||
compares it with `TransactionPort`, canonical append adapter and the business repository
|
||
resource before creating ACTIVE clients or schedulers. Add a composition RED/GREEN case where
|
||
a second DataSource causes startup rejection and Kafka/network resource count remains 0.
|
||
- [ ] Implement `mounted-secret-files-v1` in bootstrap with an explicit bounded root, exact
|
||
reference-to-directory mapping, no symlink/path escape, owner/permission checks where the
|
||
platform exposes them, atomic generation manifest read, expiry validation, redacted failure
|
||
and prompt clearing of old char/byte material. The refresh scheduler invokes
|
||
`RotatePublicationGenerationUseCase`; it never mutates a live producer object.
|
||
- [ ] Missing/wrong/expired secret blocks ACTIVE before producer/AdminClient creation. Refresh
|
||
failure may retain the old generation only until its configured expiry/safety margin, then
|
||
closes admission/readiness. DISABLED creates resolver/refresh/file-watch resource count 0.
|
||
- [ ] Gate the v2 relay and late-drain scheduler on canonical ACTIVE + `POLLING_V2` epoch + fresh
|
||
producer/topic/security readiness. Remove the legacy `relay-enabled` boolean from canonical
|
||
mode.
|
||
- [ ] Exact `LEGACY_POLLING` compatibility composition may expose `LegacyOutboxAppendPort`;
|
||
canonical `POLLING_V2` composition must assert legacy append/store/publish/relay bean count 0.
|
||
- [ ] Readiness roles stay separate:
|
||
|
||
```text
|
||
relay = producer + topic/security + DB claim + catalog
|
||
durable write = DB append + backlog capacity
|
||
direct required producer = producer/topic/security
|
||
liveness = process-internal only
|
||
```
|
||
|
||
- [ ] Add bounded hysteresis/freshness and explicit `STARTING|ACTIVE_NOT_READY|ACTIVE_READY`.
|
||
Static configuration/security mismatch fails startup; transient broker outage never starts
|
||
relay admission.
|
||
- [ ] Runtime descriptor exposes only card IDs, versions, catalog/schema/settings digests,
|
||
destination aliases/revisions, resource ID, epoch/authority, generation, readiness,
|
||
evidence status, non-guarantees and runbook IDs. Redact servers/topics where policy requires;
|
||
never expose credentials/payload/hash/raw headers.
|
||
- [ ] Replace legacy metrics with bounded dimensions for logical attempt, certainty, failure stage,
|
||
claim conflict/lease/backlog/order block/generation/late-drop. Reject event/aggregate/tenant/
|
||
key/correlation/hash/exception-message tags. One confirmed persisted transition owns the
|
||
canonical error.
|
||
- [ ] DISABLED integration test asserts client/factory/template/AdminClient/semaphore/queue/thread/
|
||
scheduler/secret resolver/network count 0.
|
||
- [ ] Verify RED then GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*MessagingCapability*Test' \
|
||
--tests '*MessagingRuntimeDescriptorTest' \
|
||
--tests '*MountedKafkaSecretMaterialResolverTest' \
|
||
--tests '*KafkaSecretRefreshSchedulerTest' \
|
||
--tests '*LatePublicationObservationSchedulerTest' \
|
||
--tests '*MessagingDisabledZeroResourceContractTest' \
|
||
--tests '*OutboxConfigTest' \
|
||
--tests '*OutboxSettingsTest' --console=plain
|
||
```
|
||
|
||
- [ ] Acceptance claim: complete dark reference graph candidate; target deployment authority remains
|
||
legacy until Task 25.
|
||
|
||
**Rollback checkpoint:** keep canonical expected-state DISABLED and `LEGACY_POLLING` ACTIVE. No DB
|
||
schema downgrade.
|
||
|
||
### Task 20: Implement and rehearse the fenced authority cutover without production switch
|
||
|
||
**Owner leaves:** `application-core`, `adapter-outbound-persistence-jpa`, `app-bootstrap`,
|
||
`adapter-outbound-messaging`
|
||
**Depends on:** Tasks 13–19
|
||
**Base template gate:** fresh or verified empty/drained V3 only
|
||
|
||
Every cutover in this task runs against a disposable rehearsal database and test broker. It proves
|
||
the code/protocol but does not change a target deployment, start its v2 relay, resume its business
|
||
writes or delete legacy runtime. Production remains `LEGACY_POLLING`.
|
||
|
||
**Files — create:**
|
||
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxWriteAdmissionControlPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelayControlPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxWriteAdmissionSnapshot.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelaySnapshot.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionEvidence.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverCommand.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverResult.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAuthorityCutoverPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCase.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionCommand.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionResult.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionUseCase.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxPreCommitRecoveryPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityCommand.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityResult.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCase.java`
|
||
- `src/application-core/src/test/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCaseTest.java`
|
||
- `src/application-core/src/test/java/dev/caskeleton/application/outbox/ResumePollingV2WriteAdmissionUseCaseTest.java`
|
||
- `src/application-core/src/test/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCaseTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxCutoverPreconditionAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxWriteAdmissionEntity.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/entity/OutboxRuntimeNodeLeaseEntity.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxWriteAdmissionJpaRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxRuntimeNodeLeaseJpaRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/PostgreSqlOutboxWriteAdmissionGuard.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxWriteAdmissionControlAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxRuntimeNodeLeaseAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapterTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxWriteAdmissionControlAdapterTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxRuntimeNodeLeaseAdapterTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapterTest.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFence.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFenceTest.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinator.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinator.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapter.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRuntimeNodeLeaseScheduler.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverJobSettings.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunner.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinatorTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinatorTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapterTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxRuntimeNodeLeaseSchedulerTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunnerTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyToV2CutoverContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxWriteAdmissionMultiNodeContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2SentinelContractTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/transaction/SpringTransactionPort.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/transaction/SpringTransactionPortTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaper.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaperTest.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapter.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRelayScheduler.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxConfigTest.java`
|
||
- `src/app-bootstrap/build.gradle`
|
||
- `src/build.gradle`
|
||
|
||
- [ ] RED application tests scope `FinalizeOutboxAuthorityCutoverUseCase` to the atomic database
|
||
finalization contract. It accepts an opaque, human-approved cutover evidence ID and never
|
||
trusts command booleans for writer/relay/producer drain. Before reconciliation it must CAS the
|
||
exact fresh attempt `CUTOVER_PENDING -> FINALIZING_V2` in the same database transaction that
|
||
commits the epoch, after taking the global write-admission/ACTIVE-epoch locks in the fixed
|
||
order and proving it is the sole nonterminal `OUTBOX_PUBLICATION` attempt;
|
||
`RECOVERING_LEGACY`, an expired attempt or a different operation owner is a hard rejection.
|
||
Rollback restores `CUTOVER_PENDING`, while a successful epoch commit records `CONSUMED_V2`.
|
||
This is a one-shot maintenance use case, not a second web endpoint.
|
||
- [ ] `OutboxLegacyToV2CutoverCoordinator` is the deployment/composition owner. Through
|
||
`OutboxWriteAdmissionControlPort`, `LegacyOutboxRelayControlPort` and
|
||
`PublicationGenerationLifecyclePort`, it freezes writes, drains writers/relay/futures,
|
||
closes/fences legacy Write, compiles/attests the canonical tuple and asks
|
||
`OutboxCutoverPreconditionPort` to persist a short-lived one-shot evidence record containing
|
||
exact node/writer/relay/producer generations, zero-active facts, epoch, manifest digest,
|
||
approver and expiry in `CUTOVER_PENDING`. Evidence creation uses the global lock order
|
||
write-admission singleton `FOR UPDATE` then ACTIVE epoch `FOR UPDATE`, requires the exact
|
||
FROZEN generation/target binding and rejects any nonterminal attempt in
|
||
`OUTBOX_PUBLICATION`; the partial unique constraint is the final concurrent-insert guard.
|
||
Bootstrap imports only application ports; it never queries repositories or Kafka adapter
|
||
internals.
|
||
- [ ] Implement the production write fence with the PostgreSQL singleton created in Task 7.
|
||
`SpringTransactionPort.inWrite` begins its transaction, acquires `FOR KEY SHARE` through
|
||
`PostgreSqlOutboxWriteAdmissionGuard`, and verifies OPEN + expected fence generation before
|
||
invoking any business action. The control adapter takes `FOR UPDATE`, waits for all older
|
||
share-holding writers to commit/rollback, writes FROZEN generation and then returns a durable
|
||
zero-active snapshot. New `inWrite` calls fail and roll back; `inNew` remains available only
|
||
for maintenance/outbox/audit and never bypasses a business write.
|
||
- [ ] Implement both explicit generation-CAS exits from FROZEN without a raw status update.
|
||
`ResumePollingV2WriteAdmissionUseCase` requires expected frozen generation, exact ACTIVE
|
||
`POLLING_V2` epoch, canonical cutover sentinel created through `OutboxAppendAdapter` and
|
||
persisted as `DELIVERY_RECORDED`, fresh target binding/readiness and no legacy runtime. It
|
||
writes `OPEN(generation+1)` once; mismatch/replay/failure leaves FROZEN.
|
||
`RecoverLegacyOutboxAuthorityUseCase` is legal only before epoch commit. Before any external
|
||
ACL mutation, its prepare operation takes the same global write-admission/ACTIVE-epoch lock
|
||
order, proves the exact attempt is the sole nonterminal authority attempt plus
|
||
FROZEN/`LEGACY_POLLING`/zero-v2-authority, then CAS-claims
|
||
`CUTOVER_PENDING -> RECOVERING_LEGACY` and atomically invalidates that attempt for v2
|
||
finalization. Choosing this branch is irreversible for that attempt; only recovery completion
|
||
or recovery-only lease takeover remains legal. The complete operation follows the separately
|
||
fenced recovery protocol below.
|
||
- [ ] Register bounded `outbox_runtime_node_lease` heartbeats for every runtime node with node ID,
|
||
source/artifact digest, write-fence protocol version, epoch and scheduler roles. Precondition
|
||
evidence requires all deployment-inventory instances to have a matching fresh lease and
|
||
rejects stale, unknown, pre-fence or missing nodes. A lease table alone does not prove the
|
||
absence of an unregistered process; target deployment inventory/provenance is also mandatory.
|
||
- [ ] `LegacyOutboxRelayControlAdapter` owns composition of existing runtime controls:
|
||
pause new `OutboxRelayScheduler` cycles, wait active cycles/claims to the finite deadline,
|
||
pause/drain the epoch-fenced `OutboxReaper`, and close
|
||
`LegacyPublicationWriteFence` so `OutboxMessagePublishAdapter` rejects every later send.
|
||
Snapshot counts/generations are bounded facts only. Reopen methods require expected component
|
||
generations plus fresh pre-commit recovery evidence and reject once ACTIVE epoch is not
|
||
`LEGACY_POLLING`; no generic boolean setter exists. Application cutover policy sees the port,
|
||
not concrete scheduler/reaper/messaging types.
|
||
- [ ] The disposable security rehearsal and actual target preflight use distinct legacy/canonical
|
||
principals. Revoke legacy exact-topic Write and require a negative Write probe while canonical
|
||
Describe/DescribeConfigs/Write stays positive. The in-process fence plus external ACL evidence
|
||
are both required; neither substitutes for the other.
|
||
- [ ] Add the exact non-web one-shot operational entrypoint
|
||
`MessagingAuthorityCutoverApplicationRunner`. It activates only for
|
||
the closed operations `legacy-to-polling-v2`, `recover-legacy-precommit`, or
|
||
`resume-polling-v2-writes`. It requires opaque operation/approval-evidence IDs plus expected
|
||
target/source/artifact/epoch/fence generation, invokes only the corresponding application use
|
||
case/coordinator, emits no payload/secret, and exits non-zero on mismatch/replay/failure. The
|
||
main cutover exits 0 only after sentinel/readiness proof and write admission
|
||
`OPEN(generation+1)`; a post-commit resume failure stays FROZEN and requires the separately
|
||
one-shot `resume-polling-v2-writes` operation. Consumed DB evidence/operation IDs make retries
|
||
non-reentrant; no controller endpoint is added.
|
||
|
||
```text
|
||
ca-skeleton.messaging.maintenance.operation
|
||
ca-skeleton.messaging.maintenance.operation-id
|
||
ca-skeleton.messaging.maintenance.approval-evidence-id
|
||
ca-skeleton.messaging.maintenance.expected-target-alias
|
||
ca-skeleton.messaging.maintenance.expected-source-digest
|
||
ca-skeleton.messaging.maintenance.expected-artifact-digest
|
||
ca-skeleton.messaging.maintenance.expected-epoch
|
||
ca-skeleton.messaging.maintenance.expected-fence-generation
|
||
```
|
||
|
||
Task 23 registers these exact maintenance-only keys and the runbook's non-web launcher
|
||
contract; none has a default that enables the runner.
|
||
- [ ] In the rehearsal harness:
|
||
deploy ACK-aware producer/v2 relay scheduler-disabled; compile the candidate tuple; prove
|
||
disposition auth/CAS negatives; execute the real PostgreSQL business-write admission freeze;
|
||
drain active writers/epoch share holders; stop legacy new claims and legacy reaper; drain
|
||
`IN_FLIGHT` to the maximum budget; audit remaining indeterminate; close/fence legacy producer
|
||
Write and DB legacy mutation.
|
||
- [ ] Real multi-node PostgreSQL tests hold old `inWrite` transactions across freeze, start new
|
||
writers during/after freeze, inject a stale/pre-fence node lease and omit a deployment
|
||
inventory member. Freeze must wait for old holders, reject new writes without partial business/
|
||
outbox state, and refuse evidence until every live instance/fence/relay/reaper/producer fact is
|
||
exact and zero-active.
|
||
- [ ] In one PostgreSQL transaction:
|
||
|
||
```text
|
||
lock OUTBOX_PUBLICATION write-admission singleton FOR UPDATE
|
||
-> lock ACTIVE LEGACY_POLLING epoch FOR UPDATE
|
||
-> assert exact FROZEN generation/target binding and sole nonterminal attempt
|
||
-> lock exact fresh cutover attempt
|
||
-> CAS CUTOVER_PENDING -> FINALIZING_V2
|
||
-> assert writer/legacy mutation fences
|
||
-> capture fixed legacy handoff watermark
|
||
-> final reconcile every row through watermark including final delta
|
||
-> assert exactly one CURRENT delivery per event, active claims 0
|
||
-> assert row count + event ID/hash manifest, unmapped/duplicate count 0
|
||
-> switch ACTIVE epoch LEGACY_POLLING -> POLLING_V2
|
||
-> append v2 cutover sentinel through the canonical append adapter
|
||
with retained V3 projection + CURRENT/READY delivery
|
||
-> mark the same attempt CONSUMED_V2
|
||
-> commit
|
||
```
|
||
|
||
- [ ] Missing, expired, reused, wrong-epoch/generation, wrong-manifest or non-zero cutover evidence
|
||
rolls back before reconciliation. `RECOVERING_LEGACY`, `RECOVERED_LEGACY`, a foreign recovery
|
||
owner or any non-`CUTOVER_PENDING` state also rejects finalization. The final transaction
|
||
marks the evidence consumed; the coordinator cannot replay it.
|
||
- [ ] Migration-only state mapping is exact:
|
||
|
||
```text
|
||
PENDING -> READY
|
||
FAILED -> RETRY_WAIT with reviewed DB-time due/budget
|
||
DEAD -> EXHAUSTED
|
||
PUBLISHED -> LEGACY_RECORDED_UNVERIFIED
|
||
IN_FLIGHT -> HOLD + remaining-indeterminate audit
|
||
```
|
||
|
||
Preserve reviewed attempt count/due/deadline and every historical observation; never
|
||
fabricate broker metadata, definite rejection, ACK observed time or `DELIVERY_RECORDED`.
|
||
- [ ] Any unknown contract/status/hash mismatch, duplicate current row, count/manifest mismatch,
|
||
active claim, fence failure or sentinel failure rolls the whole transaction back and leaves
|
||
`LEGACY_POLLING` authoritative.
|
||
- [ ] In the disposable rehearsal only, after commit start v2 relay, require sentinel
|
||
ACK/`DELIVERY_RECORDED`, prove the sentinel used the canonical append path and retained V3 NOT
|
||
NULL projection, verify legacy writer/claim/reaper/send 0, then execute the exact
|
||
FROZEN→OPEN generation CAS. Only after OPEN, exercise an ordinary `TransactionPort.inWrite`
|
||
canonical append; failure immediately freezes a new generation and fails the rehearsal.
|
||
- [ ] RED/GREEN fault cases at every numbered point, including crash before transaction, after
|
||
watermark, during final delta, before epoch switch, before/after sentinel insert and after
|
||
commit. No case permits dual authority or missing manifest row.
|
||
- [ ] Rehearse the two authority zones and both pre-commit choices:
|
||
|
||
```text
|
||
pre-commit:
|
||
keep all fences closed
|
||
-> bounded forward retry while evidence/approval remains fresh
|
||
OR
|
||
prove epoch still LEGACY_POLLING + v2 business send/sentinel authority 0 + exact inventory
|
||
-> DB-CAS exact attempt CUTOVER_PENDING -> RECOVERING_LEGACY
|
||
and atomically invalidate it for every forward finalizer
|
||
-> externally regrant legacy exact-topic Write and verify a fresh positive probe
|
||
-> re-lock attempt + epoch and revalidate RECOVERING_LEGACY owner/lease + LEGACY_POLLING
|
||
(on mismatch immediately revoke legacy Write and prove a fresh negative probe)
|
||
-> append immutable recovery/ACL audit
|
||
-> reopen in-process legacy Write fence, reaper and relay with expected generations
|
||
-> CAS write admission FROZEN -> OPEN(generation+1) and mark RECOVERED_LEGACY last
|
||
post-commit, regardless of whether a business v2 send occurred:
|
||
legacy reactivation/reverse epoch is unsupported
|
||
-> keep write admission FROZEN
|
||
-> after sentinel/readiness/canonical-projection proof, CAS OPEN(generation+1)
|
||
-> otherwise preserve backlog/schema/epoch/audit and forward-fix
|
||
```
|
||
|
||
Once `RECOVERING_LEGACY` is claimed, no forward retry or separately created attempt can commit
|
||
in `OUTBOX_PUBLICATION`, including after recovery lease expiry. Inject barrier races in both
|
||
lock orders for same-attempt finalization versus recovery prepare, different-attempt creation/
|
||
finalization versus recovery prepare/completion, plus crash/abort before and after the durable
|
||
recovery claim, external ACL regrant, post-ACL epoch revalidation, each component reopen and
|
||
final admission CAS.
|
||
Wrong epoch/inventory/ACL evidence, stale generation, partial legacy reopen, duplicate
|
||
operation, resume-before-sentinel and DB failure must never open business writes. If
|
||
post-ACL revalidation fails, immediately revoke legacy Write and require a new negative probe;
|
||
if a legacy component was reopened but final admission CAS failed, business writes stay
|
||
FROZEN and the coordinator re-fences or records the exact safe degraded state for idempotent
|
||
recovery-only retry.
|
||
- [ ] Verify:
|
||
|
||
```bash
|
||
cd src && ./gradlew :application-core:test \
|
||
--tests '*FinalizeOutboxAuthorityCutoverUseCaseTest' \
|
||
--tests '*ResumePollingV2WriteAdmissionUseCaseTest' \
|
||
--tests '*RecoverLegacyOutboxAuthorityUseCaseTest' --console=plain
|
||
cd src && ./gradlew :adapter:outbound:persistence-jpa:test \
|
||
--tests '*OutboxAuthorityCutoverAdapterTest' \
|
||
--tests '*OutboxWriteAdmissionControlAdapterTest' \
|
||
--tests '*OutboxRuntimeNodeLeaseAdapterTest' \
|
||
--tests '*OutboxPreCommitRecoveryAdapterTest' \
|
||
--tests '*SpringTransactionPortTest' --console=plain
|
||
cd src && ./gradlew :adapter:outbound:messaging:test \
|
||
--tests '*LegacyPublicationWriteFenceTest' \
|
||
--tests '*OutboxMessagePublishAdapterTest' --console=plain
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*OutboxLegacyToV2CutoverCoordinatorTest' \
|
||
--tests '*OutboxLegacyPreCommitRecoveryCoordinatorTest' \
|
||
--tests '*LegacyOutboxRelayControlAdapterTest' \
|
||
--tests '*OutboxRuntimeNodeLeaseSchedulerTest' \
|
||
--tests '*MessagingAuthorityCutoverApplicationRunnerTest' --console=plain
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*OutboxLegacyToV2CutoverContractTest' \
|
||
--tests '*OutboxWriteAdmissionMultiNodeContractTest' \
|
||
--tests '*OutboxV2SentinelContractTest' --console=plain
|
||
```
|
||
|
||
- [ ] After the disposable RED/GREEN matrix passes, schema-validate and write:
|
||
|
||
```text
|
||
src/app-bootstrap/build/messaging-evidence/cutover-rehearsal/manifest.json
|
||
```
|
||
|
||
It binds the supplied source/artifact digest, migration/card/profile/catalog/settings hashes,
|
||
disposable database/broker identity, every fault point and rollback-zone scenario, exact row/
|
||
manifest/sentinel assertions, pre-commit recovery and post-commit resume generation-CAS
|
||
scenarios, commands/timestamps, failed=0, skipped=0 and the explicit non-claim
|
||
`targetDeploymentCutOver=false`. It conforms to the common build-evidence schema.
|
||
- [ ] Acceptance claim: cutover implementation and disposable fault rehearsal candidate only.
|
||
Target deployments remain `LEGACY_POLLING`; no legacy code/config is deleted and the tuple
|
||
remains non-R2 until Wave E evidence.
|
||
|
||
**Rollback checkpoint:** discard the rehearsal database/broker. Never apply rehearsal evidence as a
|
||
target deployment switch or destructively downgrade schema.
|
||
|
||
### Wave D exit checkpoint
|
||
|
||
- [ ] Run focused owner checks and:
|
||
|
||
```bash
|
||
cd src && ./gradlew verifyMessagingContracts \
|
||
verifyMessagingJsonSchemaV1 \
|
||
verifyMessagingPollingOutboxR2 \
|
||
verifyCleanArchitectureDependencies \
|
||
verifyEnvKeys \
|
||
verifyPublicPathSnapshot \
|
||
--console=plain
|
||
```
|
||
|
||
- [ ] Keep Kafka/security cards at most `implemented-candidate`.
|
||
- [ ] Update the design ledger to `P3=IMPLEMENTED_CANDIDATE` only after the dark graph and disposable
|
||
cutover rehearsal pass; target authority is still legacy.
|
||
- [ ] Update the LLM Wiki branch-note and derived-document decision.
|
||
|
||
---
|
||
|
||
## Wave E — P4 real-service, security, fault and release qualification
|
||
|
||
### Task 21: Prove real PostgreSQL + real Kafka reference behavior
|
||
|
||
**Owner:** `app-bootstrap` qualification tests
|
||
**Depends on:** Wave D
|
||
|
||
**Files — create:**
|
||
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingKafkaR2ContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingPollingKafkaEndToEndContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingKafkaFaultContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/messaging/MessagingKafkaContainerSupport.java`
|
||
- `src/app-bootstrap/src/test/resources/messaging/evidence/messaging-evidence-schema-v1.json`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/app-bootstrap/build.gradle`
|
||
- `src/app-bootstrap/gradle.lockfile`
|
||
- `src/build.gradle`
|
||
|
||
- [ ] Add test-only:
|
||
|
||
```groovy
|
||
testImplementation 'org.testcontainers:testcontainers-kafka'
|
||
testImplementation 'org.testcontainers:testcontainers-toxiproxy'
|
||
testImplementation 'org.springframework.kafka:spring-kafka-test'
|
||
```
|
||
|
||
Pin container image digest in qualification settings and record broker/client/Spring
|
||
versions.
|
||
- [ ] Register `:app-bootstrap:messagingKafkaProducerR2` with these exact filters and
|
||
`failOnNoMatchingTests=true`:
|
||
|
||
```text
|
||
dev.caskeleton.bootstrap.integration.messaging.MessagingKafkaR2ContractTest
|
||
dev.caskeleton.bootstrap.integration.messaging.MessagingPollingKafkaEndToEndContractTest
|
||
dev.caskeleton.bootstrap.integration.messaging.MessagingKafkaFaultContractTest
|
||
```
|
||
|
||
Root `verifyMessagingKafkaProducerR2` depends on that Test task and validates its evidence.
|
||
Docker/image pull/test skip is failure, not PASS.
|
||
- [ ] After registering the task but before implementing the three tests, run RED:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:messagingKafkaProducerR2 --console=plain
|
||
```
|
||
|
||
Expected non-zero: no matching required tests or absent real-service evidence. Any unrelated
|
||
compile failure must be fixed before proceeding.
|
||
- [ ] Real Kafka RED/GREEN cases:
|
||
actual topic/partition/offset metadata; expected-topic mismatch; `acks=all`/idempotence
|
||
effective config; stable key/partition; header/record oversize; missing topic with auto-create
|
||
disabled; broker unavailable before send; leader/retriable failure; response loss/deadline/
|
||
late ACK; local buffer saturation/max-block; throttle; no per-message flush; graceful/forced
|
||
close; fatal generation recreation.
|
||
- [ ] Combined real PostgreSQL + Kafka cases:
|
||
event/delivery commit; JIT claim/admission; ACK → delivery CAS; ACK-to-DB crash/reclaim
|
||
duplicate; stale token after late ACK; outcome commit failure; late DB-commit-before-source-ACK
|
||
duplicate absorption; backlog outage/recovery; multi-worker disjoint claim/order/fairness.
|
||
- [ ] Add the rolling-compatibility golden path against the real broker: canonical append while
|
||
`LEGACY_POLLING` is active → legacy claim → broker-observed exact compiled topic, stored key and
|
||
byte-identical v1 envelope → legacy terminal `PUBLISHED` → disposable cutover maps
|
||
`LEGACY_RECORDED_UNVERIFIED` with no automatic v2 resend. Nested envelope, legacy event-type
|
||
routing for a canonical row and mixed metadata are negative cases.
|
||
- [ ] Fault injection must observe both broker event IDs and DB state at:
|
||
|
||
```text
|
||
after event commit
|
||
after claim commit
|
||
before send
|
||
after request write
|
||
after broker append before ACK receipt
|
||
after ACK before DB transition
|
||
during DB transition commit
|
||
after DB success before scheduler result
|
||
during shutdown
|
||
```
|
||
|
||
- [ ] Single-node evidence is labelled provider baseline only. It cannot satisfy RF/min ISR,
|
||
leader-loss or production security rows.
|
||
- [ ] Generate a sanitized manifest at the non-versioned exact path:
|
||
|
||
```text
|
||
src/app-bootstrap/build/messaging-evidence/real-kafka-postgresql-r2/manifest.json
|
||
```
|
||
|
||
Validate it against
|
||
`src/app-bootstrap/src/test/resources/messaging/evidence/messaging-evidence-schema-v1.json`.
|
||
Also validate the same bytes against
|
||
`src/config/messaging/evidence/build-evidence-manifest-v1.schema.json`; the lane schema may add
|
||
fields but cannot weaken the common source/artifact/scenario/failure/skip contract.
|
||
CI retains the same bytes under
|
||
`ci-artifact://messaging/{sourceDigest}/real-kafka-postgresql-r2/manifest.json`. The manifest
|
||
contains source/artifact digest supplied by human/CI, commands/timestamps,
|
||
test counts, versions/image digests, non-secret effective settings, hashes, scenarios/results,
|
||
skips/failures, unsupported claims and runbook IDs.
|
||
- [ ] Re-run GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:messagingKafkaProducerR2 \
|
||
verifyMessagingKafkaProducerR2 \
|
||
verifyMessagingPollingOutboxR2 --console=plain
|
||
```
|
||
|
||
Expected: all listed scenario IDs occur exactly once, failed=0, skipped=0, schema validation
|
||
PASS and source/artifact digests match.
|
||
- [ ] Acceptance claim: real single-node Kafka + PostgreSQL R2-candidate evidence; production tuple
|
||
remains NOT_QUALIFIED.
|
||
|
||
**Rollback checkpoint:** qualification uses disposable services. Production activation is still
|
||
blocked by Task 22.
|
||
|
||
### Task 22: Qualify SASL_SSL/SCRAM, least privilege and multi-broker topology
|
||
|
||
**Owner:** deployment/security qualification lane + `app-bootstrap` aggregator
|
||
**Depends on:** Task 21
|
||
|
||
**Files — create:**
|
||
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/qualification/messaging/MessagingSecurityR2QualificationTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/qualification/messaging/MessagingMultiBrokerR2QualificationTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/qualification/messaging/MessagingRotationShutdownQualificationTest.java`
|
||
- `src/app-bootstrap/src/test/resources/messaging/qualification/docker-compose.kafka-r2.yml`
|
||
- `src/app-bootstrap/src/test/resources/messaging/qualification/README.md`
|
||
- `src/config/messaging/evidence/messaging-release-evidence-schema-v1.json`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/app-bootstrap/build.gradle`
|
||
- `src/build.gradle`
|
||
- `src/config/messaging/release-profile-assertions.yaml`
|
||
|
||
- [ ] Register three non-ordinary Test tasks with exact filters and
|
||
`failOnNoMatchingTests=true`:
|
||
|
||
```text
|
||
:app-bootstrap:messagingSecurityR2
|
||
-> dev.caskeleton.bootstrap.qualification.messaging.MessagingSecurityR2QualificationTest
|
||
:app-bootstrap:messagingMultiBrokerR2
|
||
-> dev.caskeleton.bootstrap.qualification.messaging.MessagingMultiBrokerR2QualificationTest
|
||
:app-bootstrap:messagingRotationShutdownR2
|
||
-> dev.caskeleton.bootstrap.qualification.messaging.MessagingRotationShutdownQualificationTest
|
||
```
|
||
|
||
Root `verifyMessagingSecurityR2` depends on all three evidence validators. Missing topology,
|
||
credential fixture, certificate, Docker/image or tests fails the release task.
|
||
- [ ] After task registration but before the qualification environment/tests are complete, run RED:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:messagingSecurityR2 \
|
||
:app-bootstrap:messagingMultiBrokerR2 \
|
||
:app-bootstrap:messagingRotationShutdownR2 \
|
||
--console=plain
|
||
```
|
||
|
||
Expected non-zero for an exact missing test/topology/security prerequisite. SKIPPED is not an
|
||
accepted RED or GREEN result.
|
||
- [ ] Use a pinned three-broker topology with RF=3/min ISR=2 and production-like
|
||
SASL_SSL/SCRAM-SHA-512. Ephemeral test credentials/certificates never enter source/evidence.
|
||
- [ ] Security positive/negative cases:
|
||
trusted TLS; untrusted CA; hostname mismatch; expired/not-yet-valid cert; valid/invalid SCRAM;
|
||
missing/expired secret; production plaintext rejection; redaction; exact-topic Describe/
|
||
DescribeConfigs/Write; denied Create/Delete/Alter/other-topic Write/consumer Read. Use distinct
|
||
canonical and legacy fixture principals and prove the legacy principal's exact-topic Write can
|
||
be revoked without removing canonical Describe/DescribeConfigs/Write.
|
||
- [ ] Topology cases:
|
||
expected partitions/RF/min ISR; cleanup/retention/max bytes drift; wrong cluster/topic;
|
||
auto-create disabled; leader loss with ISR sufficient; below-min-ISR rejection/indeterminate
|
||
mapping; recovery; provisioning evidence freshness/provenance.
|
||
- [ ] Provisioning evidence includes the selected broker's
|
||
`replica.lag.time.max.ms`, the producer's effective `request.timeout.ms` and the approved
|
||
compatibility relation from design §16.6. Add a mismatch negative case; do not infer the
|
||
broker value from a client default.
|
||
- [ ] Rotation/lifecycle cases:
|
||
stop admission; bounded old drain; forced unresolved → durable INDETERMINATE/HOLD; old close;
|
||
new secret/producer/attestation; generation barrier; no old/new overlap; DB-unavailable switch
|
||
rejection; shutdown under load.
|
||
- [ ] Capacity/soak cases:
|
||
sustained drain, hot aggregate, broker throttle/outage/recovery storm, retry amplification,
|
||
producer memory/buffer/GC, DB pool/claim query, metric cardinality and late-observation drop 0.
|
||
Record numbers as selected-environment evidence, not universal repository performance claims.
|
||
- [ ] Emit and schema-validate these non-versioned exact files:
|
||
|
||
```text
|
||
src/app-bootstrap/build/messaging-evidence/security-r2/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/multi-broker-r2/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/rotation-shutdown-r2/manifest.json
|
||
```
|
||
|
||
Validate each byte-identical file against both
|
||
`src/config/messaging/evidence/build-evidence-manifest-v1.schema.json` and the stricter
|
||
`src/config/messaging/evidence/messaging-release-evidence-schema-v1.json`. Add a contract test
|
||
proving the lane schema retains every common required field and rejection rule.
|
||
CI retains byte-identical artifacts under the source-digest-qualified
|
||
`ci-artifact://messaging/` namespace. Each manifest must match exact source/artifact digest,
|
||
`qualificationEnvironmentIdentity` (fixture broker/image/principal provenance), topic/security
|
||
profile, card/settings/catalog/schema hashes, scenario set and freshness window. Any mismatch
|
||
or skip keeps all affected cards `implemented-candidate`. This identity is never reused as a
|
||
target `deploymentBindingIdentity`; only capability/profile, supported broker/client version
|
||
constraints, settings/catalog/schema and scenario-contract revisions are portable.
|
||
- [ ] Re-run GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:messagingSecurityR2 \
|
||
:app-bootstrap:messagingMultiBrokerR2 \
|
||
:app-bootstrap:messagingRotationShutdownR2 \
|
||
verifyMessagingSecurityR2 --console=plain
|
||
```
|
||
|
||
Expected: every required scenario ID exactly once, failed=0, skipped=0, all three manifests
|
||
pass `messaging-release-evidence-schema-v1.json`, and all source/artifact/profile digests
|
||
match.
|
||
- [ ] Acceptance claim: exact production security/topology candidate only after all required
|
||
scenarios PASS. No consumer/CDC claim.
|
||
|
||
**Rollback checkpoint:** failed qualification prevents release promotion; do not weaken RF/min ISR,
|
||
ACL, TLS or card requirements to make the lane green.
|
||
|
||
### Task 23: Synchronize configuration, registries, runbooks and operational truth
|
||
|
||
**Owner:** repository documentation/configuration
|
||
**Depends on:** Tasks 19–22
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/app-bootstrap/src/main/resources/application.yml`
|
||
- `src/app-bootstrap/src/test/resources/application-test.yml`
|
||
- `src/.env`
|
||
- `docs/registries/env-keys.yaml`
|
||
- `docs/registries/capabilities.yaml`
|
||
- `docs/registries/error-codes.yaml`
|
||
- `docs/registries/metrics.yaml`
|
||
- `docs/registries/secrets-classification.yaml`
|
||
- `src/application-core/README.md`
|
||
- `src/application-core/CLAUDE.md`
|
||
- `src/shared-contract/README.md`
|
||
- `src/shared-contract/CLAUDE.md`
|
||
- `src/adapter/outbound/messaging/README.md`
|
||
- `src/adapter/outbound/messaging/CLAUDE.md`
|
||
- `src/adapter/outbound/persistence-jpa/README.md`
|
||
- `src/adapter/outbound/persistence-jpa/CLAUDE.md`
|
||
- `src/adapter/inbound/web/README.md`
|
||
- `src/adapter/inbound/web/CLAUDE.md`
|
||
- `src/app-bootstrap/README.md`
|
||
- `src/app-bootstrap/CLAUDE.md`
|
||
- `docs/runbooks/outbox-publish-failed.md`
|
||
- `docs/runbooks/outbox-dead-letter.md`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/RunbookCoverageContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/outbox/OutboxStatusRegistryContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/outbox/EventPayloadPiiContractTest.java`
|
||
|
||
**Files — create:**
|
||
|
||
- `docs/runbooks/messaging-producer-unavailable-or-unauthorized.md`
|
||
- `docs/runbooks/messaging-outbox-backlog-and-stale-lease.md`
|
||
- `docs/runbooks/messaging-delivery-indeterminate-and-duplicate-burst.md`
|
||
- `docs/runbooks/messaging-schema-poison-or-record-too-large.md`
|
||
- `docs/runbooks/messaging-terminal-delivery-disposition.md`
|
||
- `docs/runbooks/messaging-topic-policy-or-partition-change.md`
|
||
- `docs/runbooks/messaging-shutdown-deploy-and-secret-rotation.md`
|
||
- `docs/runbooks/messaging-legacy-to-v2-relay-authority-cutover.md`
|
||
|
||
- [ ] Modify the three listed contract tests first, then run RED:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*RunbookCoverageContractTest' \
|
||
--tests '*OutboxStatusRegistryContractTest' \
|
||
--tests '*EventPayloadPiiContractTest' --console=plain
|
||
```
|
||
|
||
Expected non-zero because new status/key/metric/error/runbook/redaction entries are absent.
|
||
Register owner, type/default/allowlist, secret classification, validation, compatibility
|
||
impact and required test for each implemented key.
|
||
- [ ] Remove canonical reliance on:
|
||
|
||
```text
|
||
APP_MESSAGING_BROKER
|
||
APP_MESSAGING_KAFKA_BROKERS
|
||
ca-skeleton.outbox.relay-enabled
|
||
```
|
||
|
||
in the canonical graph. Retain them only as explicit R0 compatibility inputs through the
|
||
deployment observation window; legacy + canonical keys fail with no silent precedence. Final
|
||
removal belongs to Task 26. Do not add `consumer.enabled`, `cdc.enabled` or
|
||
`schemaRegistry.url`.
|
||
- [ ] Keep base skeleton default DISABLED with active contracts/destinations/resource count 0.
|
||
Deployment-specific destination/topic/security values are explicit placeholders or secret
|
||
references, never usable credentials.
|
||
- [ ] Replace producer `DEAD/dead-letter` vocabulary with `EXHAUSTED`; distinguish it from future
|
||
consumer DLT. Remove raw SQL status rewrite and fabricated consumer-dedupe claims from old
|
||
runbooks.
|
||
- [ ] Each first-R2 runbook contains detection, blast radius, guarantee degradation, safe first
|
||
response, evidence, non-destructive mitigation, destructive approval boundary,
|
||
reconciliation, recovery proof, rollback, audit and related cards/metrics/errors.
|
||
- [ ] The authority-cutover runbook contains the exact pre-commit bounded-forward-retry and
|
||
abort-to-legacy recovery state machine, the irreversible
|
||
`CUTOVER_PENDING -> RECOVERING_LEGACY` claim before external mutation, recovery-only lease
|
||
takeover, the `OUTBOX_PUBLICATION` sole-nonterminal-attempt constraint and global lock order,
|
||
same-/cross-attempt finalizer rejection, external legacy ACL regrant/positive probe, post-ACL
|
||
epoch revalidation and revoke/negative-probe compensation, expected generation ordering,
|
||
partial-reopen re-fence behavior, and the post-commit `resume-polling-v2-writes` path. It
|
||
explicitly forbids any post-commit legacy reactivation or raw admission/epoch SQL.
|
||
- [ ] Document exact state/table/class names, token/generation/audit operator API, role readiness,
|
||
no-dual-authority cutover and non-guarantees. No stub alert/dashboard references count as
|
||
evidence.
|
||
- [ ] Re-run the same three tests GREEN after registries/runbooks are complete, then verify global
|
||
drift gates:
|
||
|
||
```bash
|
||
cd src && ./gradlew verifyEnvKeys \
|
||
verifyPublicPathSnapshot \
|
||
:app-bootstrap:test \
|
||
--tests '*RunbookCoverageContractTest' \
|
||
--tests '*OutboxStatusRegistryContractTest' \
|
||
--tests '*EventPayloadPiiContractTest' \
|
||
--console=plain
|
||
```
|
||
|
||
Expected: all three contract tests PASS, no skip, and env/public-path verification PASS.
|
||
|
||
- [ ] Acceptance claim: documentation/configuration reflects actual implementation and evidence;
|
||
unexecuted lanes remain NOT_QUALIFIED.
|
||
|
||
**Rollback checkpoint:** docs describe deployed/evidenced truth, not preferred state. Never rewrite
|
||
failed evidence or instruct operators to dual-send/raw-update.
|
||
|
||
### Task 24: Freeze and aggregate the pre-cutover release candidate
|
||
|
||
**Owner:** repository-wide verification and documentation
|
||
**Depends on:** every selected Task 1–23 requirement
|
||
|
||
**Files — modify:**
|
||
|
||
- `docs/superpowers/specs/2026-07-28-messaging-production-capability-design.md`
|
||
- `docs/superpowers/plans/2026-07-28-messaging-first-r2-polling-producer.md`
|
||
- `src/config/messaging/readiness-cards.yaml`
|
||
- `src/config/messaging/release-profile-assertions.yaml`
|
||
- `src/build.gradle`
|
||
|
||
**Files — create:**
|
||
|
||
- `src/config/messaging/evidence/deployment-rollout-manifest-v1.schema.json`
|
||
- `src/config/messaging/evidence/deployment-binding-attestation-v1.schema.json`
|
||
- `src/config/messaging/evidence/final-r2-profile-v1.schema.json`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingDeploymentRolloutEvidenceContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingFinalR2ProfileContractTest.java`
|
||
|
||
- [ ] Freeze the candidate source tree and dependency locks. A human supplies the candidate
|
||
commit/source digest; CI builds the exact artifact. Agent never stages, commits or pushes.
|
||
- [ ] Make `verifyMessagingReleaseProfile` consume these exact build outputs:
|
||
|
||
```text
|
||
src/build/messaging-evidence/contracts-schema/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/polling-outbox-r2/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/cutover-rehearsal/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/real-kafka-postgresql-r2/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/security-r2/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/multi-broker-r2/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/rotation-shutdown-r2/manifest.json
|
||
```
|
||
|
||
Each producer validates its schema before writing. The aggregator verifies all required
|
||
scenario IDs, source/artifact digest, card/profile/catalog/schema/settings hashes, cluster/
|
||
qualification-environment identity, freshness, failed=0 and skipped=0, then writes:
|
||
|
||
```text
|
||
src/build/reports/messaging/release-profile/manifest.json
|
||
```
|
||
|
||
CI retains the exact bytes under a source-digest-qualified
|
||
`ci-artifact://messaging/` release-profile path.
|
||
- [ ] Revalidate every one of the seven inputs against the common build-evidence schema and its
|
||
lane-specific schema. Add contract fixtures proving a lane schema cannot omit or relax common
|
||
source/artifact/scenario/failure/skip fields.
|
||
- [ ] Implement fail-closed deployment/final gate contracts before any target cutover:
|
||
`verifyMessagingTargetBindingPreflight` validates a non-mutating target preflight;
|
||
`verifyMessagingTargetBinding` consumes the fresh target-specific topology/security/ACL
|
||
attestation created inside maintenance after the legacy Write fence;
|
||
`verifyMessagingDeploymentCutover` consumes that immutable original attestation plus the exact
|
||
local target rollout manifest; `verifyMessagingCleanupTargetBinding` consumes a distinct
|
||
cleanup-artifact attestation; `verifyMessagingFinalR2Profile` requires both attestations, the
|
||
qualified cleanup release manifest, original deployment-cutover manifest and cleanup-rollout
|
||
manifest, validates the final-profile schema and writes the final aggregate. Missing/stale/
|
||
wrong-target/wrong-digest/failed/skipped evidence is non-zero.
|
||
- [ ] Run the gate contract tests RED then GREEN with checked-in invalid/valid payload-free fixtures:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*MessagingDeploymentRolloutEvidenceContractTest' \
|
||
--tests '*MessagingFinalR2ProfileContractTest' --console=plain
|
||
```
|
||
|
||
RED is an intentionally invalid fixture accepted or a missing required validator; GREEN means
|
||
every invalid fixture is rejected and every exact valid fixture is accepted. This does not
|
||
create target rollout evidence.
|
||
- [ ] Run focused owner gates sequentially:
|
||
|
||
```bash
|
||
cd src && ./gradlew :application-core:check \
|
||
:shared-contract:check \
|
||
:adapter:outbound:messaging:check \
|
||
:adapter:outbound:persistence-jpa:check \
|
||
:adapter:inbound:web:check \
|
||
:app-bootstrap:check \
|
||
:sample-portfolio:check \
|
||
--console=plain
|
||
```
|
||
|
||
- [ ] Run Messaging gates:
|
||
|
||
```bash
|
||
cd src && ./gradlew verifyMessagingContracts \
|
||
verifyMessagingJsonSchemaV1 \
|
||
verifyMessagingPollingOutboxR2 \
|
||
verifyMessagingKafkaProducerR2 \
|
||
verifyMessagingSecurityR2 \
|
||
verifyMessagingReleaseProfile \
|
||
--console=plain
|
||
```
|
||
|
||
Expected: every exact input manifest exists and validates; no mismatch/stale/skip; aggregate
|
||
manifest PASS. Missing service/credential/image/test is non-zero, never PASS.
|
||
- [ ] Run repository gates:
|
||
|
||
```bash
|
||
cd src && ./gradlew test --console=plain
|
||
cd src && ./gradlew check --console=plain
|
||
cd src && ./gradlew verifyDependencyLocks --console=plain
|
||
cd src && ./gradlew verifyCleanArchitectureDependencies --console=plain
|
||
cd src && ./gradlew verifyPublicPathSnapshot --console=plain
|
||
cd src && ./gradlew verifyEnvKeys --console=plain
|
||
cd src && ./gradlew verifyOneTypePerFile \
|
||
verifyApplicationCoreDependencyPurity --console=plain
|
||
git diff --check
|
||
```
|
||
|
||
- [ ] Perform independent reviews for:
|
||
Clean Architecture/module ownership; schema/contract evolution; transaction/concurrency/CAS;
|
||
Kafka outcome/lifecycle; security/topology; operator endpoint; migration/cutover/rollback;
|
||
evidence/no-skip/operations. Candidate qualification requires blocker 0 and high 0.
|
||
- [ ] From the aggregate only, promote the exact artifact/card rows to `release-eligible`. Record
|
||
§0 as `P4=RELEASE_CANDIDATE_QUALIFIED_DEPLOYMENT_NOT_CUT_OVER`; target publication authority
|
||
and production runtime remain legacy. P5/P6 stay `DESIGNED_NOT_IMPLEMENTED`, P7 stays
|
||
`OPTIONAL_BACKLOG`.
|
||
- [ ] Update the branch-note with release-candidate evidence and explicitly state target
|
||
cutover/cleanup are pending. Do not make the final implementation-complete claim.
|
||
|
||
**Rollback checkpoint:** if aggregation fails, keep cards `implemented-candidate`, target
|
||
`LEGACY_POLLING`, and preserve schema/backlog/evidence. Never weaken a gate or copy evidence from
|
||
another artifact.
|
||
|
||
### Task 25: Execute the approved target deployment cutover
|
||
|
||
**Owner:** deployment coordinator + application/persistence cutover protocol
|
||
**Depends on:** Tasks 23–24, human deployment approval, exact fresh/empty-drained migration card
|
||
**Source change:** none
|
||
|
||
- [ ] Fail before maintenance unless the target matches the exact Task 24 source/artifact digest,
|
||
release profile, schema/catalog/settings hashes, supported broker/client constraints and the
|
||
checked-in cutover runbook. Do not require the target cluster/principal to equal Task 22's
|
||
qualification fixture. A live non-empty V3 target stops for a separate approved deployment
|
||
migration plan.
|
||
- [ ] Against the actual target, resolve the exact canonical secret generation and run a
|
||
non-mutating fresh topology/security preflight. Do not revoke the still-authoritative legacy
|
||
principal before maintenance. Bind the prepared exact ACL change/provenance and emit identical
|
||
sanitized bytes:
|
||
|
||
```text
|
||
src/app-bootstrap/build/messaging-evidence/target-binding-preflight/manifest.json
|
||
ci-artifact://messaging/{targetAlias}/{sourceDigest}/target-binding-preflight/manifest.json
|
||
```
|
||
|
||
`deploymentBindingIdentity` binds target cluster/topic/canonical and legacy principal
|
||
identities, canonical secret generation, provisioning provenance and prepared mutation. It
|
||
proves canonical Describe/DescribeConfigs/Write and validates the target constraints, but
|
||
explicitly records `legacyWriteRevoked=false`; it is not cutover evidence.
|
||
- [ ] Run before maintenance:
|
||
|
||
```bash
|
||
cd src && ./gradlew verifyMessagingTargetBindingPreflight --console=plain
|
||
```
|
||
|
||
Expected GREEN only for a fresh exact target/source/artifact/release/profile binding with
|
||
failed=0 and skipped=0. Fixture identity cannot satisfy this gate.
|
||
- [ ] Before target execution, run the fail-closed rollout gate once with no current rollout
|
||
artifact:
|
||
|
||
```bash
|
||
cd src && ./gradlew verifyMessagingDeploymentCutover --console=plain
|
||
```
|
||
|
||
Expected non-zero: the final in-maintenance target attestation and target rollout artifact are
|
||
absent. A stale prior-target artifact must fail for target/source/release-digest mismatch, not
|
||
satisfy this RED.
|
||
- [ ] Execute the runbook preflight:
|
||
deploy candidate with v2 relay scheduler-disabled; attest exact tuple; prove disposition
|
||
auth/CAS; freeze durable business-write admission; drain writers/epoch holders; stop/drain
|
||
legacy claim and reaper; record remaining IN_FLIGHT as indeterminate/HOLD; fence legacy
|
||
producer Write and DB mutation. Only after zero-active drain, apply the prepared ACL mutation,
|
||
require legacy exact-topic Write negative and canonical Describe/DescribeConfigs/Write
|
||
positive, then write and internally schema-validate:
|
||
|
||
```text
|
||
src/app-bootstrap/build/messaging-evidence/target-binding-attestation/manifest.json
|
||
ci-artifact://messaging/{targetAlias}/{sourceDigest}/target-binding-attestation/manifest.json
|
||
```
|
||
|
||
The one-shot precondition evidence binds this attestation digest. The maintenance runner uses
|
||
the same fail-closed validator as `verifyMessagingTargetBinding` before it may call finalization;
|
||
raw server, credential and certificate bytes are excluded.
|
||
- [ ] Invoke the exact non-web `MessagingAuthorityCutoverApplicationRunner` operation
|
||
`legacy-to-polling-v2` with opaque operation/approval-evidence IDs and expected
|
||
target/source/artifact/epoch. It calls the coordinator and
|
||
`FinalizeOutboxAuthorityCutoverUseCase`; the single transaction repeats the Task 20
|
||
`CUTOVER_PENDING -> FINALIZING_V2` CAS,
|
||
watermark/final-delta/manifest/current-delivery assertions, epoch switch, sentinel insert and
|
||
`CONSUMED_V2` transition. `RECOVERING_LEGACY` is rejected before reconciliation. Any mismatch
|
||
exits non-zero and rolls back to legacy authority. Reusing the operation or consumed evidence
|
||
ID exits non-zero without mutation.
|
||
|
||
```text
|
||
--spring.main.web-application-type=none
|
||
--ca-skeleton.messaging.maintenance.operation=legacy-to-polling-v2
|
||
--ca-skeleton.messaging.maintenance.operation-id={opaqueOperationId}
|
||
--ca-skeleton.messaging.maintenance.approval-evidence-id={opaqueApprovalEvidenceId}
|
||
--ca-skeleton.messaging.maintenance.expected-target-alias={targetAlias}
|
||
--ca-skeleton.messaging.maintenance.expected-source-digest={sourceDigest}
|
||
--ca-skeleton.messaging.maintenance.expected-artifact-digest={artifactDigest}
|
||
--ca-skeleton.messaging.maintenance.expected-epoch={legacyEpoch}
|
||
--ca-skeleton.messaging.maintenance.expected-fence-generation={openFenceGeneration}
|
||
```
|
||
- [ ] If finalization fails before epoch commit, keep every fence closed. Either retry forward within
|
||
the still-fresh bounded evidence/approval window, or execute the approved abort-to-legacy
|
||
protocol. The latter first proves epoch still `LEGACY_POLLING`, v2 business send/sentinel
|
||
authority 0 and exact inventory, then runs the prepare half of
|
||
`recover-legacy-precommit`: DB-CAS the exact attempt
|
||
`CUTOVER_PENDING -> RECOVERING_LEGACY`, bind the recovery operation/lease/evidence digest and
|
||
atomically make every forward finalizer reject it. This choice is irreversible for that
|
||
attempt, including after lease expiry. The prepare transaction holds the global
|
||
write-admission/ACTIVE-epoch locks in order and proves the partial-unique-protected attempt is
|
||
the sole nonterminal `OUTBOX_PUBLICATION` attempt. Only then may external provisioning
|
||
regrant legacy exact-topic Write and emit a fresh positive probe. The completion half
|
||
re-locks the global authority, exact attempt and epoch after that external mutation;
|
||
owner/lease, uniqueness or `LEGACY_POLLING` mismatch immediately re-revokes legacy Write,
|
||
proves a fresh negative probe and leaves writes FROZEN.
|
||
On success it appends recovery/ACL audit, reopens the in-process legacy Write fence/reaper/
|
||
relay, then CAS-opens durable writes and marks `RECOVERED_LEGACY` last. Emit:
|
||
|
||
```text
|
||
src/app-bootstrap/build/messaging-evidence/precommit-legacy-recovery/manifest.json
|
||
ci-artifact://messaging/{targetAlias}/{sourceDigest}/precommit-legacy-recovery/manifest.json
|
||
```
|
||
|
||
Validate identical bytes against the common and deployment-rollout schemas with
|
||
`outcome=ABORTED_PRECOMMIT`, exact restored legacy/write-admission generations, recovery
|
||
scenario IDs exactly once, failed=0 and skipped=0.
|
||
Any partial failure leaves writes FROZEN and is retried/re-fenced; recovery lease takeover is
|
||
recovery-only and never restores forward-finalization eligibility. Race tests must run
|
||
recovery-prepare versus same- and different-attempt creation/finalization in both lock orders,
|
||
and crash tests must cover every boundary before/after claim, ACL regrant, epoch revalidation,
|
||
component reopen and final admission CAS. The protocol never uses raw SQL. An aborted attempt
|
||
ends Task 25 without cutover; another attempt is rejected until recovery atomically records
|
||
`RECOVERED_LEGACY` and opens a new fence generation, then requires fresh preflight, approval
|
||
and target attestation.
|
||
- [ ] After epoch commit, start only v2 relay while writes remain FROZEN. Require the canonical
|
||
sentinel `DELIVERY_RECORDED`, retained V3 projection, fresh target binding/readiness and legacy
|
||
writer/claim/reaper/send 0. Then `ResumePollingV2WriteAdmissionUseCase` CAS-opens
|
||
`OPEN(generation+1)`. If the original runner dies or resume fails after commit,
|
||
`resume-polling-v2-writes` is the only recovery operation; it rechecks the same facts and
|
||
cannot reactivate legacy. After OPEN, exercise an ordinary canonical append; failure freezes a
|
||
new generation and keeps the rollout non-healthy.
|
||
- [ ] Apply the rehearsed rollback zones exactly:
|
||
pre-commit failure retains legacy DB authority but stays in maintenance until bounded forward
|
||
retry or the full audited recovery above succeeds; after epoch commit, reverse epoch and legacy
|
||
reactivation are unsupported even before the first business v2 send. Keep admission FROZEN,
|
||
preserve backlog/schema/epoch/audit and forward-fix or run the guarded v2 resume.
|
||
- [ ] Abort and fence admission on any duplicate anomaly, unexpected indeterminate, backlog/SLO
|
||
breach, late-observation drop, stale legacy mutation, topic/security drift or readiness loss.
|
||
Never automatically resend while diagnosing.
|
||
- [ ] Emit a sanitized target rollout artifact:
|
||
|
||
```text
|
||
src/app-bootstrap/build/messaging-evidence/deployment-cutover/manifest.json
|
||
ci-artifact://messaging/{targetAlias}/{sourceDigest}/deployment-cutover/manifest.json
|
||
```
|
||
|
||
It binds target environment/cluster alias, source/artifact/release manifest digest,
|
||
target-binding-attestation digest, precondition evidence digest, watermark/manifest counts,
|
||
epoch/sentinel facts, final OPEN fence generation, ordinary post-resume append probe,
|
||
commands/timestamps, rollback zone, failures/skips and operator approvals. Payload, event hash,
|
||
credential and raw IDs remain excluded. The target runner writes identical bytes to the local
|
||
handoff and retained CI URI; both validate against the common build-evidence schema and
|
||
`deployment-rollout-manifest-v1.schema.json`.
|
||
- [ ] After commit, sentinel proof and artifact handoff, run
|
||
`verifyMessagingTargetBinding verifyMessagingDeploymentCutover` GREEN. Expected: binding and
|
||
deployment schemas PASS, exact target/source/artifact/release/attestation digest match,
|
||
approval/epoch/sentinel scenario IDs present exactly once, failed=0 and skipped=0.
|
||
- [ ] Acceptance claim: the exact target uses `POLLING_V2` and the sentinel/reference path is
|
||
healthy, and durable write admission is OPEN at the recorded post-cutover generation. Legacy
|
||
source/runtime cleanup is still pending the observation/rollback window.
|
||
|
||
**Rollback checkpoint:** use only the two rehearsed zones. Schema is forward-only; dual relay
|
||
authority and destructive downgrade are forbidden.
|
||
|
||
### Task 26: Complete observation, remove legacy runtime and requalify the cleanup artifact
|
||
|
||
**Owner:** all affected leaves + repository-wide verification/Wiki
|
||
**Depends on:** Task 25, reviewed observation window, human cleanup approval
|
||
|
||
**Files — delete:**
|
||
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/core/MessageBroker.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaSender.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaMessageBroker.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaAdapterConfig.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/kafka/KafkaAdapterSettings.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/DisabledOutboxMessagePublisher.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFence.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxEnvelopeJson.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapter.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/Slf4jOutboxRelayFailureReportAdapter.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfigTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/LegacyPublicationWriteFenceTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java`
|
||
- `src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/Slf4jOutboxRelayFailureReportAdapterTest.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxMessagePublishPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelayControlPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxRelaySnapshot.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionEvidence.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxCutoverPreconditionPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverCommand.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverResult.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxAuthorityCutoverPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCase.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxPreCommitRecoveryPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityCommand.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityResult.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCase.java`
|
||
- `src/application-core/src/test/java/dev/caskeleton/application/outbox/FinalizeOutboxAuthorityCutoverUseCaseTest.java`
|
||
- `src/application-core/src/test/java/dev/caskeleton/application/outbox/RecoverLegacyOutboxAuthorityUseCaseTest.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxStorePort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxEvent.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxEventStatus.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayFailureReport.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayFailureReportPort.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayResult.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsCommand.java`
|
||
- `src/application-core/src/main/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCase.java`
|
||
- `src/application-core/src/test/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCaseTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxCutoverPreconditionAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapter.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxAuthorityCutoverAdapterTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxPreCommitRecoveryAdapterTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxClaimRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlOutboxClaimRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaper.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxStoreAdapterTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaperTest.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxReaperWiringTest.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLeaderElectionToken.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinator.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinator.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapter.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverJobSettings.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunner.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyToV2CutoverCoordinatorTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxLegacyPreCommitRecoveryCoordinatorTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/LegacyOutboxRelayControlAdapterTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/MessagingAuthorityCutoverApplicationRunnerTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxLegacyToV2CutoverContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxAppendTransactionalContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxPublisherLeaderElectionContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxRowLifecycleContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/support/conditional/EnabledIfMessagingBrokerConfigured.java`
|
||
|
||
**Files — create:**
|
||
|
||
- `src/config/messaging/legacy-runtime-denylist.txt`
|
||
- `src/config/messaging/legacy-runtime-allowlist.txt`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingLegacyRuntimeDenylistTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/messaging/MessagingLegacyActivationRunbookContractTest.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingWriteAdmissionRecoverySettings.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/MessagingWriteAdmissionRecoveryApplicationRunner.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/MessagingWriteAdmissionRecoveryApplicationRunnerTest.java`
|
||
|
||
**Files — modify:**
|
||
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/OutboxEventJpaRepository.java`
|
||
- `src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/postgresql/PostgreSqlPersistenceConfig.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfig.java`
|
||
- `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingSettings.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxMetrics.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxRelayScheduler.java`
|
||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfig.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxConfigTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxSettingsTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/messaging/MessagingCapabilityConfigTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/OutboxV2SentinelContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/adapter/outbound/OptionalAdapterBeanGatingTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/ArchitectureViolationFixtureTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/violations/application/OutboundWithoutPermissionUseCase.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/OptionalAdapterConditionalExecutionContractTest.java`
|
||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/contract/outbox/OutboxStatusRegistryContractTest.java`
|
||
- `src/app-bootstrap/src/main/resources/application.yml`
|
||
- `src/app-bootstrap/src/test/resources/application-test.yml`
|
||
- `src/sample-portfolio/src/main/resources/application.yml`
|
||
- `src/sample-portfolio/src/test/resources/application-test.yml`
|
||
- `src/.env`
|
||
- `src/config/messaging/readiness-cards.yaml`
|
||
- `src/config/messaging/profile-compatibility.yaml`
|
||
- `src/config/messaging/release-profile-assertions.yaml`
|
||
- `docs/registries/env-keys.yaml`
|
||
- `docs/registries/capabilities.yaml`
|
||
- `docs/registries/error-codes.yaml`
|
||
- `docs/registries/metrics.yaml`
|
||
- `docs/registries/secrets-classification.yaml`
|
||
- `docs/runbooks/outbox-publish-failed.md`
|
||
- `docs/runbooks/outbox-dead-letter.md`
|
||
- `docs/runbooks/messaging-producer-unavailable-or-unauthorized.md`
|
||
- `docs/runbooks/messaging-outbox-backlog-and-stale-lease.md`
|
||
- `docs/runbooks/messaging-delivery-indeterminate-and-duplicate-burst.md`
|
||
- `docs/runbooks/messaging-schema-poison-or-record-too-large.md`
|
||
- `docs/runbooks/messaging-terminal-delivery-disposition.md`
|
||
- `docs/runbooks/messaging-topic-policy-or-partition-change.md`
|
||
- `docs/runbooks/messaging-shutdown-deploy-and-secret-rotation.md`
|
||
- `docs/runbooks/messaging-legacy-to-v2-relay-authority-cutover.md`
|
||
- `src/README.md`
|
||
- `src/application-core/README.md`
|
||
- `src/application-core/CLAUDE.md`
|
||
- `src/shared-contract/README.md`
|
||
- `src/shared-contract/CLAUDE.md`
|
||
- `src/adapter/outbound/messaging/README.md`
|
||
- `src/adapter/outbound/messaging/CLAUDE.md`
|
||
- `src/adapter/outbound/persistence-jpa/README.md`
|
||
- `src/adapter/outbound/persistence-jpa/CLAUDE.md`
|
||
- `src/adapter/inbound/web/README.md`
|
||
- `src/adapter/inbound/web/CLAUDE.md`
|
||
- `src/app-bootstrap/README.md`
|
||
- `src/app-bootstrap/CLAUDE.md`
|
||
- `src/sample-portfolio/README.md`
|
||
- `src/sample-portfolio/CLAUDE.md`
|
||
- `src/app-bootstrap/build.gradle`
|
||
- `src/build.gradle`
|
||
- `docs/superpowers/specs/2026-07-28-messaging-production-capability-design.md`
|
||
- `docs/superpowers/plans/2026-07-28-messaging-first-r2-polling-producer.md`
|
||
- `/home/donghyeon/workspace/ai-tool/llm-wiki-private/raw/branch-notes/main.md` and only genuinely
|
||
derived raw documents
|
||
|
||
- [ ] Observation exit requires the reviewed duration with duplicate anomaly 0, unexpected
|
||
indeterminate 0, late-drop 0, stable backlog/readiness, no legacy mutation and successful
|
||
disposition/rotation/shutdown drills. Durable write admission must remain OPEN at the recorded
|
||
post-cutover generation except for audited drills with successful guarded resume. Any breach
|
||
postpones cleanup.
|
||
- [ ] Write `MessagingLegacyRuntimeDenylistTest` first and run RED; it must find every exact
|
||
production symbol/config key above plus legacy reaper repository methods/beans. The denylist
|
||
contains exact FQCNs and keys, including:
|
||
|
||
```text
|
||
dev.caskeleton.adapter.outbound.messaging.core.MessageBroker
|
||
dev.caskeleton.adapter.outbound.messaging.kafka.KafkaSender
|
||
dev.caskeleton.adapter.outbound.messaging.kafka.KafkaMessageBroker
|
||
dev.caskeleton.adapter.outbound.messaging.kafka.KafkaAdapterConfig
|
||
dev.caskeleton.adapter.outbound.messaging.kafka.KafkaAdapterSettings
|
||
dev.caskeleton.adapter.outbound.messaging.outbox.DisabledOutboxMessagePublisher
|
||
dev.caskeleton.adapter.outbound.messaging.outbox.LegacyPublicationWriteFence
|
||
dev.caskeleton.adapter.outbound.messaging.outbox.OutboxEnvelopeJson
|
||
dev.caskeleton.adapter.outbound.messaging.outbox.OutboxMessagePublishAdapter
|
||
dev.caskeleton.adapter.outbound.messaging.outbox.Slf4jOutboxRelayFailureReportAdapter
|
||
dev.caskeleton.application.outbox.OutboxMessagePublishPort
|
||
dev.caskeleton.application.outbox.LegacyOutboxRelayControlPort
|
||
dev.caskeleton.application.outbox.LegacyOutboxRelaySnapshot
|
||
dev.caskeleton.application.outbox.OutboxCutoverPreconditionEvidence
|
||
dev.caskeleton.application.outbox.OutboxCutoverPreconditionPort
|
||
dev.caskeleton.application.outbox.FinalizeOutboxAuthorityCutoverCommand
|
||
dev.caskeleton.application.outbox.FinalizeOutboxAuthorityCutoverResult
|
||
dev.caskeleton.application.outbox.OutboxAuthorityCutoverPort
|
||
dev.caskeleton.application.outbox.FinalizeOutboxAuthorityCutoverUseCase
|
||
dev.caskeleton.application.outbox.OutboxPreCommitRecoveryPort
|
||
dev.caskeleton.application.outbox.RecoverLegacyOutboxAuthorityCommand
|
||
dev.caskeleton.application.outbox.RecoverLegacyOutboxAuthorityResult
|
||
dev.caskeleton.application.outbox.RecoverLegacyOutboxAuthorityUseCase
|
||
dev.caskeleton.application.outbox.OutboxStorePort
|
||
dev.caskeleton.application.outbox.OutboxEvent
|
||
dev.caskeleton.application.outbox.OutboxEventStatus
|
||
dev.caskeleton.application.outbox.OutboxRelayFailureReport
|
||
dev.caskeleton.application.outbox.OutboxRelayFailureReportPort
|
||
dev.caskeleton.application.outbox.OutboxRelayResult
|
||
dev.caskeleton.application.outbox.PublishPendingOutboxEventsCommand
|
||
dev.caskeleton.application.outbox.PublishPendingOutboxEventsUseCase
|
||
dev.caskeleton.adapter.outbound.persistence.outbox.OutboxStoreAdapter
|
||
dev.caskeleton.adapter.outbound.persistence.outbox.OutboxCutoverPreconditionAdapter
|
||
dev.caskeleton.adapter.outbound.persistence.outbox.OutboxAuthorityCutoverAdapter
|
||
dev.caskeleton.adapter.outbound.persistence.outbox.OutboxPreCommitRecoveryAdapter
|
||
dev.caskeleton.adapter.outbound.persistence.outbox.OutboxClaimRepository
|
||
dev.caskeleton.adapter.outbound.persistence.postgresql.PostgreSqlOutboxClaimRepository
|
||
dev.caskeleton.adapter.outbound.persistence.outbox.OutboxReaper
|
||
dev.caskeleton.bootstrap.outbox.OutboxLeaderElectionToken
|
||
dev.caskeleton.bootstrap.outbox.OutboxLegacyToV2CutoverCoordinator
|
||
dev.caskeleton.bootstrap.outbox.OutboxLegacyPreCommitRecoveryCoordinator
|
||
dev.caskeleton.bootstrap.outbox.LegacyOutboxRelayControlAdapter
|
||
dev.caskeleton.bootstrap.outbox.MessagingAuthorityCutoverJobSettings
|
||
dev.caskeleton.bootstrap.outbox.MessagingAuthorityCutoverApplicationRunner
|
||
OutboxEventJpaRepository.deletePublishedBefore
|
||
OutboxEventJpaRepository.countGroupedByStatus
|
||
OutboxEventJpaRepository.findOldestUnpublishedOccurredAtByEventType
|
||
OutboxClaimRepository.claimEligible
|
||
outboxLeaderElection
|
||
outboxReaper
|
||
app.messaging.broker
|
||
app.messaging.kafka.brokers
|
||
APP_MESSAGING_BROKER
|
||
APP_MESSAGING_KAFKA_BROKERS
|
||
ca-skeleton.outbox.relay-enabled
|
||
ca-skeleton.messaging.maintenance.operation
|
||
ca-skeleton.messaging.maintenance.operation-id
|
||
ca-skeleton.messaging.maintenance.approval-evidence-id
|
||
ca-skeleton.messaging.maintenance.expected-target-alias
|
||
ca-skeleton.messaging.maintenance.expected-source-digest
|
||
ca-skeleton.messaging.maintenance.expected-artifact-digest
|
||
ca-skeleton.messaging.maintenance.expected-epoch
|
||
ca-skeleton.messaging.maintenance.expected-fence-generation
|
||
```
|
||
|
||
This is the complete mandatory production token/key set derived one-to-one from `Files —
|
||
delete` plus the legacy repository methods, bean names and configuration keys. The contract
|
||
test snapshots the exact eight-key Task 20 maintenance settings/registry set and asserts set
|
||
equality with these eight denylist keys before scanning source/config/runbook references. It
|
||
rejects a missing or extra maintenance key, a missing denylist entry, an unclassified deleted
|
||
production class, and any allowlist entry outside the exact sample/R0 list below.
|
||
|
||
The allowlist contains only these repository-relative paths:
|
||
|
||
```text
|
||
src/application-core/src/main/java/dev/caskeleton/application/outbox/NewOutboxEvent.java
|
||
src/application-core/src/main/java/dev/caskeleton/application/outbox/LegacyOutboxAppendPort.java
|
||
src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapter.java
|
||
src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/outbox/LegacyOutboxAppendAdapterTest.java
|
||
src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisher.java
|
||
src/sample-portfolio/src/main/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogUseCase.java
|
||
src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/event/PosterEventPublisherTest.java
|
||
src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/CreateWorkLogOutboxTest.java
|
||
src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/application/worklog/WorkLogUseCasesTest.java
|
||
src/sample-portfolio/src/test/java/dev/caskeleton/sample/portfolio/authz/WorkLogAuthorizationContractTest.java
|
||
```
|
||
|
||
No directory wildcard or silently ignored unknown path is allowed.
|
||
- [ ] Run the denylist RED before deletion:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*MessagingLegacyRuntimeDenylistTest' --console=plain
|
||
```
|
||
|
||
Expected non-zero with every still-present forbidden FQCN/key/bean reported. Missing scan
|
||
roots or a test skip is not an accepted RED.
|
||
- [ ] Delete the listed runtime/relay/reaper sources and obsolete config keys. Retain
|
||
`NewOutboxEvent`, `LegacyOutboxAppendPort` and a separately named sample-only
|
||
`LegacyOutboxAppendAdapter` only as the documented R0 fixture until standalone sample
|
||
activation. Canonical ACTIVE context must prove legacy append/store/publish/relay/reaper bean
|
||
count 0. Replace the deleted cutover runner with the disabled-by-default
|
||
`MessagingWriteAdmissionRecoveryApplicationRunner`, which accepts only
|
||
`resume-polling-v2-writes`, imports no legacy/cutover type, and uses
|
||
`ResumePollingV2WriteAdmissionUseCase` with expected FROZEN generation, ACTIVE
|
||
`POLLING_V2` epoch and fresh sentinel/readiness evidence.
|
||
|
||
```text
|
||
ca-skeleton.messaging.write-admission-recovery.operation
|
||
ca-skeleton.messaging.write-admission-recovery.operation-id
|
||
ca-skeleton.messaging.write-admission-recovery.approval-evidence-id
|
||
ca-skeleton.messaging.write-admission-recovery.expected-target-alias
|
||
ca-skeleton.messaging.write-admission-recovery.expected-epoch
|
||
ca-skeleton.messaging.write-admission-recovery.expected-fence-generation
|
||
```
|
||
|
||
No operation/default means no runner resource. Unknown or legacy operation names fail before
|
||
mutation; replay/mismatch and resume-before-sentinel remain non-zero.
|
||
- [ ] Keep retained V3 DB columns and canonical compatibility projection until a later forward
|
||
schema cleanup. Do not drop columns or rewrite history here.
|
||
- [ ] Re-run the runtime denylist GREEN across production Java, build files, YAML/env and registries;
|
||
only the exact sample/R0 source allowlist may remain. Explicitly exclude design/spec/plan/Wiki
|
||
history from raw-symbol matching: historical documentation is evidence, not an activation
|
||
surface. Compile success alone is not bean/resource absence evidence. Run the same focused
|
||
command GREEN; expect forbidden runtime match 0, canonical legacy bean/resource count 0 and
|
||
every allowlisted sample/R0 reference classified exactly.
|
||
- [ ] Run `MessagingLegacyActivationRunbookContractTest` RED before cleanup and GREEN after cleanup.
|
||
It scans operational runbooks semantically for executable legacy activation keys, commands,
|
||
dual-relay instructions or rollback-to-legacy actions, while allowing clearly labelled
|
||
historical facts and “must remain disabled/forbidden” statements. It does not raw-match this
|
||
plan/spec/branch-note.
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*MessagingLegacyActivationRunbookContractTest' --console=plain
|
||
```
|
||
- [ ] Because cleanup changes source/artifact digest, freeze a new human candidate and rerun the
|
||
Task 21 and 22 qualification lanes plus the Task 24 focused/Messaging/repository commands and
|
||
release-profile aggregator against the cleanup artifact. Do not replay Task 24's pre-cutover
|
||
status wording: production is already `POLLING_V2`. Pre-cleanup evidence is stale and cannot
|
||
qualify the cleanup artifact.
|
||
- [ ] Before deploying the cleanup artifact, run:
|
||
|
||
```bash
|
||
cd src && ./gradlew verifyMessagingFinalR2Profile --console=plain
|
||
```
|
||
|
||
Expected non-zero because the qualified cleanup release manifest and cleanup rollout artifact
|
||
do not yet form a matching final chain.
|
||
- [ ] Deploy the requalified cleanup artifact without changing the already-active `POLLING_V2`
|
||
epoch. Before deployment, rerun target topology/security/ACL probes for the cleanup source/
|
||
artifact and emit:
|
||
|
||
```text
|
||
src/app-bootstrap/build/messaging-evidence/cleanup-target-binding-attestation/manifest.json
|
||
ci-artifact://messaging/{targetAlias}/{cleanupSourceDigest}/cleanup-target-binding-attestation/manifest.json
|
||
```
|
||
|
||
Run `verifyMessagingCleanupTargetBinding` GREEN; it must bind the same target identity and
|
||
current secret generation to the cleanup release digest without overwriting the immutable
|
||
original Task 25 attestation. Then deploy and emit byte-identical sanitized evidence:
|
||
|
||
```text
|
||
src/app-bootstrap/build/messaging-evidence/cleanup-rollout/manifest.json
|
||
ci-artifact://messaging/{targetAlias}/{cleanupSourceDigest}/cleanup-rollout/manifest.json
|
||
```
|
||
|
||
It binds the target alias, cleanup source/artifact digest, qualified cleanup release-manifest
|
||
digest, cleanup-target-attestation digest, original deployment-cutover manifest digest,
|
||
unchanged epoch, recorded OPEN write-admission generation, sentinel/backlog/readiness/
|
||
legacy-bean-0 facts, approval,
|
||
commands/timestamps, failed=0 and skipped=0. Validate it against the common and
|
||
deployment-rollout schemas.
|
||
- [ ] Re-run `verifyMessagingDeploymentCutover`, `verifyMessagingCleanupTargetBinding` and
|
||
`verifyMessagingFinalR2Profile` GREEN:
|
||
|
||
```bash
|
||
cd src && ./gradlew verifyMessagingTargetBinding \
|
||
verifyMessagingDeploymentCutover \
|
||
verifyMessagingCleanupTargetBinding \
|
||
verifyMessagingFinalR2Profile --console=plain
|
||
```
|
||
|
||
The final task consumes these exact local inputs:
|
||
|
||
```text
|
||
src/build/reports/messaging/release-profile/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/target-binding-attestation/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/deployment-cutover/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/cleanup-target-binding-attestation/manifest.json
|
||
src/app-bootstrap/build/messaging-evidence/cleanup-rollout/manifest.json
|
||
```
|
||
|
||
If qualification/build cleanup removed a Task 25 local handoff, restore only the byte-identical
|
||
retained CI artifact to its exact path after verifying its recorded digest/signature and target
|
||
provenance. Never synthesize, edit or substitute a current artifact for the immutable original.
|
||
The final task writes:
|
||
|
||
```text
|
||
src/build/reports/messaging/final-r2-profile/manifest.json
|
||
```
|
||
|
||
Expected: exact target/cleanup source/artifact/release digest chain, approval/epoch/sentinel/
|
||
write-admission-OPEN/legacy-zero scenario IDs exactly once, failed=0, skipped=0 and schema
|
||
PASS. CI retains the
|
||
final bytes under the target/cleanup-source-qualified namespace.
|
||
- [ ] Run final focused, Messaging and repository gates exactly as Task 24 plus:
|
||
|
||
```bash
|
||
cd src && ./gradlew :app-bootstrap:test \
|
||
--tests '*MessagingLegacyRuntimeDenylistTest' \
|
||
--tests '*MessagingWriteAdmissionRecoveryApplicationRunnerTest' \
|
||
--tests '*MessagingDisabledZeroResourceContractTest' --console=plain
|
||
```
|
||
|
||
- [ ] Update §0 and card status from the final R2 aggregate only. First R2 may be marked complete
|
||
only after `verifyMessagingFinalR2Profile` passes; P5/P6/P7 remain unchanged.
|
||
- [ ] Before final Wiki capture, read the configured vault's `AGENTS.md`, `CLAUDE.md` and relevant
|
||
`rules/`, `.agents/`, `.claude/`, `.codex/` instructions. Resolve the branch with
|
||
`git branch --show-current`; for this plan's current `main` branch the canonical target is:
|
||
|
||
```text
|
||
/home/donghyeon/workspace/ai-tool/llm-wiki-private/raw/branch-notes/main.md
|
||
```
|
||
|
||
Record implementation, files, decisions, exact commands/results/failures, release and rollout
|
||
evidence, unsupported claims and remaining risks. Add/link derived raw documents only when
|
||
honestly produced; otherwise record “없음”. Canonical vault unavailability is an explicit
|
||
completion blocker.
|
||
- [ ] Final handoff lists changed files, behavior, exact verification counts/results, failed/not-run
|
||
lanes, release/rollout fingerprints, Wiki capture and follow-up risks.
|
||
|
||
**Rollback checkpoint:** after cleanup, do not restore legacy code for events already sent by v2.
|
||
Pause admission, preserve schema/backlog/epoch/audit and forward-fix.
|
||
|
||
---
|
||
|
||
## 8. Task dependency graph
|
||
|
||
```text
|
||
1 -> 2
|
||
|
|
||
3 -> 4 -> 5 -> 6
|
||
|
|
||
7 -> 8 -> 9 -> 10 -> 11 -> 12 -> 13
|
||
|
|
||
14 -> 15 -> 16 -> 17
|
||
| |
|
||
+-------> 18
|
||
|
|
||
19 -> 20
|
||
|
|
||
21 -> 22 -> 23 -> 24
|
||
|
|
||
25 -> 26
|
||
```
|
||
|
||
Parallel work is allowed only at non-overlapping stable boundaries:
|
||
|
||
- Task 4 shared/sample resource work may run in parallel after Task 3, but Task 5 starts only after
|
||
both resource owners are stable.
|
||
- Persistence Tasks 7–13 are sequential because they share migration/entity/repository/CAS surfaces.
|
||
- Task 18 may start after Task 12 while Tasks 15–17 proceed, but Task 19 waits for both.
|
||
- Disposable cutover rehearsal Task 20 is never parallelized with producer, persistence or
|
||
configuration changes.
|
||
- Real Kafka Task 21 and security topology setup for Task 22 may prepare in parallel only after the
|
||
final canonical artifact is frozen; their evidence aggregation remains ordered.
|
||
- Target cutover Task 25 is a serialized deployment operation after Task 24 qualification. Task 26
|
||
cleanup starts only after the reviewed observation window and requires a newly qualified artifact.
|
||
- Shared-worktree Gradle invocations remain sequential even when source subtasks are delegated.
|
||
|
||
## 9. Minimum completion matrix
|
||
|
||
| Requirement | Proving task |
|
||
| --- | --- |
|
||
| approved truth/no ACK overclaim | 1 |
|
||
| closed first-tuple registry/no future switches | 2, 24 |
|
||
| framework-free typed event/SPI | 3 |
|
||
| generic envelope + sample-owned payload schema | 4 |
|
||
| closed catalog/destination/digest/key | 5 |
|
||
| Draft 2020-12 deterministic bytes/admission | 6 |
|
||
| forward-only immutable event/delivery/journal/epoch | 7 |
|
||
| same-transaction validated append | 8 |
|
||
| exhaustive outcome + one-record relay | 9 |
|
||
| JIT claim/token/unexpired-lease CAS | 10 |
|
||
| late observation DB commit before source ACK | 11 |
|
||
| audited requeue/hold/skip/compensate | 12 |
|
||
| legacy/v2 mutual exclusion | 13 |
|
||
| finite typed config/disabled 0 | 14 |
|
||
| actual ACK-aware Spring Kafka gateway | 15 |
|
||
| admission/late queue/generation lifecycle | 16 |
|
||
| topic/security attestation | 17 |
|
||
| authenticated internal disposition endpoint | 18 |
|
||
| composition/readiness/observability | 19 |
|
||
| atomic watermark/reconcile/epoch/sentinel cutover implementation + disposable rehearsal | 20 |
|
||
| real PostgreSQL + real Kafka fault evidence | 21 |
|
||
| TLS/SASL/ACL + multi-broker RF/min ISR | 22 |
|
||
| env/registries/runbooks | 23 |
|
||
| no-skip pre-cutover release artifact + independent review | 24 |
|
||
| exact-target approved deployment cutover evidence | 25 |
|
||
| observation exit + legacy cleanup + cleanup-artifact requalification + final Wiki | 26 |
|
||
|
||
## 10. Follow-up plans after first R2
|
||
|
||
다음은 이 계획을 확장하는 checkbox가 아니라 별도 설계 승인과 실행 계획이다.
|
||
|
||
1. **Inbound Kafka + inbox + DLT/replay (P5)**
|
||
- 20번째 `adapter:inbound:messaging-kafka` leaf registry migration;
|
||
- manual ACK after application commit;
|
||
- inbox/effect identity, bounded retry, DLT ACK, replay/audit.
|
||
2. **PostgreSQL Debezium CDC (P6)**
|
||
- external Connect/Debezium asset, publication/slot/offset/WAL;
|
||
- insert-only mapping, shadow, authority-exclusive cutover/rollback and retention proof.
|
||
3. **Optional cards (P7)**
|
||
- Avro/Protobuf registry, Kafka EOS, retry topic, compaction, object-storage claim check,
|
||
alternate broker, multi-cluster, module split;
|
||
- each requirement gets an independent card/compatibility/evidence plan.
|
||
4. **Live non-empty legacy database migration**
|
||
- collect actual row volume/state distribution, lock/replication budget, data classification,
|
||
maintenance window and rollback evidence;
|
||
- separately approve either `LIVE_ADDITIVE_BACKFILL_IN_PLACE.v1` or
|
||
`COPY_AND_CUTOVER_WITH_RECONCILIATION.v1`.
|
||
|
||
## 11. Final non-negotiable assertions
|
||
|
||
- `KafkaSender.send()` return is not broker ACK.
|
||
- Kafka future success plus metadata is ACK observation; it is not consumer processing.
|
||
- `acks=all` without RF/min ISR/unclean-election evidence is not durable topology evidence.
|
||
- Kafka producer idempotence does not remove ACK-to-DB, restart, late-ACK or operator-requeue
|
||
duplicates.
|
||
- `EXHAUSTED` does not mean definitely not delivered.
|
||
- late ACK observation never rewrites authoritative delivery state.
|
||
- timestamp/random event ID is not aggregate total order.
|
||
- `outbox_event` wire bytes are immutable authority; JSONB/re-serialization is not.
|
||
- polling and CDC may never be simultaneous production dispatch authorities.
|
||
- operator mutation requires auth, expected generation/version, no active claim and immutable audit.
|
||
- disabled and configured-but-not-ready are different states.
|
||
- fake/single-node/local evidence cannot be relabelled as production R2.
|
||
- implementation completion requires exact commands/results and LLM Wiki capture.
|
||
- agent never stages, commits, amends or pushes.
|