review-followup Wave 3 라운드 2(P18): outbox canonical 통일, composition 모듈, legacy Kafka seam 삭제

- 결정 7: `LegacyOutboxAppendPort`·legacy 저장 모델·relay·Kafka R0 seam 삭제, 템플릿 JSON encoder(`JsonIntegrationEventEncoder`),
  durable operation·sample 이벤트가 `OutboxAppendPort` + encoder로, persistence-jpa V19(pre-canonical 미발행 row 거부)
- 결정 3: outbox relay·멱등성 executor 조립을 `composition:outbox-relay`·`composition:idempotency`로,
  sample이 복사한 bootstrap 인프라를 `composition:service-runtime`으로(sample `SampleIdempotencyConfig` 복사본 삭제)
- persistence-jpa outbox·멱등성 패키지 feature 기준 이동, messaging outbox·inbox leaf를 reliability / reliability-jdbc-postgresql로
- messaging outbox header를 jsonb로(P18-new-1 — `\`로 끝나는 값이 다음 header를 삼키던 결함)
- 통합(충돌 22건): P25 A-10c(전략 선택 설정 삭제)를 service-runtime `DomainContextConfig`에 적용, P25 startup 예외 통합을
  composition 설정 테스트에, P25 `CommandUseCase.handle` 이름을 PG 통합 테스트에, P15 `EventJson`은 호출자 0이 되어 삭제
- root CLAUDE.md의 legacy seam 문장 수정(P18-new-2)

검증: ./gradlew check architectureCheck verifyDependencyLocks verifyPublicPathSnapshot :app-bootstrap:verifyEnvKeys
configContractCheck :app-bootstrap:integrationTest :sample-portfolio:integrationTest :sample-portfolio:systemTest
persistence-jpa jpaPlatformContractTest·postgresql{Idempotency,OutboxAppendTransactional,OutboxConcurrentRelay,OutboxRowLifecycle}IntegrationTest
:messaging:messaging-reliability-jdbc-postgresql:integrationTest --continue 통과(700 task)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
donghyeon-ka
2026-09-29 12:30:01 +09:00
co-authored by Claude Opus 5.5
parent b5e3c03777
commit 6397f52dc8
45 changed files with 298 additions and 532 deletions
+8 -12
View File
@@ -21,20 +21,16 @@ Application의 canonical integration event를 platform publish pipeline으로
producing-service identity다.
값이 없으면 `IntegrationEventPublishPort` bridge 자체를 만들지 않는다. `spring.application.name`이나
현재 process 이름으로 추론하지 않는다. 기존 legacy `OutboxEvent`/realtime 경로는 별도 cutover가
끝날 때까지 `app.messaging.broker` 경로를 유지한다.
현재 process 이름으로 추론하지 않는다. 모든 발행(outbox relay 포함)이 이 bridge를 지나므로 messaging
모듈이 켜진 상태에서 값이 없으면 기동이 거부된다.
## Outbox canonical transport-only cutover
## Outbox
`APP_OUTBOX_CANONICAL_TRANSPORT_ENABLED` / `ca-skeleton.outbox.canonical-transport-enabled`은
기존 `outbox_event` writer/claim/status authority를 유지한 채 canonical row의 **transport만** platform
publish path로 보내는 compatibility gate다. 기본값은 `false`이며 두 번째 relay를 만들지 않는다.
`true`일 때는 `OutboxAppendPort`가 `ValidatedIntegrationEvent`의 exact envelope bytes와 canonical
metadata를 기존 outbox row에 저장하고, claim된 canonical row는 `IntegrationEventPublishPort`로 간다.
legacy row는 계속 `MessageBroker`를 사용한다. 따라서 mixed-row compatibility 기간에는 relay가 켜져
있다면 `app.messaging.broker`도 계속 필요하며, canonical path를 위해 `IntegrationEventPublishPort`도
추가로 필요하다. legacy backlog가 0이라는 별도 증거 없이 broker 요구를 제거하지 않는다.
outbox는 canonical 하나다(2026-09-28). use case가 `IntegrationEventEncoderPort`로 만든
`ValidatedIntegrationEvent`를 `OutboxAppendPort`로 append하고, relay는 claim한 row의 exact envelope
bytes를 `IntegrationEventPublishPort`로 보낸다. 예전의 `ca-skeleton.outbox.canonical-transport-enabled`
gate와 legacy `MessageBroker → KafkaSender` 경로는 삭제됐다. 발행할 contract의 logical destination마다
아래 destination profile을 선언한다(템플릿의 durable operation event는 `durable-operations`).
## Destination profile
+3 -3
View File
@@ -21,8 +21,8 @@ Flyway version `1000..1999`를 예약해서 애플리케이션의 일반적인
classpath:db/migration/messaging
```
현재 순서는 `V1001..V1005`(outbox/admin)와 `V1101`(inbox)이다. starter 테스트가 두 JDBC
모듈을 실제 combined classpath로 올려 version 중복과 예약 대역 이탈을 검사한다. 개별 DDL의
현재 순서는 `V1001..V1005`(outbox/admin)와 `V1101`(inbox)이고, 모두
`messaging-reliability-jdbc-postgresql`에 있다. starter 테스트가 그 모듈을 실제 combined classpath로 올려 version 중복과 예약 대역 이탈을 검사한다. 개별 DDL의
의미와 전환 순서는 [migration-guide.md](migration-guide.md)를 따른다.
## Outbox
@@ -39,7 +39,7 @@ COMMIT;
### relay
```text
leaseBatch(100, 30s) -- lease로 다중 relay 인스턴스 안전
claimBatch(owner, 100, 30s, now, maxAttempts) -- owner + fencing token으로 다중 relay 인스턴스 안전
→ publish (messageId 그대로)
→ CONFIRMED → markPublished
→ AMBIGUOUS → markAmbiguous (같은 messageId로 재시도 가능)
+4 -2
View File
@@ -17,8 +17,10 @@
> `BrokerFailureMatrix.shipped()` 하드코딩이 아니라 레인이 낸 증거(`BrokerCertificationEvidence`)에서
> 나온다. 증거가 없는 조합은 `NOT_COVERED`다 (MSG-014).
> **모듈 이름과 런타임 편입.** `messaging-outbox-jdbc-postgresql` / `messaging-inbox-jdbc-postgresql`은
> 이전에 `-jpa`로 불렸다. 구현은 Spring JDBC이고 SQL은 PostgreSQL 전용(`?::jsonb`,
> **모듈 이름과 런타임 편입.** 관계형 신뢰성 구현은 `messaging-reliability-jdbc-postgresql`,
> 그 port만 보는 relay·consumer 로직은 `messaging-reliability`다(2026-09-28에
> `messaging-outbox-jdbc-postgresql` / `messaging-inbox-jdbc-postgresql`을 이렇게 나눴다). 그 전에는
> `-jpa`로 불렸다. 구현은 Spring JDBC이고 SQL은 PostgreSQL 전용(`jsonb`,
> `FOR UPDATE SKIP LOCKED`, `ON CONFLICT`, `TIMESTAMPTZ`)이므로, 그 이름은 쓰지 않는 기술을
> 광고하고 vendor 중립 port(`messaging-reliability-api`)의 위치를 가렸다 (MSG-023).
>
+22 -19
View File
@@ -16,16 +16,19 @@ leaf의 정확한 id는 registry에서 읽는다. family는 **디렉터리 이
| --- | --- | --- |
| `*-api` (`core-api`, `schema-api`, `policy`, `reliability-api`, `admin-api`, `transport-spi`) | 값 타입, 포트, 정책 계약 | Java stdlib만. Spring/broker SDK/logging framework 금지 |
| `schema-*` (json/avro/protobuf), `cloudevents` | 인코딩 구현 | 해당 codec 라이브러리만. Spring 금지 |
| `runtime-core`, `security`, `observability`, `admin-runtime` | framework-free 실행 로직 | Micrometer는 `observability`에서만. 나머지는 stdlib |
| `runtime-core`, `security`, `observability`, `admin-runtime`, `reliability` | framework-free 실행 로직 | Micrometer는 `observability`에서만. 나머지는 stdlib |
| `kafka` | broker adapter | 해당 broker client. Spring은 client가 곧 Spring인 경우만 |
| `outbox-jdbc-postgresql`, `inbox-jdbc-postgresql` | 관계형 신뢰성 adapter | Spring JDBC / Spring TX. **JPA 아님** — 이름이 곧 계약이다 |
| `reliability-jdbc-postgresql` | 관계형 신뢰성 adapter (outbox·inbox repository, admin journal) | Spring JDBC / Spring TX. **JPA 아님** — 이름이 곧 계약이다 |
| `spring-boot-starter`, `spring-boot-starter-kafka` | 조립 경계 | Spring Boot autoconfigure. 여기서만 |
| `testkit` | 테스트 지원 | Testcontainers, JUnit. production leaf가 의존하지 않는다 |
`outbox`/`inbox` leaf가 `jdbc-postgresql`로 명명된 이유: 구현은 Spring JDBC이고 SQL은
`?::jsonb` · `FOR UPDATE SKIP LOCKED` · `ON CONFLICT` · `TIMESTAMPTZ`로 PostgreSQL 전용이다.
`jpa`라는 이름은 사용하지 않는 기술을 광고하고, vendor 중립 port가 어디 있는지도 숨겼다. vendor
중립 계약은 `messaging-reliability-api`에 있다.
신뢰성 기능은 세 leaf다. `messaging-reliability-api`가 vendor 중립 port(`OutboxRepository`,
`InboxRepository`)를, `messaging-reliability`가 그 port만 보는 relay·consumer 로직(stdlib)을,
`messaging-reliability-jdbc-postgresql`이 port의 관계형 구현과 migration(`V1001..V1005`, `V1101`)을
갖는다. 예전에는 `outbox-jdbc-postgresql` 하나가 relay 로직과 PG repository와 admin journal을 함께
가졌고 inbox도 같은 모양이었다(S7-#5b). 구현 leaf가 `jdbc-postgresql`로 명명된 이유: 구현은 Spring
JDBC이고 SQL은 `jsonb` · `FOR UPDATE SKIP LOCKED` · `ON CONFLICT` · `TIMESTAMPTZ`로 PostgreSQL
전용이다. `jpa`라는 이름은 사용하지 않는 기술을 광고하고, vendor 중립 port가 어디 있는지도 숨겼다.
## Public surface 정책
@@ -57,7 +60,7 @@ RabbitMQ·Pulsar·NATS·Kafka Share·Spring Cloud Stream bridge leaf와 Debezium
새 leaf를 application에 채택할 때는 starter 또는 composition dependency와 architecture edge policy,
qualification/support 상태를 함께 변경한다.
## application이 이 family에 도달하는 경로 (MSG-015, canonical bridge 구현 / legacy cutover 잔존)
## application이 이 family에 도달하는 경로 (MSG-015, cutover 완료)
리뷰(`docs/reviews/2026-08-14-messaging-module-code-review.md` §6.1·§6.3)가 정한
`application-owned port → platform anti-corruption bridge → platform` 경로는 이제 존재한다.
@@ -72,21 +75,21 @@ lease/transport/outcome normalization/observation은 platform 중앙 publish pat
package는 `messaging-core-api`와 `messaging-schema-api`만 보며 concrete runtime/transport/Kafka 타입을
직접 보지 않는다.
다만 **MSG-015의 전체 cutover가 끝난 것은 아니다.** legacy `OutboxEvent`와 realtime publisher는
schema/order/tenant 등 canonical metadata가 부족해 아직 `MessageBroker → KafkaSender` R0 경로를
사용한다. 그 metadata를 추측해 platform envelope를 만드는 것은 금지한다. 따라서 legacy outbox
storage/relay migration을 별도 change set으로 완료한 뒤에만 R0 seam을 제거한다.
**MSG-015 cutover는 끝났다(2026-09-28, 소유자 결정 7).** application의 outbox append는 canonical
`OutboxAppendPort` 하나이고, relay는 claim한 row의 validated event를 이 bridge로만 보낸다. legacy
`OutboxEvent`·`LegacyOutboxAppendPort`와 `MessageBroker → KafkaSender` R0 경로(TLS/SASL을 적용하지
않던 두 번째 producer)는 삭제했다. 남은 Kafka producer는 platform의 것 하나이고, broker security
profile(`KafkaSecurityConfigurer`)을 적용한다.
canonical bridge의 producer identity는 `app.messaging.producer-id`로 명시한다. 값이 없으면 bridge bean을
만들지 않으며 `spring.application.name`/host/pod 이름에서 identity를 발명하지 않는다.
만들지 않으며 `spring.application.name`/host/pod 이름에서 identity를 발명하지 않는다. bridge가 없으면
relay가 발행할 곳이 없으므로 messaging 모듈이 켜진 상태에서는 기동이 거부된다.
`ca-skeleton.outbox.canonical-transport-enabled=true`는 bridge 존재만으로 충분하지 않다. Stable
composition은 애플리케이션 계약 catalog/schema source를 임의로 만들지 않으므로 adopter가
`IntegrationEventEncoderPort`를 명시적으로 제공해야 한다. 템플릿은 구현체를 주지 않는다(후보였던
R1 JSON Schema encoder는 2026-09-25에 삭제). `OutboxCanonicalTransportRequirementValidator`는
publisher 또는 encoder 중 하나라도
없으면 startup을 거부한다. 즉 canonical flag가 켜졌는데 정본 encoder가 없는 상태로 성공 기동하는
경로는 없다.
`IntegrationEventEncoderPort`는 템플릿이 제공한다: `adapter:outbound:support`의
`JsonIntegrationEventEncoder`가 envelope v1(`shared-contract`의 `contracts/messaging/envelope/v1.schema.json`)
안에 payload record를 결정적 JSON으로 싣는다. 계약 catalog는 애플리케이션이 bean으로 등록한
`IntegrationEventContractContribution`뿐이고, 선언한 payload schema hash가 classpath의 schema 파일과
다르면 기동이 실패한다. payload를 JSON Schema로 매번 검증하려는 서비스는 자기 encoder bean을 두면 된다.
## Stable 승격에 필요한 증거
@@ -1,33 +0,0 @@
plugins {
id 'ca.auxiliary-source-set'
id 'ca.platform-module'
}
auxiliarySourceSets {
sourceSet('integrationTest') {
compilesAgainst 'main'
}
}
dependencies {
api project(':messaging:messaging-core-api')
api project(':messaging:messaging-reliability-api')
implementation 'org.springframework:spring-jdbc'
implementation 'org.springframework:spring-tx'
// Live-database certification. The reliability patterns are claims about transaction
// boundaries and uniqueness constraints, and only a real database can settle them.
testImplementation project(':messaging:messaging-testkit')
integrationTestImplementation 'org.testcontainers:testcontainers-postgresql'
integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter'
integrationTestImplementation 'org.postgresql:postgresql'
}
strictTestLanes {
lane('integrationTest') {
integration()
sourceSet = 'integrationTest'
description = 'Runs inbox transaction and uniqueness contracts against real PostgreSQL.'
}
}
@@ -1,111 +0,0 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.fasterxml.jackson.core:jackson-annotations:2.21=integrationTestCompileClasspath,integrationTestRuntimeClasspath
com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.github.docker-java:docker-java-api:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath
com.github.docker-java:docker-java-transport-zerodep:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath
com.github.docker-java:docker-java-transport:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs
com.github.spotbugs:spotbugs:4.10.2=spotbugs
com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.auto:auto-common:1.2.2=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs
com.google.code.gson:gson:2.13.2=spotbugs
com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.41.0=spotbugs
com.google.errorprone:error_prone_annotations:2.47.0=checkstyle
com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.guava:guava:33.5.0-jre=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.guava:guava:33.6.0-jre=checkstyle
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,testAnnotationProcessor
com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins
com.puppycrawl.tools:checkstyle:13.5.0=checkstyle
commons-beanutils:commons-beanutils:1.11.0=checkstyle
commons-codec:commons-codec:1.19.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath
commons-collections:commons-collections:3.2.2=checkstyle
commons-io:commons-io:2.20.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath
commons-io:commons-io:2.21.0=spotbugs
commons-logging:commons-logging:1.3.6=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
info.picocli:picocli:4.7.7=checkstyle
io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
io.micrometer:micrometer-commons:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.micrometer:micrometer-core:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
jaxen:jaxen:2.0.6=spotbugs
net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
net.java.dev.jna:jna:5.18.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath
net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs
org.antlr:antlr4-runtime:4.13.2=checkstyle
org.apache.bcel:bcel:6.12.0=spotbugs
org.apache.commons:commons-compress:1.28.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath
org.apache.commons:commons-lang3:3.20.0=checkstyle,integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs
org.apache.commons:commons-text:1.15.0=spotbugs
org.apache.commons:commons-text:1.3=checkstyle
org.apache.httpcomponents:httpclient:4.5.13=checkstyle
org.apache.httpcomponents:httpcore:4.4.16=checkstyle
org.apache.logging.log4j:log4j-api:2.25.5=spotbugs
org.apache.logging.log4j:log4j-core:2.25.5=spotbugs
org.apache.maven.doxia:doxia-core:1.12.0=checkstyle
org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle
org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle
org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle
org.apache.xbean:xbean-reflect:3.7=checkstyle
org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,testCompileClasspath
org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
org.checkerframework:checker-qual:3.55.1=integrationTestRuntimeClasspath
org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle
org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle
org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle
org.codehaus.plexus:plexus-utils:3.3.0=checkstyle
org.dom4j:dom4j:2.2.0=spotbugs
org.hdrhistogram:HdrHistogram:2.2.2=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
org.javassist:javassist:3.28.0-GA=checkstyle
org.jetbrains:annotations:17.0.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath
org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,integrationTestAnnotationProcessor,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-params:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-launcher:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath
org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
org.junit:junit-bom:6.1.0=spotbugs
org.latencyutils:LatencyUtils:2.0.3=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
org.mockito:mockito-core:5.20.0=mockitoAgent
org.opentest4j:opentest4j:1.3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-analysis:9.10.1=spotbugs
org.ow2.asm:asm-commons:9.10.1=spotbugs
org.ow2.asm:asm-tree:9.10.1=spotbugs
org.ow2.asm:asm-util:9.10.1=spotbugs
org.ow2.asm:asm:9.10.1=spotbugs
org.pcollections:pcollections:4.0.1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor
org.postgresql:postgresql:42.7.13=integrationTestCompileClasspath,integrationTestRuntimeClasspath
org.reflections:reflections:0.10.2=checkstyle
org.rnorth.duct-tape:duct-tape:1.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath
org.slf4j:slf4j-api:2.0.18=integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs,spotbugsSlf4j
org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j
org.slf4j:slf4j-simple:2.0.18=checkstyle
org.springframework:spring-beans:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.springframework:spring-core:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.springframework:spring-jdbc:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.springframework:spring-tx:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:testcontainers-database-commons:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath
org.testcontainers:testcontainers-jdbc:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath
org.testcontainers:testcontainers-junit-jupiter:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath
org.testcontainers:testcontainers-postgresql:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath
org.testcontainers:testcontainers:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath
org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs
empty=
@@ -22,38 +22,23 @@ public interface OutboxRepository {
void append(OutboxRecord record);
/**
* Leases a batch of publishable records.
* Claims a batch and returns the leases, each carrying its owner and fencing token, skipping rows
* that are waiting for their backoff or out of attempts.
*
* <p>Leasing rather than simply selecting is what makes multiple relay instances safe: a record
* claimed by one relay is invisible to the others until its lease expires, so the same message is
* not published concurrently by two processes.
*
* @param batchSize how many records to claim
* @param leaseDuration how long the claim holds
* @param now the current instant
* @return the leased records, oldest first
*/
List<OutboxRecord> leaseBatch(int batchSize, Duration leaseDuration, Instant now);
/**
* Claims a batch and returns the leases, each carrying its owner and fencing token.
*
* <p>The token is what a terminal write is checked against. {@link #leaseBatch} returns records
* without one, so its callers cannot prove a write belongs to their claim; it remains for
* inspection paths and is deprecated for the relay's use.
*
* @param owner this relay's stable identity
*/
List<OutboxLease> claimBatch(String owner, int batchSize, Duration leaseDuration, Instant now);
/**
* Claims a batch, skipping rows that are waiting for their backoff or out of attempts.
* <p>Claiming rather than simply selecting is what makes multiple relay instances safe: a record
* claimed by one relay is invisible to the others until its lease expires, and the token is what
* a terminal write is checked against, so a relay that stalled past its lease writes nothing.
*
* <p>Eligibility belongs in the row. A relay that decides in memory forgets on restart, and every
* replica decides for itself — so the same message is retried by as many workers as there are
* replicas, as often as they poll.
*
* @param owner this relay's stable identity
* @param batchSize how many records to claim
* @param leaseDuration how long the claim holds
* @param now the current instant
* @param maxAttempts the attempt budget; rows at or above it are not claimed
* @return the leases, oldest first
*/
List<OutboxLease> claimBatch(
String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts);
@@ -65,10 +50,10 @@ public interface OutboxRepository {
*/
OutboxTransitionResult markPublished(OutboxLease lease, Instant now);
/** Records an ambiguous outcome, only if this lease still owns the row. */
OutboxTransitionResult markAmbiguous(OutboxLease lease, String failureCode, Instant now);
/** Records an ambiguous outcome and when this row may next be claimed. */
/**
* Records an ambiguous outcome and when this row may next be claimed, only if this lease still
* owns the row.
*/
OutboxTransitionResult markAmbiguous(
OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt);
@@ -9,11 +9,13 @@ auxiliarySourceSets {
}
}
// The relational implementations of the reliability ports: the outbox repository, the inbox
// repository and the destructive-operation journal. Spring JDBC for the transaction join, and SQL
// that is PostgreSQL's (`jsonb`, `FOR UPDATE SKIP LOCKED`, `ON CONFLICT`, `TIMESTAMPTZ`) — the name
// is the contract. The relay and consumer logic that uses them is messaging-reliability.
dependencies {
api project(':messaging:messaging-core-api')
api project(':messaging:messaging-reliability-api')
api project(':messaging:messaging-policy')
api project(':messaging:messaging-observability')
// The destructive-operation journal lives here because it needs exactly what the outbox needs:
// one relational database every replica can see, and a migration lane that already exists. The
@@ -26,9 +28,7 @@ dependencies {
// Live-database certification. The reliability patterns are claims about transaction
// boundaries and uniqueness constraints, and only a real database can settle them.
testImplementation project(':messaging:messaging-testkit')
// Test-only. The relay is also driven through the real DefaultMessagePublisher: a fake publisher
// accepted the pre-encoded rows the real one re-encoded and rejected.
testImplementation project(':messaging:messaging-runtime-core')
integrationTestImplementation project(':messaging:messaging-testkit')
integrationTestImplementation 'org.testcontainers:testcontainers-postgresql'
integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter'
integrationTestImplementation 'org.postgresql:postgresql'
@@ -38,6 +38,6 @@ strictTestLanes {
lane('integrationTest') {
integration()
sourceSet = 'integrationTest'
description = 'Runs outbox and admin-journal contracts against real PostgreSQL.'
description = 'Runs outbox, inbox and admin-journal contracts against real PostgreSQL.'
}
}
@@ -71,6 +71,9 @@ class OutboxPostgresIT {
try (Connection connection = dataSource.getConnection();
Statement statement = connection.createStatement()) {
statement.execute("DROP TABLE IF EXISTS messaging_outbox");
// The inbox migration ships in the same module now, so "every migration" creates its table
// too.
statement.execute("DROP TABLE IF EXISTS messaging_inbox");
// Every migration in order, not just V1: a test that applies part of the schema certifies a
// database nobody deploys.
for (String migration : migrations()) {
@@ -90,7 +93,7 @@ class OutboxPostgresIT {
OutboxRecord record = record();
appendInTransaction(record);
List<OutboxRecord> leased = repository.leaseBatch(10, Duration.ofSeconds(30), NOW);
List<OutboxRecord> leased = claim("relay-a", NOW).stream().map(OutboxLease::record).toList();
assertThat(leased)
.singleElement()
@@ -109,8 +112,8 @@ class OutboxPostgresIT {
void aLeasedRowIsInvisibleToASecondRelayInstance() {
appendInTransaction(record());
List<OutboxRecord> first = repository.leaseBatch(10, Duration.ofSeconds(30), NOW);
List<OutboxRecord> second = repository.leaseBatch(10, Duration.ofSeconds(30), NOW);
List<OutboxLease> first = claim("relay-a", NOW);
List<OutboxLease> second = claim("relay-b", NOW);
assertThat(first).hasSize(1);
assertThat(second).as("SKIP LOCKED plus the lease keeps two relays off the same row").isEmpty();
@@ -119,10 +122,9 @@ class OutboxPostgresIT {
@Test
void anExpiredLeaseBecomesClaimableAgain() {
appendInTransaction(record());
repository.leaseBatch(10, Duration.ofSeconds(30), NOW);
claim("relay-a", NOW);
List<OutboxRecord> reclaimed =
repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(31));
List<OutboxLease> reclaimed = claim("relay-b", NOW.plusSeconds(31));
assertThat(reclaimed).hasSize(1);
}
@@ -131,18 +133,16 @@ class OutboxPostgresIT {
void anAmbiguousRowStaysClaimableUnderItsOriginalIdentity() {
OutboxRecord record = record();
appendInTransaction(record);
// Claimed rather than leased: a terminal write is checked against the lease token, so
// `markAmbiguous` takes the lease that owns the row. The identity-keyed overload this test used
// to call is gone, and with it the possibility of one relay writing an outcome onto a row
// another relay currently owns.
OutboxLease lease =
repository.claimBatch("ambiguous-relay", 10, Duration.ofSeconds(30), NOW).getFirst();
// A terminal write is checked against the lease token, so `markAmbiguous` takes the lease that
// owns the row; no identity-keyed write exists for one relay to put an outcome on a row another
// relay currently owns.
OutboxLease lease = claim("ambiguous-relay", NOW).getFirst();
repository.markAmbiguous(lease, "CONFIRM_TIMEOUT", NOW);
repository.markAmbiguous(lease, "CONFIRM_TIMEOUT", NOW, NOW);
assertThat(repository.find(record.messageId()).orElseThrow().status())
.isEqualTo(OutboxStatus.AMBIGUOUS);
assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW))
assertThat(claim("relay-b", NOW))
.singleElement()
.satisfies(retry -> assertThat(retry.messageId()).isEqualTo(record.messageId()));
}
@@ -151,24 +151,24 @@ class OutboxPostgresIT {
void aPublishedRowIsNotClaimedAgain() {
OutboxRecord record = record();
appendInTransaction(record);
repository.leaseBatch(10, Duration.ofSeconds(30), NOW);
OutboxLease lease = claim("relay-a", NOW).getFirst();
repository.markPublishedUnfenced(record.messageId(), NOW);
repository.markPublished(lease, NOW);
assertThat(repository.find(record.messageId()).orElseThrow().status())
.isEqualTo(OutboxStatus.PUBLISHED);
assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(60))).isEmpty();
assertThat(claim("relay-b", NOW.plusSeconds(60))).isEmpty();
}
@Test
void aFailedRowIsNotClaimedAgainWithoutIntervention() {
OutboxRecord record = record();
appendInTransaction(record);
repository.leaseBatch(10, Duration.ofSeconds(30), NOW);
OutboxLease lease = claim("relay-a", NOW).getFirst();
repository.markFailedUnfenced(record.messageId(), "INVALID_TOPIC", NOW);
repository.markFailed(lease, "INVALID_TOPIC", NOW);
assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(60))).isEmpty();
assertThat(claim("relay-b", NOW.plusSeconds(60))).isEmpty();
}
@Test
@@ -202,8 +202,8 @@ class OutboxPostgresIT {
OutboxRecord published = record();
OutboxRecord pending = record();
appendInTransaction(published);
repository.markPublished(claim("relay-a", NOW).getFirst(), NOW);
appendInTransaction(pending);
repository.markPublishedUnfenced(published.messageId(), NOW);
int removed = repository.purgePublishedBefore(NOW.plusSeconds(1), 100);
@@ -211,6 +211,11 @@ class OutboxPostgresIT {
assertThat(repository.find(pending.messageId())).isPresent();
}
/** Claims with an unlimited attempt budget, as a relay does on a row's first attempts. */
private List<OutboxLease> claim(String owner, Instant now) {
return repository.claimBatch(owner, 10, Duration.ofSeconds(30), now, Integer.MAX_VALUE);
}
private static OutboxRecord record() {
return record(
// The routing key is a column now, not a reserved header smuggled through the header map.
@@ -228,7 +233,8 @@ class OutboxPostgresIT {
appendInTransaction(otherKey);
// when
List<OutboxLease> claimed = repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW);
List<OutboxLease> claimed =
repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE);
// then
assertThat(claimed)
@@ -352,12 +358,13 @@ class OutboxPostgresIT {
// Relay A claims and stalls.
java.util.List<dev.caskeleton.messaging.reliability.OutboxLease> first =
repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW);
repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE);
assertThat(first).hasSize(1);
// The lease expires and relay B takes the row and publishes it.
java.util.List<dev.caskeleton.messaging.reliability.OutboxLease> second =
repository.claimBatch("relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31));
repository.claimBatch(
"relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31), Integer.MAX_VALUE);
assertThat(second).hasSize(1);
assertThat(second.get(0).token())
.as("every claim increments the token, which is what makes the older one detectable")
@@ -366,7 +373,9 @@ class OutboxPostgresIT {
.isEqualTo(dev.caskeleton.messaging.reliability.OutboxTransitionResult.APPLIED);
// Relay A finally wakes up and tries to record its own outcome.
assertThat(repository.markAmbiguous(first.get(0), "CONFIRM_TIMEOUT", NOW.plusSeconds(40)))
assertThat(
repository.markAmbiguous(
first.get(0), "CONFIRM_TIMEOUT", NOW.plusSeconds(40), NOW.plusSeconds(40)))
.as("this write used to succeed and made a published message claimable again")
.isEqualTo(dev.caskeleton.messaging.reliability.OutboxTransitionResult.STALE_LEASE);
assertThat(repository.find(record.messageId()).orElseThrow().status())
@@ -378,7 +387,7 @@ class OutboxPostgresIT {
OutboxRecord record = record();
appendInTransaction(record);
dev.caskeleton.messaging.reliability.OutboxLease lease =
repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW).get(0);
repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE).get(0);
assertThat(repository.markPublished(lease, NOW))
.isEqualTo(dev.caskeleton.messaging.reliability.OutboxTransitionResult.APPLIED);
@@ -395,9 +404,9 @@ class OutboxPostgresIT {
appendInTransaction(two);
java.util.List<dev.caskeleton.messaging.reliability.OutboxLease> a =
repository.claimBatch("relay-a", 1, Duration.ofSeconds(30), NOW);
repository.claimBatch("relay-a", 1, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE);
java.util.List<dev.caskeleton.messaging.reliability.OutboxLease> b =
repository.claimBatch("relay-b", 1, Duration.ofSeconds(30), NOW);
repository.claimBatch("relay-b", 1, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE);
assertThat(a).hasSize(1);
assertThat(b).hasSize(1);
@@ -410,9 +419,15 @@ class OutboxPostgresIT {
appendInTransaction(record);
long firstToken =
repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW).get(0).token();
repository
.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE)
.get(0)
.token();
dev.caskeleton.messaging.reliability.OutboxLease reclaimed =
repository.claimBatch("relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31)).get(0);
repository
.claimBatch(
"relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31), Integer.MAX_VALUE)
.get(0);
assertThat(reclaimed.messageId()).isEqualTo(record.messageId());
assertThat(reclaimed.token()).isEqualTo(firstToken + 1);
@@ -589,6 +604,38 @@ class OutboxPostgresIT {
.containsEntry("x-note", "line one\nline\ttwo");
}
/**
* The hand-rolled reader took a quote preceded by a backslash for an escaped quote, so a value
* ending in an escaped backslash ran on into the next header and both came back wrong.
*/
@Test
void aHeaderValueEndingInABackslashRoundTripsWithoutSwallowingTheNextHeader() {
// given
OutboxRecord record =
new OutboxRecord(
MessageId.newId(),
new DestinationName("order-events"),
new MessageType("order.created"),
new SchemaVersion(1),
ContentType.JSON,
"{}".getBytes(StandardCharsets.UTF_8),
Map.of("x-path", "C:\\exports\\", "x-next", "after"),
OutboxCanonicalMetadata.none(),
NOW.minusSeconds(60),
OutboxStatus.PENDING,
0,
Optional.empty(),
Optional.empty());
appendInTransaction(record);
// when
Map<String, String> headers = repository.find(record.messageId()).orElseThrow().headers();
// then
assertThat(headers)
.containsExactlyInAnyOrderEntriesOf(Map.of("x-path", "C:\\exports\\", "x-next", "after"));
}
private void appendInTransaction(OutboxRecord record) {
transactions.executeWithoutResult(status -> repository.append(record));
}
@@ -17,6 +17,7 @@ import dev.caskeleton.messaging.reliability.OutboxRecord;
import dev.caskeleton.messaging.reliability.OutboxRepository;
import dev.caskeleton.messaging.reliability.OutboxStatus;
import dev.caskeleton.messaging.reliability.OutboxTransitionResult;
import java.sql.Array;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
@@ -41,7 +42,7 @@ import org.springframework.transaction.support.TransactionSynchronizationManager
* already inside, which is the entire mechanism. An outbox row written on a separate connection
* commits independently of the business change and reopens the window the pattern exists to close.
*
* <p>Leasing uses {@code SELECT ... FOR UPDATE SKIP LOCKED}. Two relay instances polling the same
* <p>Claiming uses {@code SELECT ... FOR UPDATE SKIP LOCKED}. Two relay instances polling the same
* table would otherwise both claim the same rows and publish them twice; {@code SKIP LOCKED} lets
* the second instance step over what the first already holds instead of blocking behind it.
*/
@@ -55,7 +56,7 @@ public final class JdbcOutboxRepository implements OutboxRepository {
producer, occurred_at, correlation_id, causation_id, tenant,
partition_key, ordering_key, schema_subject, schema_uri,
traceparent, tracestate, baggage)
VALUES (?, ?, ?, ?, ?, ?, ?::jsonb, ?, ?, ?,
VALUES (?, ?, ?, ?, ?, ?, jsonb_object(?::text[]), ?, ?, ?,
?, ?, ?, ?, ?,
?, ?, ?, ?,
?, ?, ?)
@@ -72,38 +73,15 @@ public final class JdbcOutboxRepository implements OutboxRepository {
private static final String COLUMNS =
"""
message_id, destination, message_type, schema_version, content_type,
payload, headers, created_at, status, attempts, lease_expires_at, last_failure_code,
payload,
ARRAY(SELECT h.key FROM jsonb_each_text(headers) h ORDER BY h.key) AS header_keys,
ARRAY(SELECT h.value FROM jsonb_each_text(headers) h ORDER BY h.key) AS header_values,
created_at, status, attempts, lease_expires_at, last_failure_code,
producer, occurred_at, correlation_id, causation_id, tenant,
partition_key, ordering_key, schema_subject, schema_uri,
traceparent, tracestate, baggage\
""";
private static final String LEASE =
"""
WITH claimable AS (
SELECT message_id
FROM messaging_outbox
-- IN_FLIGHT is claimable too, but only once its lease has expired. A relay that dies
-- mid-publish leaves rows in that state; excluding them would strand those messages
-- forever, which is the one failure mode the outbox exists to prevent.
WHERE status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT')
AND (lease_expires_at IS NULL OR lease_expires_at <= ?)
ORDER BY created_at
LIMIT ?
FOR UPDATE SKIP LOCKED
)
UPDATE messaging_outbox o
SET status = 'IN_FLIGHT', lease_expires_at = ?
FROM claimable c
WHERE o.message_id = c.message_id
RETURNING o.message_id, o.destination, o.message_type, o.schema_version, o.content_type,
o.payload, o.headers, o.created_at, o.status, o.attempts,
o.lease_expires_at, o.last_failure_code,
o.producer, o.occurred_at, o.correlation_id, o.causation_id, o.tenant,
o.partition_key, o.ordering_key, o.schema_subject, o.schema_uri,
o.traceparent, o.tracestate, o.baggage
""";
/**
* The claim that records who holds the row and increments its fencing token.
*
@@ -147,7 +125,11 @@ public final class JdbcOutboxRepository implements OutboxRepository {
FROM claimable c
WHERE o.message_id = c.message_id
RETURNING o.message_id, o.destination, o.message_type, o.schema_version, o.content_type,
o.payload, o.headers, o.created_at, o.status, o.attempts,
o.payload,
ARRAY(SELECT h.key FROM jsonb_each_text(o.headers) h ORDER BY h.key) AS header_keys,
ARRAY(SELECT h.value FROM jsonb_each_text(o.headers) h ORDER BY h.key)
AS header_values,
o.created_at, o.status, o.attempts,
o.lease_expires_at, o.last_failure_code, o.lease_owner, o.lease_token,
o.producer, o.occurred_at, o.correlation_id, o.causation_id, o.tenant,
o.partition_key, o.ordering_key, o.schema_subject, o.schema_uri,
@@ -187,7 +169,7 @@ public final class JdbcOutboxRepository implements OutboxRepository {
statement.setInt(4, record.schemaVersion().value());
statement.setString(5, record.contentType().value());
statement.setBytes(6, record.payload());
statement.setString(7, toJson(record.headers()));
statement.setArray(7, connection.createArrayOf("text", flatten(record.headers())));
statement.setTimestamp(8, Timestamp.from(record.createdAt()));
statement.setString(9, record.status().name());
statement.setInt(10, record.attempts());
@@ -258,33 +240,6 @@ public final class JdbcOutboxRepository implements OutboxRepository {
}
}
@Override
public List<OutboxRecord> leaseBatch(int batchSize, Duration leaseDuration, Instant now) {
Objects.requireNonNull(leaseDuration, "leaseDuration must not be null");
Objects.requireNonNull(now, "now must not be null");
return withConnection(
connection -> {
List<OutboxRecord> leased = new ArrayList<>();
try (PreparedStatement statement = connection.prepareStatement(LEASE)) {
statement.setTimestamp(1, Timestamp.from(now));
statement.setInt(2, batchSize);
statement.setTimestamp(3, Timestamp.from(now.plus(leaseDuration)));
try (ResultSet results = statement.executeQuery()) {
while (results.next()) {
leased.add(read(results));
}
}
}
return List.copyOf(leased);
});
}
@Override
public List<OutboxLease> claimBatch(
String owner, int batchSize, Duration leaseDuration, Instant now) {
return claimBatch(owner, batchSize, leaseDuration, now, Integer.MAX_VALUE);
}
@Override
public List<OutboxLease> claimBatch(
String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts) {
@@ -328,11 +283,6 @@ public final class JdbcOutboxRepository implements OutboxRepository {
null);
}
@Override
public OutboxTransitionResult markAmbiguous(OutboxLease lease, String failureCode, Instant now) {
return markAmbiguous(lease, failureCode, now, now);
}
@Override
public OutboxTransitionResult markAmbiguous(
OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt) {
@@ -435,49 +385,6 @@ public final class JdbcOutboxRepository implements OutboxRepository {
});
}
public void markPublishedUnfenced(MessageId messageId, Instant now) {
update(
"UPDATE messaging_outbox SET status = 'PUBLISHED', published_at = ?, "
+ "lease_expires_at = NULL, attempts = attempts + 1 WHERE message_id = ?",
messageId,
now,
null);
}
public void markAmbiguousUnfenced(MessageId messageId, String failureCode, Instant now) {
// The lease is released rather than held: an ambiguous row must become claimable again so the
// relay retries it under the same message id.
update(
"UPDATE messaging_outbox SET status = 'AMBIGUOUS', lease_expires_at = NULL, "
+ "last_failure_code = ?, attempts = attempts + 1 WHERE message_id = ?",
messageId,
now,
failureCode);
}
public void markFailedUnfenced(MessageId messageId, String failureCode, Instant now) {
update(
"UPDATE messaging_outbox SET status = 'FAILED', lease_expires_at = NULL, "
+ "last_failure_code = ?, attempts = attempts + 1 WHERE message_id = ?",
messageId,
now,
failureCode);
}
public void releaseLeaseUnfenced(MessageId messageId) {
withConnection(
connection -> {
try (PreparedStatement statement =
connection.prepareStatement(
"UPDATE messaging_outbox SET status = 'PENDING', lease_expires_at = NULL "
+ "WHERE message_id = ? AND status = 'IN_FLIGHT'")) {
statement.setObject(1, messageId.value());
statement.executeUpdate();
}
return null;
});
}
@Override
public Optional<OutboxRecord> find(MessageId messageId) {
return withConnection(
@@ -526,22 +433,6 @@ public final class JdbcOutboxRepository implements OutboxRepository {
});
}
private void update(String sql, MessageId messageId, Instant now, String failureCode) {
withConnection(
connection -> {
try (PreparedStatement statement = connection.prepareStatement(sql)) {
if (failureCode == null) {
statement.setTimestamp(1, Timestamp.from(now));
} else {
statement.setString(1, failureCode);
}
statement.setObject(2, messageId.value());
statement.executeUpdate();
}
return null;
});
}
private static OutboxRecord read(ResultSet results) throws SQLException {
Timestamp lease = results.getTimestamp("lease_expires_at");
return new OutboxRecord(
@@ -551,7 +442,7 @@ public final class JdbcOutboxRepository implements OutboxRepository {
new SchemaVersion(results.getInt("schema_version")),
new ContentType(results.getString("content_type")),
results.getBytes("payload"),
fromJson(results.getString("headers")),
headers(results.getArray("header_keys"), results.getArray("header_values")),
readMetadata(results),
results.getTimestamp("created_at").toInstant(),
OutboxStatus.valueOf(results.getString("status")),
@@ -597,119 +488,36 @@ public final class JdbcOutboxRepository implements OutboxRepository {
}
/**
* Serialises headers as a flat JSON object.
* The headers as one flat key, value, key, value array for {@code jsonb_object(text[])}.
*
* <p>Hand-rolled rather than pulled from a JSON library so this module keeps no codec dependency:
* outbox headers are always flat string pairs, validated by {@code MessageHeaders} before they
* ever reach here.
* <p>PostgreSQL builds and escapes the JSON. The hand-rolled writer and reader this replaces got
* the escaping right one way and wrong the other: the reader took a quote after an escaped
* backslash for an escaped quote, so a value ending in a backslash ran on into the next header.
*/
private static String toJson(Map<String, String> headers) {
StringBuilder json = new StringBuilder("{");
boolean first = true;
private static String[] flatten(Map<String, String> headers) {
String[] flat = new String[headers.size() * 2];
int index = 0;
for (Map.Entry<String, String> entry : headers.entrySet()) {
if (!first) {
json.append(',');
}
first = false;
json.append('"')
.append(escape(entry.getKey()))
.append("\":\"")
.append(escape(entry.getValue()))
.append('"');
flat[index++] = entry.getKey();
flat[index++] = entry.getValue();
}
return json.append('}').toString();
return flat;
}
private static Map<String, String> fromJson(String json) {
/** Rebuilds the headers from the aligned key and value arrays the reads select. */
private static Map<String, String> headers(Array keys, Array values) throws SQLException {
Map<String, String> headers = new LinkedHashMap<>();
if (json == null || json.length() <= 2) {
if (keys == null || values == null) {
return headers;
}
String body = json.substring(1, json.length() - 1);
int index = 0;
while (index < body.length()) {
int keyStart = body.indexOf('"', index);
if (keyStart < 0) {
break;
}
int keyEnd = findClosingQuote(body, keyStart + 1);
int valueStart = body.indexOf('"', body.indexOf(':', keyEnd));
int valueEnd = findClosingQuote(body, valueStart + 1);
headers.put(
unescape(body.substring(keyStart + 1, keyEnd)),
unescape(body.substring(valueStart + 1, valueEnd)));
index = valueEnd + 1;
String[] names = (String[]) keys.getArray();
String[] texts = (String[]) values.getArray();
for (int index = 0; index < names.length; index++) {
headers.put(names[index], texts[index]);
}
return headers;
}
private static int findClosingQuote(String text, int from) {
for (int index = from; index < text.length(); index++) {
if (text.charAt(index) == '"' && text.charAt(index - 1) != '\\') {
return index;
}
}
return text.length();
}
/**
* JSON string escaping, including the control characters.
*
* <p>Only backslash and quote were escaped. A header value containing a newline or a NUL — which
* the value objects now refuse, but which older rows and other writers may still carry — produced
* a document PostgreSQL rejects as invalid JSONB, so the append failed at the database with an
* error naming neither the header nor the row. Control characters are the part of the JSON
* grammar that is easy to forget and impossible to get away with.
*/
private static String escape(String value) {
StringBuilder escaped = new StringBuilder(value.length() + 8);
for (int index = 0; index < value.length(); index++) {
char character = value.charAt(index);
switch (character) {
case '\\' -> escaped.append("\\\\");
case '"' -> escaped.append("\\\"");
case '\n' -> escaped.append("\\n");
case '\r' -> escaped.append("\\r");
case '\t' -> escaped.append("\\t");
case '\b' -> escaped.append("\\b");
case '\f' -> escaped.append("\\f");
default -> {
if (character < 0x20) {
escaped.append("\\u%04x".formatted((int) character));
} else {
escaped.append(character);
}
}
}
}
return escaped.toString();
}
private static String unescape(String value) {
StringBuilder plain = new StringBuilder(value.length());
for (int index = 0; index < value.length(); index++) {
char character = value.charAt(index);
if (character != '\\' || index == value.length() - 1) {
plain.append(character);
continue;
}
char next = value.charAt(++index);
switch (next) {
case 'n' -> plain.append('\n');
case 'r' -> plain.append('\r');
case 't' -> plain.append('\t');
case 'b' -> plain.append('\b');
case 'f' -> plain.append('\f');
case 'u' -> {
plain.append((char) Integer.parseInt(value.substring(index + 1, index + 5), 16));
index += 4;
}
default -> plain.append(next);
}
}
return plain.toString();
}
/** A unit of work against a borrowed connection. */
@FunctionalInterface
private interface ConnectionCallback<T> {
@@ -0,0 +1,19 @@
plugins {
id 'ca.platform-module'
}
// The outbox relay and the inbox consumer logic, framework-free. What they store and claim goes
// through the ports in messaging-reliability-api; the relational implementation of those ports is
// messaging-reliability-jdbc-postgresql. Split so a leaf's name says what it depends on: this one
// is Java stdlib plus the messaging APIs, the other is Spring JDBC and PostgreSQL SQL.
dependencies {
api project(':messaging:messaging-core-api')
api project(':messaging:messaging-reliability-api')
// The relay hands encoded rows to the publisher as EncodedMessage.
api project(':messaging:messaging-schema-api')
// Test-only. The relay is also driven through the real DefaultMessagePublisher: a fake publisher
// accepted the pre-encoded rows the real one re-encoded and rejected.
testImplementation project(':messaging:messaging-runtime-core')
testImplementation project(':messaging:messaging-policy')
}
@@ -0,0 +1,89 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor
com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs
com.github.spotbugs:spotbugs:4.10.2=spotbugs
com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor
com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor
com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs
com.google.code.gson:gson:2.13.2=spotbugs
com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.41.0=spotbugs
com.google.errorprone:error_prone_annotations:2.47.0=checkstyle
com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor
com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor
com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor
com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor
com.google.guava:guava:33.6.0-jre=checkstyle
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor
com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor
com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor
com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins
com.puppycrawl.tools:checkstyle:13.5.0=checkstyle
commons-beanutils:commons-beanutils:1.11.0=checkstyle
commons-collections:commons-collections:3.2.2=checkstyle
commons-io:commons-io:2.21.0=spotbugs
info.picocli:picocli:4.7.7=checkstyle
io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor
io.micrometer:micrometer-commons:1.16.7=testCompileClasspath,testRuntimeClasspath
io.micrometer:micrometer-core:1.16.7=testCompileClasspath,testRuntimeClasspath
io.micrometer:micrometer-observation:1.16.7=testCompileClasspath,testRuntimeClasspath
javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor
jaxen:jaxen:2.0.6=spotbugs
net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath
net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs
org.antlr:antlr4-runtime:4.13.2=checkstyle
org.apache.bcel:bcel:6.12.0=spotbugs
org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs
org.apache.commons:commons-text:1.15.0=spotbugs
org.apache.commons:commons-text:1.3=checkstyle
org.apache.httpcomponents:httpclient:4.5.13=checkstyle
org.apache.httpcomponents:httpcore:4.4.16=checkstyle
org.apache.logging.log4j:log4j-api:2.25.5=spotbugs
org.apache.logging.log4j:log4j-core:2.25.5=spotbugs
org.apache.maven.doxia:doxia-core:1.12.0=checkstyle
org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle
org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle
org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle
org.apache.xbean:xbean-reflect:3.7=checkstyle
org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath
org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath
org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle
org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle
org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle
org.codehaus.plexus:plexus-utils:3.3.0=checkstyle
org.dom4j:dom4j:2.2.0=spotbugs
org.hdrhistogram:HdrHistogram:2.2.2=testRuntimeClasspath
org.javassist:javassist:3.28.0-GA=checkstyle
org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath
org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath
org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath
org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath
org.junit:junit-bom:6.1.0=spotbugs
org.latencyutils:LatencyUtils:2.0.3=testRuntimeClasspath
org.mockito:mockito-core:5.20.0=mockitoAgent
org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-analysis:9.10.1=spotbugs
org.ow2.asm:asm-commons:9.10.1=spotbugs
org.ow2.asm:asm-tree:9.10.1=spotbugs
org.ow2.asm:asm-util:9.10.1=spotbugs
org.ow2.asm:asm:9.10.1=spotbugs
org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor
org.reflections:reflections:0.10.2=checkstyle
org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j
org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j
org.slf4j:slf4j-simple:2.0.18=checkstyle
org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs
empty=compileClasspath,runtimeClasspath
@@ -158,7 +158,7 @@ public final class OutboxRelay {
*/
public OutboxRelayReport runOnce(Instant now) {
Objects.requireNonNull(now, "now must not be null");
// claimBatch, not leaseBatch: the claim carries an owner and a fencing token, and every write
// The claim carries an owner and a fencing token, and every write
// below is conditioned on them. Without that, a relay that stalled past its lease could record
// AMBIGUOUS over the PUBLISHED another relay had already written, making a confirmed message
// claimable again.
@@ -35,17 +35,6 @@ class OutboxOperationsTest {
throw new UnsupportedOperationException();
}
@Override
public List<OutboxRecord> leaseBatch(int batchSize, Duration leaseDuration, Instant now) {
throw new UnsupportedOperationException();
}
@Override
public java.util.List<dev.caskeleton.messaging.reliability.OutboxLease> claimBatch(
String owner, int batchSize, Duration leaseDuration, Instant now) {
return java.util.List.of();
}
@Override
public java.util.List<dev.caskeleton.messaging.reliability.OutboxLease> claimBatch(
String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts) {
@@ -58,7 +47,7 @@ class OutboxOperationsTest {
String failureCode,
Instant now,
Instant nextAttemptAt) {
return markAmbiguous(lease, failureCode, now);
throw new UnsupportedOperationException();
}
@Override
@@ -73,12 +62,6 @@ class OutboxOperationsTest {
throw new UnsupportedOperationException();
}
@Override
public dev.caskeleton.messaging.reliability.OutboxTransitionResult markAmbiguous(
dev.caskeleton.messaging.reliability.OutboxLease lease, String failureCode, Instant now) {
throw new UnsupportedOperationException();
}
@Override
public dev.caskeleton.messaging.reliability.OutboxTransitionResult markFailed(
dev.caskeleton.messaging.reliability.OutboxLease lease, String failureCode, Instant now) {
@@ -199,15 +199,18 @@ class OutboxRelayTest {
assertThat(repository.find(record.messageId()).orElseThrow().status())
.isEqualTo(OutboxStatus.FAILED);
assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW)).isEmpty();
assertThat(repository.claimBatch("relay", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE))
.isEmpty();
}
@Test
void aLeasedRowIsInvisibleToAnotherRelayInstance() {
repository.append(OutboxFixtures.pending());
List<OutboxRecord> first = repository.leaseBatch(10, Duration.ofSeconds(30), NOW);
List<OutboxRecord> second = repository.leaseBatch(10, Duration.ofSeconds(30), NOW);
List<OutboxLease> first =
repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE);
List<OutboxLease> second =
repository.claimBatch("relay-b", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE);
assertThat(first).hasSize(1);
assertThat(second).isEmpty();
@@ -216,10 +219,11 @@ class OutboxRelayTest {
@Test
void anExpiredLeaseBecomesClaimableAgain() {
repository.append(OutboxFixtures.pending());
repository.leaseBatch(10, Duration.ofSeconds(30), NOW);
repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE);
List<OutboxRecord> reclaimed =
repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(31));
List<OutboxLease> reclaimed =
repository.claimBatch(
"relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31), Integer.MAX_VALUE);
assertThat(reclaimed).hasSize(1);
}
@@ -352,8 +356,7 @@ final class InMemoryOutboxRepository implements OutboxRepository {
* thing it stands in for hides exactly the defects it was written to catch, so this one follows
* the same three-state rule.
*/
@Override
public List<OutboxRecord> leaseBatch(int batchSize, Duration leaseDuration, Instant now) {
private List<OutboxRecord> lease(int batchSize, Duration leaseDuration, Instant now) {
List<OutboxRecord> leased = new ArrayList<>();
for (OutboxRecord record : List.copyOf(rows.values())) {
if (leased.size() >= batchSize) {
@@ -376,11 +379,10 @@ final class InMemoryOutboxRepository implements OutboxRepository {
return List.copyOf(leased);
}
@Override
public List<OutboxLease> claimBatch(
private List<OutboxLease> claimIgnoringBudget(
String owner, int batchSize, Duration leaseDuration, Instant now) {
List<OutboxLease> claimed = new ArrayList<>();
for (OutboxRecord record : leaseBatch(batchSize, leaseDuration, now)) {
for (OutboxRecord record : lease(batchSize, leaseDuration, now)) {
long token = tokens.merge(record.messageId(), 1L, Long::sum);
owners.put(record.messageId(), owner);
claimed.add(new OutboxLease(record, owner, token, now.plus(leaseDuration)));
@@ -408,7 +410,7 @@ final class InMemoryOutboxRepository implements OutboxRepository {
public List<OutboxLease> claimBatch(
String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts) {
List<OutboxLease> claimed = new ArrayList<>();
for (OutboxLease lease : claimBatch(owner, batchSize, leaseDuration, now)) {
for (OutboxLease lease : claimIgnoringBudget(owner, batchSize, leaseDuration, now)) {
Instant nextAttempt = nextAttempts.get(lease.messageId());
boolean waiting = nextAttempt != null && now.isBefore(nextAttempt);
if (waiting || lease.record().attempts() >= maxAttempts) {
@@ -424,11 +426,13 @@ final class InMemoryOutboxRepository implements OutboxRepository {
@Override
public OutboxTransitionResult markAmbiguous(
OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt) {
OutboxTransitionResult result = markAmbiguous(lease, failureCode, now);
if (result == OutboxTransitionResult.APPLIED) {
nextAttempts.put(lease.messageId(), nextAttemptAt);
if (!owns(lease)) {
return OutboxTransitionResult.STALE_LEASE;
}
return result;
markAmbiguousUnfenced(lease.messageId(), failureCode);
owners.remove(lease.messageId());
nextAttempts.put(lease.messageId(), nextAttemptAt);
return OutboxTransitionResult.APPLIED;
}
@Override
@@ -452,16 +456,6 @@ final class InMemoryOutboxRepository implements OutboxRepository {
return OutboxTransitionResult.APPLIED;
}
@Override
public OutboxTransitionResult markAmbiguous(OutboxLease lease, String failureCode, Instant now) {
if (!owns(lease)) {
return OutboxTransitionResult.STALE_LEASE;
}
markAmbiguousUnfenced(lease.messageId(), failureCode);
owners.remove(lease.messageId());
return OutboxTransitionResult.APPLIED;
}
@Override
public OutboxTransitionResult markFailed(OutboxLease lease, String failureCode, Instant now) {
if (!owns(lease)) {
@@ -31,8 +31,9 @@ dependencies {
// which is why MessagingCoreAutoConfiguration consumed a MessagePublisher that nothing in
// reach could produce (MSG-INT-003).
implementation project(':messaging:messaging-runtime-core')
implementation project(':messaging:messaging-outbox-jdbc-postgresql')
implementation project(':messaging:messaging-inbox-jdbc-postgresql')
// The relay and consumer logic, and the PostgreSQL stores behind their ports.
implementation project(':messaging:messaging-reliability')
implementation project(':messaging:messaging-reliability-jdbc-postgresql')
implementation project(':messaging:messaging-admin-api')
implementation project(':messaging:messaging-admin-runtime')
@@ -171,28 +171,11 @@ class MessagingOutboxRelayLifecycleTest {
throw new UnsupportedOperationException("not used by this test");
}
@Override
public List<OutboxRecord> leaseBatch(int batchSize, Duration leaseDuration, Instant now) {
throw new UnsupportedOperationException("not used by this test");
}
@Override
public List<OutboxLease> claimBatch(
String owner, int batchSize, Duration leaseDuration, Instant now) {
throw new UnsupportedOperationException("not used by this test");
}
@Override
public OutboxTransitionResult markPublished(OutboxLease lease, Instant now) {
throw new UnsupportedOperationException("not used by this test");
}
@Override
public OutboxTransitionResult markAmbiguous(
OutboxLease lease, String failureCode, Instant now) {
throw new UnsupportedOperationException("not used by this test");
}
@Override
public OutboxTransitionResult markAmbiguous(
OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt) {