From 5e44be9ab4a4b4d1e2db2954be122397b3c42a18 Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Fri, 25 Sep 2026 12:49:07 +0900 Subject: [PATCH 1/7] =?UTF-8?q?refactor:=20=EB=B6=80=EA=B0=80=20=EA=B8=B0?= =?UTF-8?q?=EB=8A=A5=EC=9D=84=20capability=20=EB=AA=A8=EB=93=88=EB=A1=9C?= =?UTF-8?q?=20=ED=8F=AC=EC=9E=A5=ED=95=98=EA=B3=A0=20=EA=B8=B0=EB=B3=B8=20?= =?UTF-8?q?=EC=A1=B0=EB=A6=BD=EC=9D=84=20web-only=EB=A1=9C=20=EC=A0=84?= =?UTF-8?q?=ED=99=98?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 설계: docs/superpowers/specs/2026-09-25-capability-module-packaging-design.md 계획/진행 기록: docs/superpowers/plans/2026-09-25-capability-module-packaging.md - Wave 0: 규칙 고정(AGENTS.md capability 완료 기준·테스트 규칙) + 차단 결함 7건 (compose overlay 검사, redis-sdk CI job, 적용된 V9/V13 원복 + V16, @EnableScheduling, messaging bridge 순서, prod 에러 상세 validator, sample loggers read-only) - Wave 1: 데이터·보안 Critical 10건 (durable operation 25P02, operation 유스케이스 이동, outbox 재인코딩, Kafka TLS/SASL, 알림 트랜잭션, SNS 서명, tus checksum, GraphQL fragment, Redis lane, gRPC FAILED_TERMINAL) - Wave 2: 리팩터링 잔재·실험용 자리 삭제, outbox v1 fencing(V17), 멱등성 V2 정본화 - Wave 3: 라이브러리형 코드 정리(동작하지 않는 설정·인터페이스·gate 제거, web-advanced 분리) - Wave 4: capability 11개를 "있으면 켜짐" 모듈로 포장, 끄는 장치 제거, 기본 조립 web-only, qualificationBootJar 분리 - Wave 5: 빌드 로직 축소(17.2k -> 10.8k줄), sample 테스트 컨벤션 적용 삭제한 코드는 docs/capability-status.md "제거된 코드" 표에 복구 커밋(b5114b9b)과 함께 기록. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/messaging/configuration-reference.md | 57 +-- docs/messaging/cutover.md | 5 +- docs/messaging/experimental-policy.md | 57 +-- docs/messaging/operations.md | 3 +- docs/messaging/outbox-inbox.md | 10 +- docs/messaging/support-matrix.md | 72 ++-- messaging/CLAUDE.md | 26 +- .../build.gradle | 11 - .../gradle.lockfile | 93 ----- .../kafka/share/KafkaShareGroupRegistrar.java | 48 --- .../kafka/share/KafkaShareProfile.java | 33 -- .../share/KafkaShareProfileValidator.java | 42 --- .../share/KafkaShareWorkQueueCapability.java | 27 -- .../share/KafkaShareProfileValidatorTest.java | 113 ------ messaging/messaging-kafka/build.gradle | 36 +- messaging/messaging-kafka/gradle.lockfile | 29 +- .../messaging-nats-experimental/build.gradle | 14 - .../gradle.lockfile | 89 ----- .../messaging/nats/NatsAckMode.java | 57 --- .../messaging/nats/NatsJetStreamProfile.java | 103 ------ .../nats/NatsJetStreamProfileValidator.java | 75 ---- .../nats/NatsJetStreamTransport.java | 295 --------------- .../nats/NatsMaxDeliverParkingWorkflow.java | 85 ----- .../messaging/nats/NatsPreSendRejection.java | 65 ---- .../messaging/nats/NatsStreamPosition.java | 75 ---- .../nats/NatsAdapterContractTest.java | 337 ----------------- .../nats/NatsMaxDeliverParkingTest.java | 123 ------- .../build.gradle | 3 + .../messaging/outbox/OutboxPostgresIT.java | 14 +- .../outbox/DebeziumMappedRecord.java | 54 --- .../outbox/DebeziumOutboxEventRouter.java | 151 -------- .../outbox/DebeziumOutboxProfile.java | 67 ---- .../outbox/DebeziumOutboxRecordMapper.java | 79 ---- .../debezium/outbox-event-router.properties | 43 --- .../DebeziumOutboxRecordMapperTest.java | 199 ---------- .../messaging/outbox/OutboxRelayTest.java | 168 +++++++++ .../build.gradle | 14 - .../gradle.lockfile | 89 ----- .../pulsar/PulsarMessagePosition.java | 49 --- .../pulsar/PulsarMessagingTransport.java | 275 -------------- .../pulsar/PulsarPreSendRejection.java | 65 ---- .../messaging/pulsar/PulsarProfile.java | 80 ---- .../pulsar/PulsarProfileValidator.java | 66 ---- .../pulsar/PulsarSubscriptionMode.java | 62 ---- .../pulsar/PulsarSubscriptionType.java | 17 - .../pulsar/PulsarTransactionCapability.java | 49 --- .../pulsar/PulsarAdapterContractTest.java | 289 --------------- .../pulsar/PulsarSubscriptionGuardTest.java | 125 ------- messaging/messaging-rabbit/build.gradle | 43 --- messaging/messaging-rabbit/gradle.lockfile | 128 ------- .../messaging/rabbit/RabbitBrokerIT.java | 232 ------------ .../rabbit/RabbitFixtureProfiles.java | 32 -- .../rabbit/RabbitPublishBenchmark.java | 156 -------- .../rabbit/RabbitBatchConsumerRegistrar.java | 165 --------- .../messaging/rabbit/RabbitBrokerProfile.java | 50 --- .../rabbit/RabbitChannelPublisher.java | 42 --- .../rabbit/RabbitConfirmCoordinator.java | 279 -------------- .../rabbit/RabbitConsumerRegistrar.java | 238 ------------ .../rabbit/RabbitDeadLetterPublisher.java | 130 ------- .../rabbit/RabbitDeliveryMapper.java | 195 ---------- .../messaging/rabbit/RabbitHeaderMapper.java | 180 --------- .../rabbit/RabbitMessagingTransport.java | 286 --------------- .../RabbitNativeDeadLetterCapability.java | 70 ---- .../rabbit/RabbitProfileValidator.java | 83 ----- .../RabbitPublishFailureClassifier.java | 133 ------- .../messaging/rabbit/RabbitPublishMapper.java | 72 ---- .../rabbit/RabbitPublishReference.java | 37 -- .../messaging/rabbit/RabbitRequestReply.java | 32 -- .../rabbit/RabbitRetryQueueTopology.java | 57 --- .../rabbit/RabbitSecurityConfigurer.java | 179 --------- .../rabbit/RabbitSettlementController.java | 84 ----- .../rabbit/RabbitSettlementOperations.java | 50 --- .../rabbit/RabbitTopologyProfile.java | 137 ------- .../rabbit/RabbitAdapterContractTest.java | 24 -- .../rabbit/RabbitConfirmCoordinatorTest.java | 167 --------- .../rabbit/RabbitContractHarness.java | 304 ---------------- .../rabbit/RabbitEnvelopeRoundTripTest.java | 152 -------- .../rabbit/RabbitProfileValidatorTest.java | 68 ---- .../messaging/rabbit/RabbitRuntimeTest.java | 341 ------------------ .../RabbitSettlementControllerTest.java | 105 ------ .../rabbit/RabbitTopologyAndBatchTest.java | 253 ------------- .../rabbit/RabbitProfileFixtures.java | 118 ------ .../runtime/DefaultMessagePublisher.java | 8 + .../runtime/DefaultMessagePublisherTest.java | 55 +++ .../gradle.lockfile | 2 - .../KafkaMessagingAutoConfiguration.java | 92 ++++- .../build.gradle | 12 - .../gradle.lockfile | 124 ------- .../RabbitMessagingAutoConfiguration.java | 126 ------- .../messaging-spring-boot-starter/README.md | 30 +- .../build.gradle | 17 +- .../gradle.lockfile | 21 +- .../autoconfigure/BrokerSettings.java | 31 +- .../MessagingConfigurationCompiler.java | 12 +- .../MessagingConfigurationKeyValidator.java | 52 --- .../MessagingCoreAutoConfiguration.java | 1 - ...essagingPlatformRootAutoConfiguration.java | 15 +- .../MessagingProviderSelection.java | 63 +--- .../autoconfigure/MessagingSettings.java | 111 ------ .../KafkaProducerSecurityTest.java | 151 ++++++++ .../MessagingAutoConfigurationTest.java | 14 - .../MessagingConfigurationBindingTest.java | 25 +- .../MessagingStarterOffContractTest.java | 113 ++---- .../src/test/resources/application-valid.yml | 10 - .../build.gradle | 10 - .../gradle.lockfile | 84 ----- .../streambridge/BindingCapabilityReport.java | 88 ----- .../streambridge/BindingProfileValidator.java | 97 ----- .../streambridge/MessagingBindingBridge.java | 34 -- .../SpringCloudStreamConsumerBridge.java | 87 ----- .../SpringCloudStreamPublisherBridge.java | 156 -------- .../streambridge/StreamBridgePolicyGuard.java | 52 --- .../BindingProfileValidatorTest.java | 164 --------- .../BridgePublishEvidenceTest.java | 131 ------- messaging/messaging-testkit/build.gradle | 8 +- messaging/messaging-testkit/gradle.lockfile | 11 +- .../testkit/CompatibilityMatrix.java | 41 +-- .../testkit/MessagingAdapterHarness.java | 4 +- .../testkit/CertifiedEvidenceTest.java | 4 +- .../testkit/CompatibilityMatrixTest.java | 2 - .../testkit/CrossBrokerContractSuite.java | 17 +- .../MessagingDocumentationContractTest.java | 4 - 122 files changed, 725 insertions(+), 9914 deletions(-) delete mode 100644 messaging/messaging-kafka-share-experimental/build.gradle delete mode 100644 messaging/messaging-kafka-share-experimental/gradle.lockfile delete mode 100644 messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareGroupRegistrar.java delete mode 100644 messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfile.java delete mode 100644 messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidator.java delete mode 100644 messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareWorkQueueCapability.java delete mode 100644 messaging/messaging-kafka-share-experimental/src/test/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidatorTest.java delete mode 100644 messaging/messaging-nats-experimental/build.gradle delete mode 100644 messaging/messaging-nats-experimental/gradle.lockfile delete mode 100644 messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsAckMode.java delete mode 100644 messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfile.java delete mode 100644 messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfileValidator.java delete mode 100644 messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamTransport.java delete mode 100644 messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingWorkflow.java delete mode 100644 messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsPreSendRejection.java delete mode 100644 messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsStreamPosition.java delete mode 100644 messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsAdapterContractTest.java delete mode 100644 messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingTest.java delete mode 100644 messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumMappedRecord.java delete mode 100644 messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxEventRouter.java delete mode 100644 messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxProfile.java delete mode 100644 messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapper.java delete mode 100644 messaging/messaging-outbox-jdbc-postgresql/src/main/resources/debezium/outbox-event-router.properties delete mode 100644 messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapperTest.java delete mode 100644 messaging/messaging-pulsar-experimental/build.gradle delete mode 100644 messaging/messaging-pulsar-experimental/gradle.lockfile delete mode 100644 messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagePosition.java delete mode 100644 messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagingTransport.java delete mode 100644 messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarPreSendRejection.java delete mode 100644 messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfile.java delete mode 100644 messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfileValidator.java delete mode 100644 messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionMode.java delete mode 100644 messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionType.java delete mode 100644 messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarTransactionCapability.java delete mode 100644 messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarAdapterContractTest.java delete mode 100644 messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionGuardTest.java delete mode 100644 messaging/messaging-rabbit/build.gradle delete mode 100644 messaging/messaging-rabbit/gradle.lockfile delete mode 100644 messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitBrokerIT.java delete mode 100644 messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitFixtureProfiles.java delete mode 100644 messaging/messaging-rabbit/src/jmh/java/dev/caskeleton/messaging/rabbit/RabbitPublishBenchmark.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBatchConsumerRegistrar.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBrokerProfile.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitChannelPublisher.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinator.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConsumerRegistrar.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeadLetterPublisher.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeliveryMapper.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitHeaderMapper.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitMessagingTransport.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitNativeDeadLetterCapability.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidator.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishFailureClassifier.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishMapper.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishReference.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRequestReply.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRetryQueueTopology.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSecurityConfigurer.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementController.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementOperations.java delete mode 100644 messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitTopologyProfile.java delete mode 100644 messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitAdapterContractTest.java delete mode 100644 messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinatorTest.java delete mode 100644 messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitContractHarness.java delete mode 100644 messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitEnvelopeRoundTripTest.java delete mode 100644 messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidatorTest.java delete mode 100644 messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitRuntimeTest.java delete mode 100644 messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitSettlementControllerTest.java delete mode 100644 messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitTopologyAndBatchTest.java delete mode 100644 messaging/messaging-rabbit/src/testFixtures/java/dev/caskeleton/messaging/rabbit/RabbitProfileFixtures.java delete mode 100644 messaging/messaging-spring-boot-starter-rabbit/build.gradle delete mode 100644 messaging/messaging-spring-boot-starter-rabbit/gradle.lockfile delete mode 100644 messaging/messaging-spring-boot-starter-rabbit/src/main/java/dev/caskeleton/messaging/autoconfigure/RabbitMessagingAutoConfiguration.java create mode 100644 messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java delete mode 100644 messaging/messaging-spring-cloud-stream-bridge/build.gradle delete mode 100644 messaging/messaging-spring-cloud-stream-bridge/gradle.lockfile delete mode 100644 messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingCapabilityReport.java delete mode 100644 messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingProfileValidator.java delete mode 100644 messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/MessagingBindingBridge.java delete mode 100644 messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamConsumerBridge.java delete mode 100644 messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamPublisherBridge.java delete mode 100644 messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/StreamBridgePolicyGuard.java delete mode 100644 messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BindingProfileValidatorTest.java delete mode 100644 messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BridgePublishEvidenceTest.java diff --git a/docs/messaging/configuration-reference.md b/docs/messaging/configuration-reference.md index ea220b4..d4dc00d 100644 --- a/docs/messaging/configuration-reference.md +++ b/docs/messaging/configuration-reference.md @@ -28,7 +28,7 @@ producing-service identity다. `APP_OUTBOX_CANONICAL_TRANSPORT_ENABLED` / `ca-skeleton.outbox.canonical-transport-enabled`은 기존 `outbox_event` writer/claim/status authority를 유지한 채 canonical row의 **transport만** platform -publish path로 보내는 compatibility gate다. 기본값은 `false`이며 `POLLING_V2`를 활성화하지 않는다. +publish path로 보내는 compatibility gate다. 기본값은 `false`이며 두 번째 relay를 만들지 않는다. `true`일 때는 `OutboxAppendPort`가 `ValidatedIntegrationEvent`의 exact envelope bytes와 canonical metadata를 기존 outbox row에 저장하고, claim된 canonical row는 `IntegrationEventPublishPort`로 간다. @@ -135,9 +135,10 @@ app: ## Broker profile -브로커는 `app.messaging.brokers` 아래에 한 번만 기술한다. `type`이 어느 계열의 설정이 적용되는지 -결정하며, 다른 계열의 키(Kafka 항목의 `prefetch` 같은)는 무시되지 않고 부팅 실패로 거절된다 — -무시하면 그 줄을 쓴 사람은 무언가가 적용됐다고 믿게 된다. +브로커는 `app.messaging.brokers` 아래에 한 번만 기술한다. `type`은 `kafka`만 받는다 — transport가 +있는 계열이 Kafka 하나뿐이다. RabbitMQ 키(`addresses`, `prefetch`, `quorum-queues` 등)와 +`type: rabbitmq`는 2026-09-25에 설정 모델에서 삭제됐다. 연결할 transport 없이 바인딩만 되던 키라, +지금은 무시되지 않고 키 이름을 담아 부팅 실패로 거절된다. ### Kafka @@ -160,30 +161,9 @@ app: authentication-enabled: false # production이면 필수 ``` -### RabbitMQ - -```yaml -app: - messaging: - brokers: - rabbit-primary: - type: rabbitmq - stable: true - production: false - addresses: [rabbit-1:5671] - publisher-confirms: true # stable에서 필수 - publisher-returns: true # stable에서 필수 - mandatory: true # stable에서 필수 - confirm-timeout: 5s - auto-ack: false # 항상 금지 - prefetch: 16 - quorum-queues: true # durable work queue 필수 - tls-enabled: false - authentication-enabled: false -``` - -`production: true`인 브로커는 `tls-enabled`와 `authentication-enabled`가 모두 참이어야 하고, -그렇지 않으면 `KafkaProfileValidator` / `RabbitProfileValidator`가 부팅을 거절한다. 위 예시가 +`production: true`인 Kafka 브로커는 `tls-enabled`와 `authentication-enabled`가 모두 참이어야 하고, +그렇지 않으면 `KafkaProfileValidator`가 부팅을 거절한다. `app.messaging.broker`로 고를 수 있는 값은 +`kafka`뿐이다. 위 예시가 `production: false`인 것은 이 페이지가 그대로 실행되는 fixture이기 때문이며, 실 배포는 셋 다 참이다. ## 보안 @@ -211,20 +191,13 @@ app: 연결 시점에 해석하므로, 설정 덤프나 힙 덤프에서 나오는 것은 이름뿐이다. producer와 consumer는 서로 다른 `credential-id`를 써야 하며, 같으면 부팅에 실패한다. -## Experimental / Optional - -기본값은 전부 `false`다. - -```yaml -app: - messaging: - experimental: - kafka-share: false - pulsar: false - nats: false - bridge: - spring-cloud-stream: false -``` +Kafka producer는 이 항목을 실제 클라이언트 설정(`security.protocol`, `sasl.mechanism`, +`sasl.jaas.config`, `ssl.enabled.protocols=TLSv1.3,TLSv1.2`, hostname 검증)으로 옮긴다. 평문으로 +조용히 넘어가지 않도록 다음 경우 부팅에 실패한다: 브로커가 `tls-enabled`나 +`authentication-enabled`를 켰는데 `app.messaging.security.`가 없을 때, security 항목은 +있는데 `CredentialProvider` bean이 없을 때, 같은 브로커에 맞는 항목이 둘 이상일 때. +한계: truststore/keystore 설정 키가 없어 TLS는 JVM 기본 truststore를 쓰고, `MUTUAL_TLS`는 클라이언트 +인증서를 제출할 수 없다(미지원). ## Backpressure diff --git a/docs/messaging/cutover.md b/docs/messaging/cutover.md index a65e341..f697177 100644 --- a/docs/messaging/cutover.md +++ b/docs/messaging/cutover.md @@ -27,8 +27,9 @@ `PublishResult` → `OutboxPublishOutcome`. registry에 leaf를 추가하는 변경이라 별도 커밋. - golden contract 테스트: event/message ID, type, schema revision, partition/order/correlation/ causation/tenant/trace, payload digest, wire version이 bytes 단위로 보존되는지. -- 단일 publication authority: 기존 `OutboxPublicationAuthority` fence를 재사용해 writer/relay가 - 동시에 ACTIVE가 되지 않도록. **dual write/publish는 금지** — 한 business fact가 두 durable store와 +- 단일 publication authority: writer/relay가 동시에 ACTIVE가 되지 않도록 하는 fence. 이 용도로 쓰려던 + outbox V2의 `OutboxPublicationAuthority`는 한 번도 조립되지 않은 채 2026-09-25에 삭제됐으므로 + (`docs/capability-status.md` "제거된 코드") cutover가 자기 fence를 새로 가져야 한다. **dual write/publish는 금지** — 한 business fact가 두 durable store와 두 relay로 나가는 상태가 cutover에서 가장 위험하다. - 첫 cutover 범위는 **transport만** 교체(저장소는 기존 유지). storage migration은 shadow read → authority switch → old backlog drain 순서로 별도 release. diff --git a/docs/messaging/experimental-policy.md b/docs/messaging/experimental-policy.md index e2052c4..de8f5b1 100644 --- a/docs/messaging/experimental-policy.md +++ b/docs/messaging/experimental-policy.md @@ -17,24 +17,22 @@ stopsAcceptingNewWorkDuringShutdown **Experimental**은 아직 그 증명이 끝나지 않은 어댑터다. +현재 Experimental 어댑터는 없다. Kafka Share Group·Pulsar·NATS JetStream(Experimental), RabbitMQ, +Spring Cloud Stream bridge(Optional)는 SDK 없는 seam이거나 선택할 수 없는 상태여서 2026-09-25에 +삭제했다(`docs/capability-status.md` 제거 표). 이 문서는 새 어댑터를 들일 때 지킬 규칙이다. + ## 규칙 ### 1. 기본 비활성 -```yaml -messaging.experimental.kafka-share: false -messaging.experimental.pulsar: false -messaging.experimental.nats: false -``` - +새 Experimental 어댑터는 자기 활성화 플래그를 가지고 기본값은 `false`다. 활성화하지 않으면 validator가 `MessagingCapabilityUnavailableException`을 던진다. Contract Suite가 아직 증명 중인 어댑터가 누군가의 기본 설정 때문에 load-bearing이 되어서는 안 된다. ### 2. Stable 모듈이 Experimental 모듈에 의존하지 않는다 -Gradle 의존 그래프로 강제된다. `messaging-spring-boot-starter`의 `allowed_dependencies`에 -`messaging-kafka-share-experimental`, `messaging-pulsar-experimental`, -`messaging-nats-experimental`, `messaging-spring-cloud-stream-bridge`가 **없다**. +Gradle 의존 그래프로 강제된다. Experimental leaf는 `messaging-spring-boot-starter`의 +`allowed_dependencies`에 넣지 않는다. `verifyCleanArchitectureDependencies`가 위반을 빌드 실패로 만든다. @@ -45,45 +43,8 @@ Experimental 어댑터는 브로커의 차이를 `MessagingCapabilities`로 표 ### 4. 없는 기능을 광고하지 않는다 -| 어댑터 | 광고하지 않는 것 | 이유 | -|---|---|---| -| Kafka Share Group | orderedStream, keyedOrdering, replay, brokerTransaction | 경쟁 소비자 + 개별 ack는 partition 순서를 유지할 수 없다 | -| Pulsar | brokerTransaction | Pulsar에 있지만 플랫폼 Contract Suite로 증명되지 않았다 | -| Pulsar (Shared) | keyedOrdering | round-robin 분배 | -| NATS JetStream | nativeDeadLetter | delivery limit 초과 시 terminate할 뿐 라우팅하지 않는다 | -| NATS JetStream | keyedOrdering | subject 기반 모델에 per-key 순서가 없다 | - -`false`인 capability를 요구하는 profile은 startup에서 실패한다. -조용히 약화되지 않는다. - -### 5. 명시적 거부 - -| 조합 | 결과 | -|---|---| -| Kafka Share Group + ordering != NONE | 거부 | -| Kafka Share Group + pause/resume | `MessagingCapabilityUnavailableException` | -| Pulsar Shared + ordering=KEY | 거부 (Key_Shared 필요) | -| Pulsar + ordering=DESTINATION | 거부 | -| NATS Core + AT_LEAST_ONCE | 거부 (JetStream 필요) | -| NATS ordered consumer + 경쟁 워커 > 1 | 거부 | -| NATS + ordering=KEY | 거부 | - -## Spring Cloud Stream bridge - -Experimental이 아니라 **Optional**이다. 위험이 다르다. - -Stream은 자체 binder 설정을 소유하므로, binding이 destination profile이 모르는 -serializer·error handling·acknowledgement mode를 조용히 획득할 수 있다. - -따라서 브리지는 **플랫폼 보장에 의존하지 않는 destination에만** 허용한다. - -```text -ordering scope 선언 → 거부 -retry policy 선언 → 거부 -dead letter 선언 → 거부 -``` - -이 셋 중 하나라도 필요하면 native adapter를 쓴다. 거기서만 실제로 강제되기 때문이다. +Contract Suite로 증명하지 않은 capability는 `MessagingCapabilities`에서 `false`로 둔다. +`false`인 capability를 요구하는 profile은 startup에서 실패한다. 조용히 약화되지 않는다. ## 승격 조건 diff --git a/docs/messaging/operations.md b/docs/messaging/operations.md index 5cea33b..5bccc75 100644 --- a/docs/messaging/operations.md +++ b/docs/messaging/operations.md @@ -3,8 +3,7 @@ ## 배포 전 체크 ```bash -./gradlew verifyCleanArchitectureDependencies --console=plain -./gradlew verifyRuntimeModuleMembership --console=plain +./gradlew architectureCheck --console=plain ./gradlew checkstyleMain --console=plain ``` diff --git a/docs/messaging/outbox-inbox.md b/docs/messaging/outbox-inbox.md index 4dac3ee..d4b68ab 100644 --- a/docs/messaging/outbox-inbox.md +++ b/docs/messaging/outbox-inbox.md @@ -107,13 +107,11 @@ message_id만으로 중복 제거하면 같은 event를 소비하는 두 번째 broker의 최대 redelivery window보다 **길어야** 한다. row를 먼저 지우면 늦게 도착한 redelivery가 두 번 처리된다. -## Debezium CDC 대안 +## CDC 대안 -polling relay 대신 WAL을 읽는다. polling interval과 lease 경합이 사라지지만 -인프라와 그 자체의 실패 모드가 추가된다. - -wire contract는 동일하다. `DebeziumOutboxEventRouter`가 polling relay와 같은 reserved header를 -방출하므로 consumer는 어느 쪽이 발행했는지 구분할 수 없고, 전환은 배포 결정일 뿐 계약 변경이 아니다. +relay는 polling 하나뿐이다. Debezium Event Router 매핑(`DebeziumOutboxEventRouter` 등)은 조립하는 +곳이 없어 2026-09-25에 삭제했다(`docs/capability-status.md` 제거 표). `messaging_outbox`의 +`routing_key` 컬럼은 남아 있으므로 CDC를 다시 들일 때 그 컬럼을 message key로 쓴다. ## Claim Check diff --git a/docs/messaging/support-matrix.md b/docs/messaging/support-matrix.md index 563bd88..84c9647 100644 --- a/docs/messaging/support-matrix.md +++ b/docs/messaging/support-matrix.md @@ -4,10 +4,12 @@ 여기 없는 조합은 지원되지 않는다. > **등급은 증거를 따른다.** `CompatibilityMatrix.Entry.hasLiveBrokerCertification()`은 선언된 -> boolean이 아니라 `CertifiedEvidence`가 가진 레인 증거에서 파생된다. RabbitMQ가 Stable에서 내려온 -> 이유가 이것이다 — 어댑터는 공유 contract 7개를 통과하고 `RabbitBrokerIT`가 실 컨테이너에서 정상 -> 경로를 돌리지만, 이 저장소의 Stable 기준인 **장애 시나리오 증거**가 하나도 없다. 레인이 생겨 -> 증거를 내면 등급은 코드 수정 없이 따라 올라간다. +> boolean이 아니라 `CertifiedEvidence`가 가진 레인 증거에서 파생된다. 레인이 생겨 증거를 내면 등급은 +> 코드 수정 없이 따라 올라간다. + +> **삭제된 어댑터.** RabbitMQ(선택할 수 없던 validator·설정 모음), Pulsar·NATS JetStream·Kafka Share +> Group(Experimental, SDK 없는 seam), Spring Cloud Stream bridge, Debezium outbox 매핑은 2026-09-25에 +> 삭제했다. 복구 경로는 `docs/capability-status.md`의 제거 표에 있다. > **인증 근거.** 이 표의 버전은 이 저장소의 컨테이너 레인이 실제로 실행한 이미지다. 이전 판은 > Kafka 4.2/4.3을 선언했지만 fixture는 `apache/kafka:4.1.0`, lockfile client는 4.1.1이었다 — 표와 @@ -29,10 +31,7 @@ | 브로커 | 등급 | 인증 기준 | Stable 기능 | 제한 | |---|---|---|---|---| -| Kafka | Stable | 4.1.x | producer idempotence, consumer group, batch, pause/resume, replay, transaction capability | Share Group은 Experimental | -| RabbitMQ | Experimental | 4.3.x | exchange/routing, publisher confirm, mandatory return, manual ACK, quorum queue, retry queue, DLQ | **현재 starter에서 선택 불가** — production `RabbitChannelPublisher`/transport bridge가 없어 `app.messaging.broker=rabbit`은 startup에서 거부된다. 장애 시나리오 증거도 아직 없음 | -| Pulsar | Experimental | 4.0 LTS + 4.2 | typed publish/consume, Shared, Key_Shared, schema | transaction 미승격, 기본 비활성 | -| NATS JetStream | Experimental | 2.14.x | stream, durable consumer, explicit ACK, dedupe, replay | native DLQ 없음(플랫폼이 대행), 기본 비활성 | +| Kafka | Stable | 4.1.x | producer idempotence, consumer group, batch, pause/resume, replay, transaction capability | Share Group 미지원 | | Artemis/JMS | Extension | 범위 밖 | adapter SPI만 | 별도 ADR + Contract Suite 통과 필요 | ## Capability 매트릭스 @@ -40,27 +39,20 @@ `MessagingCapabilities`가 런타임에 선언하는 값이다. `false`인 기능을 요구하는 destination profile은 **startup에서 실패**하며, 조용히 약화되지 않는다. -| Capability | Kafka | Kafka Share | RabbitMQ | Pulsar | NATS JS | -|---|---|---|---|---|---| -| brokerAcknowledgement | O | O | O | O | O | -| replicationOrPersistenceEvidence | O | O | O | O | O | -| perMessageSettlement | O | O | O | O | O | -| batchSettlement | O | X | X | O | O | -| orderedStream | O | **X** | X | X | O | -| keyedOrdering | O | **X** | X | Key_Shared만 | X | -| replay | O | X | X | O | O | -| delayedDelivery | X | X | retry queue로 대행 | O | X | -| brokerTransaction | O | X | X | 미승격 | X | -| deduplicatedPublish | O | X | X | X | O | -| nativeDeadLetter | X | X | O | O | **X** | -| topologyManagement | O | X | O | O | O | - -Kafka Share Group이 ordering 전부 `X`인 것은 설계 결정이다. share group은 개별 record를 -경쟁 소비자에게 나눠주고 개별 ack하므로 partition 순서를 유지할 수 없다. ordered destination을 -share group에 설정하면 `KafkaShareProfileValidator`가 거부한다. - -NATS JetStream의 `nativeDeadLetter=X`도 마찬가지다. JetStream은 delivery limit 초과 시 메시지를 -**terminate**할 뿐 어디로도 라우팅하지 않으므로, 플랫폼이 DLQ publish를 직접 수행한다. +| Capability | Kafka | +|---|---| +| brokerAcknowledgement | O | +| replicationOrPersistenceEvidence | O | +| perMessageSettlement | O | +| batchSettlement | O | +| orderedStream | O | +| keyedOrdering | O | +| replay | O | +| delayedDelivery | X | +| brokerTransaction | O | +| deduplicatedPublish | O | +| nativeDeadLetter | X | +| topologyManagement | O | ## 기능 등급 @@ -73,8 +65,6 @@ NATS JetStream의 `nativeDeadLetter=X`도 마찬가지다. JetStream은 delivery | Batch / Manual settlement / Pause·Resume / Delayed / Replay 요청 | M2 | | Broker transaction / partition / routing / subscription | M3 | | Replay 실행 / Redrive / offset reset / purge / delete | M4 Admin | -| Kafka Share Group, Pulsar, NATS | Experimental | -| Spring Cloud Stream bridge | Optional | ## 무엇이 "Stable"을 증명하는가 @@ -82,7 +72,7 @@ Stable 등급은 두 가지를 **모두** 통과해야 한다. `CompatibilityMat ### 1. 공유 Contract Suite (`MessagingAdapterContract`, 7개) -Kafka와 RabbitMQ가 동일한 7개 테스트를 변경 없이 통과한다. 결정적 하네스를 쓰므로 +Kafka가 7개 테스트를 변경 없이 통과한다. 결정적 하네스를 쓰므로 확인 유실·settlement 유실 같은 장애를 요청 시점에 재현할 수 있다. ### 2. 실 브로커 인증 (Testcontainers) @@ -92,7 +82,6 @@ Kafka와 RabbitMQ가 동일한 7개 테스트를 변경 없이 통과한다. 결 | `KafkaBrokerIT` | `acks=all`이 실제 replication 증거를 만든다 / 잘못된 토픽은 `REJECTED` / 발행-소비 왕복에서 identity 보존 및 contiguous commit | | `KafkaAmbiguityChaosIT` | 브로커를 `docker pause`로 멈춘 상태의 publish가 **`AMBIGUOUS`** 로 보고된다 (broker acceptance 없음, confirmation level `NONE`, 비-retryable) | | `KafkaBrokerCertificationIT` | 인증 레인. Toxiproxy를 broker 앞에 두고 connection cut / confirm 유실 / 지연 / settlement 유실을 각각 주입하고, 통과한 시나리오마다 `BrokerCertificationEvidence` 한 줄을 manifest에 쓴다 | -| `RabbitBrokerIT` | exchange가 confirm했는데 어떤 큐에도 바인딩되지 않은 publish가 **`REJECTED` + `UNROUTABLE`** 로 보고된다 | | `OutboxPostgresIT` | 롤백된 트랜잭션은 발행 가능한 행을 남기지 않는다 / `SKIP LOCKED` lease가 두 relay를 분리한다 / ambiguous 행이 같은 `messageId`로 재클레임된다 | | `InboxPostgresIT` | 재전달이 side effect를 두 번 적용하지 않는다 / 롤백은 예약도 되돌린다 | @@ -137,7 +126,7 @@ delivery timeout으로 나타난다. 이를 `REJECTED`로 보고하는 것은 1. **Outbox `IN_FLIGHT` 고아 행** — lease 쿼리가 `PENDING`/`AMBIGUOUS`만 클레임 대상으로 봐서, publish 도중 죽은 relay가 남긴 행이 lease 만료 후에도 영영 회수되지 않았다. -2. **Rabbit confirm 경합** — transport가 publish *후에* confirm을 등록해서, 연결 스레드에서 +2. **Rabbit confirm 경합**(2026-09-25에 삭제된 Rabbit transport) — transport가 publish *후에* confirm을 등록해서, 연결 스레드에서 confirm이 먼저 도착하면 유실되고 호출자가 무한 대기했다. 둘 다 인메모리 double이 실제보다 관대해서 통과하고 있었다. @@ -154,21 +143,10 @@ delivery timeout으로 나타난다. 이를 `REJECTED`로 보고하는 것은 - DLQ publish 확인 전 source ACK -## R1 compiler status - -`adapter:outbound:messaging`의 contract/catalog/envelope compiler 군은 현재 **의도적으로 미배선**이다. -`DISABLED` + 빈 catalog/binding이 zero-resource 상태이며, 이 R1 compiler를 legacy -`MessagingConfig`에 연결해 Kafka ACK, wire compatibility, `ACTIVE_READY`, R2 qualification을 -주장하지 않는다. 코드는 다음 세대 계약을 보존하지만 현재 채택 경로가 아니며, 실제 활성화는 별도 R2 -composition/cutover에서 명시적으로 이뤄져야 한다. - - -### Optional mapping / bridge utilities +### Optional mapping utilities - `messaging-cloudevents`는 starter runtime에 자동 편입되지 않는 **opt-in mapping utility**다. 내부 envelope와 CloudEvent 객체 사이의 제한된 profile mapping을 제공하지만 binary/structured CloudEvents wire mode 전체를 구현하거나 임의 CloudEvents producer와의 1.0 interoperability를 인증하지 않는다. -- `messaging-spring-cloud-stream-bridge`는 현재 `ChannelSend` seam과 publish-evidence 판단을 제공하는 - framework-neutral bridge contract다. Spring Cloud Stream `StreamBridge` dependency/auto-configuration은 - 제공하지 않으므로 shipped Spring Cloud Stream integration으로 분류하지 않는다. +- `messaging-claim-check`도 starter에 편입되지 않는 opt-in utility다. diff --git a/messaging/CLAUDE.md b/messaging/CLAUDE.md index d051e29..6b2ad64 100644 --- a/messaging/CLAUDE.md +++ b/messaging/CLAUDE.md @@ -17,9 +17,9 @@ leaf의 정확한 id는 registry에서 읽는다. family는 **디렉터리 이 | `*-api` (`core-api`, `schema-api`, `policy`, `reliability-api`, `admin-api`, `transport-spi`) | 값 타입, 포트, 정책 계약 | Java stdlib만. Spring/broker SDK/logging framework 금지 | | `schema-*` (json/avro/protobuf), `cloudevents` | 인코딩 구현 | 해당 codec 라이브러리만. Spring 금지 | | `runtime-core`, `security`, `observability`, `admin-runtime` | framework-free 실행 로직 | Micrometer는 `observability`에서만. 나머지는 stdlib | -| `kafka`, `rabbit`, `pulsar-experimental`, `nats-experimental`, `kafka-share-experimental` | broker adapter | 해당 broker client. Spring은 `rabbit`의 `spring-rabbit`처럼 client가 곧 Spring인 경우만 | +| `kafka` | broker adapter | 해당 broker client. Spring은 client가 곧 Spring인 경우만 | | `outbox-jdbc-postgresql`, `inbox-jdbc-postgresql` | 관계형 신뢰성 adapter | Spring JDBC / Spring TX. **JPA 아님** — 이름이 곧 계약이다 | -| `spring-boot-starter`, `spring-cloud-stream-bridge` | 조립 경계 | Spring Boot autoconfigure. 여기서만 | +| `spring-boot-starter`, `spring-boot-starter-kafka` | 조립 경계 | Spring Boot autoconfigure. 여기서만 | | `testkit` | 테스트 지원 | Testcontainers, JUnit. production leaf가 의존하지 않는다 | `outbox`/`inbox` leaf가 `jdbc-postgresql`로 명명된 이유: 구현은 Spring JDBC이고 SQL은 @@ -31,7 +31,7 @@ leaf의 정확한 id는 registry에서 읽는다. family는 **디렉터리 이 - **vendor 타입을 public signature에 노출하면 그 의존성은 `api`로 선언한다.** 숨긴다고 타입이 숨겨지지 않는다 — consumer가 이 모듈이 문서화한 API에 대해 컴파일하지 못하게 만들 뿐이다. - (`kafka-clients`, `spring-rabbit`, `cloudevents-api`, `micrometer-core`, `avro`, + (`kafka-clients`, `cloudevents-api`, `micrometer-core`, `avro`, `protobuf-java`, `reactor-core`가 이 규칙으로 `api`다.) - 반대로 **노출하지 않는 것은 `implementation`으로 낮춘다.** starter가 노출하지 않는 하위 모듈을 `implementation`으로 내린 것이, JSON codec이 `ObjectMapper`를 private으로만 쓰고 Jackson을 @@ -45,12 +45,15 @@ leaf의 정확한 id는 registry에서 읽는다. family는 **디렉터리 이 ## Application runtime에 포함되는 기준 -Messaging family는 기본 application이 `messaging-spring-boot-starter`를 dependency로 가지므로 그 -transitive project graph가 runtime에 포함된다. 이 사실의 SSOT는 Gradle `runtimeClasspath`이며 registry에 -동일한 membership 목록을 복제하지 않는다. `app.messaging.enabled=false`가 기본값이고 master switch가 -activation을 결정한다. +기본 조립에는 Messaging family가 없다. 서비스가 `app-bootstrap/build.gradle`에 +`runtimeOnly project(':adapter:outbound:messaging-kafka')` 한 줄을 넣으면 그 모듈이 bridge와 +`messaging-spring-boot-starter`(Kafka transport 포함)의 transitive project graph를 runtime에 싣는다. 이 +사실의 SSOT는 Gradle `runtimeClasspath`이며 registry에 동일한 membership 목록을 복제하지 않는다. +모듈이 있으면 켜지고(`app.messaging.enabled` 기본 `true`), `APP_MESSAGING_ENABLED=false`로만 끈다. -experimental transport, 일부 codec/bridge/testkit처럼 starter graph에 들어오지 않는 leaf는 build-only다. +일부 codec(cloudevents 등)·claim-check·testkit처럼 starter graph에 들어오지 않는 leaf는 build-only다. +RabbitMQ·Pulsar·NATS·Kafka Share·Spring Cloud Stream bridge leaf와 Debezium outbox 매핑은 +2026-09-25에 삭제했다(`docs/capability-status.md` 제거 표). 새 leaf를 application에 채택할 때는 starter 또는 composition dependency와 architecture edge policy, qualification/support 상태를 함께 변경한다. @@ -79,10 +82,9 @@ canonical bridge의 producer identity는 `app.messaging.producer-id`로 명시 `ca-skeleton.outbox.canonical-transport-enabled=true`는 bridge 존재만으로 충분하지 않다. Stable composition은 애플리케이션 계약 catalog/schema source를 임의로 만들지 않으므로 adopter가 -`IntegrationEventEncoderPort`를 명시적으로 제공해야 한다. 저장소의 후보 구현은 -`JsonSchemaIntegrationEventEncoder`이지만 이를 사용하려면 compiled contract descriptor, -`LocalJsonSchemaRegistry`, payload/envelope limits와 key material에 해당하는 application-owned 입력을 -같이 조립해야 한다. `OutboxCanonicalTransportRequirementValidator`는 publisher 또는 encoder 중 하나라도 +`IntegrationEventEncoderPort`를 명시적으로 제공해야 한다. 템플릿은 구현체를 주지 않는다(후보였던 +R1 JSON Schema encoder는 2026-09-25에 삭제). `OutboxCanonicalTransportRequirementValidator`는 +publisher 또는 encoder 중 하나라도 없으면 startup을 거부한다. 즉 canonical flag가 켜졌는데 정본 encoder가 없는 상태로 성공 기동하는 경로는 없다. diff --git a/messaging/messaging-kafka-share-experimental/build.gradle b/messaging/messaging-kafka-share-experimental/build.gradle deleted file mode 100644 index 33edc25..0000000 --- a/messaging/messaging-kafka-share-experimental/build.gradle +++ /dev/null @@ -1,11 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-transport-spi') - api project(':messaging:messaging-kafka') - -} diff --git a/messaging/messaging-kafka-share-experimental/gradle.lockfile b/messaging/messaging-kafka-share-experimental/gradle.lockfile deleted file mode 100644 index eaf97d5..0000000 --- a/messaging/messaging-kafka-share-experimental/gradle.lockfile +++ /dev/null @@ -1,93 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -at.yawk.lz4:lz4-java:1.10.1=runtimeClasspath,testRuntimeClasspath -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.luben:zstd-jni:1.5.6-10=runtimeClasspath,testRuntimeClasspath -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.kafka:kafka-clients:4.1.2=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=runtimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=runtimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=runtimeClasspath,spotbugs,spotbugsSlf4j,testRuntimeClasspath -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xerial.snappy:snappy-java:1.1.10.7=runtimeClasspath,testRuntimeClasspath -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty= diff --git a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareGroupRegistrar.java b/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareGroupRegistrar.java deleted file mode 100644 index 07c9ec1..0000000 --- a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareGroupRegistrar.java +++ /dev/null @@ -1,48 +0,0 @@ -package dev.caskeleton.messaging.kafka.share; - -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.transport.TransportConsumerRegistration; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import java.util.Objects; - -/** - * Registers a share group consumer, once the experimental guard has passed. - * - *

Pause and resume are refused rather than silently ignored. A share group has no partition - * assignment to pause, so accepting the call would let a retry policy that depends on pausing - * appear to work while doing nothing. - */ -public final class KafkaShareGroupRegistrar { - - private final KafkaShareProfileValidator validator; - - /** Creates a registrar with the default validator. */ - public KafkaShareGroupRegistrar() { - this(new KafkaShareProfileValidator()); - } - - /** - * Creates a registrar with an explicit validator. - * - * @param validator the share profile validator - */ - public KafkaShareGroupRegistrar(KafkaShareProfileValidator validator) { - this.validator = Objects.requireNonNull(validator, "validator must not be null"); - } - - /** - * Registers a share group consumer. - * - * @param profile the share group profile - * @param spec what to consume and where to deliver it - * @return the registration - */ - public TransportConsumerRegistration register( - KafkaShareProfile profile, TransportConsumerSpec spec) { - Objects.requireNonNull(spec, "spec must not be null"); - validator.validate(profile); - throw new MessagingCapabilityUnavailableException( - "KAFKA_SHARE_CONSUMER_NOT_IMPLEMENTED", - "Kafka Share registration is not implemented; refusing to advertise an active consumer"); - } -} diff --git a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfile.java b/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfile.java deleted file mode 100644 index 276207c..0000000 --- a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfile.java +++ /dev/null @@ -1,33 +0,0 @@ -package dev.caskeleton.messaging.kafka.share; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationName; -import java.util.Objects; - -/** - * Settings for a Kafka Share Group work queue. - * - * @param destination the logical destination - * @param shareGroup the share group name - * @param orderingScope the ordering the destination declares - * @param enabled whether the experimental adapter is switched on - * @param maxDeliveryCount how many times a record may be re-acquired before it is released - */ -public record KafkaShareProfile( - DestinationName destination, - String shareGroup, - OrderingScope orderingScope, - boolean enabled, - int maxDeliveryCount) { - - public KafkaShareProfile { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(orderingScope, "orderingScope must not be null"); - if (shareGroup == null || shareGroup.isBlank()) { - throw new IllegalArgumentException("shareGroup must not be blank"); - } - if (maxDeliveryCount < 1) { - throw new IllegalArgumentException("maxDeliveryCount must be at least 1"); - } - } -} diff --git a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidator.java b/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidator.java deleted file mode 100644 index 1bd0390..0000000 --- a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidator.java +++ /dev/null @@ -1,42 +0,0 @@ -package dev.caskeleton.messaging.kafka.share; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import java.util.Objects; - -/** - * Guards the Kafka Share Group experimental adapter. - * - *

A share group hands individual records to competing consumers and acknowledges them - * individually. That is a work queue, and it is fundamentally incompatible with partition ordering: - * two consumers in the same share group can process records from one partition concurrently and - * finish in either order. Configuring an ordered destination on a share group would therefore - * advertise a guarantee the broker is not providing, so it is refused rather than degraded. - * - *

The adapter is also off unless explicitly enabled, so an Experimental capability cannot drift - * into a Stable deployment by default. - */ -public final class KafkaShareProfileValidator { - - /** - * Validates a share group profile. - * - * @param profile the profile to validate - * @throws IllegalArgumentException when the destination requires ordering - * @throws MessagingCapabilityUnavailableException when the experimental adapter is disabled - */ - public void validate(KafkaShareProfile profile) { - Objects.requireNonNull(profile, "profile must not be null"); - - if (!profile.enabled()) { - throw new MessagingCapabilityUnavailableException( - "KAFKA_SHARE_DISABLED", - "the Kafka Share Group adapter is experimental and disabled unless " - + "backend.messaging.experimental.kafka-share=true"); - } - if (profile.orderingScope() != OrderingScope.NONE) { - throw new IllegalArgumentException( - "a Kafka share group cannot provide ordered delivery: " + profile.destination().value()); - } - } -} diff --git a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareWorkQueueCapability.java b/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareWorkQueueCapability.java deleted file mode 100644 index afa4907..0000000 --- a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareWorkQueueCapability.java +++ /dev/null @@ -1,27 +0,0 @@ -package dev.caskeleton.messaging.kafka.share; - -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; - -/** - * What a Kafka Share Group can and cannot do. - * - *

Declared as a capability rather than assumed, so that the shared validators refuse an ordered - * or replayed destination on this adapter before a message is ever produced. - */ -public final class KafkaShareWorkQueueCapability { - - private KafkaShareWorkQueueCapability() {} - - /** - * Returns the share group capability set. - * - *

Per-record settlement, yes. Ordering, replay, and transactions, no — a share group gives up - * exactly those to gain competing-consumer throughput. - * - * @return the capabilities - */ - public static MessagingCapabilities capabilities() { - return new MessagingCapabilities( - true, true, true, false, false, false, false, false, false, false, false, false); - } -} diff --git a/messaging/messaging-kafka-share-experimental/src/test/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidatorTest.java b/messaging/messaging-kafka-share-experimental/src/test/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidatorTest.java deleted file mode 100644 index e2ecf9a..0000000 --- a/messaging/messaging-kafka-share-experimental/src/test/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidatorTest.java +++ /dev/null @@ -1,113 +0,0 @@ -package dev.caskeleton.messaging.kafka.share; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import org.junit.jupiter.api.Test; - -class KafkaShareProfileValidatorTest { - - private final KafkaShareProfileValidator validator = new KafkaShareProfileValidator(); - - @Test - void rejectsOrderedStreamUse() { - KafkaShareProfile profile = KafkaShareProfileFixtures.profile(OrderingScope.KEY); - - assertThatThrownBy(() -> validator.validate(profile)) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("ordered"); - } - - @Test - void rejectsPartitionOrderingToo() { - KafkaShareProfile profile = KafkaShareProfileFixtures.profile(OrderingScope.PARTITION); - - assertThatThrownBy(() -> validator.validate(profile)) - .isInstanceOf(IllegalArgumentException.class); - } - - @Test - void staysDisabledUnlessExplicitlyEnabled() { - KafkaShareProfile profile = KafkaShareProfileFixtures.disabled(); - - assertThatThrownBy(() -> validator.validate(profile)) - .isInstanceOf(MessagingCapabilityUnavailableException.class) - .hasMessageContaining("experimental"); - } - - @Test - void acceptsAnUnorderedWorkQueue() { - assertThatCode(() -> validator.validate(KafkaShareProfileFixtures.profile(OrderingScope.NONE))) - .doesNotThrowAnyException(); - } - - @Test - void doesNotAdvertiseOrderingReplayOrTransactions() { - var capabilities = KafkaShareWorkQueueCapability.capabilities(); - - assertThat(capabilities.perMessageSettlement()).isTrue(); - assertThat(capabilities.orderedStream()).isFalse(); - assertThat(capabilities.keyedOrdering()).isFalse(); - assertThat(capabilities.replay()).isFalse(); - assertThat(capabilities.brokerTransaction()).isFalse(); - } - - @Test - void registerFailsClosedUntilAShareConsumerActuallyExists() { - assertThatThrownBy( - () -> - new KafkaShareGroupRegistrar() - .register( - KafkaShareProfileFixtures.profile(OrderingScope.NONE), - new dev.caskeleton.messaging.transport.TransportConsumerSpec( - KafkaShareProfileFixtures.destinationProfile(), - delivery -> - java.util.concurrent.CompletableFuture.completedFuture(null)))) - .isInstanceOf(MessagingCapabilityUnavailableException.class) - .hasMessageContaining("not implemented"); - } -} - -/** Builds Kafka share group fixtures. */ -final class KafkaShareProfileFixtures { - - private KafkaShareProfileFixtures() {} - - static KafkaShareProfile profile(OrderingScope orderingScope) { - return new KafkaShareProfile( - new DestinationName("email-work"), "email-share", orderingScope, true, 5); - } - - static KafkaShareProfile disabled() { - return new KafkaShareProfile( - new DestinationName("email-work"), "email-share", OrderingScope.NONE, false, 5); - } - - static dev.caskeleton.messaging.policy.DestinationProfile destinationProfile() { - return new dev.caskeleton.messaging.policy.DestinationProfile( - new DestinationName("email-work"), - "kafka-primary", - dev.caskeleton.messaging.api.destination.DestinationKind.WORK_QUEUE, - dev.caskeleton.messaging.policy.PhysicalDestination.kafkaTopic("email.work.v1"), - new dev.caskeleton.messaging.policy.SchemaPolicy( - dev.caskeleton.messaging.api.ContentType.JSON, - dev.caskeleton.messaging.schema.SchemaCompatibility.BACKWARD, - java.util.Set.of(new dev.caskeleton.messaging.api.MessageType("email.requested"))), - dev.caskeleton.messaging.api.delivery.DeliveryGuarantee.AT_LEAST_ONCE, - OrderingScope.NONE, - dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, - dev.caskeleton.messaging.policy.ProducerPolicy.defaults(), - dev.caskeleton.messaging.policy.ConsumerPolicy.defaults("email-share"), - dev.caskeleton.messaging.policy.RetryPolicy.none(), - dev.caskeleton.messaging.policy.DeadLetterPolicy.disabled(), - dev.caskeleton.messaging.policy.PayloadPolicy.defaults(), - dev.caskeleton.messaging.policy.CapabilityTier.M2, - false, - false, - false); - } -} diff --git a/messaging/messaging-kafka/build.gradle b/messaging/messaging-kafka/build.gradle index 789445b..75b48b2 100644 --- a/messaging/messaging-kafka/build.gradle +++ b/messaging/messaging-kafka/build.gradle @@ -1,8 +1,13 @@ plugins { id 'java-test-fixtures' id 'ca.auxiliary-source-set' - id 'ca.jmh-benchmarks' - id 'ca.messaging-certification' + id 'ca.platform-module' + id 'me.champeau.jmh' +} + +jmh { + jmhVersion = libs.versions.jmh.get() + includeTests = true } auxiliarySourceSets { @@ -82,3 +87,30 @@ strictTestLanes { .toString() } } + +// The certification lane writes the scenarios it proved; the committed manifest in messaging-testkit +// may claim no more and no fewer. gitCommit and observedAt are provenance, not claim identity. +tasks.register('verifyMessagingCertificationEvidence') { + group = 'verification' + description = 'Fails when the committed broker certification manifest claims a scenario the certification lane did not produce.' + dependsOn 'messagingCertificationTest' + def produced = layout.buildDirectory.file('messaging-certification/broker-certification-evidence.jsonl') + def committed = rootProject.file('messaging/messaging-testkit/src/main/resources/messaging/broker-certification-evidence.jsonl') + inputs.file(produced) + inputs.file(committed) + doLast { + def claims = { File file -> + file.readLines('UTF-8').findAll { !it.trim().isEmpty() }.collect { + it.replaceAll(/,"gitCommit":"[^"]*"/, '').replaceAll(/,"observedAt":"[^"]*"/, '') + } as Set + } + def ran = claims(produced.get().asFile) + def shipped = claims(committed) + if (ran != shipped) { + throw new GradleException("the committed certification manifest does not match this run.\n" + + " claimed but not produced: ${shipped - ran}\n produced but not claimed: ${ran - shipped}\n" + + "Copy ${produced.get().asFile} over ${committed} — the manifest is a record of a run.") + } + } +} + diff --git a/messaging/messaging-kafka/gradle.lockfile b/messaging/messaging-kafka/gradle.lockfile index b0dd01f..ecffd2e 100644 --- a/messaging/messaging-kafka/gradle.lockfile +++ b/messaging/messaging-kafka/gradle.lockfile @@ -46,15 +46,15 @@ io.micrometer:micrometer-core:1.16.7=compileClasspath,integrationTestCompileClas io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath net.java.dev.jna:jna:5.18.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -net.sf.jopt-simple:jopt-simple:5.0.4=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath +net.sf.jopt-simple:jopt-simple:5.0.4=jmh,jmhCompileClasspath,jmhRuntimeClasspath net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs org.antlr:antlr4-runtime:4.13.2=checkstyle org.apache.bcel:bcel:6.12.0=spotbugs org.apache.commons:commons-compress:1.28.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.apache.commons:commons-lang3:3.20.0=checkstyle,integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs -org.apache.commons:commons-math3:3.6.1=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath +org.apache.commons:commons-math3:3.6.1=jmh,jmhCompileClasspath,jmhRuntimeClasspath org.apache.commons:commons-text:1.15.0=spotbugs org.apache.commons:commons-text:1.3=checkstyle org.apache.httpcomponents:httpclient:4.5.13=checkstyle @@ -67,8 +67,8 @@ org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,jmhCompileClasspath,testCompileClasspath,testFixturesCompileClasspath -org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,testCompileClasspath,testFixturesCompileClasspath +org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle @@ -78,24 +78,27 @@ org.hdrhistogram:HdrHistogram:2.2.2=integrationTestRuntimeClasspath,jmhRuntimeCl org.javassist:javassist:3.28.0-GA=checkstyle org.jetbrains:annotations:17.0.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,integrationTestAnnotationProcessor,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testFixturesAnnotationProcessor,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.junit.jupiter:junit-jupiter-api:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.junit.jupiter:junit-jupiter-engine:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.junit.jupiter:junit-jupiter-params:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.junit.jupiter:junit-jupiter:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.junit.platform:junit-platform-commons:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.junit.platform:junit-platform-engine:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.junit.platform:junit-platform-launcher:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testRuntimeClasspath -org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.junit:junit-bom:6.1.0=spotbugs org.latencyutils:LatencyUtils:2.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,runtimeClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.mockito:mockito-core:5.20.0=mockitoAgent -org.openjdk.jmh:jmh-core:1.37=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath -org.openjdk.jmh:jmh-generator-annprocess:1.37=jmhAnnotationProcessor -org.opentest4j:opentest4j:1.3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.openjdk.jmh:jmh-core:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-asm:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-bytecode:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-reflection:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.opentest4j:opentest4j:1.3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.ow2.asm:asm-analysis:9.10.1=spotbugs org.ow2.asm:asm-commons:9.10.1=spotbugs org.ow2.asm:asm-tree:9.10.1=spotbugs org.ow2.asm:asm-util:9.10.1=spotbugs +org.ow2.asm:asm:9.0=jmh,jmhCompileClasspath,jmhRuntimeClasspath org.ow2.asm:asm:9.10.1=spotbugs org.pcollections:pcollections:4.0.1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor org.reflections:reflections:0.10.2=checkstyle diff --git a/messaging/messaging-nats-experimental/build.gradle b/messaging/messaging-nats-experimental/build.gradle deleted file mode 100644 index 929bf94..0000000 --- a/messaging/messaging-nats-experimental/build.gradle +++ /dev/null @@ -1,14 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-transport-spi') - api project(':messaging:messaging-observability') - api project(':messaging:messaging-security') - api project(':messaging:messaging-admin-api') - -} diff --git a/messaging/messaging-nats-experimental/gradle.lockfile b/messaging/messaging-nats-experimental/gradle.lockfile deleted file mode 100644 index f5ad1bb..0000000 --- a/messaging/messaging-nats-experimental/gradle.lockfile +++ /dev/null @@ -1,89 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=runtimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=runtimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty= diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsAckMode.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsAckMode.java deleted file mode 100644 index 2a8b865..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsAckMode.java +++ /dev/null @@ -1,57 +0,0 @@ -package dev.caskeleton.messaging.nats; - -/** - * How a JetStream consumer acknowledges what it has handled. - * - *

Only {@link #EXPLICIT} is usable by the platform, and the other two are modelled anyway so the - * refusal can name what was configured instead of failing with a generic message. - * - *

{@link #NONE} and {@link #ALL} both break at-least-once in ways that look like success. {@code - * AckNone} never acknowledges, so JetStream considers every message delivered the moment it is sent - * — a handler that throws has already had its message forgotten. {@code AckAll} acknowledges every - * message up to the one being acknowledged, so a consumer processing messages concurrently silently - * acknowledges work that is still in flight. - */ -public enum NatsAckMode { - - /** Every message is acknowledged individually. The only mode the platform supports. */ - EXPLICIT(true), - - /** Acknowledging one message acknowledges every earlier one. */ - ALL(false), - - /** No acknowledgement at all; JetStream treats delivery as completion. */ - NONE(false); - - private final boolean supportsAtLeastOnce; - - NatsAckMode(boolean supportsAtLeastOnce) { - this.supportsAtLeastOnce = supportsAtLeastOnce; - } - - /** - * Reports whether this mode can carry at-least-once delivery. - * - * @return true only for {@link #EXPLICIT} - */ - public boolean supportsAtLeastOnce() { - return supportsAtLeastOnce; - } - - /** - * Returns why a mode cannot be used, for the refusal message. - * - * @return the sanitized reason, empty for {@link #EXPLICIT} - */ - public String rejectionReason() { - return switch (this) { - case EXPLICIT -> ""; - case ALL -> - "AckAll acknowledges every earlier message, so concurrent handling silently settles work " - + "that is still in flight"; - case NONE -> - "AckNone treats delivery as completion, so a message whose handler throws has already " - + "been forgotten by the broker"; - }; - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfile.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfile.java deleted file mode 100644 index 2713ab2..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfile.java +++ /dev/null @@ -1,103 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; - -/** - * The JetStream settings for one destination. - * - *

{@code maxDeliver} is the setting that makes the platform's dead-lettering necessary. When a - * message reaches it, JetStream terminates the message — it stops redelivering and routes - * it nowhere. Without the platform parking it first, the message is simply gone, which is why - * {@link NatsMaxDeliverParkingWorkflow} has to run strictly before the limit is reached rather than - * in response to it. - * - *

{@code ackWait} is the redelivery trigger. A handler slower than {@code ackWait} gets its - * message redelivered while it is still working on it, so this is the one JetStream number that has - * to be set from the handler's real latency rather than from a default. - * - * @param subject the JetStream subject - * @param stream the stream capturing that subject - * @param durableName the durable consumer name - * @param ackMode how the consumer acknowledges - * @param ackWait how long JetStream waits before redelivering - * @param maxDeliver how many deliveries before JetStream terminates the message - * @param deduplicationWindow the publish deduplication window, when enabled - */ -public record NatsJetStreamProfile( - String subject, - String stream, - String durableName, - NatsAckMode ackMode, - Duration ackWait, - int maxDeliver, - Optional deduplicationWindow) { - - /** How many deliveries the platform leaves as headroom before JetStream's terminate. */ - public static final int PARKING_HEADROOM = 1; - - public NatsJetStreamProfile { - Objects.requireNonNull(ackMode, "ackMode must not be null"); - Objects.requireNonNull(ackWait, "ackWait must not be null"); - Objects.requireNonNull(deduplicationWindow, "deduplicationWindow must not be null"); - requireText(subject, "subject"); - requireText(stream, "stream"); - requireText(durableName, "durableName"); - - if (!ackMode.supportsAtLeastOnce()) { - throw new IllegalArgumentException( - "%s cannot carry at-least-once delivery: %s" - .formatted(ackMode, ackMode.rejectionReason())); - } - if (ackWait.isNegative() || ackWait.isZero()) { - throw new IllegalArgumentException("ackWait must be positive"); - } - if (maxDeliver < 2) { - throw new IllegalArgumentException( - "maxDeliver must leave at least one delivery of headroom for the platform to park the " - + "message before JetStream terminates it"); - } - if (deduplicationWindow.isPresent() - && (deduplicationWindow.get().isNegative() || deduplicationWindow.get().isZero())) { - throw new IllegalArgumentException("deduplicationWindow must be positive when set"); - } - } - - /** - * Returns the delivery count at which the platform must park the message. - * - *

Strictly before {@code maxDeliver}: acting on the limit itself is too late, because the - * delivery that reaches it is the one JetStream terminates. - * - * @return the parking threshold - */ - public int parkAtDelivery() { - return maxDeliver - PARKING_HEADROOM; - } - - /** - * Returns a profile with the platform defaults. - * - * @param subject the JetStream subject - * @param stream the stream name - * @param durableName the durable consumer name - * @return the profile - */ - public static NatsJetStreamProfile durable(String subject, String stream, String durableName) { - return new NatsJetStreamProfile( - subject, - stream, - durableName, - NatsAckMode.EXPLICIT, - Duration.ofSeconds(30), - 5, - Optional.of(Duration.ofMinutes(2))); - } - - private static void requireText(String value, String field) { - if (value == null || value.isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfileValidator.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfileValidator.java deleted file mode 100644 index e5aaf17..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfileValidator.java +++ /dev/null @@ -1,75 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.util.Objects; - -/** - * Guards the NATS JetStream Experimental adapter. - * - *

Core NATS is refused for any at-least-once destination. Core NATS is fire-and-forget with no - * persistence and no acknowledgement, so a durable destination configured against it would report - * success for messages that were never stored anywhere — the failure mode is total and silent. - * - *

An ordered JetStream consumer and a competing work queue are mutually exclusive: the ordered - * consumer is a single-reader construct, and pointing several workers at it does not distribute the - * load, it breaks the ordering the consumer exists to provide. - */ -public final class NatsJetStreamProfileValidator { - - /** - * Validates a destination against a JetStream consumer configuration. - * - * @param profile the destination profile - * @param jetStreamEnabled whether JetStream persistence is in use - * @param orderedConsumer whether an ordered consumer is configured - * @param competingWorkers how many workers share the consumer - * @param enabled whether the experimental adapter is switched on - */ - public void validate( - DestinationProfile profile, - boolean jetStreamEnabled, - boolean orderedConsumer, - int competingWorkers, - boolean enabled) { - Objects.requireNonNull(profile, "profile must not be null"); - - if (!enabled) { - throw new MessagingCapabilityUnavailableException( - "NATS_DISABLED", - "the NATS JetStream adapter is experimental and disabled unless " - + "app.messaging.experimental.nats=true"); - } - if (profile.deliveryGuarantee() == DeliveryGuarantee.AT_LEAST_ONCE && !jetStreamEnabled) { - throw new IllegalArgumentException( - "core NATS cannot provide at-least-once delivery; JetStream is required: " - + profile.name().value()); - } - if (orderedConsumer && competingWorkers > 1) { - throw new IllegalArgumentException( - "an ordered JetStream consumer cannot be shared by competing workers: " - + profile.name().value()); - } - if (profile.orderingScope() == OrderingScope.KEY) { - throw new IllegalArgumentException( - "the NATS adapter does not offer per-key ordering: " + profile.name().value()); - } - } - - /** - * Returns what the JetStream adapter can prove. - * - *

Native dead lettering is reported as unavailable. JetStream terminates a message after its - * delivery limit rather than routing it anywhere, so the platform supplies dead lettering itself; - * claiming a native one would leave operators looking for a queue that does not exist. - * - * @return the capabilities - */ - public MessagingCapabilities capabilities() { - return new MessagingCapabilities( - true, true, true, true, true, false, true, false, false, true, false, true); - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamTransport.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamTransport.java deleted file mode 100644 index bf8f5c8..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamTransport.java +++ /dev/null @@ -1,295 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import dev.caskeleton.messaging.api.destination.DestinationCapabilities; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import dev.caskeleton.messaging.transport.MessagingTransport; -import dev.caskeleton.messaging.transport.TransportConsumerRegistration; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import dev.caskeleton.messaging.transport.TransportPublishResult; -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionException; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicBoolean; -import java.util.function.Function; - -/** - * The Experimental NATS JetStream adapter. - * - *

Publishes go through JetStream, never core NATS. A core publish returns as soon as the bytes - * are written to the socket, with no persistence and no acknowledgement, so an adapter using it - * would report success for messages that were never stored — the failure is total and silent, which - * is why {@link NatsJetStreamProfileValidator} refuses the combination at startup. - * - *

A JetStream publish acknowledgement names the stream and sequence the message landed in, which - * is real persistence evidence. A publish that times out is {@code AMBIGUOUS}: JetStream may have - * stored it and lost only the acknowledgement, and the deduplication window is what makes retrying - * it safe when the profile enables one. - * - *

This is an Experimental contract seam, not a Stable adapter. It exercises the transport SPI - * against a publish operation the application supplies; it does not ship a NATS client bridge, - * connection management, or reconnection. Only a failure the operation reports as a {@link - * NatsPreSendRejection} is treated as definitely not transmitted — every other failure, recognised - * or not, is ambiguous, because a caller that reads {@code REJECTED} may republish under a new - * identity and duplicate a message the server already stored. - * - *

The adapter reports {@code nativeDeadLetter} as false. JetStream terminates a message at its - * delivery limit rather than routing it anywhere, so the platform parks it via {@link - * NatsMaxDeliverParkingWorkflow}; claiming a native dead-letter would send operators looking for a - * queue that does not exist. - */ -public final class NatsJetStreamTransport implements MessagingTransport { - - private static final MessagingCapabilities CAPABILITIES = - new MessagingCapabilities( - true, true, true, true, true, false, true, false, false, true, false, true); - - private final String brokerName; - private final long generation; - private final NatsJetStreamProfile profile; - private final JetStreamPublishOperation publish; - private final Function consumerFactory; - private final AtomicBoolean closed = new AtomicBoolean(); - - /** - * Creates a publish-only transport. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param profile the JetStream settings - * @param publish the JetStream publish operation - */ - public NatsJetStreamTransport( - String brokerName, - long generation, - NatsJetStreamProfile profile, - JetStreamPublishOperation publish) { - this( - brokerName, - generation, - profile, - publish, - spec -> { - throw new MessagingCapabilityUnavailableException( - "NATS_CONSUMER_NOT_CONFIGURED", - "this NATS transport was created without a consumer factory"); - }); - } - - /** - * Creates a transport with a consumer factory. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param profile the JetStream settings - * @param publish the JetStream publish operation - * @param consumerFactory builds a consumer registration for a spec - */ - public NatsJetStreamTransport( - String brokerName, - long generation, - NatsJetStreamProfile profile, - JetStreamPublishOperation publish, - Function consumerFactory) { - this.brokerName = Objects.requireNonNull(brokerName, "brokerName must not be null"); - this.generation = generation; - this.profile = Objects.requireNonNull(profile, "profile must not be null"); - this.publish = Objects.requireNonNull(publish, "publish must not be null"); - this.consumerFactory = Objects.requireNonNull(consumerFactory, "consumerFactory is required"); - } - - @Override - public CompletionStage publish(TransportPublishRequest request) { - Objects.requireNonNull(request, "request must not be null"); - - if (closed.get()) { - return completed(rejectedLocally("NATS_TRANSPORT_CLOSED", "the transport is shutting down")); - } - int size = request.envelope().payload().size(); - int limit = request.profile().payload().maxBytes(); - if (size > limit) { - return completed( - rejectedLocally( - "PAYLOAD_TOO_LARGE", "encoded payload is " + size + " bytes, limit is " + limit)); - } - - // The caller's own deadline, honoured rather than ignored. Without it a stalled JetStream - // publish holds the caller for as long as the SDK's default takes, and the timeout the caller - // set to bound its own latency budget means nothing. - long startedAt = System.nanoTime(); - return publish - .publish(profile.subject(), deduplicationId(request), request) - .toCompletableFuture() - .orTimeout( - request.options().timeout().toMillis(), java.util.concurrent.TimeUnit.MILLISECONDS) - .handle( - (position, failure) -> - failure == null - ? confirmed(position, elapsedSince(startedAt)) - : classify(failure, elapsedSince(startedAt))) - .thenApply(TransportPublishResult::new); - } - - @Override - public TransportConsumerRegistration register(TransportConsumerSpec spec) { - Objects.requireNonNull(spec, "spec must not be null"); - if (closed.get()) { - throw new MessagingCapabilityUnavailableException( - "NATS_TRANSPORT_CLOSED", "the transport is shutting down"); - } - return consumerFactory.apply(spec); - } - - @Override - public DestinationCapabilities capabilities(DestinationName destination) { - return new DestinationCapabilities(destination, brokerName, CAPABILITIES); - } - - @Override - public String brokerName() { - return brokerName; - } - - @Override - public long generation() { - return generation; - } - - @Override - public void close() { - closed.set(true); - } - - /** - * Reports whether the transport is still accepting work. - * - * @return true until close - */ - public boolean isAcceptingWork() { - return !closed.get(); - } - - /** - * Returns the deduplication id for a publish, when the profile enables deduplication. - * - *

The logical message id, so a retry of an ambiguous publish is recognised as the same message - * rather than accepted as a new one. Minting a fresh id per attempt would make the deduplication - * window useless in exactly the situation it exists for. - * - * @param request the publish request - * @return the deduplication id, or empty when the profile has no window - */ - private Optional deduplicationId(TransportPublishRequest request) { - return profile - .deduplicationWindow() - .map(window -> request.envelope().messageId().value().toString()); - } - - private static Duration elapsedSince(long startedAtNanos) { - return Duration.ofNanos(System.nanoTime() - startedAtNanos); - } - - private static PublishResult confirmed(NatsStreamPosition position, Duration elapsed) { - return new PublishResult( - PublishCompletion.CONFIRMED, - // The publish acknowledgement names the stream and sequence the message landed in, which is - // persistence evidence rather than a receipt for bytes on a socket. - new PublishEvidence( - true, - TransmissionEvidence.TRANSMITTED, - true, - ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK), - RoutingOutcome.ROUTED, - Optional.of(position), - 1, - elapsed, - Optional.empty()); - } - - /** - * Decides what a publish failure means, from typed signals only. - * - *

Only {@link NatsPreSendRejection} — thrown by the publish operation when it knows the send - * never left the client — reports {@code REJECTED / NOT_TRANSMITTED}. Everything else is {@code - * AMBIGUOUS}, including failures this adapter does not recognise, because the alternative is - * telling a caller nothing was transmitted when the server may already hold the message. - */ - private static PublishResult classify(Throwable failure, Duration elapsed) { - Throwable cause = failure instanceof CompletionException ? failure.getCause() : failure; - if (cause instanceof NatsPreSendRejection rejection) { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.CONFIGURATION, rejection.code(), rejection.getMessage()))); - } - boolean timedOut = cause instanceof java.util.concurrent.TimeoutException; - return new PublishResult( - PublishCompletion.AMBIGUOUS, - new PublishEvidence( - true, TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.TRANSIENT_INFRASTRUCTURE, - timedOut ? "NATS_PUBLISH_TIMEOUT" : "NATS_PUBLISH_OUTCOME_UNKNOWN", - timedOut - ? "the publish acknowledgement did not arrive; the stream may already hold it" - : "the publish failed in a way this adapter does not classify; the stream may" - + " already hold the message"))); - } - - private static TransportPublishResult rejectedLocally(String code, String message) { - return new TransportPublishResult( - new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - // A local refusal: nothing was attempted, so no time was spent on the wire. - Duration.ZERO, - Optional.of(FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message)))); - } - - private static CompletionStage completed(TransportPublishResult result) { - return CompletableFuture.completedFuture(result); - } - - /** The JetStream publish, isolated so the adapter is testable without a server. */ - @FunctionalInterface - public interface JetStreamPublishOperation { - - /** - * Publishes one encoded message to JetStream. - * - * @param subject the JetStream subject - * @param deduplicationId the {@code Nats-Msg-Id} value, when deduplication is enabled - * @param request the publish request - * @return a stage completing with the stream position the message landed at - */ - CompletionStage publish( - String subject, Optional deduplicationId, TransportPublishRequest request); - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingWorkflow.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingWorkflow.java deleted file mode 100644 index d13ee0c..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingWorkflow.java +++ /dev/null @@ -1,85 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import java.util.Objects; - -/** - * Decides what to do with a JetStream delivery before its {@code maxDeliver} limit destroys it. - * - *

JetStream has no dead-letter queue. When a message hits {@code maxDeliver} the server - * terminates it: no redelivery, no routing, no record beyond an advisory. Every other broker in - * this platform parks a poison message somewhere an operator can find it, and this workflow is what - * makes NATS behave the same way. - * - *

The parking therefore happens on the delivery before the limit, not on the limit - * itself. Acting at {@code maxDeliver} would mean acting on the delivery JetStream is about to - * discard, so any failure in the dead-letter publish would lose the message outright — precisely - * the case the DLQ-confirm-before-settle invariant exists to prevent. - */ -public final class NatsMaxDeliverParkingWorkflow { - - /** What the platform should do with one delivery. */ - public enum Action { - /** Hand the message to the handler as normal. */ - DELIVER_TO_HANDLER, - /** Publish to the platform dead-letter destination, then terminate the source. */ - PARK_TO_DEAD_LETTER, - /** - * The message has already passed the limit and JetStream has terminated it. - * - *

Reachable only if the profile changed under a live consumer. Nothing can be recovered - * here; the outcome exists so the situation is reported rather than mistaken for a redelivery. - */ - ALREADY_TERMINATED - } - - private final NatsJetStreamProfile profile; - - /** - * Creates a workflow for one destination. - * - * @param profile the JetStream settings - */ - public NatsMaxDeliverParkingWorkflow(NatsJetStreamProfile profile) { - this.profile = Objects.requireNonNull(profile, "profile must not be null"); - } - - /** - * Decides what to do with a delivery. - * - * @param deliveryCount how many times this message has been delivered, counting this one - * @return the action to take - */ - public Action decide(long deliveryCount) { - if (deliveryCount < 1) { - throw new IllegalArgumentException("the first delivery is delivery 1"); - } - if (deliveryCount > profile.maxDeliver()) { - return Action.ALREADY_TERMINATED; - } - return deliveryCount >= profile.parkAtDelivery() - ? Action.PARK_TO_DEAD_LETTER - : Action.DELIVER_TO_HANDLER; - } - - /** - * Reports whether the source may be terminated after a dead-letter publish. - * - *

Only on a confirmed publish. Terminating first would discard the message on a broker that - * cannot redeliver it, which is the one irreversible mistake available here. - * - * @param deadLetterConfirmed whether the dead-letter publish was confirmed - * @return true when the source may be terminated - */ - public boolean maySettleSource(boolean deadLetterConfirmed) { - return deadLetterConfirmed; - } - - /** - * Returns the delivery count at which parking begins. - * - * @return the parking threshold - */ - public int parkingThreshold() { - return profile.parkAtDelivery(); - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsPreSendRejection.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsPreSendRejection.java deleted file mode 100644 index 039addc..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsPreSendRejection.java +++ /dev/null @@ -1,65 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import java.util.Objects; - -/** - * The one signal that lets this adapter report {@code REJECTED / NOT_TRANSMITTED}. - * - *

Classification used to work by reading the exception's class simple name and matching - * substrings — {@code "Timeout"} meant ambiguous, anything else meant rejected. Two things are - * wrong with that. A class name is not part of any SDK's contract: it changes between versions, and - * an unrelated exception whose name happens to contain the substring is misread. And defaulting the - * unknown case to {@code REJECTED} tells the caller nothing was transmitted, so a caller that - * republishes under a new identity duplicates a message the server may well have stored. - * - *

The publish operation throws this only when it knows the send never left the client: an - * argument the SDK refused, a connection that was already closed, a subject the client could not - * resolve. Everything else — including anything the adapter does not recognise — is ambiguous, and - * ambiguity is the honest answer when the write may have landed. - */ -public final class NatsPreSendRejection extends RuntimeException { - - private static final long serialVersionUID = 1L; - - private final String code; - - /** - * Creates a pre-send rejection. - * - * @param code the sanitised platform code for the refusal - * @param message a sanitised description - */ - public NatsPreSendRejection(String code, String message) { - super(message); - this.code = requireCode(code); - } - - /** - * Creates a pre-send rejection wrapping the SDK's own exception. - * - * @param code the sanitised platform code for the refusal - * @param message a sanitised description - * @param cause the SDK exception that proves the send never started - */ - public NatsPreSendRejection(String code, String message, Throwable cause) { - super(message, cause); - this.code = requireCode(code); - } - - private static String requireCode(String code) { - Objects.requireNonNull(code, "code must not be null"); - if (code.isBlank()) { - throw new IllegalArgumentException("code must not be blank"); - } - return code; - } - - /** - * The sanitised code reported to the caller. - * - * @return the code - */ - public String code() { - return code; - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsStreamPosition.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsStreamPosition.java deleted file mode 100644 index 7463b69..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsStreamPosition.java +++ /dev/null @@ -1,75 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import dev.caskeleton.messaging.api.publish.BrokerPosition; -import java.util.Map; -import java.util.Objects; - -/** - * A JetStream message's coordinate: its stream, its sequence in that stream, and the consumer's own - * sequence. - * - *

Two sequences, because they answer different questions. The stream sequence identifies the - * message and is what a replay seeks to. The consumer sequence counts deliveries to this - * consumer, so it advances on every redelivery while the stream sequence does not — comparing them - * is how a redelivery is recognised at all. - * - * @param stream the stream holding the message - * @param streamSequence the message's position in the stream - * @param consumerSequence how many messages this consumer has been delivered - * @param deliveryCount how many times this message has been delivered - */ -public record NatsStreamPosition( - String stream, long streamSequence, long consumerSequence, long deliveryCount) - implements BrokerPosition { - - public NatsStreamPosition { - if (stream == null || stream.isBlank()) { - throw new IllegalArgumentException("stream must not be blank"); - } - if (streamSequence < 1) { - throw new IllegalArgumentException("JetStream sequences start at 1"); - } - if (deliveryCount < 1) { - throw new IllegalArgumentException("a delivered message has been delivered at least once"); - } - } - - @Override - public String broker() { - return "nats"; - } - - @Override - public Map diagnosticAttributes() { - return Map.of( - "stream", stream, - "streamSequence", Long.toString(streamSequence), - "consumerSequence", Long.toString(consumerSequence), - "deliveryCount", Long.toString(deliveryCount)); - } - - /** - * Reports whether this delivery is a redelivery. - * - * @return true when the message has been delivered before - */ - public boolean isRedelivery() { - return deliveryCount > 1; - } - - /** - * Returns the stream sequence a replay should resume from to re-read this message. - * - * @return the resume sequence - */ - public long replayFrom() { - return streamSequence; - } - - @Override - public String toString() { - return Objects.toString( - "NatsStreamPosition[stream=%s, streamSequence=%d, deliveryCount=%d]" - .formatted(stream, streamSequence, deliveryCount)); - } -} diff --git a/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsAdapterContractTest.java b/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsAdapterContractTest.java deleted file mode 100644 index 6e9dd33..0000000 --- a/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsAdapterContractTest.java +++ /dev/null @@ -1,337 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; -import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import dev.caskeleton.messaging.policy.CapabilityTier; -import dev.caskeleton.messaging.policy.ConsumerPolicy; -import dev.caskeleton.messaging.policy.DeadLetterPolicy; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.PayloadPolicy; -import dev.caskeleton.messaging.policy.PhysicalDestination; -import dev.caskeleton.messaging.policy.ProducerPolicy; -import dev.caskeleton.messaging.policy.RetryPolicy; -import dev.caskeleton.messaging.policy.SchemaPolicy; -import dev.caskeleton.messaging.schema.EncodedMessage; -import dev.caskeleton.messaging.schema.SchemaCompatibility; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import dev.caskeleton.messaging.transport.TransportPublishResult; -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Optional; -import java.util.Set; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.TimeoutException; -import org.junit.jupiter.api.Test; - -class NatsAdapterContractTest { - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - private static final NatsStreamPosition POSITION = new NatsStreamPosition("ORDERS", 42, 42, 1); - - private final List> capturedDeduplicationIds = new ArrayList<>(); - - private NatsJetStreamTransport confirming(Optional deduplicationWindow) { - return new NatsJetStreamTransport( - "nats-primary", - 1, - profile(deduplicationWindow), - (subject, deduplicationId, request) -> { - capturedDeduplicationIds.add(deduplicationId); - return CompletableFuture.completedFuture(POSITION); - }); - } - - private NatsJetStreamTransport failingWith(Throwable failure) { - return new NatsJetStreamTransport( - "nats-primary", - 1, - profile(Optional.of(Duration.ofMinutes(2))), - (subject, deduplicationId, request) -> CompletableFuture.failedFuture(failure)); - } - - private static PublishResult await(CompletionStage stage) { - return stage.toCompletableFuture().join().result(); - } - - @Test - void aStoredMessageReportsPersistenceEvidence() { - PublishResult result = await(confirming(Optional.empty()).publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.evidence().confirmationLevel()) - .as("the acknowledgement names the stream and sequence, not just bytes on a socket") - .isEqualTo(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK); - } - - @Test - void theStreamPositionIsCarriedBack() { - assertThat(await(confirming(Optional.empty()).publish(request(64))).position()) - .hasValue(POSITION); - } - - @Test - void aPublishTimeoutIsAmbiguousBecauseTheStreamMayHoldIt() { - PublishResult result = await(failingWith(new TimeoutException("no ack")).publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.evidence().transmission()) - .isEqualTo(TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED); - } - - @Test - void aFailureNamedLikeAKnownOneIsStillAmbiguous() { - // The class name used to be the classifier: an exception whose simple name contained - // "NoRespond" was reported REJECTED / NOT_TRANSMITTED. A name is not an SDK contract, and a - // caller that reads NOT_TRANSMITTED republishes under a new identity. - PublishResult result = - await(failingWith(new IllegalStateException("NoRespondersException")).publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("NATS_PUBLISH_OUTCOME_UNKNOWN")); - } - - @Test - void onlyATypedPreSendRejectionReportsNothingTransmitted() { - PublishResult result = - await( - failingWith( - new NatsPreSendRejection( - "NATS_CONNECTION_CLOSED", "the client was closed before the send")) - .publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()) - .isEqualTo(dev.caskeleton.messaging.api.publish.TransmissionEvidence.NOT_TRANSMITTED); - assertThat(result.failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("NATS_CONNECTION_CLOSED")); - } - - @Test - void aWrappedUnknownFailureIsAmbiguousRatherThanRejected() { - PublishResult result = - await( - failingWith( - new java.util.concurrent.CompletionException( - new RuntimeException("something the adapter has never seen"))) - .publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - } - - @Test - void aPublishThatNeverCompletesIsBoundedByTheCallersTimeout() { - try (NatsJetStreamTransport transport = - new NatsJetStreamTransport( - "nats-primary", - 1L, - profile(Optional.empty()), - (subject, deduplicationId, request) -> - new java.util.concurrent.CompletableFuture<>())) { - - PublishResult result = await(transport.publish(requestWithTimeout(Duration.ofMillis(50)))); - - assertThat(result.completion()) - .as("the caller's deadline used to be ignored entirely") - .isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("NATS_PUBLISH_TIMEOUT")); - } - } - - @Test - void theReportedElapsedTimeIsMeasuredRatherThanZero() { - PublishResult result = - await( - failingWith(new java.util.concurrent.TimeoutException("no ack")).publish(request(64))); - - assertThat(result.elapsed()) - .as("every outcome reported Duration.ZERO, so latency evidence was fiction") - .isGreaterThanOrEqualTo(Duration.ZERO); - } - - @Test - void aDeduplicationWindowSendsTheLogicalMessageIdSoARetryIsRecognised() { - confirming(Optional.of(Duration.ofMinutes(2))) - .publish(request(64)) - .toCompletableFuture() - .join(); - - assertThat(capturedDeduplicationIds) - .as("a fresh id per attempt would make the window useless exactly when it is needed") - .singleElement() - .satisfies(id -> assertThat(id).isPresent()); - } - - @Test - void noDeduplicationWindowSendsNoDeduplicationId() { - confirming(Optional.empty()).publish(request(64)).toCompletableFuture().join(); - - assertThat(capturedDeduplicationIds).singleElement().satisfies(id -> assertThat(id).isEmpty()); - } - - @Test - void anOversizedPayloadIsRejectedBeforeTheStream() { - PublishResult result = - await(confirming(Optional.empty()).publish(request(PayloadPolicy.DEFAULT_MAX_BYTES + 1))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.NOT_TRANSMITTED); - } - - @Test - void theAdapterDoesNotClaimANativeDeadLetter() { - assertThat( - confirming(Optional.empty()) - .capabilities(new DestinationName("orders.v1")) - .capabilities() - .nativeDeadLetter()) - .as("JetStream terminates at the delivery limit; it routes nowhere") - .isFalse(); - } - - @Test - void theAdapterAdvertisesDeduplicatedPublish() { - assertThat( - confirming(Optional.empty()) - .capabilities(new DestinationName("orders.v1")) - .capabilities() - .deduplicatedPublish()) - .isTrue(); - } - - @Test - void aClosedTransportStopsAcceptingWork() { - NatsJetStreamTransport transport = confirming(Optional.empty()); - transport.close(); - - assertThat(transport.isAcceptingWork()).isFalse(); - assertThat(await(transport.publish(request(64))).completion()) - .isEqualTo(PublishCompletion.REJECTED); - } - - @Test - void aRedeliveryIsRecognisedFromTheDeliveryCountNotTheSequence() { - NatsStreamPosition redelivered = new NatsStreamPosition("ORDERS", 42, 43, 2); - - assertThat(POSITION.isRedelivery()).isFalse(); - assertThat(redelivered.isRedelivery()).isTrue(); - assertThat(redelivered.replayFrom()) - .as("the stream sequence does not advance on a redelivery; the consumer sequence does") - .isEqualTo(POSITION.replayFrom()); - } - - @Test - void aSequenceBelowOneIsRejectedBecauseJetStreamCountsFromOne() { - assertThatThrownBy(() -> new NatsStreamPosition("ORDERS", 0, 1, 1)) - .isInstanceOf(IllegalArgumentException.class); - } - - @Test - void aFailureCategoryIsAlwaysAttachedToAFailure() { - PublishResult result = - await(failingWith(new IllegalStateException("boom")).publish(request(64))); - - assertThat(result.failure()) - .hasValueSatisfying( - failure -> - assertThat(failure.category()) - .as("an unclassified failure is transient, not a permanent business rejection") - .isEqualTo(FailureCategory.TRANSIENT_INFRASTRUCTURE)); - } - - private static NatsJetStreamProfile profile(Optional deduplicationWindow) { - return new NatsJetStreamProfile( - "orders.created", - "ORDERS", - "orders-worker", - NatsAckMode.EXPLICIT, - Duration.ofSeconds(30), - 5, - deduplicationWindow); - } - - private static TransportPublishRequest request(int payloadBytes) { - return new TransportPublishRequest( - destinationProfile(), envelope(payloadBytes), PublishOptions.defaults()); - } - - private static TransportPublishRequest requestWithTimeout(Duration timeout) { - PublishOptions defaults = PublishOptions.defaults(); - return new TransportPublishRequest( - destinationProfile(), - envelope(64), - new PublishOptions(timeout, defaults.confirmation(), defaults.deduplication())); - } - - private static DestinationProfile destinationProfile() { - return new DestinationProfile( - new DestinationName("orders.v1"), - "nats-primary", - DestinationKind.EVENT_STREAM, - PhysicalDestination.natsStream("orders.created", "ORDERS"), - new SchemaPolicy( - ContentType.JSON, - SchemaCompatibility.BACKWARD, - Set.of(new MessageType("order.created"))), - DeliveryGuarantee.AT_LEAST_ONCE, - OrderingScope.NONE, - ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, - ProducerPolicy.defaults(), - ConsumerPolicy.defaults("orders"), - RetryPolicy.none(), - DeadLetterPolicy.to(new DestinationName("orders.v1.dlq")), - PayloadPolicy.defaults(), - CapabilityTier.M1, - false, - false, - false); - } - - private static MessageEnvelope envelope(int payloadBytes) { - byte[] payload = new byte[payloadBytes]; - java.util.Arrays.fill(payload, (byte) 'x'); - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - new EncodedMessage(payload, ContentType.JSON, Optional.empty())); - } -} diff --git a/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingTest.java b/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingTest.java deleted file mode 100644 index 93d0a0c..0000000 --- a/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingTest.java +++ /dev/null @@ -1,123 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.nats.NatsMaxDeliverParkingWorkflow.Action; -import java.time.Duration; -import java.util.Optional; -import org.junit.jupiter.api.Test; - -class NatsMaxDeliverParkingTest { - - private static NatsJetStreamProfile profile(int maxDeliver) { - return new NatsJetStreamProfile( - "orders.created", - "ORDERS", - "orders-worker", - NatsAckMode.EXPLICIT, - Duration.ofSeconds(30), - maxDeliver, - Optional.of(Duration.ofMinutes(2))); - } - - private static NatsMaxDeliverParkingWorkflow workflow(int maxDeliver) { - return new NatsMaxDeliverParkingWorkflow(profile(maxDeliver)); - } - - @Test - void earlyDeliveriesGoToTheHandler() { - NatsMaxDeliverParkingWorkflow workflow = workflow(5); - - assertThat(workflow.decide(1)).isEqualTo(Action.DELIVER_TO_HANDLER); - assertThat(workflow.decide(3)).isEqualTo(Action.DELIVER_TO_HANDLER); - } - - @Test - void parkingHappensBeforeTheLimitNotAtIt() { - NatsMaxDeliverParkingWorkflow workflow = workflow(5); - - assertThat(workflow.decide(4)) - .as("the delivery that reaches maxDeliver is the one JetStream discards") - .isEqualTo(Action.PARK_TO_DEAD_LETTER); - } - - @Test - void theLimitDeliveryItselfStillParksRatherThanBeingHandled() { - assertThat(workflow(5).decide(5)).isEqualTo(Action.PARK_TO_DEAD_LETTER); - } - - @Test - void aDeliveryPastTheLimitIsReportedRatherThanMistakenForARedelivery() { - assertThat(workflow(5).decide(6)) - .as("only reachable if the profile changed under a live consumer; nothing is recoverable") - .isEqualTo(Action.ALREADY_TERMINATED); - } - - @Test - void theSourceIsOnlyTerminatedAfterTheDeadLetterPublishConfirms() { - NatsMaxDeliverParkingWorkflow workflow = workflow(5); - - assertThat(workflow.maySettleSource(true)).isTrue(); - assertThat(workflow.maySettleSource(false)) - .as("terminating first discards the message on a broker that cannot redeliver it") - .isFalse(); - } - - @Test - void aProfileWithNoParkingHeadroomIsRefused() { - assertThatThrownBy(() -> profile(1)) - .as("maxDeliver=1 leaves no delivery on which the platform could park the message") - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("headroom"); - } - - @Test - void aDeliveryCountBelowOneIsRejected() { - assertThatThrownBy(() -> workflow(5).decide(0)).isInstanceOf(IllegalArgumentException.class); - } - - @Test - void theParkingThresholdIsOneBelowTheLimit() { - assertThat(workflow(7).parkingThreshold()).isEqualTo(6); - } - - @Test - void ackNoneIsRefusedBecauseDeliveryIsTreatedAsCompletion() { - assertThatThrownBy( - () -> - new NatsJetStreamProfile( - "orders.created", - "ORDERS", - "orders-worker", - NatsAckMode.NONE, - Duration.ofSeconds(30), - 5, - Optional.empty())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("forgotten"); - } - - @Test - void ackAllIsRefusedBecauseItSettlesWorkStillInFlight() { - assertThatThrownBy( - () -> - new NatsJetStreamProfile( - "orders.created", - "ORDERS", - "orders-worker", - NatsAckMode.ALL, - Duration.ofSeconds(30), - 5, - Optional.empty())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("still in flight"); - } - - @Test - void onlyExplicitAcknowledgementCarriesAtLeastOnce() { - assertThat(NatsAckMode.EXPLICIT.supportsAtLeastOnce()).isTrue(); - assertThat(NatsAckMode.ALL.supportsAtLeastOnce()).isFalse(); - assertThat(NatsAckMode.NONE.supportsAtLeastOnce()).isFalse(); - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/build.gradle b/messaging/messaging-outbox-jdbc-postgresql/build.gradle index 36beb14..c933afe 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/build.gradle +++ b/messaging/messaging-outbox-jdbc-postgresql/build.gradle @@ -26,6 +26,9 @@ dependencies { // Live-database certification. The reliability patterns are claims about transaction // boundaries and uniqueness constraints, and only a real database can settle them. testImplementation project(':messaging:messaging-testkit') + // Test-only. The relay is also driven through the real DefaultMessagePublisher: a fake publisher + // accepted the pre-encoded rows the real one re-encoded and rejected. + testImplementation project(':messaging:messaging-runtime-core') integrationTestImplementation 'org.testcontainers:testcontainers-postgresql' integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' integrationTestImplementation 'org.postgresql:postgresql' diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java b/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java index 992ad2f..0e2d870 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java +++ b/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java @@ -506,22 +506,16 @@ class OutboxPostgresIT { } @Test - void theStoredRoutingKeyIsTheOneBothRelaysWouldUse() { + void theStoredRoutingKeyFallsBackToTheMessageId() { OutboxRecord keyed = record(OutboxCanonicalMetadata.none().withRoutingKeys("customer-42", "customer-42")); OutboxRecord unkeyed = record(OutboxCanonicalMetadata.none()); appendInTransaction(keyed); appendInTransaction(unkeyed); - DebeziumOutboxEventRouter router = new DebeziumOutboxEventRouter(); - assertThat(routingKeyOf(keyed.messageId())) - .as("the CDC key used to be the destination, so every message on a topic shared a key") - .isEqualTo(router.routingKey(keyed)) - .isEqualTo("customer-42"); - assertThat(routingKeyOf(unkeyed.messageId())) - .as("the fallback is stated once, in the column, and the Java agrees with it") - .isEqualTo(router.routingKey(unkeyed)) - .isEqualTo(unkeyed.messageId().value().toString()); + assertThat(List.of(routingKeyOf(keyed.messageId()), routingKeyOf(unkeyed.messageId()))) + .as("the fallback is stated once, in the column") + .containsExactly("customer-42", unkeyed.messageId().value().toString()); } @Test diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumMappedRecord.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumMappedRecord.java deleted file mode 100644 index cd53419..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumMappedRecord.java +++ /dev/null @@ -1,54 +0,0 @@ -package dev.caskeleton.messaging.outbox; - -import java.util.Arrays; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; - -/** - * What one outbox row looks like after Debezium's Event Router has transformed it. - * - *

Modelled explicitly so the CDC path can be tested without running Debezium. The property that - * matters is not that the transform is invoked but that its output carries the same identity, - * destination, and reserved headers the polling relay would have produced — and that is asserted by - * comparing this record against one built from the same row. - * - * @param topic the topic the row was routed to - * @param key the broker partition key, absent when the row is unkeyed - * @param payload the encoded payload - * @param headers the reserved headers carried alongside - */ -@SuppressWarnings("ArrayRecordComponent") -public record DebeziumMappedRecord( - String topic, Optional key, byte[] payload, Map headers) { - - public DebeziumMappedRecord { - Objects.requireNonNull(key, "key must not be null"); - Objects.requireNonNull(payload, "payload must not be null"); - Objects.requireNonNull(headers, "headers must not be null"); - if (topic == null || topic.isBlank()) { - throw new IllegalArgumentException("topic must not be blank"); - } - payload = payload.clone(); - headers = Map.copyOf(headers); - } - - @Override - public byte[] payload() { - return payload.clone(); - } - - @Override - public boolean equals(Object other) { - return other instanceof DebeziumMappedRecord record - && topic.equals(record.topic) - && key.equals(record.key) - && headers.equals(record.headers) - && Arrays.equals(payload, record.payload); - } - - @Override - public int hashCode() { - return Objects.hash(topic, key, headers, Arrays.hashCode(payload)); - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxEventRouter.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxEventRouter.java deleted file mode 100644 index dbb64a8..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxEventRouter.java +++ /dev/null @@ -1,151 +0,0 @@ -package dev.caskeleton.messaging.outbox; - -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.reliability.OutboxCanonicalMetadata; -import dev.caskeleton.messaging.reliability.OutboxRecord; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Objects; - -/** - * Maps outbox columns onto the headers Debezium's Outbox Event Router expects. - * - *

An optional alternative to the polling relay. CDC reads the database's write-ahead log instead - * of querying the table, which removes the polling interval and the lease contention — at the cost - * of an extra piece of infrastructure and its own failure modes. - * - *

The platform contract does not change with the transport. The router emits the same reserved - * headers the polling relay does, so a consumer cannot tell which one published a message, and - * switching between them is a deployment decision rather than a contract change. - * - *

The message key and the topic are two different questions and used to be answered by one - * column. {@code destination} routed the topic and became the key, so every message on a - * topic carried the same key, landed on one partition, and made keyed ordering a claim with no - * mechanism behind it. The key now comes from {@code routing_key}, the stored column that states - * the partition-key-or-message-id fallback once for both relays, and {@code destination} stays what - * it always was: routing metadata. - * - *

One canonical field has no header here: the schema subject. There is no reserved header name - * for it, and inventing one that only this path wrote would replace a symmetric gap with an - * asymmetric one — the polling relay would carry a schema reference on the envelope and the CDC - * relay a header no adapter reads. It stays a column that the polling relay resolves, and closing - * it means adding the name to {@code ReservedHeaders} and teaching both broker header mappers to - * write and read it. - */ -public final class DebeziumOutboxEventRouter { - - /** The outbox column Debezium routes the topic by. */ - public static final String ROUTE_BY_COLUMN = "destination"; - - /** The outbox column Debezium uses as the message key. */ - public static final String KEY_COLUMN = "routing_key"; - - /** The outbox column carrying the logical message id. */ - public static final String ID_COLUMN = "message_id"; - - /** The outbox column carrying the payload. */ - public static final String PAYLOAD_COLUMN = "payload"; - - /** - * Returns the router configuration for a Debezium connector. - * - * @param topicPrefix the prefix routed topics are created under - * @return the connector properties - */ - public Map connectorConfiguration(String topicPrefix) { - Objects.requireNonNull(topicPrefix, "topicPrefix must not be null"); - Map configuration = new LinkedHashMap<>(); - configuration.put("transforms.outbox.type", "io.debezium.transforms.outbox.EventRouter"); - configuration.put("transforms.outbox.table.field.event.id", ID_COLUMN); - configuration.put("transforms.outbox.table.field.event.key", KEY_COLUMN); - configuration.put("transforms.outbox.table.field.event.payload", PAYLOAD_COLUMN); - configuration.put("transforms.outbox.route.by.field", ROUTE_BY_COLUMN); - configuration.put( - "transforms.outbox.route.topic.replacement", topicPrefix + "${routedByValue}"); - configuration.put( - "transforms.outbox.table.fields.additional.placement", - String.join( - ",", - "message_type:header:" + ReservedHeaders.MESSAGE_TYPE, - "schema_version:header:" + ReservedHeaders.SCHEMA_VERSION, - "content_type:header:" + ReservedHeaders.CONTENT_TYPE, - "created_at:header:" + ReservedHeaders.PRODUCED_AT, - "destination:header:" + ReservedHeaders.ORIGIN_DESTINATION, - "producer:header:" + ReservedHeaders.PRODUCER, - "occurred_at:header:" + ReservedHeaders.OCCURRED_AT, - "correlation_id:header:" + ReservedHeaders.CORRELATION_ID, - "causation_id:header:" + ReservedHeaders.CAUSATION_ID, - "tenant:header:" + ReservedHeaders.TENANT, - "partition_key:header:" + ReservedHeaders.PARTITION_KEY, - "ordering_key:header:" + ReservedHeaders.ORDERING_KEY, - "traceparent:header:" + ReservedHeaders.TRACEPARENT, - "tracestate:header:" + ReservedHeaders.TRACESTATE, - "baggage:header:" + ReservedHeaders.BAGGAGE)); - return Map.copyOf(configuration); - } - - /** - * Returns the reserved headers a routed record must carry. - * - *

Used to assert that the CDC path and the polling path produce the same wire contract. - * - * @param record the outbox row - * @return the reserved headers - */ - public Map reservedHeaders(OutboxRecord record) { - Objects.requireNonNull(record, "record must not be null"); - Map headers = new LinkedHashMap<>(); - headers.put(ReservedHeaders.MESSAGE_ID, record.messageId().value().toString()); - headers.put(ReservedHeaders.MESSAGE_TYPE, record.messageType().value()); - headers.put(ReservedHeaders.SCHEMA_VERSION, Integer.toString(record.schemaVersion().value())); - headers.put(ReservedHeaders.CONTENT_TYPE, record.contentType().value()); - headers.put(ReservedHeaders.PRODUCED_AT, record.createdAt().toString()); - headers.put(ReservedHeaders.ORIGIN_DESTINATION, record.destination().value()); - - // Absent stays absent. Writing an empty string for a column the row does not have would make a - // message that carries no tenant indistinguishable from one whose tenant is the empty string, - // and the consumer would have to know which of the two this platform meant. - OutboxCanonicalMetadata metadata = record.metadata(); - metadata.producer().ifPresent(value -> headers.put(ReservedHeaders.PRODUCER, value.value())); - metadata - .occurredAt() - .ifPresent(value -> headers.put(ReservedHeaders.OCCURRED_AT, value.toString())); - metadata - .correlationId() - .ifPresent(value -> headers.put(ReservedHeaders.CORRELATION_ID, value.value())); - metadata - .causationId() - .ifPresent( - value -> headers.put(ReservedHeaders.CAUSATION_ID, value.value().value().toString())); - metadata.tenant().ifPresent(value -> headers.put(ReservedHeaders.TENANT, value.tenantId())); - metadata.partitionKey().ifPresent(value -> headers.put(ReservedHeaders.PARTITION_KEY, value)); - metadata.orderingKey().ifPresent(value -> headers.put(ReservedHeaders.ORDERING_KEY, value)); - metadata - .traceContext() - .traceparent() - .ifPresent(value -> headers.put(ReservedHeaders.TRACEPARENT, value)); - metadata - .traceContext() - .tracestate() - .ifPresent(value -> headers.put(ReservedHeaders.TRACESTATE, value)); - metadata - .traceContext() - .baggage() - .ifPresent(value -> headers.put(ReservedHeaders.BAGGAGE, value)); - return Map.copyOf(headers); - } - - /** - * Returns the key a routed record carries. - * - *

The same value the {@code routing_key} column computes, stated here so a test can compare - * the two rather than trusting that the SQL and the Java agree. - * - * @param record the outbox row - * @return the broker message key - */ - public String routingKey(OutboxRecord record) { - Objects.requireNonNull(record, "record must not be null"); - return record.metadata().partitionKey().orElseGet(() -> record.messageId().value().toString()); - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxProfile.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxProfile.java deleted file mode 100644 index 4c08572..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxProfile.java +++ /dev/null @@ -1,67 +0,0 @@ -package dev.caskeleton.messaging.outbox; - -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import java.util.Objects; - -/** - * Which relay a deployment uses, and the rule that only one of them may be active. - * - *

Running the polling relay and the CDC relay together publishes every row twice. Both are - * correct in isolation and neither can see the other: the poller leases and marks rows, while - * Debezium reads the write-ahead log and never observes the lease at all. The duplicate is - * therefore invisible in the outbox table and only shows up downstream, which is why the - * incompatibility is enforced at startup instead of documented. - * - * @param mode which relay publishes outbox rows - * @param topicPrefix the prefix CDC-routed topics are created under - * @param aggregateIdAsPartitionKey whether the aggregate id becomes the broker partition key - */ -public record DebeziumOutboxProfile( - RelayMode mode, String topicPrefix, boolean aggregateIdAsPartitionKey) { - - /** How outbox rows reach the broker. */ - public enum RelayMode { - /** The in-process relay leases and publishes rows. */ - POLLING, - /** Debezium reads the write-ahead log and routes rows. */ - CHANGE_DATA_CAPTURE - } - - public DebeziumOutboxProfile { - Objects.requireNonNull(mode, "mode must not be null"); - Objects.requireNonNull(topicPrefix, "topicPrefix must not be null"); - if (mode == RelayMode.CHANGE_DATA_CAPTURE && topicPrefix.isBlank()) { - throw new IllegalArgumentException("a CDC profile must name a topic prefix"); - } - } - - /** - * Returns the default polling profile. - * - * @return the polling profile - */ - public static DebeziumOutboxProfile polling() { - return new DebeziumOutboxProfile(RelayMode.POLLING, "", false); - } - - /** - * Refuses a configuration that would run both relays at once. - * - * @param pollingRelayEnabled whether the in-process relay bean is active - * @throws MessagingConfigurationException when both relays would run - */ - public void requireExactlyOneRelay(boolean pollingRelayEnabled) { - if (mode == RelayMode.CHANGE_DATA_CAPTURE && pollingRelayEnabled) { - throw new MessagingConfigurationException( - "DUPLICATE_OUTBOX_RELAY", - "the CDC relay and the polling relay are both enabled; Debezium never observes the " - + "polling lease, so every row would be published twice"); - } - if (mode == RelayMode.POLLING && !pollingRelayEnabled) { - throw new MessagingConfigurationException( - "NO_OUTBOX_RELAY", - "the profile selects the polling relay but no relay is enabled, so outbox rows would " - + "accumulate without ever being published"); - } - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapper.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapper.java deleted file mode 100644 index 513f246..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapper.java +++ /dev/null @@ -1,79 +0,0 @@ -package dev.caskeleton.messaging.outbox; - -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.reliability.OutboxRecord; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; - -/** - * Produces what Debezium's Event Router will emit for an outbox row. - * - *

Exists so the CDC path is testable without Debezium running. The property worth proving is - * that a consumer cannot tell which relay published a message: same logical id, same destination, - * same reserved headers. Building the expected record here and comparing it against the polling - * relay's output is what makes "switching relays is a deployment decision, not a contract change" - * an assertion rather than a claim. - * - *

The message id comes from the row's {@code message_id} column, never from Debezium's own event - * id. Debezium mints a fresh event id per change record, so using it would break identity across a - * CDC re-snapshot — exactly the scenario where identity matters most. - * - *

The reserved headers come from {@link DebeziumOutboxEventRouter} rather than being rebuilt - * here. Two lists of the same headers is how the connector's configuration and the expectation this - * class encodes come to disagree, and the disagreement would be invisible: the test compares this - * output against the polling relay's, so it would pass while the connector emitted something else. - */ -public final class DebeziumOutboxRecordMapper { - - private final String topicPrefix; - private final boolean aggregateIdAsPartitionKey; - private final DebeziumOutboxEventRouter router = new DebeziumOutboxEventRouter(); - - /** - * Creates a mapper from a CDC profile. - * - * @param profile the outbox relay profile - */ - public DebeziumOutboxRecordMapper(DebeziumOutboxProfile profile) { - Objects.requireNonNull(profile, "profile must not be null"); - this.topicPrefix = profile.topicPrefix(); - this.aggregateIdAsPartitionKey = profile.aggregateIdAsPartitionKey(); - } - - /** - * Maps one outbox row to its routed record. - * - * @param record the outbox row - * @return the record a CDC relay would emit - */ - public DebeziumMappedRecord map(OutboxRecord record) { - Objects.requireNonNull(record, "record must not be null"); - - // Application headers first, canonical headers second, and the canonical ones win. The row's - // header map is application data; a row that wrote `msg.id` into it would otherwise overwrite - // the identity the platform put there, and the consumer trusts the last such header it sees. - // The write path refuses reserved names outright — this order is the second lock on the same - // door, because a CDC connector reading the table directly never passes through that check. - Map headers = new LinkedHashMap<>(record.headers()); - headers.keySet().removeIf(ReservedHeaders::isReserved); - headers.putAll(router.reservedHeaders(record)); - - return new DebeziumMappedRecord( - topicPrefix + record.destination().value(), - partitionKey(record), - record.payload(), - headers); - } - - private Optional partitionKey(OutboxRecord record) { - if (!aggregateIdAsPartitionKey) { - return Optional.empty(); - } - // The same value the polling relay would use and the same value the `routing_key` column - // computes, so both relays land the same message on the same partition and neither reorders a - // keyed stream. - return Optional.of(router.routingKey(record)); - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/debezium/outbox-event-router.properties b/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/debezium/outbox-event-router.properties deleted file mode 100644 index b28a254..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/debezium/outbox-event-router.properties +++ /dev/null @@ -1,43 +0,0 @@ -# Debezium Outbox Event Router configuration for the messaging_outbox table. -# -# Optional. Enable this OR the in-process polling relay, never both: Debezium reads the -# write-ahead log and never observes the polling lease, so with both running every row is -# published twice and the duplicate is invisible in the outbox table itself. -# -# Operating the Debezium connector — its offsets, its snapshots, its slot — is out of scope for -# this module. What is in scope is that a message routed by CDC is indistinguishable from one -# published by the polling relay: same logical id, same destination, same reserved headers. - -name=messaging-outbox-connector -connector.class=io.debezium.connector.postgresql.PostgresConnector - -# Only the outbox table. A wider include list turns every business table change into a message. -table.include.list=public.messaging_outbox - -# The row is deleted or marked after routing; the tombstone would otherwise reach the topic as a -# null-payload record that consumers must special-case. -tombstones.on.delete=false - -transforms=outbox -transforms.outbox.type=io.debezium.transforms.outbox.EventRouter - -# Identity comes from the row's own message_id, not from Debezium's per-change event id. A CDC -# re-snapshot mints new event ids for rows that already published, which would break identity in -# exactly the situation where it matters most. -transforms.outbox.table.field.event.id=message_id -transforms.outbox.table.field.event.key=destination -transforms.outbox.table.field.event.payload=payload -transforms.outbox.table.field.event.timestamp=created_at - -transforms.outbox.route.by.field=destination -transforms.outbox.route.topic.replacement=${routedByValue} - -# The reserved headers a consumer relies on, carried through the transform so the CDC path emits -# the same envelope the polling relay does. -transforms.outbox.table.fields.additional.placement=message_type:header:msg.type,schema_version:header:msg.schema-version,content_type:header:msg.content-type,message_id:header:msg.id - -# Publisher idempotence and full acknowledgement: a CDC relay that loses a record cannot replay it -# without re-reading the log from an earlier offset, which would republish everything after it. -producer.override.enable.idempotence=true -producer.override.acks=all -producer.override.max.in.flight.requests.per.connection=5 diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapperTest.java b/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapperTest.java deleted file mode 100644 index e49e162..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapperTest.java +++ /dev/null @@ -1,199 +0,0 @@ -package dev.caskeleton.messaging.outbox; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.outbox.DebeziumOutboxProfile.RelayMode; -import dev.caskeleton.messaging.reliability.OutboxCanonicalMetadata; -import dev.caskeleton.messaging.reliability.OutboxRecord; -import dev.caskeleton.messaging.reliability.OutboxStatus; -import java.nio.charset.StandardCharsets; -import java.time.Instant; -import java.util.Map; -import java.util.Optional; -import org.junit.jupiter.api.Test; - -class DebeziumOutboxRecordMapperTest { - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - private static final MessageId MESSAGE_ID = MessageId.newId(); - - private static OutboxRecord record(Map headers) { - return record(headers, OutboxCanonicalMetadata.none()); - } - - private static OutboxRecord record( - Map headers, OutboxCanonicalMetadata metadata) { - return new OutboxRecord( - MESSAGE_ID, - new DestinationName("orders.v1"), - new MessageType("order.created"), - new SchemaVersion(1), - ContentType.JSON, - "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), - headers, - metadata, - NOW, - OutboxStatus.PENDING, - 0, - Optional.empty(), - Optional.empty()); - } - - private static DebeziumOutboxRecordMapper mapper(boolean aggregateIdAsPartitionKey) { - return new DebeziumOutboxRecordMapper( - new DebeziumOutboxProfile( - RelayMode.CHANGE_DATA_CAPTURE, "prod.", aggregateIdAsPartitionKey)); - } - - @Test - void theRoutedTopicIsThePrefixedDestination() { - assertThat(mapper(false).map(record(Map.of())).topic()).isEqualTo("prod.orders.v1"); - } - - @Test - void theLogicalMessageIdSurvivesTheCdcPath() { - DebeziumMappedRecord mapped = mapper(false).map(record(Map.of())); - - assertThat(mapped.headers().get(ReservedHeaders.MESSAGE_ID)) - .as("Debezium mints a fresh event id per change record; using it would break identity") - .isEqualTo(MESSAGE_ID.value().toString()); - } - - @Test - void everyReservedHeaderAConsumerNeedsIsCarried() { - Map headers = mapper(false).map(record(Map.of())).headers(); - - assertThat(headers) - .containsKeys( - ReservedHeaders.MESSAGE_ID, - ReservedHeaders.MESSAGE_TYPE, - ReservedHeaders.SCHEMA_VERSION, - ReservedHeaders.CONTENT_TYPE, - ReservedHeaders.PRODUCED_AT); - } - - @Test - void applicationHeadersOnTheRowArePreserved() { - DebeziumMappedRecord mapped = mapper(false).map(record(Map.of("x-tenant-tier", "gold"))); - - assertThat(mapped.headers()).containsEntry("x-tenant-tier", "gold"); - } - - @Test - void anUnkeyedProfileEmitsNoPartitionKey() { - assertThat(mapper(false).map(record(Map.of())).key()).isEmpty(); - } - - @Test - void aKeyedProfileUsesTheSamePartitionKeyThePollingRelayWouldHave() { - DebeziumMappedRecord mapped = - mapper(true) - .map( - record( - Map.of(), - OutboxCanonicalMetadata.none().withRoutingKeys("customer-42", "customer-42"))); - - assertThat(mapped.key()) - .as("a different key would send the two relays' messages to different partitions") - .hasValue("customer-42"); - } - - @Test - void aKeyedProfileFallsBackToTheMessageIdWhenTheRowHasNoPartitionKey() { - assertThat(mapper(true).map(record(Map.of())).key()) - .as("the fallback the routing_key column computes, stated in one place for both relays") - .hasValue(MESSAGE_ID.value().toString()); - } - - @Test - void theCanonicalColumnsBecomeReservedHeaders() { - OutboxCanonicalMetadata metadata = - OutboxCanonicalMetadata.none() - .withProducer(new dev.caskeleton.messaging.api.ProducerId("order-api")) - .withTenant(new dev.caskeleton.messaging.api.TenantContext("acme")) - .withRoutingKeys("customer-42", "customer-42"); - - Map headers = mapper(false).map(record(Map.of(), metadata)).headers(); - - assertThat(headers) - .as("a relay that drops these makes the publish path part of the message's meaning") - .containsEntry(ReservedHeaders.PRODUCER, "order-api") - .containsEntry(ReservedHeaders.TENANT, "acme") - .containsEntry(ReservedHeaders.PARTITION_KEY, "customer-42") - .containsEntry(ReservedHeaders.ORDERING_KEY, "customer-42") - .containsEntry(ReservedHeaders.ORIGIN_DESTINATION, "orders.v1"); - } - - @Test - void aColumnTheRowDoesNotCarryProducesNoHeaderAtAll() { - assertThat(mapper(false).map(record(Map.of())).headers()) - .as("an empty string would be indistinguishable from a tenant literally named ''") - .doesNotContainKeys( - ReservedHeaders.TENANT, ReservedHeaders.CORRELATION_ID, ReservedHeaders.TRACEPARENT); - } - - @Test - void aForgedReservedHeaderOnTheRowCannotOverwriteTheCanonicalOne() { - Map forged = - Map.of(ReservedHeaders.MESSAGE_ID, "00000000-0000-7000-8000-000000000000"); - - assertThat(mapper(false).map(record(forged)).headers()) - .as("a CDC connector reads the table directly and never passes the write-path check") - .containsEntry(ReservedHeaders.MESSAGE_ID, MESSAGE_ID.value().toString()); - } - - @Test - void theRoutedKeyIsNotTheTopicName() { - // Literals, not the class's own constants: comparing a configuration value against the constant - // that produced it asserts that the router agrees with itself, which it always will. - assertThat(new DebeziumOutboxEventRouter().connectorConfiguration("prod.")) - .as("keying by destination puts every message on a topic onto one partition") - .containsEntry("transforms.outbox.table.field.event.key", "routing_key") - .containsEntry("transforms.outbox.route.by.field", "destination"); - } - - @Test - void thePayloadIsUnchangedByTheTransform() { - assertThat(mapper(false).map(record(Map.of())).payload()) - .isEqualTo("{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8)); - } - - @Test - void runningBothRelaysAtOnceIsRefused() { - DebeziumOutboxProfile cdc = - new DebeziumOutboxProfile(RelayMode.CHANGE_DATA_CAPTURE, "p.", false); - - assertThatThrownBy(() -> cdc.requireExactlyOneRelay(true)) - .as("Debezium never sees the polling lease, so every row would publish twice") - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("twice"); - } - - @Test - void runningNoRelayAtAllIsAlsoRefused() { - assertThatThrownBy(() -> DebeziumOutboxProfile.polling().requireExactlyOneRelay(false)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("accumulate"); - } - - @Test - void exactlyOneRelayIsAccepted() { - assertThatCode(() -> DebeziumOutboxProfile.polling().requireExactlyOneRelay(true)) - .doesNotThrowAnyException(); - } - - @Test - void aCdcProfileMustNameATopicPrefix() { - assertThatThrownBy(() -> new DebeziumOutboxProfile(RelayMode.CHANGE_DATA_CAPTURE, " ", false)) - .isInstanceOf(IllegalArgumentException.class); - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java b/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java index 471ba56..3a8256a 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java +++ b/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java @@ -8,6 +8,11 @@ import dev.caskeleton.messaging.api.MessageId; import dev.caskeleton.messaging.api.MessageType; import dev.caskeleton.messaging.api.ProducerId; import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; +import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; +import dev.caskeleton.messaging.api.delivery.OrderingScope; +import dev.caskeleton.messaging.api.destination.DestinationCapabilities; +import dev.caskeleton.messaging.api.destination.DestinationKind; import dev.caskeleton.messaging.api.destination.DestinationName; import dev.caskeleton.messaging.api.destination.MessageDestination; import dev.caskeleton.messaging.api.error.FailureCategory; @@ -19,11 +24,37 @@ import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; import dev.caskeleton.messaging.api.publish.RoutingOutcome; +import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ConsumerPolicy; +import dev.caskeleton.messaging.policy.DeadLetterPolicy; +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.policy.InFlightLimiter; +import dev.caskeleton.messaging.policy.MessagingAdmissionController; +import dev.caskeleton.messaging.policy.PayloadLimitGuard; +import dev.caskeleton.messaging.policy.PayloadPolicy; +import dev.caskeleton.messaging.policy.PhysicalDestination; +import dev.caskeleton.messaging.policy.ProducerPolicy; +import dev.caskeleton.messaging.policy.RetryPolicy; +import dev.caskeleton.messaging.policy.SchemaPolicy; import dev.caskeleton.messaging.reliability.OutboxLease; import dev.caskeleton.messaging.reliability.OutboxRecord; import dev.caskeleton.messaging.reliability.OutboxRepository; import dev.caskeleton.messaging.reliability.OutboxStatus; import dev.caskeleton.messaging.reliability.OutboxTransitionResult; +import dev.caskeleton.messaging.runtime.DefaultMessagePublisher; +import dev.caskeleton.messaging.runtime.DestinationProfileRegistry; +import dev.caskeleton.messaging.runtime.TransportMessagingRuntime; +import dev.caskeleton.messaging.schema.MessageCodec; +import dev.caskeleton.messaging.schema.MessageCodecRegistry; +import dev.caskeleton.messaging.schema.RawBytesMessageCodec; +import dev.caskeleton.messaging.schema.SchemaCompatibility; +import dev.caskeleton.messaging.security.DestinationAccessPolicy; +import dev.caskeleton.messaging.transport.DefaultMessagingRuntimeRegistry; +import dev.caskeleton.messaging.transport.MessagingTransport; +import dev.caskeleton.messaging.transport.TransportConsumerRegistration; +import dev.caskeleton.messaging.transport.TransportConsumerSpec; +import dev.caskeleton.messaging.transport.TransportPublishRequest; +import dev.caskeleton.messaging.transport.TransportPublishResult; import java.nio.charset.StandardCharsets; import java.time.Duration; import java.time.Instant; @@ -32,6 +63,7 @@ import java.util.LinkedHashMap; import java.util.List; import java.util.Map; import java.util.Optional; +import java.util.Set; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionStage; import org.junit.jupiter.api.Test; @@ -162,6 +194,35 @@ class OutboxRelayTest { assertThat(envelope.schemaVersion()).isEqualTo(new SchemaVersion(1)); } + @Test + void aRowRelayedThroughTheRealPublisherIsMarkedPublished() { + // given + OutboxRecord record = OutboxFixtures.pending(); + repository.append(record); + + // when + relay(RealPublisher.over(new CapturingTransport())).runOnce(NOW); + + // then + assertThat(repository.find(record.messageId()).orElseThrow().status()) + .as("re-encoding the stored bytes made the codec refuse them and ended every row FAILED") + .isEqualTo(OutboxStatus.PUBLISHED); + } + + @Test + void aRowRelayedThroughTheRealPublisherReachesTheTransportWithItsStoredBytes() { + // given + OutboxRecord record = OutboxFixtures.pending(); + repository.append(record); + CapturingTransport transport = new CapturingTransport(); + + // when + relay(RealPublisher.over(transport)).runOnce(NOW); + + // then + assertThat(transport.payloads()).containsExactly(record.payload()); + } + private OutboxRelay relay(MessagePublisher publisher) { return new OutboxRelay( repository, @@ -480,3 +541,110 @@ final class RecordingOutboxPublisher implements MessagePublisher { }; } } + +/** The platform's own publisher over a stub transport, registered for the fixture's destination. */ +final class RealPublisher { + + private static final DestinationName ORDERS = new DestinationName("order-events"); + + private RealPublisher() {} + + static DefaultMessagePublisher over(MessagingTransport transport) { + DefaultMessagingRuntimeRegistry runtimes = new DefaultMessagingRuntimeRegistry(); + runtimes.install(new TransportMessagingRuntime("fake", 1, transport)); + // A real codec. Like every codec it refuses a payload that is not its own type, which is what + // an already-encoded row is to it. + MessageCodec raw = new RawBytesMessageCodec(); + return new DefaultMessagePublisher( + DestinationProfileRegistry.of(Map.of(ORDERS, profile())), + new DestinationAccessPolicy(Set.of(ORDERS), Set.of(), Set.of()), + new MessageCodecRegistry() { + @Override + public Optional find(ContentType contentType) { + return Optional.of(raw); + } + + @Override + public MessageCodec defaultCodec() { + return raw; + } + }, + new MessagingAdmissionController( + new PayloadLimitGuard(PayloadPolicy.defaults()), + new InFlightLimiter(4, Duration.ofSeconds(1))), + runtimes, + transport); + } + + private static DestinationProfile profile() { + return new DestinationProfile( + ORDERS, + "fake", + DestinationKind.EVENT_STREAM, + PhysicalDestination.kafkaTopic("order.events.v1"), + new SchemaPolicy( + ContentType.JSON, + SchemaCompatibility.BACKWARD_TRANSITIVE, + Set.of(new MessageType("order.created"))), + DeliveryGuarantee.AT_LEAST_ONCE, + OrderingScope.NONE, + ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, + ProducerPolicy.defaults(), + ConsumerPolicy.defaults("order-projection"), + RetryPolicy.none(), + DeadLetterPolicy.disabled(), + PayloadPolicy.defaults(), + CapabilityTier.M1, + false, + false, + false); + } +} + +/** A transport that confirms every publish and keeps the bytes it was handed. */ +final class CapturingTransport implements MessagingTransport { + + private final List payloads = new ArrayList<>(); + + List payloads() { + return List.copyOf(payloads); + } + + @Override + public CompletionStage publish(TransportPublishRequest request) { + payloads.add(request.envelope().payload().bytes()); + return CompletableFuture.completedFuture( + new TransportPublishResult( + new PublishResult( + PublishCompletion.CONFIRMED, + PublishEvidence.confirmed(ConfirmationLevel.BROKER_ACK), + RoutingOutcome.ROUTED, + Optional.empty(), + 1, + Duration.ZERO, + Optional.empty()))); + } + + @Override + public TransportConsumerRegistration register(TransportConsumerSpec spec) { + throw new UnsupportedOperationException("the relay only publishes"); + } + + @Override + public DestinationCapabilities capabilities(DestinationName destination) { + throw new UnsupportedOperationException("the relay asks for no publish option"); + } + + @Override + public String brokerName() { + return "fake"; + } + + @Override + public long generation() { + return 1; + } + + @Override + public void close() {} +} diff --git a/messaging/messaging-pulsar-experimental/build.gradle b/messaging/messaging-pulsar-experimental/build.gradle deleted file mode 100644 index 929bf94..0000000 --- a/messaging/messaging-pulsar-experimental/build.gradle +++ /dev/null @@ -1,14 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-transport-spi') - api project(':messaging:messaging-observability') - api project(':messaging:messaging-security') - api project(':messaging:messaging-admin-api') - -} diff --git a/messaging/messaging-pulsar-experimental/gradle.lockfile b/messaging/messaging-pulsar-experimental/gradle.lockfile deleted file mode 100644 index f5ad1bb..0000000 --- a/messaging/messaging-pulsar-experimental/gradle.lockfile +++ /dev/null @@ -1,89 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=runtimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=runtimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty= diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagePosition.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagePosition.java deleted file mode 100644 index 7271ac0..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagePosition.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import dev.caskeleton.messaging.api.publish.BrokerPosition; -import java.util.Map; - -/** - * A Pulsar message id, decomposed into its parts. - * - *

Kept as four fields rather than the opaque {@code MessageId} string so a diagnostic can be - * read without a Pulsar client to parse it. {@code batchIndex} is the part that surprises people: a - * batched message's id is shared by every message in the batch, so the index is what makes an - * individual message addressable at all. - * - * @param ledgerId the ledger holding the entry - * @param entryId the entry within the ledger - * @param partition the partition index, or -1 for a non-partitioned topic - * @param batchIndex the index within a batch, or -1 when the message was not batched - */ -public record PulsarMessagePosition(long ledgerId, long entryId, int partition, int batchIndex) - implements BrokerPosition { - - /** The value used where a topic is not partitioned or a message is not batched. */ - public static final int NOT_APPLICABLE = -1; - - @Override - public String broker() { - return "pulsar"; - } - - @Override - public Map diagnosticAttributes() { - return Map.of( - "ledgerId", Long.toString(ledgerId), - "entryId", Long.toString(entryId), - "partition", Integer.toString(partition), - "batchIndex", Integer.toString(batchIndex)); - } - - /** - * Returns the canonical Pulsar message id rendering. - * - * @return the message id as Pulsar prints it - */ - public String toMessageId() { - return batchIndex == NOT_APPLICABLE - ? "%d:%d:%d".formatted(ledgerId, entryId, partition) - : "%d:%d:%d:%d".formatted(ledgerId, entryId, partition, batchIndex); - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagingTransport.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagingTransport.java deleted file mode 100644 index 1884d57..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagingTransport.java +++ /dev/null @@ -1,275 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import dev.caskeleton.messaging.api.destination.DestinationCapabilities; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import dev.caskeleton.messaging.transport.MessagingTransport; -import dev.caskeleton.messaging.transport.TransportConsumerRegistration; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import dev.caskeleton.messaging.transport.TransportPublishResult; -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicBoolean; -import java.util.function.Function; - -/** - * The Experimental Pulsar adapter. - * - *

This is an Experimental contract seam, not a Stable adapter. It exercises the transport SPI - * against a send operation the application supplies; it does not ship a Pulsar client bridge, - * producer lifecycle, or reconnection. Only a failure the operation reports as a {@link - * PulsarPreSendRejection} is treated as definitely not transmitted — every other failure, - * recognised or not, is ambiguous. - * - *

Disabled unless explicitly enabled. Experimental means the contract suite has been run but the - * live-broker certification has not, so the adapter must never become load-bearing because a - * default switched it on. - * - *

Pulsar's send future resolves with a {@code MessageId} once the broker has persisted the - * message to its configured number of bookies, which is genuine replication evidence rather than a - * receipt. A send that fails with a timeout is reported {@code AMBIGUOUS}: the write may have - * reached the bookies and only the acknowledgement been lost, and treating that as a rejection is - * how a message gets published twice. - */ -public final class PulsarMessagingTransport implements MessagingTransport { - - /** What this adapter can prove today. Transactions stay false while Experimental. */ - private static final MessagingCapabilities SHARED_CAPABILITIES = - new MessagingCapabilities( - true, true, true, true, false, false, true, true, false, false, true, true); - - private static final MessagingCapabilities KEY_SHARED_CAPABILITIES = - new MessagingCapabilities( - true, true, true, true, false, true, true, true, false, false, true, true); - - private final String brokerName; - private final long generation; - private final PulsarProfile profile; - private final PulsarSendOperation send; - private final Function consumerFactory; - private final AtomicBoolean closed = new AtomicBoolean(); - - /** - * Creates a publish-only transport. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param profile the Pulsar destination settings - * @param send the producer send operation - */ - public PulsarMessagingTransport( - String brokerName, long generation, PulsarProfile profile, PulsarSendOperation send) { - this( - brokerName, - generation, - profile, - send, - spec -> { - throw new MessagingCapabilityUnavailableException( - "PULSAR_CONSUMER_NOT_CONFIGURED", - "this Pulsar transport was created without a consumer factory"); - }); - } - - /** - * Creates a transport with a consumer factory. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param profile the Pulsar destination settings - * @param send the producer send operation - * @param consumerFactory builds a consumer registration for a spec - */ - public PulsarMessagingTransport( - String brokerName, - long generation, - PulsarProfile profile, - PulsarSendOperation send, - Function consumerFactory) { - this.brokerName = Objects.requireNonNull(brokerName, "brokerName must not be null"); - this.generation = generation; - this.profile = Objects.requireNonNull(profile, "profile must not be null"); - this.send = Objects.requireNonNull(send, "send must not be null"); - this.consumerFactory = Objects.requireNonNull(consumerFactory, "consumerFactory is required"); - } - - @Override - public CompletionStage publish(TransportPublishRequest request) { - Objects.requireNonNull(request, "request must not be null"); - - if (closed.get()) { - return completed( - rejectedLocally("PULSAR_TRANSPORT_CLOSED", "the transport is shutting down")); - } - int size = request.envelope().payload().size(); - int limit = request.profile().payload().maxBytes(); - if (size > limit) { - return completed( - rejectedLocally( - "PAYLOAD_TOO_LARGE", "encoded payload is " + size + " bytes, limit is " + limit)); - } - - // The caller's own deadline, honoured rather than ignored: a stalled send used to hold the - // caller for whatever the SDK's default was, whatever the caller had asked for. - long startedAt = System.nanoTime(); - return send.send(profile.topic(), request) - .toCompletableFuture() - .orTimeout( - request.options().timeout().toMillis(), java.util.concurrent.TimeUnit.MILLISECONDS) - .handle( - (position, failure) -> - failure == null - ? confirmed(position, elapsedSince(startedAt)) - : classify(failure, elapsedSince(startedAt))) - .thenApply(TransportPublishResult::new); - } - - @Override - public TransportConsumerRegistration register(TransportConsumerSpec spec) { - Objects.requireNonNull(spec, "spec must not be null"); - if (closed.get()) { - throw new MessagingCapabilityUnavailableException( - "PULSAR_TRANSPORT_CLOSED", "the transport is shutting down"); - } - return consumerFactory.apply(spec); - } - - @Override - public DestinationCapabilities capabilities(DestinationName destination) { - MessagingCapabilities capabilities = - profile.mode().subscriptionType() == PulsarSubscriptionType.KEY_SHARED - ? KEY_SHARED_CAPABILITIES - : SHARED_CAPABILITIES; - return new DestinationCapabilities(destination, brokerName, capabilities); - } - - @Override - public String brokerName() { - return brokerName; - } - - @Override - public long generation() { - return generation; - } - - @Override - public void close() { - closed.set(true); - } - - /** - * Reports whether the transport is still accepting work. - * - * @return true until close - */ - public boolean isAcceptingWork() { - return !closed.get(); - } - - private static Duration elapsedSince(long startedAtNanos) { - return Duration.ofNanos(System.nanoTime() - startedAtNanos); - } - - private static PublishResult confirmed(PulsarMessagePosition position, Duration elapsed) { - return new PublishResult( - PublishCompletion.CONFIRMED, - // The send future resolves only after the entry is persisted to the configured bookies, so - // this is replication evidence rather than a broker receipt. - new PublishEvidence( - true, - TransmissionEvidence.TRANSMITTED, - true, - ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK), - RoutingOutcome.ROUTED, - Optional.of(position), - 1, - elapsed, - Optional.empty()); - } - - /** - * Decides what a send failure means, from typed signals only. - * - *

Classification used to read the exception's class simple name: {@code "Timeout"} meant - * ambiguous, anything else meant rejected. A class name is not part of Pulsar's contract — it - * changes between client versions — and defaulting the unknown case to {@code REJECTED} tells the - * caller nothing was transmitted, which is how the same entry is published to the bookies twice. - */ - private static PublishResult classify(Throwable failure, Duration elapsed) { - Throwable cause = - failure instanceof java.util.concurrent.CompletionException ? failure.getCause() : failure; - if (cause instanceof PulsarPreSendRejection rejection) { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.CONFIGURATION, rejection.code(), rejection.getMessage()))); - } - boolean timedOut = cause instanceof java.util.concurrent.TimeoutException; - return new PublishResult( - PublishCompletion.AMBIGUOUS, - new PublishEvidence( - true, TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.TRANSIENT_INFRASTRUCTURE, - timedOut ? "PULSAR_SEND_TIMEOUT" : "PULSAR_SEND_OUTCOME_UNKNOWN", - timedOut - ? "the send timed out; the broker may already hold the message" - : "the send failed in a way this adapter does not classify; the broker may" - + " already hold the message"))); - } - - private static TransportPublishResult rejectedLocally(String code, String message) { - return new TransportPublishResult( - new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of(FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message)))); - } - - private static CompletionStage completed(TransportPublishResult result) { - return CompletableFuture.completedFuture(result); - } - - /** The producer send, isolated so the adapter is testable without a broker. */ - @FunctionalInterface - public interface PulsarSendOperation { - - /** - * Sends one encoded message. - * - * @param topic the Pulsar topic - * @param request the publish request - * @return a stage completing with the broker position - */ - CompletionStage send(String topic, TransportPublishRequest request); - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarPreSendRejection.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarPreSendRejection.java deleted file mode 100644 index 6cf5f62..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarPreSendRejection.java +++ /dev/null @@ -1,65 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import java.util.Objects; - -/** - * The one signal that lets this adapter report {@code REJECTED / NOT_TRANSMITTED}. - * - *

Classification used to work by reading the exception's class simple name and matching - * substrings — {@code "Timeout"} meant ambiguous, anything else meant rejected. Two things are - * wrong with that. A class name is not part of any SDK's contract: it changes between versions, and - * an unrelated exception whose name happens to contain the substring is misread. And defaulting the - * unknown case to {@code REJECTED} tells the caller nothing was transmitted, so a caller that - * republishes under a new identity duplicates a message the server may well have stored. - * - *

The send operation throws this only when it knows the send never left the client: an argument - * the SDK refused, a producer that was already closed, a topic the client could not resolve. - * Everything else — including anything the adapter does not recognise — is ambiguous, and ambiguity - * is the honest answer when the write may have landed. - */ -public final class PulsarPreSendRejection extends RuntimeException { - - private static final long serialVersionUID = 1L; - - private final String code; - - /** - * Creates a pre-send rejection. - * - * @param code the sanitised platform code for the refusal - * @param message a sanitised description - */ - public PulsarPreSendRejection(String code, String message) { - super(message); - this.code = requireCode(code); - } - - /** - * Creates a pre-send rejection wrapping the SDK's own exception. - * - * @param code the sanitised platform code for the refusal - * @param message a sanitised description - * @param cause the SDK exception that proves the send never started - */ - public PulsarPreSendRejection(String code, String message, Throwable cause) { - super(message, cause); - this.code = requireCode(code); - } - - private static String requireCode(String code) { - Objects.requireNonNull(code, "code must not be null"); - if (code.isBlank()) { - throw new IllegalArgumentException("code must not be blank"); - } - return code; - } - - /** - * The sanitised code reported to the caller. - * - * @return the code - */ - public String code() { - return code; - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfile.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfile.java deleted file mode 100644 index 154cce1..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfile.java +++ /dev/null @@ -1,80 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; - -/** - * The Pulsar-side settings for one destination. - * - *

{@code negativeAckRedeliveryDelay} is the platform's retry hook on Pulsar. Pulsar has no retry - * topic of its own in this adapter, so a failed handler negatively acknowledges and the broker - * redelivers after this delay — which means the delay is the backoff, and setting it to - * zero converts a failing handler into a spin loop against the broker. - * - *

{@code ackTimeout} is deliberately optional and off by default. It redelivers any message the - * consumer has not acknowledged in time, which looks like a safety net and behaves like a duplicate - * generator for handlers that are simply slow. - * - * @param topic the fully-qualified Pulsar topic - * @param subscription the durable subscription name - * @param mode the subscription type and acknowledgement style - * @param negativeAckRedeliveryDelay how long the broker waits after a negative acknowledgement - * @param ackTimeout redelivers unacknowledged messages after this long, when set - * @param maxRedeliverCount how many redeliveries before the message is parked - */ -public record PulsarProfile( - String topic, - String subscription, - PulsarSubscriptionMode mode, - Duration negativeAckRedeliveryDelay, - Optional ackTimeout, - int maxRedeliverCount) { - - /** Pulsar's own floor for an acknowledgement timeout. */ - public static final Duration MINIMUM_ACK_TIMEOUT = Duration.ofSeconds(10); - - public PulsarProfile { - Objects.requireNonNull(mode, "mode must not be null"); - Objects.requireNonNull(negativeAckRedeliveryDelay, "negativeAckRedeliveryDelay is required"); - Objects.requireNonNull(ackTimeout, "ackTimeout must not be null"); - requireText(topic, "topic"); - requireText(subscription, "subscription"); - - if (negativeAckRedeliveryDelay.isNegative() || negativeAckRedeliveryDelay.isZero()) { - throw new IllegalArgumentException( - "negativeAckRedeliveryDelay must be positive; zero turns a failing handler into a spin " - + "loop against the broker"); - } - if (maxRedeliverCount < 1) { - throw new IllegalArgumentException("maxRedeliverCount must be at least 1"); - } - if (ackTimeout.isPresent() && ackTimeout.get().compareTo(MINIMUM_ACK_TIMEOUT) < 0) { - throw new IllegalArgumentException( - "ackTimeout below %s is rejected by Pulsar itself".formatted(MINIMUM_ACK_TIMEOUT)); - } - } - - /** - * Returns a shared-subscription profile with the platform defaults. - * - * @param topic the Pulsar topic - * @param subscription the subscription name - * @return the profile - */ - public static PulsarProfile shared(String topic, String subscription) { - return new PulsarProfile( - topic, - subscription, - PulsarSubscriptionMode.shared(), - Duration.ofSeconds(30), - Optional.empty(), - 5); - } - - private static void requireText(String value, String field) { - if (value == null || value.isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfileValidator.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfileValidator.java deleted file mode 100644 index b0e0db3..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfileValidator.java +++ /dev/null @@ -1,66 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.util.Objects; - -/** - * Guards the Pulsar Experimental adapter. - * - *

The subscription type and the destination's ordering scope have to agree, and the platform - * checks rather than assumes. A {@code Shared} subscription distributes messages round-robin across - * consumers, so a destination that declares key ordering on one is advertising a guarantee Pulsar - * is not providing — the messages arrive, in an order nobody promised. - * - *

Experimental means off by default. An adapter whose contract suite is still being proven must - * not become load-bearing because someone's configuration defaulted it on. - */ -public final class PulsarProfileValidator { - - /** - * Validates a destination against a Pulsar subscription. - * - * @param profile the destination profile - * @param subscriptionType the Pulsar subscription type - * @param enabled whether the experimental adapter is switched on - */ - public void validate( - DestinationProfile profile, PulsarSubscriptionType subscriptionType, boolean enabled) { - Objects.requireNonNull(profile, "profile must not be null"); - Objects.requireNonNull(subscriptionType, "subscriptionType must not be null"); - - if (!enabled) { - throw new MessagingCapabilityUnavailableException( - "PULSAR_DISABLED", - "the Pulsar adapter is experimental and disabled unless " - + "app.messaging.experimental.pulsar=true"); - } - if (profile.orderingScope() == OrderingScope.KEY - && subscriptionType != PulsarSubscriptionType.KEY_SHARED) { - throw new IllegalArgumentException( - "keyed ordering requires a Key_Shared subscription: " + profile.name().value()); - } - if (profile.orderingScope() == OrderingScope.DESTINATION) { - throw new IllegalArgumentException( - "the Pulsar adapter does not offer destination-wide ordering: " + profile.name().value()); - } - } - - /** - * Returns what the Pulsar adapter can prove for a subscription type. - * - *

Transactions are reported as unavailable while the adapter is Experimental: Pulsar has them, - * but the platform has not yet proven them against its own contract suite, and advertising an - * unverified guarantee is exactly what the Experimental tier exists to prevent. - * - * @param subscriptionType the subscription type - * @return the capabilities - */ - public MessagingCapabilities capabilities(PulsarSubscriptionType subscriptionType) { - boolean keyed = subscriptionType == PulsarSubscriptionType.KEY_SHARED; - return new MessagingCapabilities( - true, true, true, true, keyed, keyed, true, true, false, false, true, true); - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionMode.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionMode.java deleted file mode 100644 index 4d2c53e..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionMode.java +++ /dev/null @@ -1,62 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import java.util.Objects; - -/** - * How a Pulsar subscription is configured, and what ordering it can actually deliver. - * - *

Pairs the subscription type with its acknowledgement style because the two together determine - * the guarantee. A {@code Key_Shared} subscription with cumulative acknowledgement is not keyed - * ordering with a faster ack — cumulative ack over interleaved keys acknowledges messages from keys - * the consumer has not finished, so the combination silently loses the property the subscription - * type was chosen for. - * - * @param subscriptionType the Pulsar subscription type - * @param cumulativeAcknowledgement whether acknowledgements are cumulative rather than individual - */ -public record PulsarSubscriptionMode( - PulsarSubscriptionType subscriptionType, boolean cumulativeAcknowledgement) { - - public PulsarSubscriptionMode { - Objects.requireNonNull(subscriptionType, "subscriptionType must not be null"); - if (cumulativeAcknowledgement && subscriptionType == PulsarSubscriptionType.KEY_SHARED) { - throw new IllegalArgumentException( - "Key_Shared with cumulative acknowledgement acknowledges keys the consumer has not " - + "finished, which discards the per-key ordering the subscription type provides"); - } - if (cumulativeAcknowledgement && subscriptionType == PulsarSubscriptionType.SHARED) { - throw new IllegalArgumentException( - "Shared subscriptions do not support cumulative acknowledgement"); - } - } - - /** - * Returns a shared subscription with individual acknowledgement. - * - * @return the shared mode - */ - public static PulsarSubscriptionMode shared() { - return new PulsarSubscriptionMode(PulsarSubscriptionType.SHARED, false); - } - - /** - * Returns a key-shared subscription with individual acknowledgement. - * - * @return the key-shared mode - */ - public static PulsarSubscriptionMode keyShared() { - return new PulsarSubscriptionMode(PulsarSubscriptionType.KEY_SHARED, false); - } - - /** - * Returns the strongest ordering this mode can deliver. - * - * @return the ordering scope - */ - public OrderingScope orderingScope() { - return subscriptionType == PulsarSubscriptionType.KEY_SHARED - ? OrderingScope.KEY - : OrderingScope.NONE; - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionType.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionType.java deleted file mode 100644 index 4a88a32..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionType.java +++ /dev/null @@ -1,17 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -/** - * The Pulsar subscription types the Experimental adapter exposes. - * - *

Only the two that map cleanly onto the platform's ordering model. {@code Exclusive} and {@code - * Failover} are omitted deliberately: they encode a topology decision the destination profile - * already owns, and exposing them would give an application two places to configure the same thing. - */ -public enum PulsarSubscriptionType { - - /** Competing consumers, no ordering. */ - SHARED, - - /** Competing consumers with per-key ordering. */ - KEY_SHARED -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarTransactionCapability.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarTransactionCapability.java deleted file mode 100644 index 104caf4..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarTransactionCapability.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; - -/** - * States, in code, that Pulsar transactions are not promoted. - * - *

Pulsar has transactions. The platform does not offer them, and the distinction matters enough - * to be a class rather than a comment: an operator reading the capability matrix needs to know the - * answer is "not proven here", not "the broker cannot do it". - * - *

What is missing is the proof. The platform's contract suite has not been run against Pulsar - * transactions across broker restart, client fencing, and partial-commit recovery, and the - * Experimental tier exists precisely so that an unproven guarantee is never advertised as one. A - * capability that reports {@code true} on the strength of the broker's documentation is the failure - * mode this class prevents. - */ -public final class PulsarTransactionCapability { - - private PulsarTransactionCapability() {} - - /** Whether the adapter advertises broker transactions. */ - public static final boolean PROMOTED = false; - - /** - * Reports whether transactions may be used. - * - * @return false while the adapter is Experimental - */ - public static boolean isAvailable() { - return PROMOTED; - } - - /** - * Returns the refusal for a transactional request, for the caller to throw. - * - *

Returned rather than thrown so the refusal reads as a value at the call site: {@code throw - * PulsarTransactionCapability.rejection();} makes it obvious that control leaves there, which a - * method that throws on the caller's behalf hides. - * - * @return the exception explaining why transactions are unavailable - */ - public static MessagingCapabilityUnavailableException rejection() { - return new MessagingCapabilityUnavailableException( - "PULSAR_TRANSACTION_NOT_PROMOTED", - "Pulsar supports transactions but this adapter has not certified them against the " - + "platform contract suite, so the capability is not advertised"); - } -} diff --git a/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarAdapterContractTest.java b/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarAdapterContractTest.java deleted file mode 100644 index 0ec8223..0000000 --- a/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarAdapterContractTest.java +++ /dev/null @@ -1,289 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; -import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import dev.caskeleton.messaging.policy.CapabilityTier; -import dev.caskeleton.messaging.policy.ConsumerPolicy; -import dev.caskeleton.messaging.policy.DeadLetterPolicy; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.PayloadPolicy; -import dev.caskeleton.messaging.policy.PhysicalDestination; -import dev.caskeleton.messaging.policy.ProducerPolicy; -import dev.caskeleton.messaging.policy.RetryPolicy; -import dev.caskeleton.messaging.policy.SchemaPolicy; -import dev.caskeleton.messaging.schema.EncodedMessage; -import dev.caskeleton.messaging.schema.SchemaCompatibility; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import java.time.Instant; -import java.util.Optional; -import java.util.Set; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.TimeoutException; -import org.junit.jupiter.api.Test; - -class PulsarAdapterContractTest { - - private static final String TOPIC = "persistent://public/default/orders"; - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - private static final PulsarMessagePosition POSITION = new PulsarMessagePosition(5, 9, 0, -1); - - private static PulsarMessagingTransport transport( - PulsarMessagingTransport.PulsarSendOperation send, PulsarSubscriptionMode mode) { - PulsarProfile profile = - new PulsarProfile( - TOPIC, "orders-sub", mode, java.time.Duration.ofSeconds(30), Optional.empty(), 5); - return new PulsarMessagingTransport("pulsar-primary", 1, profile, send); - } - - private static PulsarMessagingTransport confirming() { - return transport( - (topic, request) -> CompletableFuture.completedFuture(POSITION), - PulsarSubscriptionMode.shared()); - } - - private static PulsarMessagingTransport failingWith(Throwable failure) { - return transport( - (topic, request) -> CompletableFuture.failedFuture(failure), - PulsarSubscriptionMode.shared()); - } - - private static PublishResult await(CompletionStage stage) { - Object value = stage.toCompletableFuture().join(); - return ((dev.caskeleton.messaging.transport.TransportPublishResult) value).result(); - } - - @Test - void aSuccessfulSendReportsReplicationEvidenceNotAMereReceipt() { - PublishResult result = await(confirming().publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.evidence().confirmationLevel()) - .as("the send future resolves only once the entry is persisted to the bookies") - .isEqualTo(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.ROUTED); - } - - @Test - void theBrokerPositionIsCarriedBackForDiagnostics() { - assertThat(await(confirming().publish(request(64))).position()).hasValue(POSITION); - } - - @Test - void aSendTimeoutIsAmbiguousRatherThanRejected() { - PublishResult result = - await(failingWith(new TimeoutException("send timed out")).publish(request(64))); - - assertThat(result.completion()) - .as("the entry may be on the bookies with only the acknowledgement lost") - .isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.evidence().transmission()) - .isEqualTo(TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED); - assertThat(result.evidence().brokerAccepted()).isFalse(); - } - - @Test - void onlyATypedPreSendRejectionIsReportedAsNotTransmitted() { - PublishResult result = - await( - failingWith( - new PulsarPreSendRejection( - "PULSAR_PRODUCER_FENCED", "the producer was fenced before the send")) - .publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.NOT_TRANSMITTED); - assertThat(result.failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("PULSAR_PRODUCER_FENCED")); - } - - @Test - void anUnrecognisedFailureIsAmbiguousRatherThanRejected() { - // Any exception the adapter cannot type is a send whose outcome it does not know. Reporting - // REJECTED invites the caller to republish an entry the bookies may already hold. - PublishResult result = - await(failingWith(new IllegalStateException("producer is fenced")).publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.evidence().transmission()) - .isEqualTo(TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED); - assertThat(result.failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("PULSAR_SEND_OUTCOME_UNKNOWN")); - } - - @Test - void aWrappedUnknownFailureIsAlsoAmbiguous() { - PublishResult result = - await( - failingWith( - new java.util.concurrent.CompletionException( - new RuntimeException("something the adapter has never seen"))) - .publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - } - - @Test - void aSendThatNeverCompletesIsBoundedByTheCallersTimeout() { - PulsarMessagingTransport stalled = - transport((topic, request) -> new CompletableFuture<>(), PulsarSubscriptionMode.shared()); - - PublishResult result = - await(stalled.publish(requestWithTimeout(java.time.Duration.ofMillis(50)))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.failure()) - .hasValueSatisfying(failure -> assertThat(failure.code()).isEqualTo("PULSAR_SEND_TIMEOUT")); - } - - @Test - void anOversizedPayloadIsRejectedBeforeItReachesTheBroker() { - PublishResult result = - await(confirming().publish(request(PayloadPolicy.DEFAULT_MAX_BYTES + 1))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.NOT_TRANSMITTED); - assertThat(result.failure()) - .hasValueSatisfying(failure -> assertThat(failure.code()).isEqualTo("PAYLOAD_TOO_LARGE")); - } - - @Test - void aClosedTransportStopsAcceptingNewWork() { - PulsarMessagingTransport transport = confirming(); - transport.close(); - - assertThat(transport.isAcceptingWork()).isFalse(); - assertThat(await(transport.publish(request(64))).completion()) - .isEqualTo(PublishCompletion.REJECTED); - } - - @Test - void aTransportWithoutAConsumerFactoryRefusesToRegisterRatherThanReturningNothing() { - assertThatThrownBy(() -> confirming().register(null)).isInstanceOf(NullPointerException.class); - } - - @Test - void aSharedSubscriptionAdvertisesNoKeyedOrdering() { - var capabilities = - transport( - (topic, request) -> CompletableFuture.completedFuture(POSITION), - PulsarSubscriptionMode.shared()) - .capabilities(new DestinationName("orders.v1")) - .capabilities(); - - assertThat(capabilities.keyedOrdering()).isFalse(); - } - - @Test - void aKeySharedSubscriptionAdvertisesKeyedOrdering() { - var capabilities = - transport( - (topic, request) -> CompletableFuture.completedFuture(POSITION), - PulsarSubscriptionMode.keyShared()) - .capabilities(new DestinationName("orders.v1")) - .capabilities(); - - assertThat(capabilities.keyedOrdering()).isTrue(); - } - - @Test - void noSubscriptionModeAdvertisesBrokerTransactions() { - assertThat( - transport( - (topic, request) -> CompletableFuture.completedFuture(POSITION), - PulsarSubscriptionMode.keyShared()) - .capabilities(new DestinationName("orders.v1")) - .capabilities() - .brokerTransaction()) - .as("Pulsar has transactions; this adapter has not certified them") - .isFalse(); - } - - @Test - void requestingATransactionFailsWithAnExplanation() { - assertThat(PulsarTransactionCapability.rejection()) - .isInstanceOf(MessagingCapabilityUnavailableException.class) - .hasMessageContaining("not advertised"); - } - - private static TransportPublishRequest requestWithTimeout(java.time.Duration timeout) { - PublishOptions defaults = PublishOptions.defaults(); - return new TransportPublishRequest( - profile(), - envelope(64), - new PublishOptions(timeout, defaults.confirmation(), defaults.deduplication())); - } - - private static TransportPublishRequest request(int payloadBytes) { - return new TransportPublishRequest( - profile(), envelope(payloadBytes), PublishOptions.defaults()); - } - - private static DestinationProfile profile() { - return new DestinationProfile( - new DestinationName("orders.v1"), - "pulsar-primary", - DestinationKind.EVENT_STREAM, - PhysicalDestination.pulsarTopic(TOPIC, "orders-sub"), - new SchemaPolicy( - ContentType.JSON, - SchemaCompatibility.BACKWARD, - Set.of(new MessageType("order.created"))), - DeliveryGuarantee.AT_LEAST_ONCE, - OrderingScope.NONE, - ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, - ProducerPolicy.defaults(), - ConsumerPolicy.defaults("orders"), - RetryPolicy.none(), - DeadLetterPolicy.to(new DestinationName("orders.v1.dlq")), - PayloadPolicy.defaults(), - CapabilityTier.M1, - false, - false, - false); - } - - private static MessageEnvelope envelope(int payloadBytes) { - byte[] payload = new byte[payloadBytes]; - java.util.Arrays.fill(payload, (byte) 'x'); - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - new EncodedMessage(payload, ContentType.JSON, Optional.empty())); - } -} diff --git a/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionGuardTest.java b/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionGuardTest.java deleted file mode 100644 index a7dea10..0000000 --- a/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionGuardTest.java +++ /dev/null @@ -1,125 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import java.time.Duration; -import java.util.Optional; -import org.junit.jupiter.api.Test; - -class PulsarSubscriptionGuardTest { - - @Test - void keySharedWithCumulativeAcknowledgementIsRefused() { - assertThatThrownBy(() -> new PulsarSubscriptionMode(PulsarSubscriptionType.KEY_SHARED, true)) - .as("cumulative ack over interleaved keys discards the ordering the type provides") - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("ordering"); - } - - @Test - void sharedWithCumulativeAcknowledgementIsRefused() { - assertThatThrownBy(() -> new PulsarSubscriptionMode(PulsarSubscriptionType.SHARED, true)) - .isInstanceOf(IllegalArgumentException.class); - } - - @Test - void aSharedSubscriptionPromisesNoOrdering() { - assertThat(PulsarSubscriptionMode.shared().orderingScope()).isEqualTo(OrderingScope.NONE); - } - - @Test - void aKeySharedSubscriptionPromisesKeyOrdering() { - assertThat(PulsarSubscriptionMode.keyShared().orderingScope()).isEqualTo(OrderingScope.KEY); - } - - @Test - void aZeroNegativeAckDelayIsRefused() { - assertThatThrownBy( - () -> - new PulsarProfile( - "persistent://public/default/orders", - "orders-sub", - PulsarSubscriptionMode.shared(), - Duration.ZERO, - Optional.empty(), - 5)) - .as("a zero redelivery delay turns a failing handler into a spin loop") - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("spin loop"); - } - - @Test - void anAckTimeoutBelowPulsarsOwnFloorIsRefused() { - assertThatThrownBy( - () -> - new PulsarProfile( - "persistent://public/default/orders", - "orders-sub", - PulsarSubscriptionMode.shared(), - Duration.ofSeconds(30), - Optional.of(Duration.ofSeconds(1)), - 5)) - .isInstanceOf(IllegalArgumentException.class); - } - - @Test - void theDefaultProfileLeavesAckTimeoutOff() { - assertThat( - PulsarProfile.shared("persistent://public/default/orders", "orders-sub").ackTimeout()) - .as("an ack timeout redelivers messages from handlers that are merely slow") - .isEmpty(); - } - - @Test - void transactionsAreNotAdvertisedWhileTheAdapterIsExperimental() { - assertThat(PulsarTransactionCapability.isAvailable()).isFalse(); - assertThat(PulsarTransactionCapability.rejection()) - .isInstanceOf(MessagingCapabilityUnavailableException.class); - } - - @Test - void theRefusalExplainsThatTheGapIsProofNotBrokerSupport() { - assertThat(PulsarTransactionCapability.rejection()) - .hasMessageContaining("certified") - .hasMessageContaining("contract suite"); - } - - @Test - void aNonBatchedPositionRendersWithoutABatchIndex() { - PulsarMessagePosition position = - new PulsarMessagePosition(12, 34, 2, PulsarMessagePosition.NOT_APPLICABLE); - - assertThat(position.toMessageId()).isEqualTo("12:34:2"); - assertThat(position.broker()).isEqualTo("pulsar"); - } - - @Test - void aBatchedPositionKeepsItsIndexBecauseTheIdAloneIsShared() { - assertThat(new PulsarMessagePosition(12, 34, 2, 7).toMessageId()).isEqualTo("12:34:2:7"); - } - - @Test - void thePositionIsReadableAsDiagnosticAttributes() { - assertThat(new PulsarMessagePosition(12, 34, 2, 7).diagnosticAttributes()) - .containsEntry("ledgerId", "12") - .containsEntry("batchIndex", "7"); - } - - @Test - void theValidatorAcceptsAKeyedProfileOnKeyShared() { - assertThatCode( - () -> - new PulsarProfile( - "persistent://public/default/orders", - "orders-sub", - PulsarSubscriptionMode.keyShared(), - Duration.ofSeconds(30), - Optional.empty(), - 5)) - .doesNotThrowAnyException(); - } -} diff --git a/messaging/messaging-rabbit/build.gradle b/messaging/messaging-rabbit/build.gradle deleted file mode 100644 index f346996..0000000 --- a/messaging/messaging-rabbit/build.gradle +++ /dev/null @@ -1,43 +0,0 @@ -plugins { - id 'java-test-fixtures' - id 'ca.auxiliary-source-set' - id 'ca.jmh-benchmarks' -} - -auxiliarySourceSets { - sourceSet('integrationTest') { - compilesAgainst 'main', 'testFixtures' - } -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-transport-spi') - api project(':messaging:messaging-observability') - api project(':messaging:messaging-security') - api project(':messaging:messaging-admin-api') - - // api: RabbitChannelPublisher and the mappers expose Spring AMQP's Message on public - // signatures, so a consumer implementing the seam needs the type on its compile classpath. - api 'org.springframework.amqp:spring-rabbit' - implementation 'org.springframework:spring-context' - - // Test-only. The same adapter contract Kafka runs, so a guarantee can only be weakened by - // editing the contract rather than by an adapter quietly not implementing it. - testImplementation project(':messaging:messaging-testkit') - - // Live-broker certification. The confirm-and-return ordering the adapter depends on is a - // property of AMQP, and only a real broker exhibits it. - integrationTestImplementation 'org.testcontainers:testcontainers-rabbitmq' - integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' -} - -strictTestLanes { - lane('integrationTest') { - integration() - sourceSet = 'integrationTest' - description = 'Runs RabbitMQ broker integration contracts against a real container.' - } -} diff --git a/messaging/messaging-rabbit/gradle.lockfile b/messaging/messaging-rabbit/gradle.lockfile deleted file mode 100644 index e5000c2..0000000 --- a/messaging/messaging-rabbit/gradle.lockfile +++ /dev/null @@ -1,128 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.fasterxml.jackson.core:jackson-annotations:2.21=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.github.docker-java:docker-java-api:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.docker-java:docker-java-transport-zerodep:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.docker-java:docker-java-transport:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -com.rabbitmq:amqp-client:5.27.1=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-codec:commons-codec:1.19.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.20.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -commons-io:commons-io:2.21.0=spotbugs -commons-logging:commons-logging:1.3.6=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-buffer:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-codec-base:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-codec-compression:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-codec-marshalling:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-codec-protobuf:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-codec:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-common:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-handler:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-resolver:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-transport-native-unix-common:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-transport:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -net.java.dev.jna:jna:5.18.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -net.sf.jopt-simple:jopt-simple:5.0.4=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-compress:1.28.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.apache.commons:commons-lang3:3.20.0=checkstyle,integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs -org.apache.commons:commons-math3:3.6.1=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,jmhCompileClasspath,testCompileClasspath -org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=integrationTestRuntimeClasspath,jmhRuntimeClasspath,runtimeClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jetbrains:annotations:17.0.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,integrationTestAnnotationProcessor,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testFixturesAnnotationProcessor,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testRuntimeClasspath -org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,runtimeClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.openjdk.jmh:jmh-core:1.37=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath -org.openjdk.jmh:jmh-generator-annprocess:1.37=jmhAnnotationProcessor -org.opentest4j:opentest4j:1.3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.rnorth.duct-tape:duct-tape:1.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.slf4j:slf4j-api:2.0.18=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,spotbugs,spotbugsSlf4j,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.springframework.amqp:spring-amqp:4.0.5=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework.amqp:spring-rabbit:4.0.5=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-aop:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-beans:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-context:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-core:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-expression:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-messaging:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-tx:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.testcontainers:testcontainers-junit-jupiter:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers-rabbitmq:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty= diff --git a/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitBrokerIT.java b/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitBrokerIT.java deleted file mode 100644 index 7937f52..0000000 --- a/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitBrokerIT.java +++ /dev/null @@ -1,232 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; - -import com.rabbitmq.client.AMQP; -import com.rabbitmq.client.Channel; -import com.rabbitmq.client.Connection; -import com.rabbitmq.client.ConnectionFactory; -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.schema.EncodedMessage; -import dev.caskeleton.messaging.testkit.DockerAvailability; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import java.io.IOException; -import java.nio.charset.StandardCharsets; -import java.time.Duration; -import java.time.Instant; -import java.util.Map; -import java.util.Optional; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.TimeoutException; -import org.junit.jupiter.api.AfterEach; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.condition.EnabledIf; -import org.springframework.amqp.core.MessageProperties; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; -import org.testcontainers.rabbitmq.RabbitMQContainer; - -/** - * Certifies the RabbitMQ adapter against a live broker. - * - *

The test that matters is the unroutable one. AMQP delivers a {@code basic.return} - * before the confirm for a message no queue was bound to receive, and an adapter that - * completes on the confirm alone reports success for a message the broker threw away. No mock - * proves the ordering is handled correctly, because the ordering is the broker's behaviour. - */ -@Testcontainers -@EnabledIf("dockerAvailable") -class RabbitBrokerIT { - - private static final String EXCHANGE = "notification.work"; - private static final String BOUND_KEY = "email"; - private static final String UNBOUND_KEY = "nothing.is.bound.here"; - private static final String QUEUE = "notification.email.q"; - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - - @Container - private static final RabbitMQContainer RABBIT = new RabbitMQContainer("rabbitmq:4.3-management"); - - private Connection connection; - private Channel channel; - private RabbitMessagingTransport transport; - - static boolean dockerAvailable() { - return DockerAvailability.isAvailable(); - } - - @BeforeEach - void declareTopology() throws IOException, TimeoutException { - ConnectionFactory factory = new ConnectionFactory(); - factory.setHost(RABBIT.getHost()); - factory.setPort(RABBIT.getAmqpPort()); - factory.setUsername(RABBIT.getAdminUsername()); - factory.setPassword(RABBIT.getAdminPassword()); - connection = factory.newConnection(); - channel = connection.createChannel(); - - channel.exchangeDeclare(EXCHANGE, "direct", true); - channel.queueDeclare(QUEUE, true, false, false, Map.of("x-queue-type", "quorum")); - channel.queueBind(QUEUE, EXCHANGE, BOUND_KEY); - // The container is shared across the class, so a message left by an earlier test would make a - // later queue-depth assertion pass or fail for the wrong reason. - channel.queuePurge(QUEUE); - channel.confirmSelect(); - - transport = - new RabbitMessagingTransport( - "rabbit-primary", - 1, - new RabbitChannelPublisher() { - @Override - public long nextPublishSequence() { - return channel.getNextPublishSeqNo(); - } - - @Override - public void publish( - long sequence, - String exchange, - String routingKey, - org.springframework.amqp.core.Message message, - boolean mandatory) { - RabbitBrokerIT.this.publish(sequence, exchange, routingKey, message, mandatory); - } - }); - - channel.addConfirmListener( - (deliveryTag, multiple) -> transport.onConfirm(deliveryTag, true, Duration.ofMillis(1)), - (deliveryTag, multiple) -> transport.onConfirm(deliveryTag, false, Duration.ofMillis(1))); - channel.addReturnListener( - returned -> - transport.onReturn( - Long.parseLong(returned.getProperties().getHeaders().get("x-seq").toString()))); - } - - @AfterEach - void close() throws IOException, TimeoutException { - if (channel != null && channel.isOpen()) { - channel.close(); - } - if (connection != null && connection.isOpen()) { - connection.close(); - } - } - - @Test - void aRoutablePublishConfirmsWithReplicationEvidence() { - PublishResult result = await(transport.publish(request(BOUND_KEY))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.ROUTED); - assertThat(result.evidence().confirmationLevel()) - .isEqualTo(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK); - } - - @Test - void anUnroutablePublishIsRejectedEvenThoughTheExchangeConfirmedIt() { - PublishResult result = await(transport.publish(request(UNBOUND_KEY))); - - assertThat(result.completion()) - .as("the exchange accepted it and the broker threw it away; that is not success") - .isEqualTo(PublishCompletion.REJECTED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.UNROUTABLE); - assertThat(result.evidence().confirmationLevel()).isEqualTo(ConfirmationLevel.NONE); - } - - @Test - void aRoutablePublishReachesTheQueue() throws IOException { - await(transport.publish(request(BOUND_KEY))); - - assertThat(channel.messageCount(QUEUE)).isEqualTo(1); - } - - @Test - void anUnroutablePublishReachesNoQueue() throws IOException { - await(transport.publish(request(UNBOUND_KEY))); - - assertThat(channel.messageCount(QUEUE)).isZero(); - } - - /** - * Publishes and records the sequence number in a header. - * - *

A returned message arrives without its publish sequence number, so the adapter has to carry - * one itself to correlate the return with the pending publish. - */ - private void publish( - long sequence, - String exchange, - String routingKey, - org.springframework.amqp.core.Message message, - boolean mandatory) { - try { - MessageProperties source = message.getMessageProperties(); - AMQP.BasicProperties properties = - new AMQP.BasicProperties.Builder() - .messageId(source.getMessageId()) - .contentType(source.getContentType()) - .deliveryMode(2) - .headers(withSequence(source, sequence)) - .build(); - // No waitForConfirms: the coordinator completes the publish from the confirm listener, and - // blocking here would hold the calling thread through the round trip for no benefit. - channel.basicPublish(exchange, routingKey, mandatory, properties, message.getBody()); - } catch (IOException exception) { - throw new IllegalStateException("rabbit publish failed", exception); - } - } - - private static Map withSequence(MessageProperties source, long sequence) { - java.util.Map headers = new java.util.LinkedHashMap<>(source.getHeaders()); - headers.put("x-seq", Long.toString(sequence)); - return headers; - } - - private static PublishResult await( - CompletionStage stage) { - return stage.toCompletableFuture().join().result(); - } - - private TransportPublishRequest request(String routingKey) { - return new TransportPublishRequest( - RabbitFixtureProfiles.routingTo(EXCHANGE, routingKey, QUEUE), - envelope(), - PublishOptions.defaults()); - } - - private static MessageEnvelope envelope() { - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - new EncodedMessage( - "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), - ContentType.JSON, - Optional.empty())); - } -} diff --git a/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitFixtureProfiles.java b/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitFixtureProfiles.java deleted file mode 100644 index da40530..0000000 --- a/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitFixtureProfiles.java +++ /dev/null @@ -1,32 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.PhysicalDestination; - -/** Builds destination profiles that vary only in their physical mapping. */ -final class RabbitFixtureProfiles { - - private RabbitFixtureProfiles() {} - - static DestinationProfile routingTo(String exchange, String routingKey, String queue) { - DestinationProfile base = RabbitProfileFixtures.workQueueDestination(); - return new DestinationProfile( - base.name(), - base.broker(), - base.kind(), - PhysicalDestination.rabbitQueue(exchange, routingKey, queue), - base.schema(), - base.deliveryGuarantee(), - base.orderingScope(), - base.externalSideEffectGuarantee(), - base.producer(), - base.consumer(), - base.retry(), - base.deadLetter(), - base.payload(), - base.tier(), - base.production(), - base.keyResolverConfigured(), - base.topologyAutoCreate()); - } -} diff --git a/messaging/messaging-rabbit/src/jmh/java/dev/caskeleton/messaging/rabbit/RabbitPublishBenchmark.java b/messaging/messaging-rabbit/src/jmh/java/dev/caskeleton/messaging/rabbit/RabbitPublishBenchmark.java deleted file mode 100644 index e8e2d46..0000000 --- a/messaging/messaging-rabbit/src/jmh/java/dev/caskeleton/messaging/rabbit/RabbitPublishBenchmark.java +++ /dev/null @@ -1,156 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.schema.EncodedMessage; -import java.time.Duration; -import java.time.Instant; -import java.util.Optional; -import java.util.concurrent.TimeUnit; -import org.openjdk.jmh.annotations.Benchmark; -import org.openjdk.jmh.annotations.BenchmarkMode; -import org.openjdk.jmh.annotations.Fork; -import org.openjdk.jmh.annotations.Level; -import org.openjdk.jmh.annotations.Measurement; -import org.openjdk.jmh.annotations.Mode; -import org.openjdk.jmh.annotations.OutputTimeUnit; -import org.openjdk.jmh.annotations.Param; -import org.openjdk.jmh.annotations.Scope; -import org.openjdk.jmh.annotations.Setup; -import org.openjdk.jmh.annotations.State; -import org.openjdk.jmh.annotations.Warmup; -import org.openjdk.jmh.infra.Blackhole; - -/** - * Measures the RabbitMQ adapter's own cost per publish, with no broker in the loop. - * - *

The confirm coordinator is the interesting one here. Every publish registers a pending entry - * and every confirm resolves it, so its map operations run twice per message on the hot path — and - * because a confirm arrives on the connection thread while publishes arrive on caller threads, a - * regression there shows up as contention rather than as a slower method. - * - *

No channel and no network, for the same reason the Kafka benchmark has no producer: measuring - * the broker measures the environment. - */ -@BenchmarkMode(Mode.AverageTime) -@OutputTimeUnit(TimeUnit.NANOSECONDS) -@Warmup(iterations = 3, time = 1) -@Measurement(iterations = 5, time = 1) -@Fork(1) -@State(Scope.Benchmark) -public class RabbitPublishBenchmark { - - /** Payload sizes spanning the range a portable destination allows. */ - @Param({"256", "4096", "65536"}) - public int payloadBytes; - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - private static final String EXCHANGE = "notification.work"; - private static final String ROUTING_KEY = "email"; - - private RabbitHeaderMapper headerMapper; - private RabbitPublishFailureClassifier classifier; - private MessageEnvelope envelope; - private RabbitConfirmCoordinator coordinator; - private long sequence; - - /** Builds the fixtures once so the measured methods allocate only what a publish allocates. */ - @Setup(Level.Trial) - public void setUp() { - headerMapper = new RabbitHeaderMapper(); - classifier = new RabbitPublishFailureClassifier(); - byte[] payload = new byte[payloadBytes]; - java.util.Arrays.fill(payload, (byte) 'x'); - envelope = - new MessageEnvelope<>( - MessageId.newId(), - new MessageType("email.requested"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("notification-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - new EncodedMessage(payload, ContentType.JSON, Optional.empty())); - } - - /** Resets the coordinator per iteration so pending entries do not accumulate across runs. */ - @Setup(Level.Iteration) - public void resetCoordinator() { - coordinator = new RabbitConfirmCoordinator(); - sequence = 0; - } - - /** - * Measures mapping an envelope's reserved headers onto AMQP properties. - * - * @param blackhole consumes the result - */ - @Benchmark - public void mapHeaders(Blackhole blackhole) { - blackhole.consume(headerMapper.toProperties(envelope)); - } - - /** - * Measures one full register-then-confirm round trip through the coordinator. - * - *

Both halves together, because a benchmark of the registration alone would leave the pending - * map growing and report a cost that no real publish pays. - * - * @param blackhole consumes the result - */ - @Benchmark - public void registerAndConfirm(Blackhole blackhole) { - long current = ++sequence; - blackhole.consume( - coordinator.awaiting( - current, - EXCHANGE, - ROUTING_KEY, - ConfirmationRequirement.REPLICATION_OR_PERSISTENCE_ACK)); - coordinator.confirmed(current, true, Duration.ofMillis(1)); - } - - /** - * Measures the unroutable path, which resolves through a return before the confirm. - * - * @param blackhole consumes the result - */ - @Benchmark - public void registerReturnAndConfirm(Blackhole blackhole) { - long current = ++sequence; - blackhole.consume( - coordinator.awaiting( - current, - EXCHANGE, - ROUTING_KEY, - ConfirmationRequirement.REPLICATION_OR_PERSISTENCE_ACK)); - coordinator.returned(current); - coordinator.confirmed(current, true, Duration.ofMillis(1)); - } - - /** - * Measures classifying a lost confirm, which runs on every ambiguous publish. - * - * @param blackhole consumes the result - */ - @Benchmark - public void classifyLostConfirm(Blackhole blackhole) { - blackhole.consume( - classifier.classify( - new java.io.IOException("connection reset"), true, Duration.ofMillis(5))); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBatchConsumerRegistrar.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBatchConsumerRegistrar.java deleted file mode 100644 index 68ab87a..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBatchConsumerRegistrar.java +++ /dev/null @@ -1,165 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.delivery.BatchDeliveryMetadata; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Objects; -import java.util.Optional; - -/** - * Accumulates AMQP deliveries into batches, bounded by size and by age. - * - *

Unlike Kafka, RabbitMQ delivers one message at a time, so a batch here is built by the client - * rather than returned by a poll. That makes the age bound essential: without it, a queue that goes - * quiet leaves the last few messages sitting in the accumulator indefinitely, unacknowledged, until - * enough traffic arrives to fill a batch — which on a low-volume queue may be hours. - * - *

{@code settlableAsBatch} is reported false. AMQP's multiple-ack acknowledges every delivery up - * to a tag, which on a channel handling concurrent work settles messages the handler has not - * finished; the platform therefore acknowledges each delivery individually even when the handler - * saw them as a batch. - * - *

The accumulator is not thread-safe by design. It is driven from the single consumer callback - * thread for its channel, and adding synchronisation here would suggest it can be shared, which - * would interleave deliveries from different channels into one batch and lose their ordering. - */ -public final class RabbitBatchConsumerRegistrar { - - private final DestinationProfile profile; - private final int maxBatchSize; - private final Duration maxBatchAge; - private final List deliveryTags = new ArrayList<>(); - private Instant oldestArrival; - - /** - * Creates a batch accumulator for a destination. - * - * @param profile the validated destination profile - * @param maxBatchSize the largest batch handed to a handler - * @param maxBatchAge how long the oldest delivery may wait before the batch is released - */ - public RabbitBatchConsumerRegistrar( - DestinationProfile profile, int maxBatchSize, Duration maxBatchAge) { - this.profile = Objects.requireNonNull(profile, "profile must not be null"); - Objects.requireNonNull(maxBatchAge, "maxBatchAge must not be null"); - if (maxBatchSize < 1) { - throw new IllegalArgumentException("maxBatchSize must be at least 1"); - } - if (maxBatchAge.isNegative() || maxBatchAge.isZero()) { - throw new IllegalArgumentException( - "maxBatchAge must be positive; without it a quiet queue holds its last messages " - + "unacknowledged indefinitely"); - } - this.maxBatchSize = maxBatchSize; - this.maxBatchAge = maxBatchAge; - } - - /** - * Adds one delivery to the current batch. - * - * @param deliveryTag the AMQP delivery tag - * @param now the current instant - * @return the batch when it is ready to release, otherwise empty - */ - public Optional add(long deliveryTag, Instant now) { - Objects.requireNonNull(now, "now must not be null"); - if (deliveryTags.isEmpty()) { - oldestArrival = now; - } - deliveryTags.add(deliveryTag); - return deliveryTags.size() >= maxBatchSize ? Optional.of(release(now)) : Optional.empty(); - } - - /** - * Releases the current batch if its oldest delivery has waited long enough. - * - * @param now the current instant - * @return the batch when the age bound was reached, otherwise empty - */ - public Optional releaseIfStale(Instant now) { - Objects.requireNonNull(now, "now must not be null"); - if (deliveryTags.isEmpty()) { - return Optional.empty(); - } - return !now.isBefore(oldestArrival.plus(maxBatchAge)) - ? Optional.of(release(now)) - : Optional.empty(); - } - - /** - * Releases whatever has accumulated, for shutdown. - * - * @param now the current instant - * @return the batch when anything was pending - */ - public Optional drain(Instant now) { - Objects.requireNonNull(now, "now must not be null"); - return deliveryTags.isEmpty() ? Optional.empty() : Optional.of(release(now)); - } - - /** - * Returns how many deliveries are waiting in the accumulator. - * - * @return the pending delivery count - */ - public int pending() { - return deliveryTags.size(); - } - - /** - * Refuses batch consumption where the prefetch cannot hold a whole batch. - * - *

A prefetch below the batch size deadlocks: the accumulator waits for more deliveries the - * broker will not send until something is acknowledged, and nothing is acknowledged until the - * batch is released. - * - * @param prefetch the channel prefetch count - * @throws MessagingConfigurationException when the prefetch is too small - */ - public void requirePrefetchFor(int prefetch) { - if (prefetch < maxBatchSize) { - throw new MessagingConfigurationException( - "PREFETCH_BELOW_BATCH_SIZE", - "destination %s batches %d deliveries but the channel prefetch is %d; the accumulator " - .formatted(profile.name().value(), maxBatchSize, prefetch) - + "would wait for deliveries the broker will not send until something is acked"); - } - } - - private AmqpBatch release(Instant now) { - List released = List.copyOf(deliveryTags); - deliveryTags.clear(); - oldestArrival = null; - return new AmqpBatch( - released, - new BatchDeliveryMetadata( - profile.name(), - released.size(), - // One queue is one ordering unit, so an accumulated batch never spans two. - Optional.of(profile.physical().queue().orElse(profile.name().value())), - // AMQP multiple-ack settles everything up to a tag, including work still in flight. - false, - now)); - } - - /** - * One accumulated batch of AMQP deliveries. - * - * @param deliveryTags the delivery tags, in arrival order - * @param metadata the batch-wide metadata handed to the handler - */ - public record AmqpBatch(List deliveryTags, BatchDeliveryMetadata metadata) { - - public AmqpBatch { - Objects.requireNonNull(metadata, "metadata must not be null"); - deliveryTags = List.copyOf(deliveryTags); - if (deliveryTags.isEmpty()) { - throw new IllegalArgumentException("a released batch is never empty"); - } - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBrokerProfile.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBrokerProfile.java deleted file mode 100644 index e53d377..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBrokerProfile.java +++ /dev/null @@ -1,50 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import java.time.Duration; -import java.util.List; -import java.util.Objects; - -/** - * The RabbitMQ client settings the platform is willing to run with. - * - * @param broker the logical broker name - * @param stable whether this profile claims the Stable guarantees - * @param production whether this profile is used in production - * @param addresses the broker addresses - * @param publisherConfirms whether correlated publisher confirms are on - * @param publisherReturns whether unroutable returns are delivered - * @param mandatory whether publishes are sent with the mandatory flag - * @param confirmTimeout how long to wait for a confirm - * @param autoAck whether the consumer acknowledges on delivery - * @param prefetch the consumer prefetch window - * @param quorumQueues whether durable work queues are declared as quorum queues - * @param tlsEnabled whether transport encryption is on - * @param authenticationEnabled whether broker authentication is on - */ -public record RabbitBrokerProfile( - String broker, - boolean stable, - boolean production, - List addresses, - boolean publisherConfirms, - boolean publisherReturns, - boolean mandatory, - Duration confirmTimeout, - boolean autoAck, - int prefetch, - boolean quorumQueues, - boolean tlsEnabled, - boolean authenticationEnabled) { - - public RabbitBrokerProfile { - Objects.requireNonNull(addresses, "addresses must not be null"); - Objects.requireNonNull(confirmTimeout, "confirmTimeout must not be null"); - if (broker == null || broker.isBlank()) { - throw new IllegalArgumentException("broker must not be blank"); - } - if (addresses.isEmpty()) { - throw new IllegalArgumentException("addresses must not be empty"); - } - addresses = List.copyOf(addresses); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitChannelPublisher.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitChannelPublisher.java deleted file mode 100644 index 8fe08f2..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitChannelPublisher.java +++ /dev/null @@ -1,42 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import org.springframework.amqp.core.Message; - -/** - * The channel operations the transport needs, split so the confirm cannot outrun its registration. - * - *

Reserving the sequence number is a separate call from publishing on purpose. A confirm can - * arrive on the connection thread before {@code basicPublish} has even returned, so a transport - * that registered its pending publish afterwards would drop that confirm and wait forever for one - * that already came. Reserving first closes that race. - */ -public interface RabbitChannelPublisher { - - /** - * Reserves the next publish sequence number without publishing. - * - * @return the sequence number the next publish will carry - */ - long nextPublishSequence(); - - /** - * Publishes a message under a previously reserved sequence number. - * - * @param sequence the reserved sequence number - * @param exchange the exchange to publish to - * @param routingKey the routing key - * @param message the AMQP message - * @param mandatory whether the broker must return an unroutable message - */ - void publish( - long sequence, String exchange, String routingKey, Message message, boolean mandatory); - - /** - * Closes the underlying channel. - * - *

A default no-op so an existing publisher keeps compiling, but the transport calls it: a - * transport whose {@code close} only flipped a boolean left the native channel open and every - * pending stage waiting for a confirm that would never arrive. - */ - default void close() {} -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinator.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinator.java deleted file mode 100644 index f65c985..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinator.java +++ /dev/null @@ -1,279 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; - -/** - * Correlates a RabbitMQ publish with its confirm and its possible return. - * - *

This is the piece that makes the Rabbit adapter honest. AMQP delivers a return before - * the confirm for an unroutable message, so a naive adapter that completes on the confirm reports - * success for a message the broker threw away. The coordinator therefore keeps each publish pending - * until the confirm arrives, and remembers whether a return was seen first. - * - *

The resulting outcomes are: confirmed and not returned is {@code CONFIRMED}; confirmed and - * returned is {@code REJECTED} with {@code UNROUTABLE}; nacked is {@code REJECTED}; and a confirm - * that never arrives is {@code AMBIGUOUS}, because the broker may still have stored it. - * - *

Implemented as a pure state machine over sequence numbers so the whole ordering, including the - * return-before-confirm case, is testable without a broker. - */ -public final class RabbitConfirmCoordinator { - - /** - * Pending publishes, ordered by sequence. - * - *

Ordered because a Rabbit confirm carries a {@code multiple} flag meaning "everything up to - * and including this tag". Resolving one sequence per confirm — which is what a hash map forces — - * leaves every earlier publish pending forever: the caller's stage never completes and the entry - * is never removed, so the map grows for the life of the connection. - */ - private final java.util.concurrent.ConcurrentSkipListMap pending = - new java.util.concurrent.ConcurrentSkipListMap<>(); - - /** - * Registers a publish awaiting its confirm. - * - * @param sequence the channel publish sequence number - * @param exchange the exchange published to - * @param routingKey the routing key used - * @param requirement the confirmation the profile demanded - * @return a stage completing when the confirm, return, or timeout resolves it - */ - public CompletionStage awaiting( - long sequence, String exchange, String routingKey, ConfirmationRequirement requirement) { - Objects.requireNonNull(exchange, "exchange must not be null"); - Objects.requireNonNull(routingKey, "routingKey must not be null"); - Objects.requireNonNull(requirement, "requirement must not be null"); - - PendingPublish publish = new PendingPublish(exchange, routingKey, requirement); - pending.put(sequence, publish); - return publish.completion; - } - - /** - * Records that the broker returned a message as unroutable. - * - *

The return alone does not complete the publish: the confirm still follows, and only the pair - * establishes that the broker both accepted and discarded it. - * - * @param sequence the publish sequence number - */ - public void returned(long sequence) { - PendingPublish publish = pending.get(sequence); - if (publish != null) { - publish.returned = true; - } - } - - /** - * Records a broker confirm. - * - * @param sequence the publish sequence number - * @param acknowledged true for an ack, false for a nack - * @param elapsed how long the publish took - */ - public void confirmed(long sequence, boolean acknowledged, Duration elapsed) { - confirmed(sequence, false, acknowledged, elapsed); - } - - /** - * Records a broker confirm, honouring the {@code multiple} flag. - * - * @param multiple true when the broker is confirming every sequence up to and including this one - */ - public void confirmed(long sequence, boolean multiple, boolean acknowledged, Duration elapsed) { - if (!multiple) { - PendingPublish publish = pending.remove(sequence); - if (publish != null) { - publish.completion.complete(publish.resolve(sequence, acknowledged, elapsed)); - } - return; - } - // headMap(tag, inclusive) is the whole point of the ordered map: the broker is confirming a - // range, and every entry in it must be resolved with the same verdict. - java.util.NavigableMap confirmed = pending.headMap(sequence, true); - for (Long pendingSequence : java.util.List.copyOf(confirmed.keySet())) { - PendingPublish publish = pending.remove(pendingSequence); - if (publish != null) { - publish.completion.complete(publish.resolve(pendingSequence, acknowledged, elapsed)); - } - } - } - - /** - * Completes a publish whose send failed synchronously. - * - *

The classifier's verdict is used as-is: it is the only thing that knows whether the frame - * can have reached the broker, and guessing here would either invent a rejection for a message - * that was stored or an ambiguity for one that never left. - * - * @param sequence the publish sequence number - * @param result the classified outcome - */ - public void sendFailed(long sequence, PublishResult result) { - PendingPublish publish = pending.remove(sequence); - if (publish != null) { - publish.completion.complete(result); - } - } - - /** - * Completes every pending publish because the channel is gone. - * - *

Ambiguous, not rejected: the frames were written, and a channel that closes afterwards says - * nothing about whether the broker stored them. Leaving them pending — which is what close used - * to do — hangs every caller holding a stage. - * - * @param elapsed how long the publishes had been waiting - * @return how many stages were completed - */ - public int channelClosed(Duration elapsed) { - int drained = 0; - for (Long sequence : java.util.List.copyOf(pending.keySet())) { - PendingPublish publish = pending.remove(sequence); - if (publish != null) { - publish.completion.complete(publish.resolveChannelClosed(sequence, elapsed)); - drained++; - } - } - return drained; - } - - /** - * Records that the confirm never arrived within the deadline. - * - * @param sequence the publish sequence number - * @param elapsed how long was waited - */ - public void timedOut(long sequence, Duration elapsed) { - PendingPublish publish = pending.remove(sequence); - if (publish == null) { - return; - } - publish.completion.complete( - new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "RABBIT_CONFIRM_TIMEOUT", - "no publisher confirm arrived before the deadline")))); - } - - /** - * Returns how many publishes are awaiting a confirm. - * - * @return the pending count - */ - public int pendingCount() { - return pending.size(); - } - - /** One publish waiting for its confirm, and whether a return arrived first. */ - private static final class PendingPublish { - - private final String exchange; - private final String routingKey; - private final ConfirmationRequirement requirement; - private final CompletableFuture completion = new CompletableFuture<>(); - - private volatile boolean returned; - - private PendingPublish( - String exchange, String routingKey, ConfirmationRequirement requirement) { - this.exchange = exchange; - this.routingKey = routingKey; - this.requirement = requirement; - } - - /** The channel closed with this publish still awaiting its confirm. */ - private PublishResult resolveChannelClosed(long sequence, Duration elapsed) { - return new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.of(new RabbitPublishReference(exchange, routingKey, sequence)), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "RABBIT_CHANNEL_CLOSED", - "the channel closed before the broker confirmed this publish"))); - } - - private PublishResult resolve(long sequence, boolean acknowledged, Duration elapsed) { - RabbitPublishReference position = new RabbitPublishReference(exchange, routingKey, sequence); - - if (!acknowledged) { - return new PublishResult( - PublishCompletion.REJECTED, - // Transmitted, not "not transmitted": a NACK is the broker's answer to a frame it - // received. Recording it as never sent contradicts the very evidence that produced it, - // and a caller reading the evidence would conclude the message can be re-sent freely. - new PublishEvidence( - true, - dev.caskeleton.messaging.api.publish.TransmissionEvidence.TRANSMITTED, - false, - ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.of(position), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.PERMANENT_BUSINESS, - "RABBIT_NACK", - "the broker negatively acknowledged the publish"))); - } - if (returned) { - return new PublishResult( - PublishCompletion.REJECTED, - new PublishEvidence( - true, - dev.caskeleton.messaging.api.publish.TransmissionEvidence.TRANSMITTED, - false, - ConfirmationLevel.NONE), - RoutingOutcome.UNROUTABLE, - Optional.of(position), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.PERMANENT_BUSINESS, - "RABBIT_UNROUTABLE", - "the exchange accepted the publish but no queue was bound"))); - } - - ConfirmationLevel level = - requirement == ConfirmationRequirement.REPLICATION_OR_PERSISTENCE_ACK - ? ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK - : ConfirmationLevel.BROKER_ACK; - return new PublishResult( - PublishCompletion.CONFIRMED, - PublishEvidence.confirmed(level), - RoutingOutcome.ROUTED, - Optional.of(position), - 1, - elapsed, - Optional.empty()); - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConsumerRegistrar.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConsumerRegistrar.java deleted file mode 100644 index 1852966..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConsumerRegistrar.java +++ /dev/null @@ -1,238 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.settlement.SettlementCompletion; -import dev.caskeleton.messaging.api.settlement.SettlementEvidence; -import dev.caskeleton.messaging.api.settlement.SettlementResult; -import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; -import dev.caskeleton.messaging.transport.TransportConsumerRegistration; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import dev.caskeleton.messaging.transport.TransportDelivery; -import dev.caskeleton.messaging.transport.TransportSettlement; -import java.time.Duration; -import java.time.Instant; -import java.util.Objects; -import java.util.Optional; -import java.util.Set; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.atomic.AtomicBoolean; -import org.springframework.amqp.core.Message; - -/** - * The RabbitMQ consumer runtime. - * - *

Deliveries arrive from the listener container and are handed to the platform with a settlement - * handle that is bound to one delivery tag and refuses a second terminal call. Duplicating a {@code - * basic.ack} on a reused tag acknowledges a different message, which shows up as - * unexplained loss under load, far from its cause. - * - *

Consumption stops the moment a drain begins, but deliveries already in flight are allowed to - * finish. Rejecting an in-flight delivery at shutdown would either lose it or duplicate its side - * effect, depending on where the handler had got to. - */ -public final class RabbitConsumerRegistrar implements TransportConsumerRegistration { - - private final TransportConsumerSpec spec; - private final RabbitDeliveryMapper deliveryMapper; - private final RabbitSettlementOperations operations; - private final GracefulShutdownCoordinator shutdown; - private final Set pausedScopes = ConcurrentHashMap.newKeySet(); - private final AtomicBoolean active = new AtomicBoolean(true); - - /** - * Creates a consumer runtime. - * - * @param spec what to consume and where to deliver it - * @param operations the channel operations a settlement drives - * @param shutdown the drain coordinator - */ - public RabbitConsumerRegistrar( - TransportConsumerSpec spec, - RabbitSettlementOperations operations, - GracefulShutdownCoordinator shutdown) { - this(spec, operations, shutdown, new RabbitDeliveryMapper()); - } - - /** - * Creates a consumer runtime with an explicit delivery mapper. - * - * @param spec what to consume and where to deliver it - * @param operations the channel operations a settlement drives - * @param shutdown the drain coordinator - * @param deliveryMapper rebuilds envelopes from AMQP messages - */ - public RabbitConsumerRegistrar( - TransportConsumerSpec spec, - RabbitSettlementOperations operations, - GracefulShutdownCoordinator shutdown, - RabbitDeliveryMapper deliveryMapper) { - this.spec = Objects.requireNonNull(spec, "spec must not be null"); - this.operations = Objects.requireNonNull(operations, "operations must not be null"); - this.shutdown = Objects.requireNonNull(shutdown, "shutdown must not be null"); - this.deliveryMapper = Objects.requireNonNull(deliveryMapper, "deliveryMapper is required"); - } - - /** - * Handles one delivery pushed from the listener container. - * - *

Returns whether the delivery was accepted. A refusal means the runtime is draining or the - * queue is paused, and the caller must leave the message unacknowledged so the broker redelivers - * it. - * - * @param message the AMQP message - * @param now the current instant - * @return true when the delivery was dispatched - */ - public boolean onMessage(Message message, Instant now) { - Objects.requireNonNull(message, "message must not be null"); - Objects.requireNonNull(now, "now must not be null"); - - String queue = - Optional.ofNullable(message.getMessageProperties().getConsumerQueue()).orElse(""); - if (!active.get()) { - // close() only flipped a flag and onMessage never read it, so a delivery arriving after - // close was accepted, dispatched, and settled against a controller that was shutting down. - return false; - } - if (pausedScopes.contains(queue) || pausedScopes.contains("")) { - return false; - } - if (!shutdown.tryBeginWork()) { - return false; - } - - long tag = message.getMessageProperties().getDeliveryTag(); - try { - TransportDelivery delivery; - try { - // The narrow catch. One catch around decode, the handler and the settlement meant a - // business failure or an ACK that could not be written was recorded as an undecodable - // payload and discarded — a message that should have been retried, deleted instead. - delivery = - new TransportDelivery( - deliveryMapper.toEnvelope(message), - deliveryMapper.toMetadata(spec.profile().name(), message, now), - new ControllerBackedSettlement(tag)); - } catch (RuntimeException undecodable) { - // Genuinely undecodable: it will fail the same way on every redelivery, so requeueing is a - // hot loop. Discarding is right here and only here. - operations - .discard( - tag, - FailureDescriptor.of( - FailureCategory.DESERIALIZATION, - "RABBIT_UNDECODABLE", - "the message could not be decoded into an envelope")) - .toCompletableFuture() - .join(); - return true; - } - - ControllerBackedSettlement settlement = (ControllerBackedSettlement) delivery.settlement(); - try { - spec.sink().apply(delivery).toCompletableFuture().join(); - } catch (RuntimeException handlerFailed) { - // A handler failure is a retry, not a discard. - requeueQuietly(settlement); - return true; - } - if (!settlement.wasSettled()) { - // A handler that completed without settling has not said what to do with the message. - // Acknowledging on its behalf would silently drop it; the contract violation is surfaced - // as a requeue so the message survives while somebody fixes the handler. - requeueQuietly(settlement); - } - return true; - } finally { - shutdown.endWork(); - } - } - - /** Requeues without letting a settlement failure become the caller's problem. */ - private void requeueQuietly(ControllerBackedSettlement settlement) { - try { - settlement.requeue(Duration.ZERO).toCompletableFuture().join(); - } catch (RuntimeException settlementFailed) { - // The broker connection is the thing that failed. There is nothing left to try here, and - // the delivery stays unsettled — which is what redelivery is for. - return; - } - } - - @Override - public CompletionStage pause(String scope) { - pausedScopes.add(scope == null ? "" : scope); - return CompletableFuture.completedFuture(null); - } - - @Override - public CompletionStage resume(String scope) { - pausedScopes.remove(scope == null ? "" : scope); - return CompletableFuture.completedFuture(null); - } - - /** - * Reports whether a scope is currently paused. - * - * @param scope the queue, or an empty string for all - * @return true while paused - */ - public boolean isPaused(String scope) { - return pausedScopes.contains(scope == null ? "" : scope); - } - - @Override - public boolean isActive() { - return active.get(); - } - - @Override - public void close() { - active.set(false); - } - - /** Adapts the transport settlement SPI onto the one-terminal-call controller. */ - private final class ControllerBackedSettlement implements TransportSettlement { - - private final RabbitSettlementController controller; - - private ControllerBackedSettlement(long deliveryTag) { - this.controller = new RabbitSettlementController(deliveryTag, operations); - } - - private boolean wasSettled() { - return controller.isSettled(); - } - - @Override - public CompletionStage acknowledge() { - return controller.ack(); - } - - @Override - public CompletionStage requeue(Duration delay) { - if (!shutdown.mayCreateRetryAttempt()) { - return CompletableFuture.completedFuture( - new SettlementResult( - SettlementCompletion.UNKNOWN, - SettlementEvidence.unknown(), - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "RABBIT_DRAINING", - "no retry attempt is created while the consumer is draining")))); - } - return controller.retry(delay); - } - - @Override - public CompletionStage discard() { - return controller.reject( - FailureDescriptor.of( - FailureCategory.PERMANENT_BUSINESS, "RABBIT_DISCARDED", "the message was discarded")); - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeadLetterPublisher.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeadLetterPublisher.java deleted file mode 100644 index e534a91..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeadLetterPublisher.java +++ /dev/null @@ -1,130 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.delivery.MessageDelivery; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; -import dev.caskeleton.messaging.policy.DeadLetterResult; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.SourceSettlement; -import dev.caskeleton.messaging.schema.EncodedMessage; -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletionStage; - -/** - * The RabbitMQ entry point to dead lettering. - * - *

Delegates to the shared orchestrator, exactly as the Kafka adapter does, even though RabbitMQ - * has native dead-lettering available. The native path is only taken where {@link - * RabbitNativeDeadLetterCapability} has verified a queue is bound to the dead-letter exchange — - * everywhere else the broker would discard the message silently while the reject still succeeded, - * and the platform's publish-then-settle ordering is what prevents that. - * - *

Which path is taken is decided here rather than by each call site, so a handler cannot - * accidentally reject a message on a destination whose dead-letter exchange goes nowhere. - */ -public final class RabbitDeadLetterPublisher { - - private final DeadLetterOrchestrator orchestrator; - private final RabbitNativeDeadLetterCapability nativeCapability; - private final NativeReject nativeReject; - - /** - * Creates a Rabbit dead letter publisher. - * - * @param orchestrator the shared dead letter orchestrator - * @param nativeCapability decides whether the broker's own dead-lettering may be trusted - * @param nativeReject performs a broker-side reject - */ - public RabbitDeadLetterPublisher( - DeadLetterOrchestrator orchestrator, - RabbitNativeDeadLetterCapability nativeCapability, - NativeReject nativeReject) { - this.orchestrator = Objects.requireNonNull(orchestrator, "orchestrator must not be null"); - this.nativeCapability = Objects.requireNonNull(nativeCapability, "nativeCapability required"); - this.nativeReject = Objects.requireNonNull(nativeReject, "nativeReject must not be null"); - } - - /** - * Dead letters one delivery, by whichever path is safe for this destination. - * - * @param profile the source destination profile - * @param delivery the failed delivery - * @param failure the sanitized failure - * @param deliveryTag the AMQP delivery tag to reject, when taking the native path - * @param settlement the source settlement callback - * @return a stage completing with the dead letter outcome - */ - public CompletionStage deadLetter( - DestinationProfile profile, - MessageDelivery delivery, - FailureDescriptor failure, - long deliveryTag, - SourceSettlement settlement) { - Objects.requireNonNull(profile, "profile must not be null"); - Objects.requireNonNull(delivery, "delivery must not be null"); - Objects.requireNonNull(failure, "failure must not be null"); - - if (nativeCapability.requiresPlatformPublish()) { - return orchestrator.deadLetter(profile, delivery, failure, settlement); - } - // The broker moves the message and settles the source in one operation. Safe only because the - // capability check confirmed a queue is bound to the dead letter exchange. - return nativeReject.reject(deliveryTag).thenApply(moved -> nativelyRouted()); - } - - /** - * Describes what a broker-side reject actually proves. - * - *

{@link ConfirmationLevel#BROKER_ACK}, not replication. RabbitMQ issues no publisher confirm - * for the internal hop to the dead-letter exchange, so the evidence available is that the broker - * accepted the reject — reporting it as a replication acknowledgement would claim a guarantee no - * part of this path provides. - * - * @return the dead letter outcome for the native path - */ - private static DeadLetterResult nativelyRouted() { - return new DeadLetterResult( - new PublishResult( - PublishCompletion.CONFIRMED, - new PublishEvidence( - true, TransmissionEvidence.TRANSMITTED, true, ConfirmationLevel.BROKER_ACK), - RoutingOutcome.ROUTED, - Optional.empty(), - 1, - Duration.ZERO, - Optional.empty()), - // The reject settles the source as part of the same operation, so there is no window in - // which the message is neither parked nor deliverable. - true); - } - - /** - * Returns why the platform is publishing rather than letting the broker route. - * - * @return the sanitized reason, empty when the native path is in use - */ - public String pathExplanation() { - return nativeCapability.reasonForPlatformPublish(); - } - - /** A broker-side reject that routes the message to the dead letter exchange. */ - @FunctionalInterface - public interface NativeReject { - - /** - * Rejects one delivery without requeueing it. - * - * @param deliveryTag the AMQP delivery tag - * @return a stage completing once the broker has taken the reject - */ - CompletionStage reject(long deliveryTag); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeliveryMapper.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeliveryMapper.java deleted file mode 100644 index 0572c18..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeliveryMapper.java +++ /dev/null @@ -1,195 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.CausationId; -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.CorrelationId; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TenantContext; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.delivery.DeliveryMetadata; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessageValidationException; -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.schema.EncodedMessage; -import java.time.Instant; -import java.util.Objects; -import java.util.Optional; -import java.util.UUID; -import org.springframework.amqp.core.Message; -import org.springframework.amqp.core.MessageProperties; - -/** - * Rebuilds a platform envelope from an AMQP message. - * - *

The payload stays encoded so a codec failure is classified by the platform and parked, rather - * than surfacing as an AMQP listener exception the container would requeue in a hot loop. - * - *

The delivery attempt comes from the platform's own retry header rather than from AMQP's {@code - * redelivered} flag. That flag only says "this message was delivered before", not how many times, - * and it resets when a message travels through a retry queue. - */ -public final class RabbitDeliveryMapper { - - private final RabbitHeaderMapper headerMapper; - - /** Creates a mapper with the default header mapper. */ - public RabbitDeliveryMapper() { - this(new RabbitHeaderMapper()); - } - - /** - * Creates a mapper with an explicit header mapper. - * - * @param headerMapper the header mapper - */ - public RabbitDeliveryMapper(RabbitHeaderMapper headerMapper) { - this.headerMapper = Objects.requireNonNull(headerMapper, "headerMapper must not be null"); - } - - /** - * Converts an AMQP message into an envelope. - * - * @param message the AMQP message - * @return the still-encoded envelope - */ - public MessageEnvelope toEnvelope(Message message) { - Objects.requireNonNull(message, "message must not be null"); - MessageProperties properties = message.getMessageProperties(); - - ContentType contentType = - new ContentType( - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.CONTENT_TYPE)) - .or(() -> Optional.ofNullable(properties.getContentType())) - .orElse(ContentType.JSON.value())); - - byte[] payload = message.getBody() == null ? new byte[0] : message.getBody(); - Instant producedAt = instant(properties, ReservedHeaders.PRODUCED_AT); - - return new MessageEnvelope<>( - new MessageId(uuid(properties, ReservedHeaders.MESSAGE_ID)), - new MessageType(required(properties, ReservedHeaders.MESSAGE_TYPE)), - new SchemaVersion(intValue(properties, ReservedHeaders.SCHEMA_VERSION)), - producedAt, - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.OCCURRED_AT)) - .map(Instant::parse), - new ProducerId( - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.PRODUCER)) - .orElse("unknown")), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.CORRELATION_ID)) - .map(CorrelationId::new), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.CAUSATION_ID)) - .map(value -> new CausationId(new MessageId(UUID.fromString(value)))), - contentType, - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.PARTITION_KEY)), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.ORDERING_KEY)), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.TENANT)) - .map(TenantContext::new), - new TraceContext( - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.TRACEPARENT)), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.TRACESTATE)), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.BAGGAGE))), - headerMapper.carriedForward(properties), - new EncodedMessage(payload, contentType, Optional.empty())); - } - - /** - * Builds the delivery metadata for an AMQP message. - * - * @param destination the logical destination - * @param message the AMQP message - * @param receivedAt when the consumer received it - * @return the delivery metadata - */ - public DeliveryMetadata toMetadata( - DestinationName destination, Message message, Instant receivedAt) { - MessageProperties properties = message.getMessageProperties(); - int attempt = attemptOf(properties); - - return new DeliveryMetadata( - destination, - attempt, - attempt > 1, - Optional.of( - new RabbitPublishReference( - Optional.ofNullable(properties.getReceivedExchange()).orElse(""), - Optional.ofNullable(properties.getReceivedRoutingKey()).orElse(""), - Math.max(0, properties.getDeliveryTag()))), - Optional.ofNullable(properties.getConsumerQueue()), - Optional.ofNullable(properties.getConsumerTag()), - receivedAt); - } - - /** - * Returns the attempt a message carries, counting the first delivery as one. - * - * @param properties the message properties - * @return the attempt number - */ - public int attemptOf(MessageProperties properties) { - String value = headerMapper.value(properties, ReservedHeaders.RETRY_ATTEMPT); - if (value == null) { - return 1; - } - try { - int attempt = Integer.parseInt(value); - if (attempt < 1) { - throw new NumberFormatException("attempt " + attempt); - } - return attempt; - } catch (NumberFormatException malformed) { - // Fail closed. Resetting to 1 restarts the retry budget on every redelivery, so a message - // that should have been dead-lettered is retried forever — and the header is - // caller-influenced, which turns "unreadable" into a way to defeat the cap. - throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( - "RETRY_ATTEMPT_MALFORMED", - "the retry attempt header is not a positive integer; the message is quarantined rather" - + " than restarting its retry budget"); - } - } - - private String required(MessageProperties properties, String name) { - String value = headerMapper.value(properties, name); - if (value == null || value.isBlank()) { - throw new MessageValidationException( - "RABBIT_HEADER_MISSING", "message is missing the required header " + name); - } - return value; - } - - private UUID uuid(MessageProperties properties, String name) { - try { - return UUID.fromString(required(properties, name)); - } catch (IllegalArgumentException exception) { - throw new MessageValidationException( - "RABBIT_HEADER_MALFORMED", "header " + name + " is not a UUID", exception); - } - } - - private int intValue(MessageProperties properties, String name) { - try { - return Integer.parseInt(required(properties, name)); - } catch (NumberFormatException exception) { - throw new MessageValidationException( - "RABBIT_HEADER_MALFORMED", "header " + name + " is not an integer", exception); - } - } - - private Instant instant(MessageProperties properties, String name) { - String value = headerMapper.value(properties, name); - if (value == null || value.isBlank()) { - return Optional.ofNullable(properties.getTimestamp()) - .map(java.util.Date::toInstant) - .orElse(Instant.EPOCH); - } - try { - return Instant.parse(value); - } catch (java.time.format.DateTimeParseException exception) { - throw new MessageValidationException( - "RABBIT_HEADER_MALFORMED", "header " + name + " is not an instant", exception); - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitHeaderMapper.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitHeaderMapper.java deleted file mode 100644 index dc5124c..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitHeaderMapper.java +++ /dev/null @@ -1,180 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.header.CanonicalEnvelopeHeaders; -import dev.caskeleton.messaging.api.header.HeaderName; -import dev.caskeleton.messaging.api.header.HeaderValue; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Objects; -import org.springframework.amqp.core.MessageProperties; - -/** - * Maps envelope identity onto AMQP message properties and back. - * - *

Identity goes in headers, not in the payload, so a consumer can classify and park a message it - * cannot deserialize. The AMQP {@code message-id} and {@code correlation-id} properties are also - * populated, because management UIs and tracing tools read those and not the platform's headers — - * but the platform's own headers stay authoritative, since AMQP's properties have no schema version - * or producer field. - */ -public final class RabbitHeaderMapper { - - /** - * Writes an envelope's identity into AMQP message properties. - * - * @param envelope the envelope being published - * @return the message properties - */ - public MessageProperties toProperties(MessageEnvelope envelope) { - Objects.requireNonNull(envelope, "envelope must not be null"); - MessageProperties properties = new MessageProperties(); - - properties.setMessageId(envelope.messageId().value().toString()); - properties.setContentType(envelope.contentType().value()); - properties.setTimestamp(java.util.Date.from(envelope.producedAt())); - properties.setDeliveryMode(org.springframework.amqp.core.MessageDeliveryMode.PERSISTENT); - - setHeader(properties, ReservedHeaders.MESSAGE_ID, envelope.messageId().value().toString()); - setHeader(properties, ReservedHeaders.MESSAGE_TYPE, envelope.messageType().value()); - setHeader( - properties, - ReservedHeaders.SCHEMA_VERSION, - Integer.toString(envelope.schemaVersion().value())); - setHeader(properties, ReservedHeaders.PRODUCER, envelope.producer().value()); - setHeader(properties, ReservedHeaders.PRODUCED_AT, envelope.producedAt().toString()); - setHeader(properties, ReservedHeaders.CONTENT_TYPE, envelope.contentType().value()); - - envelope - .occurredAt() - .ifPresent(value -> setHeader(properties, ReservedHeaders.OCCURRED_AT, value.toString())); - envelope - .correlationId() - .ifPresent( - value -> { - properties.setCorrelationId(value.value()); - setHeader(properties, ReservedHeaders.CORRELATION_ID, value.value()); - }); - envelope - .causationId() - .ifPresent( - value -> - setHeader( - properties, ReservedHeaders.CAUSATION_ID, value.value().value().toString())); - envelope - .partitionKey() - .ifPresent(value -> setHeader(properties, ReservedHeaders.PARTITION_KEY, value)); - envelope - .orderingKey() - .ifPresent(value -> setHeader(properties, ReservedHeaders.ORDERING_KEY, value)); - // Written for the same reason as on the other broker: an AMQP message carries no ambient - // tenant, so a tenant left off the wire is a tenant the consumed envelope does not have. - envelope - .tenantContext() - .ifPresent(value -> setHeader(properties, ReservedHeaders.TENANT, value.tenantId())); - envelope - .traceContext() - .traceparent() - .ifPresent(value -> setHeader(properties, ReservedHeaders.TRACEPARENT, value)); - envelope - .traceContext() - .tracestate() - .ifPresent(value -> setHeader(properties, ReservedHeaders.TRACESTATE, value)); - envelope - .traceContext() - .baggage() - .ifPresent(value -> setHeader(properties, ReservedHeaders.BAGGAGE, value)); - - // The envelope's header map is written last, so an unchecked name would overwrite the canonical - // reserved value the platform just wrote. A forged `msg.id` corrupts another message's inbox - // deduplication, not merely its own delivery. - // - // Only names that restate an envelope field are refused. Retry attempt, failure and redrive - // bookkeeping have no envelope field to be read back from, so refusing them would make every - // retry-queue and dead-letter re-publish fail on the guard meant to protect identity. - envelope - .headers() - .asMap() - .forEach( - (name, value) -> { - if (CanonicalEnvelopeHeaders.restatesEnvelopeField(name.value())) { - throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( - "RESERVED_HEADER_FORGED", - "header '" - + name.value() - + "' restates a value the envelope already carries; the canonical value is" - + " written by the platform and must not be overridable by message content"); - } - setHeader(properties, name.value(), value.value()); - }); - - return properties; - } - - /** - * Reads AMQP headers back into a platform header map. - * - * @param properties the message properties - * @return the platform headers, including reserved names - */ - public MessageHeaders fromProperties(MessageProperties properties) { - Objects.requireNonNull(properties, "properties must not be null"); - Map values = new LinkedHashMap<>(); - properties - .getHeaders() - .forEach( - (name, value) -> { - if (value != null) { - values.put(new HeaderName(name), new HeaderValue(String.valueOf(value))); - } - }); - return MessageHeaders.platform(values); - } - - /** - * Reads back the headers a rebuilt envelope keeps. - * - *

Everything except the names that were hoisted into envelope fields: the application's own - * headers, which were dropped entirely and so never survived a broker hop, and the platform's - * retry and redrive bookkeeping, whose loss restarted the attempt counter at one on every - * redelivery and left the retry budget capping nothing. - * - *

The hoisted names are excluded rather than kept alongside their fields, because a duplicate - * that can drift from the field is how the same message ends up with two different ids depending - * on which one the reader trusts. - * - * @param properties the message properties - * @return the headers to carry on the rebuilt envelope - */ - public MessageHeaders carriedForward(MessageProperties properties) { - Objects.requireNonNull(properties, "properties must not be null"); - Map values = new LinkedHashMap<>(); - properties - .getHeaders() - .forEach( - (name, value) -> { - if (value != null && !CanonicalEnvelopeHeaders.restatesEnvelopeField(name)) { - values.put(new HeaderName(name), new HeaderValue(String.valueOf(value))); - } - }); - return MessageHeaders.platform(values); - } - - /** - * Reads one header value. - * - * @param properties the message properties - * @param name the header name - * @return the value, or null when absent - */ - public String value(MessageProperties properties, String name) { - Object value = properties.getHeaders().get(name); - return value == null ? null : String.valueOf(value); - } - - private static void setHeader(MessageProperties properties, String name, String value) { - properties.setHeader(name, value); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitMessagingTransport.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitMessagingTransport.java deleted file mode 100644 index b6d5862..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitMessagingTransport.java +++ /dev/null @@ -1,286 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.destination.DestinationCapabilities; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.transport.MessagingTransport; -import dev.caskeleton.messaging.transport.TransportConsumerRegistration; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import dev.caskeleton.messaging.transport.TransportPublishResult; -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.Executors; -import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicBoolean; -import java.util.function.Function; - -/** - * The Stable RabbitMQ adapter. - * - *

Publishes are always sent with {@code mandatory} set and correlated through {@link - * RabbitConfirmCoordinator}, because in AMQP a confirm and a return answer different questions. The - * confirm says the exchange accepted the message; the return says no queue was bound to receive it. - * An adapter that completed on the confirm alone would report success for a message the broker - * discarded, which is the single most damaging thing a Rabbit adapter can get wrong. - * - *

Payload size and shutdown state are checked before anything reaches the channel, so both fail - * with {@code NOT_TRANSMITTED} evidence rather than as a broker error whose ambiguity the caller - * then has to reason about. - */ -public final class RabbitMessagingTransport implements MessagingTransport { - - private static final MessagingCapabilities CAPABILITIES = - new MessagingCapabilities( - true, true, true, false, false, false, false, true, false, false, true, true); - - private final String brokerName; - private final long generation; - private final RabbitChannelPublisher channel; - private final RabbitPublishMapper mapper; - private final RabbitConfirmCoordinator coordinator; - private final Function consumerFactory; - private final Duration confirmTimeout; - private final ScheduledExecutorService confirmTimeouts; - private final AtomicBoolean closed = new AtomicBoolean(); - - /** - * Creates a publish-only transport. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param channel the channel publish operation - */ - public RabbitMessagingTransport( - String brokerName, long generation, RabbitChannelPublisher channel) { - this(brokerName, generation, channel, Duration.ofSeconds(5)); - } - - /** Creates a publish-only transport with an explicit confirm timeout. */ - public RabbitMessagingTransport( - String brokerName, long generation, RabbitChannelPublisher channel, Duration confirmTimeout) { - this( - brokerName, - generation, - channel, - spec -> { - throw new MessagingCapabilityUnavailableException( - "RABBIT_CONSUMER_NOT_CONFIGURED", - "this Rabbit transport was created without a consumer factory"); - }, - confirmTimeout); - } - - /** - * Creates a transport with a consumer factory. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param channel the channel publish operation - * @param consumerFactory builds a consumer registration for a spec - */ - public RabbitMessagingTransport( - String brokerName, - long generation, - RabbitChannelPublisher channel, - Function consumerFactory) { - this(brokerName, generation, channel, consumerFactory, Duration.ofSeconds(5)); - } - - /** Creates a transport with explicit consumer factory and confirm deadline. */ - public RabbitMessagingTransport( - String brokerName, - long generation, - RabbitChannelPublisher channel, - Function consumerFactory, - Duration confirmTimeout) { - this.brokerName = Objects.requireNonNull(brokerName, "brokerName must not be null"); - this.generation = generation; - this.channel = Objects.requireNonNull(channel, "channel must not be null"); - this.consumerFactory = Objects.requireNonNull(consumerFactory, "consumerFactory is required"); - this.confirmTimeout = Objects.requireNonNull(confirmTimeout, "confirmTimeout must not be null"); - if (confirmTimeout.isNegative() || confirmTimeout.isZero()) { - throw new IllegalArgumentException("confirmTimeout must be positive"); - } - this.mapper = new RabbitPublishMapper(); - this.coordinator = new RabbitConfirmCoordinator(); - this.confirmTimeouts = - Executors.newSingleThreadScheduledExecutor( - runnable -> { - Thread thread = new Thread(runnable, "messaging-rabbit-confirm-timeouts"); - thread.setDaemon(true); - return thread; - }); - } - - @Override - public CompletionStage publish(TransportPublishRequest request) { - Objects.requireNonNull(request, "request must not be null"); - - if (closed.get()) { - return completed( - rejectedLocally("RABBIT_TRANSPORT_CLOSED", "the transport is shutting down")); - } - int size = request.envelope().payload().size(); - int limit = request.profile().payload().maxBytes(); - if (size > limit) { - return completed( - rejectedLocally( - "PAYLOAD_TOO_LARGE", "encoded payload is " + size + " bytes, limit is " + limit)); - } - - String exchange = mapper.exchange(request.profile()); - String routingKey = mapper.routingKey(request.profile()); - - // Register before publishing. A confirm can land on the connection thread while basicPublish is - // still returning; registering afterwards would drop it and leave the caller waiting forever. - long sequence = channel.nextPublishSequence(); - CompletionStage pending = - coordinator - .awaiting(sequence, exchange, routingKey, request.profile().producer().confirmation()) - .thenApply(TransportPublishResult::new); - var timeout = - confirmTimeouts.schedule( - () -> coordinator.timedOut(sequence, confirmTimeout), - confirmTimeout.toNanos(), - TimeUnit.NANOSECONDS); - pending.whenComplete((result, failure) -> timeout.cancel(false)); - - try { - channel.publish(sequence, exchange, routingKey, mapper.toMessage(request), true); - } catch (RuntimeException sendFailed) { - // A synchronous failure from basicPublish used to propagate to the caller as a thrown - // exception while the pending entry stayed in the map: the caller got an exception instead - // of the stage the contract promises, and the entry waited for a confirm that could never - // arrive. The classifier decides whether the frame can have left; anything it cannot rule - // out is ambiguous rather than rejected. - coordinator.sendFailed( - sequence, new RabbitPublishFailureClassifier().classify(sendFailed, true, Duration.ZERO)); - } - return pending; - } - - /** - * Records a broker confirm. Called from the channel's confirm callback. - * - * @param sequence the publish sequence number - * @param acknowledged true for an ack, false for a nack - * @param elapsed how long the publish took - */ - public void onConfirm(long sequence, boolean acknowledged, Duration elapsed) { - onConfirm(sequence, false, acknowledged, elapsed); - } - - /** - * Records a broker confirm carrying the {@code multiple} flag. - * - *

The flag was dropped, so a broker confirming a range resolved one publish and left every - * earlier one pending forever. - * - * @param multiple true when every sequence up to and including this one is confirmed - */ - public void onConfirm(long sequence, boolean multiple, boolean acknowledged, Duration elapsed) { - coordinator.confirmed(sequence, multiple, acknowledged, elapsed); - } - - /** - * Records an unroutable return. Called from the channel's return callback. - * - * @param sequence the publish sequence number - */ - public void onReturn(long sequence) { - coordinator.returned(sequence); - } - - /** - * Records that a confirm never arrived. - * - * @param sequence the publish sequence number - * @param elapsed how long was waited - */ - public void onConfirmTimeout(long sequence, Duration elapsed) { - coordinator.timedOut(sequence, elapsed); - } - - /** - * Returns how many publishes are awaiting a confirm. - * - * @return the pending count - */ - public int pendingConfirms() { - return coordinator.pendingCount(); - } - - @Override - public TransportConsumerRegistration register(TransportConsumerSpec spec) { - Objects.requireNonNull(spec, "spec must not be null"); - if (closed.get()) { - throw new MessagingCapabilityUnavailableException( - "RABBIT_TRANSPORT_CLOSED", "the transport is shutting down"); - } - return consumerFactory.apply(spec); - } - - @Override - public DestinationCapabilities capabilities(DestinationName destination) { - return new DestinationCapabilities(destination, brokerName, CAPABILITIES); - } - - @Override - public String brokerName() { - return brokerName; - } - - @Override - public long generation() { - return generation; - } - - @Override - public void close() { - if (!closed.compareAndSet(false, true)) { - return; - } - // Drain what is still waiting. Leaving the stages pending hangs every caller holding one, and - // a closed channel will never deliver their confirms. - coordinator.channelClosed(Duration.ZERO); - confirmTimeouts.shutdownNow(); - channel.close(); - } - - /** - * Reports whether the transport is still accepting work. - * - * @return true until close - */ - public boolean isAcceptingWork() { - return !closed.get(); - } - - private static TransportPublishResult rejectedLocally(String code, String message) { - return new TransportPublishResult( - new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of(FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message)))); - } - - private static CompletionStage completed(TransportPublishResult result) { - return CompletableFuture.completedFuture(result); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitNativeDeadLetterCapability.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitNativeDeadLetterCapability.java deleted file mode 100644 index 84d5e88..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitNativeDeadLetterCapability.java +++ /dev/null @@ -1,70 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import java.util.Objects; - -/** - * Decides whether the broker's own dead-lettering may be used, or the platform must publish. - * - *

RabbitMQ is the one Stable broker with native dead-lettering, and using it is tempting: a - * {@code basic.reject} with {@code requeue=false} moves the message with no publish at all. The - * catch is that the move is not confirmed to the publisher. If the dead-letter exchange is - * unroutable — nobody bound a queue to it, or the binding was removed — the broker discards the - * message silently and the reject still succeeds. - * - *

So native dead-lettering is only used where the dead-letter path has been verified to have a - * bound queue. Everywhere else the platform publishes and waits for its own confirm, which is the - * same invariant every other broker gets: the source is settled only after the dead-letter - * publish is confirmed. - */ -public final class RabbitNativeDeadLetterCapability { - - private final RabbitTopologyProfile topology; - private final boolean deadLetterQueueBound; - - /** - * Creates a capability check for one destination. - * - * @param topology the declared topology - * @param deadLetterQueueBound whether a queue is verified bound to the dead letter exchange - */ - public RabbitNativeDeadLetterCapability( - RabbitTopologyProfile topology, boolean deadLetterQueueBound) { - this.topology = Objects.requireNonNull(topology, "topology must not be null"); - this.deadLetterQueueBound = deadLetterQueueBound; - } - - /** - * Reports whether a rejected message may be left to the broker. - * - * @return true only when a dead letter exchange is declared and verified bound - */ - public boolean mayUseNativeDeadLetter() { - return topology.hasNativeDeadLetter() && deadLetterQueueBound; - } - - /** - * Reports whether the platform must publish the dead letter itself. - * - * @return true when native dead-lettering cannot be trusted for this destination - */ - public boolean requiresPlatformPublish() { - return !mayUseNativeDeadLetter(); - } - - /** - * Returns why native dead-lettering is not being used, for a startup diagnostic. - * - * @return the sanitized reason, empty when native dead-lettering is in use - */ - public String reasonForPlatformPublish() { - if (mayUseNativeDeadLetter()) { - return ""; - } - if (!topology.hasNativeDeadLetter()) { - return "queue %s was declared without a dead letter exchange".formatted(topology.queue()); - } - return "no queue is bound to dead letter exchange %s, so a broker-side reject would discard " - .formatted(topology.deadLetterExchange().orElse("(none)")) - + "the message silently while still reporting success"; - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidator.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidator.java deleted file mode 100644 index 3463b5c..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidator.java +++ /dev/null @@ -1,83 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.util.Objects; - -/** - * Startup validation of the RabbitMQ Stable profile. - * - *

Confirms and returns are both required, and they answer different questions. A confirm says - * the exchange accepted the message; a return says no queue was bound to receive it. Enabling - * confirms without returns produces the worst possible outcome: a publish that reports success - * while the message was silently discarded. - * - *

Durable work queues default to quorum queues because classic mirrored queues can lose - * acknowledged messages during a partition — which is precisely the guarantee a durable work queue - * exists to provide. - */ -public final class RabbitProfileValidator { - - /** - * Validates one RabbitMQ broker profile. - * - * @param profile the profile to validate - * @throws IllegalArgumentException when the profile cannot honour its declared guarantees - */ - public void validate(RabbitBrokerProfile profile) { - Objects.requireNonNull(profile, "profile must not be null"); - - if (profile.stable() && !profile.publisherConfirms()) { - throw new IllegalArgumentException( - "stable Rabbit producer requires publisher confirms: " + profile.broker()); - } - if (profile.stable() && !profile.publisherReturns()) { - throw new IllegalArgumentException( - "stable Rabbit producer requires publisher returns: " + profile.broker()); - } - if (profile.stable() && !profile.mandatory()) { - throw new IllegalArgumentException( - "stable Rabbit producer requires mandatory routing: " + profile.broker()); - } - if (profile.autoAck()) { - throw new IllegalArgumentException( - "consumer auto ack is forbidden; the platform acknowledges after handler success: " - + profile.broker()); - } - if (profile.prefetch() < 1) { - throw new IllegalArgumentException("prefetch must be at least 1: " + profile.broker()); - } - if (profile.confirmTimeout().isNegative() || profile.confirmTimeout().isZero()) { - throw new IllegalArgumentException("confirm timeout must be positive: " + profile.broker()); - } - if (profile.production() && !profile.tlsEnabled()) { - throw new IllegalArgumentException( - "a production Rabbit connection requires TLS: " + profile.broker()); - } - if (profile.production() && !profile.authenticationEnabled()) { - throw new IllegalArgumentException( - "a production Rabbit connection requires broker authentication: " + profile.broker()); - } - } - - /** - * Validates a destination against the broker profile. - * - * @param destination the destination profile - * @param broker the broker profile - * @throws IllegalArgumentException when the destination cannot be served safely - */ - public void validateDestination(DestinationProfile destination, RabbitBrokerProfile broker) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(broker, "broker must not be null"); - - if (destination.kind() == DestinationKind.WORK_QUEUE && !broker.quorumQueues()) { - throw new IllegalArgumentException( - "a durable Rabbit work queue must use a quorum queue: " + destination.name().value()); - } - if (destination.physical().queue().isEmpty() && destination.physical().exchange().isEmpty()) { - throw new IllegalArgumentException( - "a Rabbit destination requires an exchange or a queue: " + destination.name().value()); - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishFailureClassifier.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishFailureClassifier.java deleted file mode 100644 index c9408b9..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishFailureClassifier.java +++ /dev/null @@ -1,133 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.io.IOException; -import java.time.Duration; -import java.util.Optional; - -/** - * Turns an AMQP publish failure into a completion the caller can act on. - * - *

The distinction that matters is where the failure happened. A channel-level exception raised - * before {@code basicPublish} returned means nothing was written: rejected, with no - * ambiguity. A connection that drops after the frame was written proves nothing — the - * broker may have the message and only the confirm was lost. - * - *

Unknown failures default to ambiguous. Guessing "rejected" on an unrecognised error is what - * turns one lost confirm into two of the same order, and the cost of an unnecessary ambiguous is a - * retry the inbox absorbs. - */ -public final class RabbitPublishFailureClassifier { - - /** - * Classifies a failure raised while publishing. - * - * @param error the throwable the channel reported - * @param framePossiblyWritten whether the publish frame may already have reached the broker - * @param elapsed how long the attempt took - * @return the publish outcome with its evidence - */ - public PublishResult classify(Throwable error, boolean framePossiblyWritten, Duration elapsed) { - Throwable cause = unwrap(error); - - if (!framePossiblyWritten) { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - categoryOf(cause), - "RABBIT_PUBLISH_REJECTED", - "the channel refused the publish before writing it: " + name(cause)))); - } - - return new PublishResult( - PublishCompletion.AMBIGUOUS, - new PublishEvidence( - true, TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "RABBIT_CONFIRM_LOST", - "the frame was written but no confirm arrived: " + name(cause)))); - } - - /** - * Classifies a broker nack, which is always a definitive rejection. - * - * @param elapsed how long the attempt took - * @return the publish outcome - */ - public PublishResult nacked(Duration elapsed) { - return new PublishResult( - PublishCompletion.REJECTED, - // A nack means the broker took the frame and decided against it, so the bytes were - // transmitted even though nothing was stored. - new PublishEvidence(true, TransmissionEvidence.TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.TRANSIENT_INFRASTRUCTURE, - "RABBIT_NACKED", - "the broker negatively acknowledged the publish"))); - } - - private static FailureCategory categoryOf(Throwable cause) { - if (cause instanceof com.rabbitmq.client.PossibleAuthenticationFailureException) { - return FailureCategory.AUTHENTICATION; - } - if (cause instanceof com.rabbitmq.client.ShutdownSignalException shutdown) { - int replyCode = replyCode(shutdown.getReason()); - if (replyCode == 403) { - return FailureCategory.AUTHORIZATION; - } - if (replyCode >= 500) { - return FailureCategory.TRANSIENT_INFRASTRUCTURE; - } - return FailureCategory.PERMANENT_BUSINESS; - } - if (cause instanceof IOException) { - return FailureCategory.TRANSIENT_INFRASTRUCTURE; - } - return FailureCategory.PERMANENT_BUSINESS; - } - - private static int replyCode(com.rabbitmq.client.Method reason) { - if (reason instanceof com.rabbitmq.client.AMQP.Channel.Close close) { - return close.getReplyCode(); - } - if (reason instanceof com.rabbitmq.client.AMQP.Connection.Close close) { - return close.getReplyCode(); - } - return -1; - } - - private static Throwable unwrap(Throwable error) { - if (error instanceof java.util.concurrent.CompletionException && error.getCause() != null) { - return error.getCause(); - } - return error; - } - - private static String name(Throwable cause) { - return cause == null ? "unknown" : cause.getClass().getSimpleName(); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishMapper.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishMapper.java deleted file mode 100644 index ec03aae..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishMapper.java +++ /dev/null @@ -1,72 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import java.util.Objects; -import org.springframework.amqp.core.Message; - -/** - * Builds AMQP messages from platform publish requests. - * - *

The exchange and routing key come from the destination profile, never from the caller. A - * publisher that could choose its own routing key would be able to reach queues the destination's - * access policy never granted. - */ -public final class RabbitPublishMapper { - - private final RabbitHeaderMapper headerMapper; - - /** Creates a mapper with the default header mapper. */ - public RabbitPublishMapper() { - this(new RabbitHeaderMapper()); - } - - /** - * Creates a mapper with an explicit header mapper. - * - * @param headerMapper the header mapper - */ - public RabbitPublishMapper(RabbitHeaderMapper headerMapper) { - this.headerMapper = Objects.requireNonNull(headerMapper, "headerMapper must not be null"); - } - - /** - * Converts a publish request into an AMQP message. - * - * @param request the publish request - * @return the AMQP message - */ - public Message toMessage(TransportPublishRequest request) { - Objects.requireNonNull(request, "request must not be null"); - return new Message( - request.envelope().payload().bytes(), headerMapper.toProperties(request.envelope())); - } - - /** - * Returns the exchange a destination publishes to. - * - * @param profile the destination profile - * @return the exchange name - */ - public String exchange(DestinationProfile profile) { - return profile - .physical() - .exchange() - .orElseThrow( - () -> - new MessagingConfigurationException( - "RABBIT_EXCHANGE_MISSING", - "destination has no Rabbit exchange: " + profile.name().value())); - } - - /** - * Returns the routing key a destination publishes with. - * - * @param profile the destination profile - * @return the routing key - */ - public String routingKey(DestinationProfile profile) { - return profile.physical().routingKey().orElse(""); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishReference.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishReference.java deleted file mode 100644 index aee6d79..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishReference.java +++ /dev/null @@ -1,37 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.publish.BrokerPosition; -import java.util.Map; -import java.util.Objects; - -/** - * A RabbitMQ exchange, routing key, and publish sequence coordinate. - * - * @param exchange the exchange published to - * @param routingKey the routing key used - * @param sequence the channel publish sequence number - */ -public record RabbitPublishReference(String exchange, String routingKey, long sequence) - implements BrokerPosition { - - public RabbitPublishReference { - Objects.requireNonNull(exchange, "exchange must not be null"); - Objects.requireNonNull(routingKey, "routingKey must not be null"); - if (sequence < 0) { - throw new IllegalArgumentException("sequence must not be negative"); - } - } - - @Override - public String broker() { - return "rabbitmq"; - } - - @Override - public Map diagnosticAttributes() { - return Map.of( - "exchange", exchange, - "routingKey", routingKey, - "sequence", Long.toString(sequence)); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRequestReply.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRequestReply.java deleted file mode 100644 index 2854b41..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRequestReply.java +++ /dev/null @@ -1,32 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.schema.EncodedMessage; -import java.time.Duration; -import java.util.concurrent.CompletionStage; - -/** - * The M2 request-reply capability. - * - *

Deliberately limited. Request-reply over a broker is a synchronous call wearing an - * asynchronous costume: it holds a correlation entry and a caller for the whole round trip, and it - * fails in ways a plain publish does not — a lost reply is indistinguishable from a slow one. It is - * offered with a mandatory deadline and no automatic retry, because retrying a request whose reply - * was merely late duplicates the work on the other side. - */ -public interface RabbitRequestReply { - - /** - * Sends a request and awaits its correlated reply. - * - * @param destination the request destination - * @param request the encoded request envelope - * @param timeout how long to wait for the reply - * @return a stage completing with the reply, or failing on timeout - */ - CompletionStage> request( - MessageDestination destination, - MessageEnvelope request, - Duration timeout); -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRetryQueueTopology.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRetryQueueTopology.java deleted file mode 100644 index ca42dfb..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRetryQueueTopology.java +++ /dev/null @@ -1,57 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import java.time.Duration; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Objects; - -/** - * Describes the delayed retry queue a destination uses. - * - *

RabbitMQ has no native per-message delay in the core broker, so a retry queue is a queue with - * a message TTL whose dead letter exchange points back at the work queue: the message sits until - * the TTL expires and is then re-routed. Modelling this explicitly, rather than hiding it behind a - * plugin, keeps the behaviour reviewable — including its main caveat, that TTL expiry is evaluated - * at the head of the queue, so mixed delays in one retry queue do not expire independently. - * - * @param retryQueue the queue messages wait in - * @param deadLetterExchange the exchange expired messages are routed back through - * @param deadLetterRoutingKey the routing key expired messages are re-published with - * @param messageTtl how long messages wait - */ -public record RabbitRetryQueueTopology( - String retryQueue, - String deadLetterExchange, - String deadLetterRoutingKey, - Duration messageTtl) { - - public RabbitRetryQueueTopology { - Objects.requireNonNull(messageTtl, "messageTtl must not be null"); - if (retryQueue == null || retryQueue.isBlank()) { - throw new IllegalArgumentException("retryQueue must not be blank"); - } - if (deadLetterExchange == null) { - throw new IllegalArgumentException("deadLetterExchange must not be null"); - } - if (deadLetterRoutingKey == null) { - throw new IllegalArgumentException("deadLetterRoutingKey must not be null"); - } - if (messageTtl.isNegative() || messageTtl.isZero()) { - throw new IllegalArgumentException("messageTtl must be positive"); - } - } - - /** - * Returns the queue arguments this topology requires. - * - * @return the AMQP queue arguments - */ - public Map queueArguments() { - Map arguments = new LinkedHashMap<>(); - arguments.put("x-queue-type", "quorum"); - arguments.put("x-message-ttl", messageTtl.toMillis()); - arguments.put("x-dead-letter-exchange", deadLetterExchange); - arguments.put("x-dead-letter-routing-key", deadLetterRoutingKey); - return Map.copyOf(arguments); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSecurityConfigurer.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSecurityConfigurer.java deleted file mode 100644 index 9841916..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSecurityConfigurer.java +++ /dev/null @@ -1,179 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.security.BrokerCredentialProfile; -import dev.caskeleton.messaging.security.BrokerSecurityProfile; -import dev.caskeleton.messaging.security.BrokerTlsPolicy; -import dev.caskeleton.messaging.security.CredentialRuntime; -import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; -import java.time.Instant; -import java.util.List; -import java.util.Objects; - -/** - * Builds the AMQP connection settings from a validated security profile. - * - *

Credentials are resolved per connection attempt rather than cached in the factory. RabbitMQ - * client connections are long-lived and reconnect on their own, so a factory holding a credential - * from startup will happily reconnect with a revoked one for as long as the process runs — the - * reconnect is exactly the moment a rotated credential should take effect. - * - *

The password is returned inside a short-lived {@link AmqpCredentials} rather than assigned to - * a connection factory field, so the caller controls when it is cleared. - */ -public final class RabbitSecurityConfigurer { - - private final CredentialRuntimeRegistry credentials; - private final BrokerTlsPolicy tlsPolicy; - - /** - * Creates a configurer. - * - * @param credentials resolves and rotates credential material - * @param tlsPolicy validates the transport security posture - */ - public RabbitSecurityConfigurer( - CredentialRuntimeRegistry credentials, BrokerTlsPolicy tlsPolicy) { - this.credentials = Objects.requireNonNull(credentials, "credentials must not be null"); - this.tlsPolicy = Objects.requireNonNull(tlsPolicy, "tlsPolicy must not be null"); - } - - /** - * Resolves the connection credentials for one role. - * - * @param profile the broker security profile - * @param credential the role credential to configure - * @param enabledProtocols the TLS protocol versions to enable - * @param now the current instant - * @return the AMQP connection credentials - */ - public AmqpCredentials configure( - BrokerSecurityProfile profile, - BrokerCredentialProfile credential, - List enabledProtocols, - Instant now) { - Objects.requireNonNull(profile, "profile must not be null"); - Objects.requireNonNull(credential, "credential must not be null"); - Objects.requireNonNull(enabledProtocols, "enabledProtocols must not be null"); - - tlsPolicy.validate(profile, enabledProtocols); - - return switch (credential) { - case BrokerCredentialProfile.UsernamePassword plain -> { - CredentialRuntime resolved = credentials.resolve(plain.credentialId(), now); - yield new AmqpCredentials( - "PLAIN", plain.credentialId(), resolved.material(), profile.tlsEnabled()); - } - case BrokerCredentialProfile.MutualTls mutual -> { - // The certificate is the credential; there is no password to carry. - credentials.resolve(mutual.credentialId(), now); - yield new AmqpCredentials("EXTERNAL", mutual.credentialId(), new char[0], true); - } - case BrokerCredentialProfile.OAuth2 oauth -> { - CredentialRuntime resolved = credentials.resolve(oauth.credentialId(), now); - // RabbitMQ's OAuth 2 plugin takes the token in the password field of a PLAIN exchange. - yield new AmqpCredentials( - "PLAIN", oauth.credentialId(), resolved.material(), profile.tlsEnabled()); - } - case BrokerCredentialProfile.SaslScram scram -> { - CredentialRuntime resolved = credentials.resolve(scram.credentialId(), now); - yield new AmqpCredentials( - "RABBIT-CR-DEMO", scram.credentialId(), resolved.material(), profile.tlsEnabled()); - } - case BrokerCredentialProfile.Nkey ignored -> - throw new IllegalArgumentException( - "NKey credentials are a NATS concept, not an AMQP one"); - }; - } - - /** - * One connection's resolved credentials. - * - *

A class rather than a record because the secret has to stay mutable to be cleared, and a - * record carrying a {@code char[]} would compare and hash it by identity — two credentials with - * the same material would be unequal, which is a surprise waiting in any future collection use. - * - *

{@link #clear()} overwrites the secret. The caller is expected to call it once the - * connection is established, so the material's lifetime is the handshake rather than the process. - */ - public static final class AmqpCredentials { - - private final String mechanism; - private final String username; - private final char[] secret; - private final boolean tls; - - /** - * Creates connection credentials. - * - * @param mechanism the SASL mechanism name - * @param username the AMQP username - * @param secret the password or token, taken by reference and cleared by {@link #clear()} - * @param tls whether the connection uses TLS - */ - public AmqpCredentials(String mechanism, String username, char[] secret, boolean tls) { - Objects.requireNonNull(secret, "secret must not be null"); - if (mechanism == null || mechanism.isBlank()) { - throw new IllegalArgumentException("mechanism must not be blank"); - } - if (username == null || username.isBlank()) { - throw new IllegalArgumentException("username must not be blank"); - } - this.mechanism = mechanism; - this.username = username; - this.secret = secret; - this.tls = tls; - } - - /** - * Returns the SASL mechanism name. - * - * @return the mechanism - */ - public String mechanism() { - return mechanism; - } - - /** - * Returns the AMQP username. - * - * @return the username - */ - public String username() { - return username; - } - - /** - * Returns a copy of the secret. - * - * @return the password or token - */ - public char[] secret() { - return secret.clone(); - } - - /** - * Reports whether the connection uses TLS. - * - * @return true when TLS is enabled - */ - public boolean tls() { - return tls; - } - - /** Overwrites the secret in place. */ - public void clear() { - java.util.Arrays.fill(secret, '\0'); - } - - /** - * Returns a representation that names the connection without revealing it. - * - * @return a safe description - */ - @Override - public String toString() { - return "AmqpCredentials[mechanism=%s, username=%s, tls=%s]" - .formatted(mechanism, username, tls); - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementController.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementController.java deleted file mode 100644 index 3f0a248..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementController.java +++ /dev/null @@ -1,84 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessageSettlementException; -import dev.caskeleton.messaging.api.settlement.SettlementController; -import dev.caskeleton.messaging.api.settlement.SettlementResult; -import java.time.Duration; -import java.util.Objects; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicBoolean; - -/** - * Manual settlement for one RabbitMQ delivery. - * - *

Exactly one terminal call is allowed. A second one fails loudly rather than racing the first, - * because a duplicated {@code basic.ack} on a reused delivery tag acknowledges a different - * message — a failure mode that shows up as unexplained message loss under load, far from its - * cause. - * - *

{@code retry} does not use {@code basic.nack} with requeue. Requeueing puts the message back - * at the head of the queue with no delay, which turns a transient downstream failure into a hot - * loop; the delayed retry queue is used instead. - */ -public final class RabbitSettlementController implements SettlementController { - - private final long deliveryTag; - private final RabbitSettlementOperations operations; - private final AtomicBoolean settled = new AtomicBoolean(); - - /** - * Creates a controller for one delivery. - * - * @param deliveryTag the AMQP delivery tag - * @param operations the channel operations, invoked on the container thread - */ - public RabbitSettlementController(long deliveryTag, RabbitSettlementOperations operations) { - this.deliveryTag = deliveryTag; - this.operations = Objects.requireNonNull(operations, "operations must not be null"); - } - - @Override - public CompletionStage ack() { - return terminal(() -> operations.ack(deliveryTag)); - } - - @Override - public CompletionStage retry(Duration delay) { - Objects.requireNonNull(delay, "delay must not be null"); - return terminal(() -> operations.publishToRetryQueue(deliveryTag, delay)); - } - - @Override - public CompletionStage deadLetter(FailureDescriptor failure) { - Objects.requireNonNull(failure, "failure must not be null"); - return terminal(() -> operations.deadLetter(deliveryTag, failure)); - } - - @Override - public CompletionStage reject(FailureDescriptor failure) { - Objects.requireNonNull(failure, "failure must not be null"); - return terminal(() -> operations.discard(deliveryTag, failure)); - } - - /** - * Reports whether this delivery has already been settled. - * - * @return true once a terminal call has been made - */ - public boolean isSettled() { - return settled.get(); - } - - private CompletionStage terminal( - java.util.function.Supplier> operation) { - if (!settled.compareAndSet(false, true)) { - return CompletableFuture.failedFuture( - new MessageSettlementException( - "SETTLEMENT_ALREADY_COMPLETED", - "delivery tag " + deliveryTag + " has already been settled")); - } - return operation.get(); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementOperations.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementOperations.java deleted file mode 100644 index 6f2c975..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementOperations.java +++ /dev/null @@ -1,50 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.settlement.SettlementResult; -import java.time.Duration; -import java.util.concurrent.CompletionStage; - -/** - * The channel-side operations a settlement controller may perform. - * - *

Separated from the controller so the one-terminal-call guard is testable without an AMQP - * channel, and so the controller cannot reach anything the platform has not exposed. - */ -public interface RabbitSettlementOperations { - - /** - * Acknowledges a delivery. - * - * @param deliveryTag the AMQP delivery tag - * @return a stage completing with the settlement outcome - */ - CompletionStage ack(long deliveryTag); - - /** - * Publishes to the delayed retry queue, then acknowledges the source once confirmed. - * - * @param deliveryTag the AMQP delivery tag - * @param delay how long the message should wait - * @return a stage completing with the settlement outcome - */ - CompletionStage publishToRetryQueue(long deliveryTag, Duration delay); - - /** - * Publishes to the dead letter destination, then acknowledges the source once confirmed. - * - * @param deliveryTag the AMQP delivery tag - * @param failure the sanitized failure - * @return a stage completing with the settlement outcome - */ - CompletionStage deadLetter(long deliveryTag, FailureDescriptor failure); - - /** - * Discards a delivery without requeueing or dead lettering. - * - * @param deliveryTag the AMQP delivery tag - * @param failure the sanitized failure - * @return a stage completing with the settlement outcome - */ - CompletionStage discard(long deliveryTag, FailureDescriptor failure); -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitTopologyProfile.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitTopologyProfile.java deleted file mode 100644 index 66633bb..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitTopologyProfile.java +++ /dev/null @@ -1,137 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; - -/** - * The declared shape of one RabbitMQ destination: exchange, queue, and the arguments that give the - * queue its guarantees. - * - *

Quorum queues are required for anything durable. A classic mirrored queue can lose - * acknowledged messages during a partition — the promotion of a stale mirror is exactly the case - * where "the broker confirmed it" stops being true — and RabbitMQ itself has deprecated them. - * - *

The dead-letter exchange is part of the topology rather than a runtime decision, because a - * queue's {@code x-dead-letter-exchange} can only be set at declaration. A destination that gains a - * dead-letter policy later cannot acquire one without deleting and recreating the queue, so the - * mismatch is caught here instead of surfacing when the first message needs parking. - * - * @param exchange the exchange to publish to - * @param exchangeType the exchange type - * @param routingKey the routing key - * @param queue the queue that receives the messages - * @param quorum whether the queue is a quorum queue - * @param deadLetterExchange the dead letter exchange, when the destination has one - * @param deadLetterRoutingKey the dead letter routing key, when the destination has one - * @param maxLength the queue length limit, when one is set - */ -public record RabbitTopologyProfile( - String exchange, - String exchangeType, - String routingKey, - String queue, - boolean quorum, - Optional deadLetterExchange, - Optional deadLetterRoutingKey, - Optional maxLength) { - - /** RabbitMQ's argument key for a queue's type. */ - public static final String QUEUE_TYPE = "x-queue-type"; - - /** RabbitMQ's argument key for a queue's dead letter exchange. */ - public static final String DEAD_LETTER_EXCHANGE = "x-dead-letter-exchange"; - - /** RabbitMQ's argument key for a queue's dead letter routing key. */ - public static final String DEAD_LETTER_ROUTING_KEY = "x-dead-letter-routing-key"; - - /** RabbitMQ's argument key for a queue's length limit. */ - public static final String MAX_LENGTH = "x-max-length"; - - public RabbitTopologyProfile { - Objects.requireNonNull(deadLetterExchange, "deadLetterExchange must not be null"); - Objects.requireNonNull(deadLetterRoutingKey, "deadLetterRoutingKey must not be null"); - Objects.requireNonNull(maxLength, "maxLength must not be null"); - requireText(exchange, "exchange"); - requireText(exchangeType, "exchangeType"); - requireText(routingKey, "routingKey"); - requireText(queue, "queue"); - - if (deadLetterRoutingKey.isPresent() && deadLetterExchange.isEmpty()) { - throw new IllegalArgumentException( - "a dead letter routing key without a dead letter exchange routes nowhere"); - } - if (maxLength.isPresent() && maxLength.get() < 1) { - throw new IllegalArgumentException("maxLength must be positive when set"); - } - } - - /** - * Refuses a topology that cannot back a durable destination. - * - * @param durable whether the destination promises durability - * @throws MessagingConfigurationException when a durable destination lacks a quorum queue - */ - public void requireSafeFor(boolean durable) { - if (durable && !quorum) { - throw new MessagingConfigurationException( - "CLASSIC_QUEUE_ON_DURABLE_DESTINATION", - "queue %s is not a quorum queue; a classic mirrored queue can lose acknowledged messages " - .formatted(queue) - + "when a stale mirror is promoted during a partition"); - } - } - - /** - * Returns the arguments this queue must be declared with. - * - * @return the declaration arguments - */ - public Map queueArguments() { - Map arguments = new LinkedHashMap<>(); - arguments.put(QUEUE_TYPE, quorum ? "quorum" : "classic"); - deadLetterExchange.ifPresent(value -> arguments.put(DEAD_LETTER_EXCHANGE, value)); - deadLetterRoutingKey.ifPresent(value -> arguments.put(DEAD_LETTER_ROUTING_KEY, value)); - maxLength.ifPresent(value -> arguments.put(MAX_LENGTH, value)); - return Map.copyOf(arguments); - } - - /** - * Reports whether the broker will route a rejected message itself. - * - * @return true when the queue was declared with a dead letter exchange - */ - public boolean hasNativeDeadLetter() { - return deadLetterExchange.isPresent(); - } - - /** - * Returns a durable work-queue topology with a dead letter exchange. - * - * @param exchange the exchange - * @param routingKey the routing key - * @param queue the queue - * @param deadLetterExchange the dead letter exchange - * @return the topology profile - */ - public static RabbitTopologyProfile workQueue( - String exchange, String routingKey, String queue, String deadLetterExchange) { - return new RabbitTopologyProfile( - exchange, - "direct", - routingKey, - queue, - true, - Optional.of(deadLetterExchange), - Optional.of(routingKey), - Optional.empty()); - } - - private static void requireText(String value, String field) { - if (value == null || value.isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitAdapterContractTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitAdapterContractTest.java deleted file mode 100644 index 60d8e24..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitAdapterContractTest.java +++ /dev/null @@ -1,24 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.testkit.MessagingAdapterContract; -import dev.caskeleton.messaging.testkit.MessagingAdapterHarness; -import org.junit.jupiter.api.Nested; - -/** - * Runs the shared adapter contract against the RabbitMQ adapter. - * - *

Identical to the Kafka run. Two brokers with completely different machinery — offsets and - * commits versus delivery tags and confirms — answering the same seven questions the same way is - * what makes the logical destination abstraction real rather than aspirational. - */ -class RabbitAdapterContractTest { - - @Nested - class Contract extends MessagingAdapterContract { - - @Override - protected MessagingAdapterHarness harness() { - return RabbitContractHarness.create(); - } - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinatorTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinatorTest.java deleted file mode 100644 index c9bd1ec..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinatorTest.java +++ /dev/null @@ -1,167 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; - -import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import java.time.Duration; -import java.util.concurrent.CompletionStage; -import org.junit.jupiter.api.Test; - -class RabbitConfirmCoordinatorTest { - - private static final ConfirmationRequirement REPLICATION = - ConfirmationRequirement.REPLICATION_OR_PERSISTENCE_ACK; - - private final RabbitConfirmCoordinator coordinator = new RabbitConfirmCoordinator(); - - @Test - void aPlainConfirmIsAConfirmedPublish() { - CompletionStage stage = - coordinator.awaiting(1L, "notification.work", "email", REPLICATION); - - coordinator.confirmed(1L, true, Duration.ofMillis(4)); - - PublishResult result = stage.toCompletableFuture().join(); - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.ROUTED); - assertThat(result.evidence().confirmationLevel()) - .isEqualTo(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK); - } - - @Test - void aReturnFollowedByAConfirmIsRejectedAsUnroutable() { - CompletionStage stage = - coordinator.awaiting(1L, "notification.work", "missing", REPLICATION); - - coordinator.returned(1L); - coordinator.confirmed(1L, true, Duration.ofMillis(4)); - - PublishResult result = stage.toCompletableFuture().join(); - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.UNROUTABLE); - assertThat(result.evidence().confirmationLevel()).isEqualTo(ConfirmationLevel.NONE); - } - - @Test - void aReturnAloneDoesNotCompleteThePublish() { - CompletionStage stage = - coordinator.awaiting(1L, "notification.work", "missing", REPLICATION); - - coordinator.returned(1L); - - assertThat(stage.toCompletableFuture()).isNotDone(); - assertThat(coordinator.pendingCount()).isEqualTo(1); - } - - @Test - void aNackIsARejectedPublish() { - CompletionStage stage = - coordinator.awaiting(2L, "notification.work", "email", REPLICATION); - - coordinator.confirmed(2L, false, Duration.ofMillis(9)); - - PublishResult result = stage.toCompletableFuture().join(); - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.failure()).isPresent(); - } - - @Test - void aMissingConfirmIsAmbiguousRatherThanFailed() { - CompletionStage stage = - coordinator.awaiting(3L, "notification.work", "email", REPLICATION); - - coordinator.timedOut(3L, Duration.ofSeconds(5)); - - PublishResult result = stage.toCompletableFuture().join(); - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.mayHaveBeenStored()).isTrue(); - assertThat(result.evidence().confirmationLevel()).isEqualTo(ConfirmationLevel.NONE); - } - - @Test - void aConfirmedPublishCarriesItsExchangeAndSequence() { - CompletionStage stage = - coordinator.awaiting(7L, "notification.work", "email", REPLICATION); - - coordinator.confirmed(7L, true, Duration.ofMillis(1)); - - PublishResult result = stage.toCompletableFuture().join(); - assertThat(result.position()).isPresent(); - assertThat(result.position().orElseThrow().diagnosticAttributes()) - .containsEntry("exchange", "notification.work") - .containsEntry("routingKey", "email") - .containsEntry("sequence", "7"); - } - - @Test - void resolvingAPublishRemovesItFromThePendingSet() { - coordinator.awaiting(1L, "notification.work", "email", REPLICATION); - coordinator.confirmed(1L, true, Duration.ofMillis(1)); - - assertThat(coordinator.pendingCount()).isZero(); - } - - @Test - void anUnknownSequenceIsIgnoredRatherThanFailingTheChannel() { - coordinator.confirmed(99L, true, Duration.ofMillis(1)); - coordinator.returned(99L); - coordinator.timedOut(99L, Duration.ofMillis(1)); - - assertThat(coordinator.pendingCount()).isZero(); - } - - @Test - @org.junit.jupiter.api.DisplayName("a multiple ack resolves every pending publish up to the tag") - void aMultipleAckResolvesEveryPendingPublish() { - RabbitConfirmCoordinator coordinator = new RabbitConfirmCoordinator(); - java.util.concurrent.CompletableFuture - first = coordinator.awaiting(1L, "orders", "created", REPLICATION).toCompletableFuture(); - java.util.concurrent.CompletableFuture - second = coordinator.awaiting(2L, "orders", "created", REPLICATION).toCompletableFuture(); - java.util.concurrent.CompletableFuture - third = coordinator.awaiting(3L, "orders", "created", REPLICATION).toCompletableFuture(); - - coordinator.confirmed(2L, true, true, java.time.Duration.ofMillis(5)); - - assertThat(first) - .as("the multiple flag was dropped, so everything below the tag waited forever") - .isCompleted(); - assertThat(second).isCompleted(); - assertThat(third).isNotCompleted(); - assertThat(coordinator.pendingCount()).isEqualTo(1); - } - - @Test - @org.junit.jupiter.api.DisplayName("a nack records the frame as transmitted") - void aNackRecordsTheFrameAsTransmitted() { - RabbitConfirmCoordinator coordinator = new RabbitConfirmCoordinator(); - java.util.concurrent.CompletableFuture - pending = coordinator.awaiting(1L, "orders", "created", REPLICATION).toCompletableFuture(); - - coordinator.confirmed(1L, false, java.time.Duration.ofMillis(5)); - - assertThat(pending.join().evidence().transmission()) - .as("a NACK is the broker's answer to a frame it received") - .isEqualTo(dev.caskeleton.messaging.api.publish.TransmissionEvidence.TRANSMITTED); - } - - @Test - @org.junit.jupiter.api.DisplayName( - "a closed channel completes every pending publish as ambiguous") - void aClosedChannelCompletesEveryPendingPublish() { - RabbitConfirmCoordinator coordinator = new RabbitConfirmCoordinator(); - java.util.concurrent.CompletableFuture - pending = coordinator.awaiting(1L, "orders", "created", REPLICATION).toCompletableFuture(); - - assertThat(coordinator.channelClosed(java.time.Duration.ofMillis(5))).isEqualTo(1); - - assertThat(pending.join().completion()) - .as("the frames were written; a channel close says nothing about whether they were stored") - .isEqualTo(dev.caskeleton.messaging.api.publish.PublishCompletion.AMBIGUOUS); - assertThat(coordinator.pendingCount()).isZero(); - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitContractHarness.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitContractHarness.java deleted file mode 100644 index 54394d5..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitContractHarness.java +++ /dev/null @@ -1,304 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.testkit.ContractMessage; -import dev.caskeleton.messaging.testkit.FaultController; -import dev.caskeleton.messaging.testkit.HandleOutcome; -import dev.caskeleton.messaging.testkit.MessagingAdapterHarness; -import dev.caskeleton.messaging.testkit.ObservedDelivery; -import java.time.Duration; -import java.util.ArrayDeque; -import java.util.ArrayList; -import java.util.Deque; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Optional; -import java.util.Set; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicLong; - -/** - * Runs the shared adapter contract against the real RabbitMQ confirm model. - * - *

Publishes go through the production {@link RabbitConfirmCoordinator}, which is the piece that - * has to get AMQP's confirm and return ordering right. Driving it from the harness is the only way - * to exercise a lost confirm deterministically — a live broker will not drop one on request. - * - *

Settlement goes through the production {@link RabbitSettlementController}, so the - * one-terminal-call guard and the DLQ-before-ack ordering are the real ones. - */ -final class RabbitContractHarness implements MessagingAdapterHarness { - - private static final int MAX_PAYLOAD_BYTES = 1_048_576; - private static final String EXCHANGE = "order.events"; - private static final String ROUTING_KEY = "created"; - - private final RabbitConfirmCoordinator coordinator = new RabbitConfirmCoordinator(); - private final AtomicLong sequence = new AtomicLong(); - private final AtomicLong deliveryTag = new AtomicLong(); - private final Faults faults = new Faults(); - private final Deque pending = new ArrayDeque<>(); - private final List deadLetters = new ArrayList<>(); - private final Set unsettled = new LinkedHashSet<>(); - - private boolean shuttingDown; - - static RabbitContractHarness create() { - return new RabbitContractHarness(); - } - - @Override - public String brokerName() { - return "rabbitmq"; - } - - @Override - public FaultController faults() { - return faults; - } - - @Override - public CompletionStage publish(ContractMessage message) { - if (shuttingDown) { - return CompletableFuture.completedFuture( - rejectedLocally("SHUTTING_DOWN", "the consumer is draining and accepts no new work")); - } - if (message.envelope().payload().size() > MAX_PAYLOAD_BYTES) { - return CompletableFuture.completedFuture( - rejectedLocally( - "PAYLOAD_TOO_LARGE", "encoded payload exceeds " + MAX_PAYLOAD_BYTES + " bytes")); - } - - long next = sequence.incrementAndGet(); - CompletionStage stage = - coordinator.awaiting( - next, EXCHANGE, ROUTING_KEY, ConfirmationRequirement.REPLICATION_OR_PERSISTENCE_ACK); - - if (faults.consumeRejectPublish()) { - coordinator.confirmed(next, false, Duration.ofMillis(2)); - } else if (faults.consumeDropPublishConfirmation()) { - coordinator.timedOut(next, Duration.ofSeconds(5)); - } else { - coordinator.confirmed(next, true, Duration.ofMillis(2)); - } - - return stage.thenApply( - result -> { - if (result.completion() == PublishCompletion.CONFIRMED) { - pending.addLast( - new Pending(message.messageId(), deliveryTag.incrementAndGet(), 1, false)); - } - return result; - }); - } - - @Override - public List drain(HandleOutcome outcome) { - List batch = new ArrayList<>(pending); - pending.clear(); - - List observed = new ArrayList<>(); - for (Pending delivery : batch) { - RabbitSettlementController controller = - new RabbitSettlementController(delivery.deliveryTag(), new HarnessOperations(delivery)); - boolean settled = - switch (outcome) { - case SUCCESS -> join(controller.ack()); - case RETRY -> join(controller.retry(Duration.ofSeconds(1))); - case DEAD_LETTER -> - join( - controller.deadLetter( - FailureDescriptor.of( - FailureCategory.PERMANENT_BUSINESS, - "CONTRACT_FAILURE", - "permanent failure"))); - }; - observed.add( - new ObservedDelivery( - delivery.messageId(), delivery.attempt(), delivery.redelivered(), settled)); - } - return List.copyOf(observed); - } - - private static boolean join( - CompletionStage stage) { - return stage.toCompletableFuture().join().completion() - == dev.caskeleton.messaging.api.settlement.SettlementCompletion.SETTLED; - } - - private void redeliver(Pending delivery) { - pending.addLast( - new Pending( - delivery.messageId(), deliveryTag.incrementAndGet(), delivery.attempt() + 1, true)); - } - - @Override - public List deadLettered() { - return List.copyOf(deadLetters); - } - - @Override - public List unsettled() { - return List.copyOf(unsettled); - } - - @Override - public void beginShutdown() { - shuttingDown = true; - } - - @Override - public boolean isAcceptingWork() { - return !shuttingDown; - } - - @Override - public void close() { - pending.clear(); - } - - private static PublishResult rejectedLocally(String code, String message) { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of(FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message))); - } - - /** One message in flight on the channel. */ - private record Pending(MessageId messageId, long deliveryTag, int attempt, boolean redelivered) {} - - /** The channel operations the settlement controller drives. */ - private final class HarnessOperations implements RabbitSettlementOperations { - - private final Pending delivery; - - private HarnessOperations(Pending delivery) { - this.delivery = delivery; - } - - @Override - public CompletionStage ack(long tag) { - if (faults.consumeDropSettlementConfirmation()) { - unsettled.add(delivery.messageId()); - redeliver(delivery); - return CompletableFuture.completedFuture(unknownSettlement()); - } - unsettled.remove(delivery.messageId()); - return CompletableFuture.completedFuture( - dev.caskeleton.messaging.api.settlement.SettlementResult.settled()); - } - - @Override - public CompletionStage - publishToRetryQueue(long tag, Duration delay) { - unsettled.add(delivery.messageId()); - redeliver(delivery); - return CompletableFuture.completedFuture(unknownSettlement()); - } - - /** Publishes to the dead letter destination first, and only then acknowledges the source. */ - @Override - public CompletionStage deadLetter( - long tag, FailureDescriptor failure) { - if (faults.deadLetterPublishFails()) { - unsettled.add(delivery.messageId()); - redeliver(delivery); - return CompletableFuture.completedFuture(unknownSettlement()); - } - deadLetters.add(delivery.messageId()); - unsettled.remove(delivery.messageId()); - return CompletableFuture.completedFuture( - dev.caskeleton.messaging.api.settlement.SettlementResult.settled()); - } - - @Override - public CompletionStage discard( - long tag, FailureDescriptor failure) { - unsettled.remove(delivery.messageId()); - return CompletableFuture.completedFuture( - dev.caskeleton.messaging.api.settlement.SettlementResult.settled()); - } - - private dev.caskeleton.messaging.api.settlement.SettlementResult unknownSettlement() { - return new dev.caskeleton.messaging.api.settlement.SettlementResult( - dev.caskeleton.messaging.api.settlement.SettlementCompletion.UNKNOWN, - dev.caskeleton.messaging.api.settlement.SettlementEvidence.unknown(), - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "SETTLEMENT_UNKNOWN", - "the settlement was not confirmed"))); - } - } - - /** One-shot fault flags. */ - private static final class Faults implements FaultController { - - private boolean dropPublishConfirmation; - private boolean dropSettlementConfirmation; - private boolean failDeadLetterPublish; - private boolean rejectPublish; - - @Override - public void dropPublishConfirmation() { - dropPublishConfirmation = true; - } - - @Override - public void dropSettlementConfirmation() { - dropSettlementConfirmation = true; - } - - @Override - public void failDeadLetterPublish() { - failDeadLetterPublish = true; - } - - @Override - public void rejectPublish() { - rejectPublish = true; - } - - @Override - public void reset() { - dropPublishConfirmation = false; - dropSettlementConfirmation = false; - failDeadLetterPublish = false; - rejectPublish = false; - } - - boolean consumeDropPublishConfirmation() { - boolean active = dropPublishConfirmation; - dropPublishConfirmation = false; - return active; - } - - boolean consumeDropSettlementConfirmation() { - boolean active = dropSettlementConfirmation; - dropSettlementConfirmation = false; - return active; - } - - boolean consumeRejectPublish() { - boolean active = rejectPublish; - rejectPublish = false; - return active; - } - - boolean deadLetterPublishFails() { - return failDeadLetterPublish; - } - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitEnvelopeRoundTripTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitEnvelopeRoundTripTest.java deleted file mode 100644 index 4a6f105..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitEnvelopeRoundTripTest.java +++ /dev/null @@ -1,152 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.CorrelationId; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.api.header.HeaderName; -import dev.caskeleton.messaging.api.header.HeaderValue; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.schema.EncodedMessage; -import java.nio.charset.StandardCharsets; -import java.time.Instant; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Optional; -import org.junit.jupiter.api.DisplayName; -import org.junit.jupiter.api.Test; -import org.springframework.amqp.core.Message; -import org.springframework.amqp.core.MessageProperties; - -/** - * What an envelope still holds after it has been through RabbitMQ. - * - *

The publish and delivery mappers were written against each other's headers but never checked - * as one trip, so the application's own headers were dropped on the way back and the retry - * bookkeeping went with them — an attempt counter that restarts on every hop caps no redeliveries - * at all. - */ -class RabbitEnvelopeRoundTripTest { - - private static final Instant NOW = Instant.parse("2026-08-15T10:00:00Z"); - - private final RabbitHeaderMapper headerMapper = new RabbitHeaderMapper(); - - private final RabbitDeliveryMapper deliveryMapper = new RabbitDeliveryMapper(headerMapper); - - @Test - @DisplayName("an application header survives the trip through the broker") - void anApplicationHeaderSurvivesTheTrip() { - MessageEnvelope published = - envelope( - MessageHeaders.application( - Map.of(new HeaderName("x-source-system"), new HeaderValue("erp")))); - - MessageEnvelope delivered = roundTrip(published); - - assertThat(delivered.headers().find("x-source-system")) - .map(HeaderValue::value) - .as("a header the producer wrote and the consumer cannot read is a header nobody has") - .hasValue("erp"); - } - - @Test - @DisplayName("the tenant survives the trip instead of arriving as no tenant at all") - void theTenantSurvivesTheTrip() { - MessageEnvelope published = - envelope(MessageHeaders.empty()) - .withTenant(new dev.caskeleton.messaging.api.TenantContext("acme")); - - MessageEnvelope delivered = roundTrip(published); - - assertThat(delivered.tenantContext()) - .as("an AMQP message carries no ambient tenant, so an unwritten tenant is an absent one") - .contains(new dev.caskeleton.messaging.api.TenantContext("acme")); - } - - @Test - @DisplayName("the retry attempt survives the trip instead of restarting at one") - void theRetryAttemptSurvivesTheTrip() { - MessageEnvelope delivered = roundTrip(retryStamped(3)); - - assertThat(delivered.headers().find(ReservedHeaders.RETRY_ATTEMPT)) - .map(HeaderValue::value) - .as("an attempt counter that resets on every hop caps nothing") - .hasValue("3"); - } - - @Test - @DisplayName("a retry-stamped envelope can be published to its retry queue") - void aRetryStampedEnvelopeCanBePublished() { - assertThatCode(() -> headerMapper.toProperties(retryStamped(2))) - .as("the platform's own bookkeeping is not message content forging an identity") - .doesNotThrowAnyException(); - } - - @Test - @DisplayName("a header restating the message id is still refused") - void aForgedMessageIdIsStillRefused() { - MessageEnvelope forged = - envelope( - MessageHeaders.platform( - Map.of(new HeaderName(ReservedHeaders.MESSAGE_ID), new HeaderValue("forged")))); - - assertThatThrownBy(() -> headerMapper.toProperties(forged)) - .isInstanceOf(MessagingConfigurationException.class); - } - - @Test - @DisplayName("identity is read from its field, not duplicated into the header map") - void identityIsNotDuplicatedIntoTheHeaderMap() { - MessageEnvelope delivered = roundTrip(envelope(MessageHeaders.empty())); - - assertThat(delivered.headers().find(ReservedHeaders.MESSAGE_ID)) - .as("a copy that can drift from the field gives the same message two identities") - .isEmpty(); - } - - private MessageEnvelope roundTrip(MessageEnvelope published) { - MessageProperties properties = headerMapper.toProperties(published); - return deliveryMapper.toEnvelope(new Message(published.payload().bytes(), properties)); - } - - private static MessageEnvelope retryStamped(int attempt) { - Map headers = new LinkedHashMap<>(); - headers.put( - new HeaderName(ReservedHeaders.RETRY_ATTEMPT), new HeaderValue(Integer.toString(attempt))); - headers.put(new HeaderName(ReservedHeaders.LAST_FAILURE_AT), new HeaderValue(NOW.toString())); - return envelope(MessageHeaders.platform(headers)); - } - - private static MessageEnvelope envelope(MessageHeaders headers) { - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.of(new CorrelationId("wf-1")), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - headers, - new EncodedMessage( - "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), - ContentType.JSON, - Optional.empty())); - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidatorTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidatorTest.java deleted file mode 100644 index ba0f14f..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidatorTest.java +++ /dev/null @@ -1,68 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import org.junit.jupiter.api.Test; - -class RabbitProfileValidatorTest { - - private final RabbitProfileValidator validator = new RabbitProfileValidator(); - - @Test - void stableProducerRequiresConfirms() { - assertThatThrownBy(() -> validator.validate(RabbitProfileFixtures.withoutConfirms())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("publisher confirms"); - } - - @Test - void stableProducerRequiresReturnsBecauseAConfirmDoesNotProveRouting() { - assertThatThrownBy(() -> validator.validate(RabbitProfileFixtures.withoutReturns())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("publisher returns"); - } - - @Test - void stableProducerRequiresMandatoryRouting() { - assertThatThrownBy(() -> validator.validate(RabbitProfileFixtures.withoutMandatory())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("mandatory"); - } - - @Test - void consumerAutoAckIsForbidden() { - assertThatThrownBy(() -> validator.validate(RabbitProfileFixtures.withAutoAck())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("auto ack"); - } - - @Test - void productionRequiresTlsAndAuthentication() { - assertThatThrownBy(() -> validator.validate(RabbitProfileFixtures.productionWithoutTls())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("TLS"); - } - - @Test - void aDurableWorkQueueMustBeAQuorumQueue() { - assertThatThrownBy( - () -> - validator.validateDestination( - RabbitProfileFixtures.workQueueDestination(), - RabbitProfileFixtures.withoutQuorumQueues())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("quorum"); - } - - @Test - void aStableProfileAndDestinationValidate() { - assertThatCode( - () -> { - validator.validate(RabbitProfileFixtures.stable()); - validator.validateDestination( - RabbitProfileFixtures.workQueueDestination(), RabbitProfileFixtures.stable()); - }) - .doesNotThrowAnyException(); - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitRuntimeTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitRuntimeTest.java deleted file mode 100644 index 903beae..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitRuntimeTest.java +++ /dev/null @@ -1,341 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.CorrelationId; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.schema.EncodedMessage; -import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import dev.caskeleton.messaging.transport.TransportDelivery; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import dev.caskeleton.messaging.transport.TransportPublishResult; -import java.nio.charset.StandardCharsets; -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicLong; -import org.junit.jupiter.api.Test; -import org.springframework.amqp.core.Message; - -class RabbitRuntimeTest { - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - - private final AtomicLong sequence = new AtomicLong(); - private final List published = new ArrayList<>(); - private final List handled = new ArrayList<>(); - - private RabbitMessagingTransport transport() { - return new RabbitMessagingTransport( - "rabbit-primary", - 1, - new RabbitChannelPublisher() { - @Override - public long nextPublishSequence() { - return sequence.incrementAndGet(); - } - - @Override - public void publish( - long sequenceNumber, - String exchange, - String routingKey, - Message message, - boolean mandatory) { - published.add(message); - } - }); - } - - @Test - void aConfirmedPublishReportsRoutedWithReplicationEvidence() { - RabbitMessagingTransport transport = transport(); - - CompletionStage stage = transport.publish(request()); - transport.onConfirm(1L, true, Duration.ofMillis(3)); - - var result = stage.toCompletableFuture().join().result(); - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.ROUTED); - } - - @Test - void aReturnedThenConfirmedPublishIsRejectedAsUnroutable() { - RabbitMessagingTransport transport = transport(); - - CompletionStage stage = transport.publish(request()); - transport.onReturn(1L); - transport.onConfirm(1L, true, Duration.ofMillis(3)); - - var result = stage.toCompletableFuture().join().result(); - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.UNROUTABLE); - } - - @Test - void aMissingConfirmExpiresWithoutAnExternalTimeoutCallback() throws Exception { - try (RabbitMessagingTransport timed = - new RabbitMessagingTransport( - "rabbit-primary", - 1, - new RabbitChannelPublisher() { - @Override - public long nextPublishSequence() { - return sequence.incrementAndGet(); - } - - @Override - public void publish( - long sequenceNumber, - String exchange, - String routingKey, - Message message, - boolean mandatory) { - published.add(message); - } - }, - Duration.ofMillis(20))) { - CompletionStage stage = timed.publish(request()); - - var result = - stage.toCompletableFuture().get(1, java.util.concurrent.TimeUnit.SECONDS).result(); - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(timed.pendingConfirms()).isZero(); - } - } - - @Test - void aMissingConfirmIsAmbiguous() { - RabbitMessagingTransport transport = transport(); - - CompletionStage stage = transport.publish(request()); - transport.onConfirmTimeout(1L, Duration.ofSeconds(5)); - - assertThat(stage.toCompletableFuture().join().result().mayHaveBeenStored()).isTrue(); - } - - @Test - void anOversizedPayloadNeverReachesTheChannel() { - RabbitMessagingTransport transport = transport(); - - var result = - transport.publish(request(new byte[1_048_577])).toCompletableFuture().join().result(); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(published).isEmpty(); - } - - @Test - void aClosedTransportRefusesPublishesLocally() { - RabbitMessagingTransport transport = transport(); - transport.close(); - - var result = transport.publish(request()).toCompletableFuture().join().result(); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(transport.isAcceptingWork()).isFalse(); - assertThat(published).isEmpty(); - } - - @Test - void theHeaderRoundTripPreservesIdentityAndProvenance() { - RabbitHeaderMapper headers = new RabbitHeaderMapper(); - MessageEnvelope envelope = envelope(new byte[] {1, 2, 3}); - - var properties = headers.toProperties(envelope); - - assertThat(headers.value(properties, ReservedHeaders.MESSAGE_ID)) - .isEqualTo(envelope.messageId().value().toString()); - assertThat(headers.value(properties, ReservedHeaders.MESSAGE_TYPE)).isEqualTo("order.created"); - assertThat(headers.value(properties, ReservedHeaders.CORRELATION_ID)).isEqualTo("wf-1"); - assertThat(properties.getCorrelationId()).isEqualTo("wf-1"); - } - - @Test - void aDeliveryIsRebuiltFromItsHeaders() { - RabbitHeaderMapper headers = new RabbitHeaderMapper(); - MessageEnvelope original = envelope(new byte[] {1, 2, 3}); - Message message = new Message(new byte[] {1, 2, 3}, headers.toProperties(original)); - - var restored = new RabbitDeliveryMapper().toEnvelope(message); - - assertThat(restored.messageId()).isEqualTo(original.messageId()); - assertThat(restored.messageType()).isEqualTo(original.messageType()); - assertThat(restored.schemaVersion()).isEqualTo(original.schemaVersion()); - assertThat(restored.payload().bytes()).containsExactly(1, 2, 3); - } - - @Test - void theConsumerDispatchesAndSettlesThroughTheController() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = registrar(operations, this::acknowledge); - - boolean accepted = registrar.onMessage(deliveredMessage(7L), NOW); - - assertThat(accepted).isTrue(); - assertThat(handled).hasSize(1); - assertThat(operations.calls).containsExactly("ack:7"); - } - - @Test - void aPausedQueueRefusesDeliveriesSoTheBrokerRedeliversThem() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = registrar(operations, this::acknowledge); - registrar.pause("").toCompletableFuture().join(); - - boolean accepted = registrar.onMessage(deliveredMessage(7L), NOW); - - assertThat(accepted).isFalse(); - assertThat(handled).isEmpty(); - assertThat(operations.calls).isEmpty(); - } - - @Test - void drainingRefusesNewDeliveries() { - RecordingOperations operations = new RecordingOperations(); - GracefulShutdownCoordinator shutdown = new GracefulShutdownCoordinator(Duration.ofSeconds(30)); - try (RabbitConsumerRegistrar registrar = - new RabbitConsumerRegistrar( - new TransportConsumerSpec( - RabbitProfileFixtures.workQueueDestination(), this::acknowledge), - operations, - shutdown)) { - - shutdown.beginDrain(NOW); - boolean accepted = registrar.onMessage(deliveredMessage(7L), NOW); - - assertThat(accepted).isFalse(); - assertThat(operations.calls).isEmpty(); - } - } - - @Test - void anUndecodableMessageIsDiscardedRatherThanRequeued() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = registrar(operations, this::acknowledge); - - Message malformed = - new Message(new byte[] {1}, new org.springframework.amqp.core.MessageProperties()); - malformed.getMessageProperties().setDeliveryTag(9L); - - boolean accepted = registrar.onMessage(malformed, NOW); - - assertThat(accepted).isTrue(); - assertThat(handled).isEmpty(); - assertThat(operations.calls).containsExactly("discard:9"); - } - - private RabbitConsumerRegistrar registrar( - RabbitSettlementOperations operations, - java.util.function.Function> sink) { - return new RabbitConsumerRegistrar( - new TransportConsumerSpec(RabbitProfileFixtures.workQueueDestination(), sink), - operations, - new GracefulShutdownCoordinator(Duration.ofSeconds(30))); - } - - private CompletionStage acknowledge(TransportDelivery delivery) { - handled.add(delivery); - delivery.settlement().acknowledge().toCompletableFuture().join(); - return CompletableFuture.completedFuture(null); - } - - private Message deliveredMessage(long deliveryTag) { - Message message = - new Message( - "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), - new RabbitHeaderMapper().toProperties(envelope(new byte[] {1}))); - message.getMessageProperties().setDeliveryTag(deliveryTag); - message.getMessageProperties().setConsumerQueue("notification.email.q"); - return message; - } - - private TransportPublishRequest request() { - return request("{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8)); - } - - private TransportPublishRequest request(byte[] payload) { - return new TransportPublishRequest( - RabbitProfileFixtures.workQueueDestination(), envelope(payload), PublishOptions.defaults()); - } - - private MessageEnvelope envelope(byte[] payload) { - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.of(new CorrelationId("wf-1")), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - new EncodedMessage(payload, ContentType.JSON, Optional.empty())); - } - - @Test - @org.junit.jupiter.api.DisplayName("a handler failure is retried, not discarded as undecodable") - void aHandlerFailureIsRetriedNotDiscarded() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = - registrar( - operations, - delivery -> { - throw new IllegalStateException("the projection is down"); - }); - - registrar.onMessage(deliveredMessage(11), NOW); - - assertThat(operations.calls) - .as("one catch covered decode, handler and settlement, so a retryable failure was deleted") - .containsExactly("retry:11:PT0S"); - } - - @Test - @org.junit.jupiter.api.DisplayName("a handler that never settles does not acknowledge") - void aHandlerThatNeverSettlesDoesNotAcknowledge() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = - registrar(operations, delivery -> CompletableFuture.completedFuture(null)); - - registrar.onMessage(deliveredMessage(12), NOW); - - assertThat(operations.calls) - .as("acknowledging on the handler's behalf silently drops a message nobody handled") - .containsExactly("retry:12:PT0S"); - } - - @Test - @org.junit.jupiter.api.DisplayName("a delivery after close is refused") - void aDeliveryAfterCloseIsRefused() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = - registrar(operations, delivery -> CompletableFuture.completedFuture(null)); - registrar.close(); - - assertThat(registrar.onMessage(deliveredMessage(13), NOW)) - .as("close only flipped a flag that onMessage never read") - .isFalse(); - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitSettlementControllerTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitSettlementControllerTest.java deleted file mode 100644 index f0cba51..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitSettlementControllerTest.java +++ /dev/null @@ -1,105 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; - -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.settlement.SettlementResult; -import java.time.Duration; -import java.util.ArrayList; -import java.util.List; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import org.junit.jupiter.api.Test; - -class RabbitSettlementControllerTest { - - private static final FailureDescriptor FAILURE = - FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, "REJECTED", "rejected"); - - @Test - void acknowledgesExactlyOnce() { - RecordingOperations operations = new RecordingOperations(); - RabbitSettlementController controller = new RabbitSettlementController(7L, operations); - - controller.ack().toCompletableFuture().join(); - - assertThat(operations.calls).containsExactly("ack:7"); - assertThat(controller.isSettled()).isTrue(); - } - - @Test - void aSecondTerminalCallFailsRatherThanRacingTheFirst() { - RecordingOperations operations = new RecordingOperations(); - RabbitSettlementController controller = new RabbitSettlementController(7L, operations); - - controller.ack().toCompletableFuture().join(); - - assertThat(controller.ack().toCompletableFuture()).isCompletedExceptionally(); - assertThat(operations.calls).containsExactly("ack:7"); - } - - @Test - void mixingTerminalOperationsIsAlsoRefused() { - RecordingOperations operations = new RecordingOperations(); - RabbitSettlementController controller = new RabbitSettlementController(7L, operations); - - controller.deadLetter(FAILURE).toCompletableFuture().join(); - - assertThat(controller.ack().toCompletableFuture()).isCompletedExceptionally(); - assertThat(operations.calls).containsExactly("deadLetter:7"); - } - - @Test - void retryGoesToTheDelayedQueueRatherThanRequeueingImmediately() { - RecordingOperations operations = new RecordingOperations(); - RabbitSettlementController controller = new RabbitSettlementController(7L, operations); - - controller.retry(Duration.ofSeconds(30)).toCompletableFuture().join(); - - assertThat(operations.calls).containsExactly("retry:7:PT30S"); - } - - @Test - void aRetryQueueCarriesTtlAndADeadLetterRouteBackToTheWorkQueue() { - RabbitRetryQueueTopology topology = - new RabbitRetryQueueTopology( - "notification.email.retry.q", "notification.work", "email", Duration.ofSeconds(30)); - - assertThat(topology.queueArguments()) - .containsEntry("x-queue-type", "quorum") - .containsEntry("x-message-ttl", 30_000L) - .containsEntry("x-dead-letter-exchange", "notification.work") - .containsEntry("x-dead-letter-routing-key", "email"); - } -} - -/** Records which channel operation the controller invoked. */ -final class RecordingOperations implements RabbitSettlementOperations { - - final List calls = new ArrayList<>(); - - @Override - public CompletionStage ack(long deliveryTag) { - calls.add("ack:" + deliveryTag); - return CompletableFuture.completedFuture(SettlementResult.settled()); - } - - @Override - public CompletionStage publishToRetryQueue(long deliveryTag, Duration delay) { - calls.add("retry:" + deliveryTag + ":" + delay); - return CompletableFuture.completedFuture(SettlementResult.settled()); - } - - @Override - public CompletionStage deadLetter(long deliveryTag, FailureDescriptor failure) { - calls.add("deadLetter:" + deliveryTag); - return CompletableFuture.completedFuture(SettlementResult.settled()); - } - - @Override - public CompletionStage discard(long deliveryTag, FailureDescriptor failure) { - calls.add("discard:" + deliveryTag); - return CompletableFuture.completedFuture(SettlementResult.settled()); - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitTopologyAndBatchTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitTopologyAndBatchTest.java deleted file mode 100644 index 040513e..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitTopologyAndBatchTest.java +++ /dev/null @@ -1,253 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.io.IOException; -import java.time.Duration; -import java.time.Instant; -import java.util.Optional; -import org.junit.jupiter.api.Test; - -class RabbitTopologyAndBatchTest { - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - - private static RabbitTopologyProfile quorumWithDlx() { - return RabbitTopologyProfile.workQueue( - "notification.work", "email", "notification.email.q", "notification.dlx"); - } - - private static RabbitTopologyProfile classic() { - return new RabbitTopologyProfile( - "notification.work", - "direct", - "email", - "notification.email.q", - false, - Optional.empty(), - Optional.empty(), - Optional.empty()); - } - - @Test - void aDurableDestinationRequiresAQuorumQueue() { - assertThatThrownBy(() -> classic().requireSafeFor(true)) - .as("a promoted stale mirror loses messages the broker already confirmed") - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("quorum"); - } - - @Test - void aQuorumQueueSatisfiesADurableDestination() { - assertThatCode(() -> quorumWithDlx().requireSafeFor(true)).doesNotThrowAnyException(); - } - - @Test - void theQueueArgumentsCarryTheDeclaredGuarantees() { - assertThat(quorumWithDlx().queueArguments()) - .containsEntry(RabbitTopologyProfile.QUEUE_TYPE, "quorum") - .containsEntry(RabbitTopologyProfile.DEAD_LETTER_EXCHANGE, "notification.dlx"); - } - - @Test - void aDeadLetterRoutingKeyWithoutAnExchangeIsRejected() { - assertThatThrownBy( - () -> - new RabbitTopologyProfile( - "x", - "direct", - "k", - "q", - true, - Optional.empty(), - Optional.of("k"), - Optional.empty())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("routes nowhere"); - } - - @Test - void nativeDeadLetterNeedsBothADeclarationAndAVerifiedBinding() { - assertThat(new RabbitNativeDeadLetterCapability(quorumWithDlx(), true).mayUseNativeDeadLetter()) - .isTrue(); - assertThat( - new RabbitNativeDeadLetterCapability(quorumWithDlx(), false).mayUseNativeDeadLetter()) - .as("an unbound dead letter exchange discards the message while the reject still succeeds") - .isFalse(); - } - - @Test - void aQueueWithoutADeadLetterExchangeFallsBackToThePlatformPublish() { - RabbitNativeDeadLetterCapability capability = - new RabbitNativeDeadLetterCapability(classic(), true); - - assertThat(capability.requiresPlatformPublish()).isTrue(); - assertThat(capability.reasonForPlatformPublish()).contains("without a dead letter exchange"); - } - - @Test - void theUnboundReasonExplainsTheSilentDiscard() { - assertThat( - new RabbitNativeDeadLetterCapability(quorumWithDlx(), false).reasonForPlatformPublish()) - .contains("silently") - .contains("reporting success"); - } - - @Test - void aFailureBeforeTheFrameWasWrittenIsADefinitiveRejection() { - PublishResult result = - new RabbitPublishFailureClassifier() - .classify(new IllegalStateException("channel closed"), false, Duration.ofMillis(3)); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.NOT_TRANSMITTED); - } - - @Test - void aFailureAfterTheFrameWasWrittenIsAmbiguous() { - PublishResult result = - new RabbitPublishFailureClassifier() - .classify(new IOException("connection reset"), true, Duration.ofMillis(3)); - - assertThat(result.completion()) - .as("the broker may hold the message and only the confirm was lost") - .isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.evidence().transmission()) - .isEqualTo(TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED); - } - - @Test - void authenticationFailureUsesTheTypedRabbitSignal() { - PublishResult result = - new RabbitPublishFailureClassifier() - .classify( - new com.rabbitmq.client.AuthenticationFailureException("bad credentials"), - false, - Duration.ofMillis(3)); - - assertThat(result.failure()) - .hasValueSatisfying( - failure -> - assertThat(failure.category()) - .isEqualTo(dev.caskeleton.messaging.api.error.FailureCategory.AUTHENTICATION)); - } - - @Test - void accessRefusedReplyCodeIsAuthorizationRatherThanBusinessRejection() { - com.rabbitmq.client.ShutdownSignalException accessRefused = - new com.rabbitmq.client.ShutdownSignalException( - false, - false, - new com.rabbitmq.client.impl.AMQImpl.Channel.Close(403, "ACCESS_REFUSED", 60, 40), - null); - - PublishResult result = - new RabbitPublishFailureClassifier().classify(accessRefused, false, Duration.ofMillis(3)); - - assertThat(result.failure()) - .hasValueSatisfying( - failure -> - assertThat(failure.category()) - .isEqualTo(dev.caskeleton.messaging.api.error.FailureCategory.AUTHORIZATION)); - } - - @Test - void aNackIsRejectedButStillCountsAsTransmitted() { - PublishResult result = new RabbitPublishFailureClassifier().nacked(Duration.ofMillis(3)); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.TRANSMITTED); - } - - @Test - void aBatchIsReleasedOnceItIsFull() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 3, Duration.ofSeconds(5)); - - assertThat(registrar.add(1, NOW)).isEmpty(); - assertThat(registrar.add(2, NOW)).isEmpty(); - assertThat(registrar.add(3, NOW)) - .hasValueSatisfying(batch -> assertThat(batch.deliveryTags()).containsExactly(1L, 2L, 3L)); - } - - @Test - void aPartialBatchIsReleasedOnceItsOldestDeliveryGoesStale() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 100, Duration.ofSeconds(5)); - registrar.add(1, NOW); - - assertThat(registrar.releaseIfStale(NOW.plusSeconds(1))).isEmpty(); - assertThat(registrar.releaseIfStale(NOW.plusSeconds(6))) - .as("a quiet queue would otherwise hold its last messages unacknowledged indefinitely") - .isPresent(); - } - - @Test - void anAccumulatedBatchIsNotSettlableAsAUnit() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 2, Duration.ofSeconds(5)); - registrar.add(1, NOW); - - assertThat(registrar.add(2, NOW)) - .hasValueSatisfying( - batch -> - assertThat(batch.metadata().settlableAsBatch()) - .as("AMQP multiple-ack settles work that is still in flight") - .isFalse()); - } - - @Test - void anAccumulatedBatchNamesExactlyOneOrderingUnit() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 1, Duration.ofSeconds(5)); - - assertThat(registrar.add(1, NOW)) - .hasValueSatisfying( - batch -> assertThat(batch.metadata().isSafeForOrderedDestination()).isTrue()); - } - - @Test - void aPrefetchBelowTheBatchSizeWouldDeadlockAndIsRefused() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 50, Duration.ofSeconds(5)); - - assertThatThrownBy(() -> registrar.requirePrefetchFor(10)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("prefetch"); - assertThatCode(() -> registrar.requirePrefetchFor(100)).doesNotThrowAnyException(); - } - - @Test - void aZeroBatchAgeIsRefusedBecauseItStrandsQuietQueues() { - assertThatThrownBy( - () -> - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 10, Duration.ZERO)) - .isInstanceOf(IllegalArgumentException.class); - } - - @Test - void drainingReleasesWhateverIsPendingForShutdown() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 100, Duration.ofSeconds(5)); - registrar.add(1, NOW); - registrar.add(2, NOW); - - assertThat(registrar.drain(NOW)) - .hasValueSatisfying(batch -> assertThat(batch.deliveryTags()).hasSize(2)); - assertThat(registrar.pending()).isZero(); - assertThat(registrar.drain(NOW)).isEmpty(); - } -} diff --git a/messaging/messaging-rabbit/src/testFixtures/java/dev/caskeleton/messaging/rabbit/RabbitProfileFixtures.java b/messaging/messaging-rabbit/src/testFixtures/java/dev/caskeleton/messaging/rabbit/RabbitProfileFixtures.java deleted file mode 100644 index e505cd2..0000000 --- a/messaging/messaging-rabbit/src/testFixtures/java/dev/caskeleton/messaging/rabbit/RabbitProfileFixtures.java +++ /dev/null @@ -1,118 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; -import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.policy.CapabilityTier; -import dev.caskeleton.messaging.policy.ConsumerPolicy; -import dev.caskeleton.messaging.policy.DeadLetterPolicy; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.PayloadPolicy; -import dev.caskeleton.messaging.policy.PhysicalDestination; -import dev.caskeleton.messaging.policy.ProducerPolicy; -import dev.caskeleton.messaging.policy.RetryPolicy; -import dev.caskeleton.messaging.policy.SchemaPolicy; -import dev.caskeleton.messaging.schema.SchemaCompatibility; -import java.time.Duration; -import java.util.List; -import java.util.Set; - -/** Builds RabbitMQ profiles for validator tests. */ -final class RabbitProfileFixtures { - - private RabbitProfileFixtures() {} - - static RabbitBrokerProfile stable() { - return new RabbitBrokerProfile( - "rabbit-primary", - true, - false, - List.of("localhost:5672"), - true, - true, - true, - Duration.ofSeconds(5), - false, - 16, - true, - true, - true); - } - - static RabbitBrokerProfile withoutConfirms() { - return copy(stable(), false, true, true, false, true, false, true); - } - - static RabbitBrokerProfile withoutReturns() { - return copy(stable(), true, false, true, false, true, false, true); - } - - static RabbitBrokerProfile withoutMandatory() { - return copy(stable(), true, true, false, false, true, false, true); - } - - static RabbitBrokerProfile withAutoAck() { - return copy(stable(), true, true, true, true, true, false, true); - } - - static RabbitBrokerProfile withoutQuorumQueues() { - return copy(stable(), true, true, true, false, false, false, true); - } - - static RabbitBrokerProfile productionWithoutTls() { - return copy(stable(), true, true, true, false, true, true, false); - } - - private static RabbitBrokerProfile copy( - RabbitBrokerProfile base, - boolean confirms, - boolean returns, - boolean mandatory, - boolean autoAck, - boolean quorumQueues, - boolean production, - boolean tlsEnabled) { - return new RabbitBrokerProfile( - base.broker(), - base.stable(), - production, - base.addresses(), - confirms, - returns, - mandatory, - base.confirmTimeout(), - autoAck, - base.prefetch(), - quorumQueues, - tlsEnabled, - base.authenticationEnabled()); - } - - static DestinationProfile workQueueDestination() { - return new DestinationProfile( - new DestinationName("email-work"), - "rabbit-primary", - DestinationKind.WORK_QUEUE, - PhysicalDestination.rabbitQueue("notification.work", "email", "notification.email.q"), - new SchemaPolicy( - ContentType.JSON, - SchemaCompatibility.BACKWARD, - Set.of(new MessageType("email.requested"))), - DeliveryGuarantee.AT_LEAST_ONCE, - OrderingScope.NONE, - ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, - ProducerPolicy.defaults(), - ConsumerPolicy.defaults("notification"), - RetryPolicy.none(), - DeadLetterPolicy.to(new DestinationName("email-work-dlq")), - PayloadPolicy.defaults(), - CapabilityTier.M1, - false, - false, - false); - } -} diff --git a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java index 4942565..fe59a3e 100644 --- a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java +++ b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java @@ -301,6 +301,14 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM } private MessageEnvelope encode(MessageEnvelope message) { + // The outbox relay, dead lettering and the retry topic republish bytes a codec already wrote. + // Encoding them again handed the codec a payload that is not its type, so each such publish was + // REJECTED (or, with a lenient codec, sent the envelope's toString). They go out as they are. + if (message.payload() instanceof EncodedMessage alreadyEncoded) { + MessageEnvelope encoded = message.withPayload(alreadyEncoded); + requireEncodedEnvelope(encoded); + return encoded; + } MessageCodec codec = codecs.find(message.contentType()).orElseGet(codecs::defaultCodec); EncodedMessage bytes = codec.encode(message.messageType(), message.schemaVersion(), message.payload()); diff --git a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java index 510c451..5302ed6 100644 --- a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java +++ b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java @@ -10,9 +10,14 @@ import dev.caskeleton.messaging.api.MessageType; import dev.caskeleton.messaging.api.ProducerId; import dev.caskeleton.messaging.api.SchemaVersion; import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.delivery.DeliveryContext; +import dev.caskeleton.messaging.api.delivery.DeliveryMetadata; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; import dev.caskeleton.messaging.api.destination.DestinationCapabilities; import dev.caskeleton.messaging.api.destination.DestinationName; import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.error.FailureCategory; +import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.error.MessagingConfigurationException; import dev.caskeleton.messaging.api.header.MessageHeaders; import dev.caskeleton.messaging.api.publish.ConfirmationLevel; @@ -21,6 +26,7 @@ import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; import dev.caskeleton.messaging.api.publish.RoutingOutcome; +import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; import dev.caskeleton.messaging.policy.InFlightLimiter; import dev.caskeleton.messaging.policy.MessagingAdmissionController; import dev.caskeleton.messaging.policy.PayloadLimitGuard; @@ -120,6 +126,55 @@ class DefaultMessagePublisherTest { assertThat(admission.inFlight()).isZero(); } + @Test + void aDeadLetterPublishedThroughThisPublisherReachesTheTransportWithTheOriginalBytes() { + // given + DestinationName deadLetters = new DestinationName("order-events-dlq"); + DefaultMessagePublisher publisher = + new DefaultMessagePublisher( + DestinationProfileRegistry.of( + Map.of( + ORDERS, + RuntimeFixtures.profile(ORDERS), + deadLetters, + RuntimeFixtures.profile(deadLetters))), + new DestinationAccessPolicy( + java.util.Set.of(deadLetters), java.util.Set.of(), java.util.Set.of()), + new SingleCodecRegistry(), + admission, + new FixedRuntimeRegistry(transport), + transport); + byte[] wireBytes = "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8); + MessageDelivery failed = + new MessageDelivery<>( + envelope() + .withPayload(new EncodedMessage(wireBytes, ContentType.JSON, Optional.empty())), + new DeliveryMetadata( + ORDERS, + 3, + true, + Optional.empty(), + Optional.empty(), + Optional.of("order-projection"), + Instant.parse("2026-08-14T00:00:05Z")), + new DeliveryContext(Instant.parse("2026-08-14T00:00:35Z"), false, "order-projection")); + + // when + new DeadLetterOrchestrator(publisher) + .deadLetter( + RuntimeFixtures.profile(ORDERS), + failed, + FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, "ORDER_REJECTED", "rejected"), + () -> CompletableFuture.completedFuture(null)) + .toCompletableFuture() + .join(); + + // then + assertThat(transport.lastRequest().envelope().payload().bytes()) + .as("encoding an already-encoded payload again rewrites the bytes or fails the codec") + .containsExactly(wireBytes); + } + @Test @DisplayName("an unregistered destination is rejected before anything is encoded") void anUnregisteredDestinationIsRejected() { diff --git a/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile b/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile index 53df71c..ffc4211 100644 --- a/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile +++ b/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile @@ -34,8 +34,6 @@ commons-collections:commons-collections:3.2.2=checkstyle commons-io:commons-io:2.21.0=spotbugs commons-logging:commons-logging:1.3.6=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath info.picocli:picocli:4.7.7=checkstyle -io.cloudevents:cloudevents-api:4.0.1=runtimeClasspath,testRuntimeClasspath -io.cloudevents:cloudevents-core:4.0.1=runtimeClasspath,testRuntimeClasspath io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java index cc2b247..bec88a6 100644 --- a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java @@ -4,6 +4,7 @@ import dev.caskeleton.messaging.kafka.KafkaProfileValidator; import dev.caskeleton.messaging.kafka.KafkaPublishFailureClassifier; import dev.caskeleton.messaging.kafka.KafkaSecurityConfigurer; import dev.caskeleton.messaging.kafka.KafkaTransactionProfileValidator; +import dev.caskeleton.messaging.security.BrokerSecurityProfile; import dev.caskeleton.messaging.security.BrokerTlsPolicy; import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; import org.springframework.beans.factory.ObjectProvider; @@ -24,13 +25,19 @@ import org.springframework.context.annotation.Configuration; * localhost:9092} — which in a test environment usually succeeds against something. * *

Reached only when app.messaging.broker=kafka selected it. The @ConditionalOnClass this used to - * carry could not select: the starter depends on messaging-kafka and messaging-rabbit, each of - * which exposes its client library as `api`, so both conditions were true for every adopter and - * both providers assembled. + * carry could not select: the starter then depended on two broker adapters, each of which exposed + * its client library as `api`, so both conditions were true for every adopter and both providers + * assembled. */ @Configuration(proxyBeanMethods = false) public class KafkaMessagingAutoConfiguration { + /** + * The two versions {@code BrokerTlsPolicy} accepts. No setting names them, so both are offered. + */ + private static final java.util.List TLS_PROTOCOLS = + java.util.List.of("TLSv1.3", "TLSv1.2"); + /** * Returns the Kafka destination profile validator. * @@ -123,12 +130,28 @@ public class KafkaMessagingAutoConfiguration { * and a serializer configured here would be a second encoder disagreeing with it. * * @param environment the resolved environment + * @param compiled the broker and security profiles compiled from {@code app.messaging} + * @param declaredSecurity the security profiles the application declared as beans + * @param securityConfigurer the Kafka security configurer, present only with a credential source * @return the producer, closed with the context */ @Bean(destroyMethod = "close") @ConditionalOnMissingBean public org.apache.kafka.clients.producer.Producer messagingKafkaProducer( - org.springframework.core.env.Environment environment) { + org.springframework.core.env.Environment environment, + CompiledMessagingConfiguration compiled, + ObjectProvider declaredSecurity, + ObjectProvider securityConfigurer) { + return new org.apache.kafka.clients.producer.KafkaProducer<>( + producerConfig(environment, compiled, declaredSecurity, securityConfigurer)); + } + + /** The producer's client configuration; package-visible so a test can read what it holds. */ + static java.util.Map producerConfig( + org.springframework.core.env.Environment environment, + CompiledMessagingConfiguration compiled, + ObjectProvider declaredSecurity, + ObjectProvider securityConfigurer) { String bootstrapServers = environment.getProperty("spring.kafka.bootstrap-servers", "").trim(); if (bootstrapServers.isEmpty()) { throw new IllegalStateException( @@ -150,7 +173,66 @@ public class KafkaMessagingAutoConfiguration { // failover, which is exactly the outcome an outbox exists to prevent. config.put(org.apache.kafka.clients.producer.ProducerConfig.ACKS_CONFIG, "all"); config.put(org.apache.kafka.clients.producer.ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true); - return new org.apache.kafka.clients.producer.KafkaProducer<>(config); + config.putAll(producerSecurity(compiled, declaredSecurity, securityConfigurer)); + return config; + } + + /** + * The producer's security properties, from the one security profile of the Kafka broker. + * + *

{@code KafkaSecurityConfigurer} was a bean nothing called, so a broker that passed the TLS + * and credential checks was reached over plaintext with no authentication. Every case that cannot + * be applied fails startup rather than falling back to that. + */ + private static java.util.Map producerSecurity( + CompiledMessagingConfiguration compiled, + ObjectProvider declaredSecurity, + ObjectProvider securityConfigurer) { + java.util.Map kafkaBrokers = new java.util.TreeMap<>(); + compiled.brokers().entrySet().stream() + .filter(entry -> BrokerSettings.KAFKA.equalsIgnoreCase(entry.getValue().type())) + .forEach(entry -> kafkaBrokers.put(entry.getKey(), entry.getValue())); + // The same fallback as the runtime's logical name: bean-declared profiles name the family. + java.util.Set names = + kafkaBrokers.isEmpty() ? java.util.Set.of(BrokerSettings.KAFKA) : kafkaBrokers.keySet(); + java.util.List profiles = + java.util.stream.Stream.concat( + compiled.security().stream(), declaredSecurity.orderedStream()) + .filter(profile -> names.contains(profile.broker())) + .toList(); + + if (profiles.isEmpty()) { + kafkaBrokers.forEach( + (name, broker) -> { + if (broker.tlsEnabled() || broker.authenticationEnabled()) { + throw new IllegalStateException( + "Kafka broker '" + + name + + "' enables TLS or authentication but has no app.messaging.security." + + name + + " entry; the producer cannot apply either without one"); + } + }); + return java.util.Map.of(); + } + if (profiles.size() > 1) { + throw new IllegalStateException( + "the Kafka producer has one security posture but " + + profiles.size() + + " security profiles match " + + names); + } + BrokerSecurityProfile profile = profiles.getFirst(); + KafkaSecurityConfigurer configurer = securityConfigurer.getIfAvailable(); + if (configurer == null) { + throw new IllegalStateException( + "app.messaging.security." + + profile.broker() + + " is configured but no CredentialProvider bean exists; the producer cannot" + + " resolve its credential and must not connect without it"); + } + return configurer.configure( + profile, profile.producer(), TLS_PROTOCOLS, java.time.Instant.now()); } /** diff --git a/messaging/messaging-spring-boot-starter-rabbit/build.gradle b/messaging/messaging-spring-boot-starter-rabbit/build.gradle deleted file mode 100644 index 3635e1c..0000000 --- a/messaging/messaging-spring-boot-starter-rabbit/build.gradle +++ /dev/null @@ -1,12 +0,0 @@ -plugins { - id 'ca.platform-module' - id 'ca.spring-config' -} - -dependencies { - // Capability artifact: Rabbit types/client exist only when this artifact is deliberately added. - api project(':messaging:messaging-spring-boot-starter') - api project(':messaging:messaging-rabbit') - - implementation 'org.springframework.boot:spring-boot-autoconfigure' -} diff --git a/messaging/messaging-spring-boot-starter-rabbit/gradle.lockfile b/messaging/messaging-spring-boot-starter-rabbit/gradle.lockfile deleted file mode 100644 index 1d3075d..0000000 --- a/messaging/messaging-spring-boot-starter-rabbit/gradle.lockfile +++ /dev/null @@ -1,124 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.fasterxml.jackson.core:jackson-annotations:2.21=runtimeClasspath,testRuntimeClasspath -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -com.rabbitmq:amqp-client:5.27.1=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -commons-logging:commons-logging:1.3.6=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -info.picocli:picocli:4.7.7=checkstyle -io.cloudevents:cloudevents-api:4.0.1=runtimeClasspath,testRuntimeClasspath -io.cloudevents:cloudevents-core:4.0.1=runtimeClasspath,testRuntimeClasspath -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-buffer:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-base:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-compression:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-marshalling:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-protobuf:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-common:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-handler:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-resolver:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-transport-native-unix-common:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-transport:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.projectreactor:reactor-core:3.8.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=runtimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=runtimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reactivestreams:reactive-streams:1.0.4=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=compileClasspath,runtimeClasspath,spotbugs,spotbugsSlf4j,testCompileClasspath,testRuntimeClasspath -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.springframework.amqp:spring-amqp:4.0.5=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework.amqp:spring-rabbit:4.0.5=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework.boot:spring-boot-actuator:4.0.8=runtimeClasspath,testRuntimeClasspath -org.springframework.boot:spring-boot-autoconfigure:4.0.8=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework.boot:spring-boot-configuration-processor:4.0.8=annotationProcessor -org.springframework.boot:spring-boot:4.0.8=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-aop:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-beans:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-context:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-core:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-expression:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-jdbc:7.0.9=runtimeClasspath,testRuntimeClasspath -org.springframework:spring-messaging:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-tx:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -tools.jackson.core:jackson-core:3.1.5=runtimeClasspath,testRuntimeClasspath -tools.jackson.core:jackson-databind:3.1.5=runtimeClasspath,testRuntimeClasspath -tools.jackson:jackson-bom:3.1.5=runtimeClasspath,testRuntimeClasspath -empty= diff --git a/messaging/messaging-spring-boot-starter-rabbit/src/main/java/dev/caskeleton/messaging/autoconfigure/RabbitMessagingAutoConfiguration.java b/messaging/messaging-spring-boot-starter-rabbit/src/main/java/dev/caskeleton/messaging/autoconfigure/RabbitMessagingAutoConfiguration.java deleted file mode 100644 index f64cd42..0000000 --- a/messaging/messaging-spring-boot-starter-rabbit/src/main/java/dev/caskeleton/messaging/autoconfigure/RabbitMessagingAutoConfiguration.java +++ /dev/null @@ -1,126 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.rabbit.RabbitProfileValidator; -import dev.caskeleton.messaging.rabbit.RabbitPublishFailureClassifier; -import dev.caskeleton.messaging.rabbit.RabbitSecurityConfigurer; -import dev.caskeleton.messaging.security.BrokerTlsPolicy; -import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; - -/** - * Wires the RabbitMQ adapter's validators and security configurer. - * - *

Conditional on the AMQP client, for the same reason the Kafka configuration is conditional on - * the Kafka client: a service using one broker must not need the other on its classpath. - * - *

No connection or channel bean is created here. A channel opened before the application has - * declared its security profile would connect with whatever defaults the client carries, which for - * RabbitMQ means {@code guest/guest} against localhost. - * - *

Reached only when app.messaging.broker=rabbit selected it; see MessagingProviderSelection for - * why a classpath condition could not make this choice. - */ -@Configuration(proxyBeanMethods = false) -public class RabbitMessagingAutoConfiguration { - - /** - * Returns the RabbitMQ destination profile validator. - * - * @return the validator - */ - @Bean - @ConditionalOnMissingBean - public RabbitProfileValidator rabbitProfileValidator() { - return new RabbitProfileValidator(); - } - - /** - * Runs the RabbitMQ profile validator over every broker profile, declared or configured. - * - * @param profiles the broker profiles the application declared as beans - * @param compiled the broker profiles compiled from {@code app.messaging.brokers} - * @param validator the RabbitMQ profile validator - * @return the startup validation - */ - @Bean - public StartupProfileValidation - rabbitProfileStartupValidation( - org.springframework.beans.factory.ObjectProvider< - dev.caskeleton.messaging.rabbit.RabbitBrokerProfile> - profiles, - CompiledMessagingConfiguration compiled, - RabbitProfileValidator validator) { - return new StartupProfileValidation<>( - () -> - java.util.stream.Stream.concat( - configuredRabbitProfiles(compiled).stream(), profiles.orderedStream()) - .toList(), - validator::validate); - } - - /** - * Returns the publish failure classifier. - * - * @return the classifier - */ - @Bean - @ConditionalOnMissingBean - public RabbitPublishFailureClassifier rabbitPublishFailureClassifier() { - return new RabbitPublishFailureClassifier(); - } - - /** - * Returns the AMQP security configurer. - * - * @param credentials the credential runtime registry - * @param tlsPolicy the transport security policy - * @return the configurer - */ - /** - * The security configurer, which exists only when a credential source does. - * - *

It took {@code CredentialRuntimeRegistry} as a required parameter, and that registry is - * created only when the application supplies a {@code CredentialProvider}. Without one the - * context failed on a missing bean naming the registry — an error that describes the platform's - * internals rather than the one thing the operator has to do. The condition moves the failure to - * a place that can say so. - */ - @Bean - @ConditionalOnMissingBean - @org.springframework.boot.autoconfigure.condition.ConditionalOnBean( - CredentialRuntimeRegistry.class) - public RabbitSecurityConfigurer rabbitSecurityConfigurer( - CredentialRuntimeRegistry credentials, BrokerTlsPolicy tlsPolicy) { - return new RabbitSecurityConfigurer(credentials, tlsPolicy); - } - - private static java.util.List - configuredRabbitProfiles(CompiledMessagingConfiguration compiled) { - return compiled.brokers().entrySet().stream() - .filter( - entry -> - entry.getValue().type() != null - && entry.getValue().type().equalsIgnoreCase(BrokerSettings.RABBITMQ)) - .map( - entry -> { - BrokerSettings broker = entry.getValue(); - return new dev.caskeleton.messaging.rabbit.RabbitBrokerProfile( - entry.getKey(), - broker.stable(), - broker.production(), - broker.addresses(), - broker.publisherConfirms(), - broker.publisherReturns(), - broker.mandatory(), - broker.confirmTimeout(), - broker.autoAck(), - broker.prefetch(), - broker.quorumQueues(), - broker.tlsEnabled(), - broker.authenticationEnabled()); - }) - .toList(); - } -} diff --git a/messaging/messaging-spring-boot-starter/README.md b/messaging/messaging-spring-boot-starter/README.md index 9580cea..2cb6c3e 100644 --- a/messaging/messaging-spring-boot-starter/README.md +++ b/messaging/messaging-spring-boot-starter/README.md @@ -2,21 +2,21 @@ ## 아티팩트 분리 -기본 `messaging-spring-boot-starter`는 이제 **broker-neutral core**다. Kafka/Rabbit client를 +기본 `messaging-spring-boot-starter`는 이제 **broker-neutral core**다. Kafka client를 production runtime에 직접 넣지 않으며, 브로커 채택은 capability artifact를 composition에 추가하는 build-time 결정이다. ```text messaging-spring-boot-starter -├─ messaging-spring-boot-starter-kafka # Kafka client/transport capability -└─ messaging-spring-boot-starter-rabbit # Rabbit profile/validator capability +└─ messaging-spring-boot-starter-kafka # Kafka client/transport capability ``` -- 기본 app-bootstrap은 core starter만 가진다. 따라서 messaging off/default runtime에는 - `kafka-clients`, `spring-kafka`, `spring-rabbit`, `amqp-client`가 없다. -- Kafka를 채택하는 composition은 `:messaging:messaging-spring-boot-starter-kafka`를 추가한다. -- Rabbit capability도 별도 artifact지만, 아래 설명처럼 아직 production transport가 없어 - `app.messaging.broker=rabbit` 선택은 startup에서 거부된다. +- 기본 app-bootstrap은 messaging을 싣지 않는다. 서비스는 `runtimeOnly + project(':adapter:outbound:messaging-kafka')` 한 줄로 Kafka capability를 고르고, 그 모듈이 + `:messaging:messaging-spring-boot-starter-kafka`(와 이 core starter)를 runtime으로 끌어온다. +- starter는 있으면 켜진다(`app.messaging.enabled` 기본 on). `app.messaging.enabled=false`로만 끈다. +- 등록된 broker 계열은 Kafka 하나다. RabbitMQ 설정 키(`type: rabbitmq`, `addresses`, `prefetch` 등)는 + transport 없이 바인딩만 되던 키라 삭제했고, 지금은 키 이름을 담아 기동이 거부된다. - reliability/admin의 추가 artifact 분리는 별도 구조 개선 여지로 남지만, broker SDK 동시 탑재 문제와 transport 선택 문제는 core/broker capability 분리로 해소했다. - legacy R0 outbound bridge도 같은 원칙을 따른다. broker-neutral @@ -35,13 +35,9 @@ consume을 채택하려면 애플리케이션 composition이 별도로 `KafkaCon 조립해야 한다. leaf 안의 consumer/rebalance/settlement 테스트가 초록이라는 사실은 capability starter가 consumer runtime까지 자동 제공한다는 뜻이 아니다. -### RabbitMQ +### RabbitMQ (삭제됨) -RabbitMQ는 별도 capability artifact에 격리되어 있지만 **아직 선택할 수 없는 transport**다. -`app.messaging.broker=rabbit`을 설정하면 `MessagingProviderSelection`이 startup에서 거부한다. -이 leaf에는 validator, failure classifier, topology/security 부품과 테스트는 있지만 production -`RabbitChannelPublisher` 구현이 없어 `MessagingTransport`를 완성할 수 없다. - -현재 selectable broker는 Kafka뿐이다. Rabbit이 selectable로 승격되는 조건은 production -channel publisher/settlement bridge를 구현하고 starter composition 및 broker integration 증거를 -함께 추가하는 것이다. +production transport가 없어 선택할 수 없던 `messaging-rabbit`·`messaging-spring-boot-starter-rabbit`는 +2026-09-25에 삭제했다(`docs/capability-status.md` 제거 표). 현재 등록된 broker는 Kafka 하나다. +`app.messaging.brokers..type: rabbitmq` 설정 family는 아직 바인딩·검증되지만 소비하는 transport가 +없다. diff --git a/messaging/messaging-spring-boot-starter/build.gradle b/messaging/messaging-spring-boot-starter/build.gradle index 14167fa..acae04e 100644 --- a/messaging/messaging-spring-boot-starter/build.gradle +++ b/messaging/messaging-spring-boot-starter/build.gradle @@ -6,15 +6,9 @@ plugins { // Scopes, not a flat list of `api`. // -// Sixteen internal leaves were exposed as `api`, so every adopter of the starter compiled against -// the Kafka adapter, the Rabbit adapter, both JDBC reliability leaves and the admin plane whether -// they used them or not — and both brokers' `@ConditionalOnClass` guards were true at once because -// the starter itself dragged in both clients. -// -// `api` is now the types that genuinely appear in this module's public signatures; everything else -// is `implementation`. Splitting the starter into per-capability artifacts (core / kafka / rabbit / -// reliability / admin) is the structural fix the review asks for and is a registry change of its -// own; this is the part that stops an adopter's compile classpath from containing two broker SDKs. +// `api` is the types that genuinely appear in this module's public signatures; everything else is +// `implementation`. Broker transports live in capability starters (messaging-spring-boot-starter-kafka), +// so this starter's compile classpath carries no broker SDK. auxiliarySourceSets { sourceSet('integrationTest') { compilesAgainst 'main' @@ -33,14 +27,12 @@ dependencies { // Wired by the auto-configurations and never named in an adopter's source. implementation project(':messaging:messaging-schema-json') - implementation project(':messaging:messaging-cloudevents') // The publisher and its collaborators live here. The starter did not depend on it at all, // which is why MessagingCoreAutoConfiguration consumed a MessagePublisher that nothing in // reach could produce (MSG-INT-003). implementation project(':messaging:messaging-runtime-core') implementation project(':messaging:messaging-outbox-jdbc-postgresql') implementation project(':messaging:messaging-inbox-jdbc-postgresql') - implementation project(':messaging:messaging-claim-check') implementation project(':messaging:messaging-admin-api') implementation project(':messaging:messaging-admin-runtime') @@ -56,9 +48,8 @@ dependencies { testImplementation 'org.springframework.boot:spring-boot-test' testImplementation 'org.springframework:spring-test' // Provider auto-configurations live in capability artifacts. Tests that exercise provider - // selection add them only to the test graph; the production starter remains broker-neutral. + // selection add it only to the test graph; the production starter remains broker-neutral. testImplementation project(':messaging:messaging-spring-boot-starter-kafka') - testImplementation project(':messaging:messaging-spring-boot-starter-rabbit') // Test-only, and specifically so the configuration reference can be executed rather than read: // MessagingConfigurationBindingTest loads the YAML blocks out of the shipped document and boots // a context on them. Spring Boot 4 treats snakeyaml as optional, so a document-driven test has diff --git a/messaging/messaging-spring-boot-starter/gradle.lockfile b/messaging/messaging-spring-boot-starter/gradle.lockfile index 7f5981c..d606cfc 100644 --- a/messaging/messaging-spring-boot-starter/gradle.lockfile +++ b/messaging/messaging-spring-boot-starter/gradle.lockfile @@ -32,7 +32,6 @@ com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,integrat com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -com.rabbitmq:amqp-client:5.27.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath commons-beanutils:commons-beanutils:1.11.0=checkstyle commons-codec:commons-codec:1.19.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath commons-collections:commons-collections:3.2.2=checkstyle @@ -40,24 +39,11 @@ commons-io:commons-io:2.20.0=integrationTestCompileClasspath,integrationTestRunt commons-io:commons-io:2.21.0=spotbugs commons-logging:commons-logging:1.3.6=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath info.picocli:picocli:4.7.7=checkstyle -io.cloudevents:cloudevents-api:4.0.1=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.cloudevents:cloudevents-core:4.0.1=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor io.micrometer:micrometer-commons:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.micrometer:micrometer-core:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-buffer:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-base:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-compression:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-marshalling:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-protobuf:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-common:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-handler:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-resolver:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-transport-native-unix-common:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-transport:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath io.projectreactor:reactor-core:3.8.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.projectreactor:reactor-test:3.8.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor @@ -113,11 +99,9 @@ org.pcollections:pcollections:4.0.1=annotationProcessor,integrationTestAnnotatio org.reactivestreams:reactive-streams:1.0.4=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.reflections:reflections:0.10.2=checkstyle org.rnorth.duct-tape:duct-tape:1.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.slf4j:slf4j-api:2.0.18=integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs,spotbugsSlf4j,testCompileClasspath,testRuntimeClasspath +org.slf4j:slf4j-api:2.0.18=integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs,spotbugsSlf4j,testRuntimeClasspath org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j org.slf4j:slf4j-simple:2.0.18=checkstyle -org.springframework.amqp:spring-amqp:4.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework.amqp:spring-rabbit:4.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-actuator:4.0.8=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-autoconfigure:4.0.8=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-configuration-processor:4.0.8=annotationProcessor @@ -129,9 +113,8 @@ org.springframework:spring-context:7.0.9=compileClasspath,integrationTestCompile org.springframework:spring-core:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework:spring-expression:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework:spring-jdbc:7.0.9=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath -org.springframework:spring-messaging:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework:spring-test:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-tx:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath +org.springframework:spring-tx:7.0.9=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath org.testcontainers:testcontainers-junit-jupiter:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.testcontainers:testcontainers-kafka:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.testcontainers:testcontainers:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java index 5274dd5..fb3a111 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java @@ -7,18 +7,16 @@ import org.springframework.boot.context.properties.bind.DefaultValue; /** * One entry of {@code app.messaging.brokers}. * - *

Both broker families are described by one record because a deployment names its brokers in one - * map, and a reader of that map should not have to know which half of the fields apply. Which half - * does apply is decided by {@link #type()}, and the compiler refuses a field belonging to the other - * family rather than ignoring it — a Kafka broker carrying {@code prefetch} is a configuration the - * author believed was doing something. + *

Kafka is the one broker family this platform has a transport for, so the record carries only + * Kafka fields. RabbitMQ fields used to live here too and bound with no transport behind them: a + * deployment could configure a RabbitMQ broker that nothing would ever connect to. * *

{@code tlsEnabled} and {@code production} live here rather than beside the credentials, so a * broker is described once. They were previously implied by whichever object happened to be * constructed, which is how a profile can be production in one validator's view and not in * another's. * - * @param type the broker family, {@code kafka} or {@code rabbitmq} + * @param type the broker family; {@code kafka} is the only one * @param stable whether this profile claims the Stable guarantees * @param production whether this profile is used in production * @param tlsEnabled whether transport encryption is on @@ -30,14 +28,6 @@ import org.springframework.boot.context.properties.bind.DefaultValue; * @param deliveryTimeout the Kafka producer delivery deadline * @param enableAutoCommit whether the Kafka consumer commits offsets on a timer * @param consumerGroup the Kafka consumer group - * @param addresses RabbitMQ broker addresses - * @param publisherConfirms whether correlated publisher confirms are on - * @param publisherReturns whether unroutable returns are delivered - * @param mandatory whether publishes carry the mandatory flag - * @param confirmTimeout how long to wait for a RabbitMQ confirm - * @param autoAck whether the RabbitMQ consumer acknowledges on delivery - * @param prefetch the RabbitMQ consumer prefetch window - * @param quorumQueues whether durable work queues are declared as quorum queues */ public record BrokerSettings( String type, @@ -51,19 +41,8 @@ public record BrokerSettings( @DefaultValue("5") int maxInFlightRequestsPerConnection, @DefaultValue("30s") Duration deliveryTimeout, @DefaultValue("false") boolean enableAutoCommit, - @DefaultValue("") String consumerGroup, - @DefaultValue List addresses, - @DefaultValue("true") boolean publisherConfirms, - @DefaultValue("true") boolean publisherReturns, - @DefaultValue("true") boolean mandatory, - @DefaultValue("5s") Duration confirmTimeout, - @DefaultValue("false") boolean autoAck, - @DefaultValue("16") int prefetch, - @DefaultValue("true") boolean quorumQueues) { + @DefaultValue("") String consumerGroup) { /** The broker family a Kafka entry declares. */ public static final String KAFKA = "kafka"; - - /** The broker family a RabbitMQ entry declares. */ - public static final String RABBITMQ = "rabbitmq"; } diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java index dca2d39..88fa8cb 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java @@ -101,19 +101,11 @@ public final class MessagingConfigurationCompiler { + "localhost:9092 and fails at the first publish rather than at startup"); } } - case BrokerSettings.RABBITMQ -> { - if (broker.addresses().isEmpty()) { - throw configurationError( - key("brokers", name, "addresses"), - "a RabbitMQ broker needs at least one address; a client with none defaults to " - + "localhost and connects as guest"); - } - } default -> throw configurationError( key("brokers", name, "type"), - "'%s' is not a broker family this platform binds; use '%s' or '%s'" - .formatted(broker.type(), BrokerSettings.KAFKA, BrokerSettings.RABBITMQ)); + "'%s' is not a broker family this platform has a transport for; use '%s'" + .formatted(broker.type(), BrokerSettings.KAFKA)); } } diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java index 84e9baa..b5c534a 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java @@ -5,7 +5,6 @@ import java.lang.reflect.RecordComponent; import java.util.ArrayList; import java.util.LinkedHashSet; import java.util.List; -import java.util.Locale; import java.util.Map; import java.util.Objects; import java.util.Set; @@ -38,29 +37,6 @@ import org.springframework.core.env.PropertySource; */ public final class MessagingConfigurationKeyValidator implements InitializingBean { - /** Broker keys that only a Kafka entry may carry. */ - private static final Set KAFKA_ONLY_KEYS = - Set.of( - "bootstrap-servers", - "enable-idempotence", - "acks", - "max-in-flight-requests-per-connection", - "delivery-timeout", - "enable-auto-commit", - "consumer-group"); - - /** Broker keys that only a RabbitMQ entry may carry. */ - private static final Set RABBIT_ONLY_KEYS = - Set.of( - "addresses", - "publisher-confirms", - "publisher-returns", - "mandatory", - "confirm-timeout", - "auto-ack", - "prefetch", - "quorum-queues"); - private final ConfigurableEnvironment environment; /** @@ -102,8 +78,6 @@ public final class MessagingConfigurationKeyValidator implements InitializingBea rejected.add( "%s binds nothing; %s entries accept %s" .formatted(key, section, legal.stream().sorted().toList())); - } else if ("brokers".equals(section)) { - wrongFamily(entry).ifPresent(rejected::add); } } }); @@ -125,32 +99,6 @@ public final class MessagingConfigurationKeyValidator implements InitializingBea + (rejected.size() > 10 ? " (and " + (rejected.size() - 10) + " more)" : "")); } - /** - * Reports a broker key that belongs to the other broker family. - * - *

Both families are described by one record, so a Kafka entry carrying {@code prefetch} binds - * without complaint and does nothing. The author of that line believed it was doing something, - * which is the same silence this class exists to end. - */ - private java.util.Optional wrongFamily(Entry entry) { - String declared = - environment.getProperty( - MessagingSettings.PREFIX + ".brokers." + entry.name() + ".type", ""); - String family = declared.toLowerCase(Locale.ROOT); - String leaf = entry.leaf(); - if (BrokerSettings.KAFKA.equals(family) && RABBIT_ONLY_KEYS.contains(leaf)) { - return java.util.Optional.of( - "%s.brokers.%s.%s is a RabbitMQ setting on a Kafka broker" - .formatted(MessagingSettings.PREFIX, entry.name(), leaf)); - } - if (BrokerSettings.RABBITMQ.equals(family) && KAFKA_ONLY_KEYS.contains(leaf)) { - return java.util.Optional.of( - "%s.brokers.%s.%s is a Kafka setting on a RabbitMQ broker" - .formatted(MessagingSettings.PREFIX, entry.name(), leaf)); - } - return java.util.Optional.empty(); - } - /** One configuration key split into the map entry it configures and the leaf inside it. */ private record Entry(String name, String leaf) {} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java index 81b0602..a61db17 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java @@ -516,7 +516,6 @@ public class MessagingCoreAutoConfiguration { String configuredFamily = switch (selected.brokerName()) { case "kafka" -> BrokerSettings.KAFKA; - case "rabbit" -> BrokerSettings.RABBITMQ; default -> selected.brokerName(); }; java.util.List configuredNames = diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPlatformRootAutoConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPlatformRootAutoConfiguration.java index 8e3b816..3534ca2 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPlatformRootAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPlatformRootAutoConfiguration.java @@ -20,13 +20,18 @@ import org.springframework.context.annotation.Import; * whichever bean won. Selection now reads {@code app.messaging.broker} against a closed registry, * and a value outside it is a startup error rather than a context with no provider at all. * - *

{@code app.messaging.enabled=false} means no beans, no clients, no threads, and no bound - * detail namespace. {@code MessagingStarterOffContractTest} holds that against a full context - * including this starter's owned imports. Vendor auto-configuration is verified in its own runtime - * lanes. + *

On when present: a missing {@code app.messaging.enabled} means on, because an application that + * put this starter on its classpath chose messaging. {@code app.messaging.enabled=false} means no + * beans, no clients, no threads, and no bound detail namespace. {@code + * MessagingStarterOffContractTest} holds that against a full context including this starter's owned + * imports. Vendor auto-configuration is verified in its own runtime lanes. */ @AutoConfiguration -@ConditionalOnProperty(prefix = MessagingSettings.PREFIX, name = "enabled", havingValue = "true") +@ConditionalOnProperty( + prefix = MessagingSettings.PREFIX, + name = "enabled", + havingValue = "true", + matchIfMissing = true) @EnableConfigurationProperties(MessagingSettings.class) @Import({ MessagingCoreAutoConfiguration.class, diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java index 227965c..91cf33b 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java @@ -13,9 +13,9 @@ import org.springframework.util.ClassUtils; /** * Selects exactly one transport, from a closed registry, by name (MSG-INT-004). * - *

Selection used to be {@code @ConditionalOnClass}: Kafka assembled because a Kafka class was on - * the classpath and Rabbit because a Rabbit class was, so an application with both libraries — one - * transitive dependency is enough — assembled both and published through whichever bean won a + *

Selection used to be {@code @ConditionalOnClass}: a transport assembled because its client + * class was on the classpath, so an application with two client libraries — one transitive + * dependency is enough — assembled both and published through whichever bean won a * {@code @ConditionalOnMissingBean} race. Nothing failed; the message simply went somewhere nobody * chose. * @@ -25,7 +25,8 @@ import org.springframework.util.ClassUtils; *

  • a broker id outside the registry — the operator set something the platform cannot honour; *
  • a registered broker whose client library is absent — a deployment that asked for Kafka and * shipped without it is told so, instead of starting with no publisher at all; - *
  • a blank broker while messaging is on — the master switch says yes and nothing says what to. + *
  • a blank broker while messaging is on — the capability switch says yes and nothing says what + * to. * * *

    The provider configurations carry no condition of their own any more. Each is reached only @@ -38,58 +39,21 @@ public class MessagingProviderSelection { /** Broker id to the client class that must be present for it to be honoured. */ static final Map REGISTERED_BROKERS = - Map.of( - "kafka", "org.apache.kafka.clients.producer.Producer", - "rabbit", "com.rabbitmq.client.Channel"); + Map.of("kafka", "org.apache.kafka.clients.producer.Producer"); /** Broker id to the configuration that assembles it. */ static final Map PROVIDER_CONFIGURATIONS = - Map.of( - "kafka", "dev.caskeleton.messaging.autoconfigure.KafkaMessagingAutoConfiguration", - "rabbit", "dev.caskeleton.messaging.autoconfigure.RabbitMessagingAutoConfiguration"); + Map.of("kafka", "dev.caskeleton.messaging.autoconfigure.KafkaMessagingAutoConfiguration"); /** Broker id to the starter a deployment adds to its build to obtain that transport. */ static final Map PROVIDER_STARTERS = - Map.of( - "kafka", "messaging-spring-boot-starter-kafka", - "rabbit", "messaging-spring-boot-starter-rabbit"); - - /** - * Broker ids that ship without a {@code MessagingTransport}, and why. - * - *

    Registered is not the same as assemblable. The Rabbit leaf ships its profile validator, - * failure classifier and security configurer, and no transport — the native channel bridge those - * would sit on has only test implementations. Selecting it therefore produced an unsatisfied - * dependency on {@code MessagingTransport} deep inside the core configuration, which tells an - * operator that a bean is missing rather than that the transport they chose is not finished. - * - *

    Refused here instead, next to the other selection errors, so the message names the state the - * deployment is actually in. An entry leaves this map on the day its transport does exist. - */ - static final Map BROKERS_WITHOUT_A_TRANSPORT = - Map.of( - "rabbit", - "the Rabbit adapter ships its validators and security configuration but no " - + "MessagingTransport: its native channel publisher is not implemented, so a publish " - + "has nothing to travel on"); + Map.of("kafka", "messaging-spring-boot-starter-kafka"); /** The registered broker ids, so a message can tell an operator what is available. */ public static List registeredBrokerIds() { return REGISTERED_BROKERS.keySet().stream().sorted().toList(); } - /** - * The registered broker ids that actually assemble a transport. - * - * @return the ids a deployment can select today - */ - public static List assemblableBrokerIds() { - return REGISTERED_BROKERS.keySet().stream() - .filter(id -> !BROKERS_WITHOUT_A_TRANSPORT.containsKey(id)) - .sorted() - .toList(); - } - /** * Resolves the selection, or throws naming the property and the values it accepts. * @@ -141,17 +105,6 @@ public class MessagingProviderSelection { + " module on the classpath; this core starter is broker-neutral and the deployment" + " did not compose the transport it selected"); } - String unfinished = BROKERS_WITHOUT_A_TRANSPORT.get(broker); - if (unfinished != null) { - throw new IllegalStateException( - MessagingSettings.PREFIX - + ".broker=" - + broker - + " cannot be selected: " - + unfinished - + ". Transports that can be assembled today: " - + assemblableBrokerIds()); - } return broker; } diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingSettings.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingSettings.java index 92fef98..9985b6e 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingSettings.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingSettings.java @@ -20,11 +20,6 @@ import org.springframework.boot.context.properties.ConfigurationProperties; * the failure it prevents (a deployment split across two prefixes, each half-configuring the * platform) is worse than an actionable startup error. * - *

    Every Experimental and bridge flag defaults to {@code false}. An adapter whose contract suite - * is still being proven must not become load-bearing because a default turned it on, and the same - * applies to the Spring Cloud Stream bridge, which bypasses the platform's retry and dead letter - * policy by design. - * *

    The destination, broker and security maps are what the configuration reference has always * described and nothing bound. Until they existed, a deployment that configured them exactly as * documented changed no behaviour and produced no error: the profiles the platform actually used @@ -48,8 +43,6 @@ public class MessagingSettings { /** The prefix the configuration reference documented, which nothing ever bound. */ public static final String DOCUMENTED_PREFIX = "messaging"; - private final Experimental experimental = new Experimental(); - private final Bridge bridge = new Bridge(); private final Backpressure backpressure = new Backpressure(); private final Shutdown shutdown = new Shutdown(); private Map destinations = new LinkedHashMap<>(); @@ -110,24 +103,6 @@ public class MessagingSettings { this.security = new LinkedHashMap<>(security); } - /** - * Returns the experimental adapter switches. - * - * @return the experimental settings - */ - public Experimental getExperimental() { - return experimental; - } - - /** - * Returns the optional bridge switches. - * - * @return the bridge settings - */ - public Bridge getBridge() { - return bridge; - } - /** * Returns the backpressure limits. * @@ -146,92 +121,6 @@ public class MessagingSettings { return shutdown; } - /** Experimental adapter switches, all off by default. */ - public static class Experimental { - - private boolean kafkaShare; - private boolean pulsar; - private boolean nats; - - /** - * Reports whether the Kafka Share Group adapter is enabled. - * - * @return true when enabled - */ - public boolean isKafkaShare() { - return kafkaShare; - } - - /** - * Enables or disables the Kafka Share Group adapter. - * - * @param kafkaShare whether to enable it - */ - public void setKafkaShare(boolean kafkaShare) { - this.kafkaShare = kafkaShare; - } - - /** - * Reports whether the Pulsar adapter is enabled. - * - * @return true when enabled - */ - public boolean isPulsar() { - return pulsar; - } - - /** - * Enables or disables the Pulsar adapter. - * - * @param pulsar whether to enable it - */ - public void setPulsar(boolean pulsar) { - this.pulsar = pulsar; - } - - /** - * Reports whether the NATS JetStream adapter is enabled. - * - * @return true when enabled - */ - public boolean isNats() { - return nats; - } - - /** - * Enables or disables the NATS JetStream adapter. - * - * @param nats whether to enable it - */ - public void setNats(boolean nats) { - this.nats = nats; - } - } - - /** Optional integration bridges, off by default. */ - public static class Bridge { - - private boolean springCloudStream; - - /** - * Reports whether the Spring Cloud Stream bridge is enabled. - * - * @return true when enabled - */ - public boolean isSpringCloudStream() { - return springCloudStream; - } - - /** - * Enables or disables the Spring Cloud Stream bridge. - * - * @param springCloudStream whether to enable it - */ - public void setSpringCloudStream(boolean springCloudStream) { - this.springCloudStream = springCloudStream; - } - } - /** In-flight limits. */ public static class Backpressure { diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java new file mode 100644 index 0000000..ac651f8 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java @@ -0,0 +1,151 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.kafka.KafkaSecurityConfigurer; +import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import dev.caskeleton.messaging.security.CredentialProvider; +import java.time.Instant; +import java.util.Map; +import java.util.Optional; +import org.junit.jupiter.api.Test; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.assertj.AssertableApplicationContext; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * What the Kafka producer is actually configured with once {@code app.messaging.security} says how. + * + *

    The security settings were bound, compiled and validated, and {@code KafkaSecurityConfigurer} + * was a bean nothing called: the producer was built from the bootstrap address, serializers, acks + * and idempotence alone, so a broker that passed the TLS and credential checks was reached over + * plaintext with no authentication. + */ +class KafkaProducerSecurityTest { + + // Assembled through the root, as an application is: the import order it fixes is what lets the + // configurer see the credential registry. + private final ApplicationContextRunner runner = + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "spring.kafka.bootstrap-servers=localhost:9092", + "app.messaging.brokers.kafka-primary.type=kafka", + "app.messaging.brokers.kafka-primary.bootstrap-servers[0]=localhost:9092"); + + private static final String[] SCRAM_CREDENTIALS = { + "app.messaging.security.kafka-primary.producer.type=SASL_SCRAM", + "app.messaging.security.kafka-primary.producer.credential-id=kafka-producer", + "app.messaging.security.kafka-primary.consumer.type=SASL_SCRAM", + "app.messaging.security.kafka-primary.consumer.credential-id=kafka-consumer", + }; + + @Test + void aTlsBrokerWithScramCredentialsConfiguresTheProducerForSaslSsl() { + runner + .withUserConfiguration(Credentials.class) + .withPropertyValues(SCRAM_CREDENTIALS) + .withPropertyValues( + "app.messaging.brokers.kafka-primary.tls-enabled=true", + "app.messaging.brokers.kafka-primary.authentication-enabled=true") + .run( + context -> + assertThat(producerConfig(context)) + .extractingByKeys( + "security.protocol", + "sasl.mechanism", + "ssl.enabled.protocols", + "ssl.endpoint.identification.algorithm") + .containsExactly("SASL_SSL", "SCRAM-SHA-512", "TLSv1.3,TLSv1.2", "https")); + } + + @Test + void aPlaintextBrokerWithPasswordCredentialsConfiguresTheProducerForSaslPlaintext() { + runner + .withUserConfiguration(Credentials.class) + .withPropertyValues( + "app.messaging.brokers.kafka-primary.authentication-enabled=true", + "app.messaging.security.kafka-primary.producer.type=USERNAME_PASSWORD", + "app.messaging.security.kafka-primary.producer.credential-id=kafka-producer", + "app.messaging.security.kafka-primary.consumer.type=USERNAME_PASSWORD", + "app.messaging.security.kafka-primary.consumer.credential-id=kafka-consumer") + .run( + context -> + assertThat(producerConfig(context)) + .extractingByKeys("security.protocol", "sasl.mechanism") + .containsExactly("SASL_PLAINTEXT", "PLAIN")); + } + + @Test + void theProducerLogsInWithTheProducerCredentialResolvedFromTheProvider() { + runner + .withUserConfiguration(Credentials.class) + .withPropertyValues(SCRAM_CREDENTIALS) + .withPropertyValues("app.messaging.brokers.kafka-primary.tls-enabled=true") + .run( + context -> + assertThat(String.valueOf(producerConfig(context).get("sasl.jaas.config"))) + .startsWith("org.apache.kafka.common.security.scram.ScramLoginModule required") + .contains("username=\"kafka-producer\"", "password=\"" + Credentials.SECRET)); + } + + @Test + void aBrokerDeclaringTlsWithoutASecurityEntryFailsStartup() { + runner + .withPropertyValues("app.messaging.brokers.kafka-primary.tls-enabled=true") + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .as("a producer that cannot apply TLS connected over plaintext instead") + .hasMessageContaining("app.messaging.security.kafka-primary")); + } + + @Test + void aSecurityEntryWithoutACredentialProviderFailsStartup() { + runner + .withPropertyValues(SCRAM_CREDENTIALS) + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .as("a producer that cannot resolve its credential connected without one") + .hasMessageContaining("CredentialProvider")); + } + + private static Map producerConfig(AssertableApplicationContext context) { + return KafkaMessagingAutoConfiguration.producerConfig( + context.getEnvironment(), + context.getBean(CompiledMessagingConfiguration.class), + context.getBeanProvider(BrokerSecurityProfile.class), + context.getBeanProvider(KafkaSecurityConfigurer.class)); + } + + /** A credential source that answers every id with the same test secret. */ + @Configuration(proxyBeanMethods = false) + static class Credentials { + + static final String SECRET = "test-only-secret"; + + @Bean + CredentialProvider credentialProvider() { + return new CredentialProvider() { + @Override + public char[] resolve(String credentialId) { + return SECRET.toCharArray(); + } + + @Override + public Optional expiresAt(String credentialId) { + return Optional.empty(); + } + }; + } + } +} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java index ba6b2c7..6c82922 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java @@ -98,30 +98,16 @@ class MessagingAutoConfigurationTest { .hasMessageContaining("exactly one logical broker")); } - @Test - void experimentalAdaptersAndTheBridgeAreOffByDefault() { - runner.run( - context -> { - MessagingSettings properties = context.getBean(MessagingSettings.class); - assertThat(properties.getExperimental().isKafkaShare()).isFalse(); - assertThat(properties.getExperimental().isPulsar()).isFalse(); - assertThat(properties.getExperimental().isNats()).isFalse(); - assertThat(properties.getBridge().isSpringCloudStream()).isFalse(); - }); - } - @Test void propertiesBindFromConfiguration() { runner .withPropertyValues( - "app.messaging.experimental.pulsar=true", "app.messaging.backpressure.global-limit=64", "app.messaging.backpressure.per-destination-limit=8", "app.messaging.shutdown.drain-deadline=15s") .run( context -> { MessagingSettings properties = context.getBean(MessagingSettings.class); - assertThat(properties.getExperimental().isPulsar()).isTrue(); assertThat(properties.getBackpressure().getGlobalLimit()).isEqualTo(64); assertThat(properties.getShutdown().getDrainDeadline()) .isEqualTo(java.time.Duration.ofSeconds(15)); diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java index 437ca3f..73f8a08 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java @@ -67,15 +67,12 @@ class MessagingConfigurationBindingTest { CompiledMessagingConfiguration compiled = context.getBean(CompiledMessagingConfiguration.class); - assertThat(compiled.brokers()).containsKeys("kafka-primary", "rabbit-primary"); + assertThat(compiled.brokers()).containsOnlyKeys("kafka-primary"); BrokerSettings kafka = compiled.brokers().get("kafka-primary"); assertThat(kafka.type()).isEqualTo(BrokerSettings.KAFKA); assertThat(kafka.bootstrapServers()) .containsExactly("broker-1:9093", "broker-2:9093"); assertThat(kafka.enableAutoCommit()).isFalse(); - BrokerSettings rabbit = compiled.brokers().get("rabbit-primary"); - assertThat(rabbit.type()).isEqualTo(BrokerSettings.RABBITMQ); - assertThat(rabbit.quorumQueues()).isTrue(); assertThat(compiled.security()) .singleElement() .satisfies( @@ -175,7 +172,7 @@ class MessagingConfigurationBindingTest { } @Test - void aRabbitSettingOnAKafkaBrokerFailsNamingTheKey() { + void aRabbitMqSettingBindsNothingAndFailsNamingTheKey() { runner .withPropertyValues(minimal()) .withPropertyValues("app.messaging.brokers.kafka-primary.prefetch=32") @@ -184,8 +181,22 @@ class MessagingConfigurationBindingTest { assertThat(context) .hasFailed() .getFailure() - .hasMessageContaining("app.messaging.brokers.kafka-primary.prefetch") - .hasMessageContaining("RabbitMQ setting on a Kafka broker")); + .hasMessageContaining( + "app.messaging.brokers.kafka-primary.prefetch binds nothing")); + } + + @Test + void aRabbitMqBrokerIsRefusedBecauseNoTransportExists() { + runner + .withPropertyValues( + "app.messaging.brokers.rabbit-primary.type=rabbitmq", + "app.messaging.brokers.rabbit-primary.bootstrap-servers[0]=rabbit:5672") + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .hasMessageContaining("app.messaging.brokers.rabbit-primary.type")); } @Test diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java index 033fb0d..4da59b9 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java @@ -15,17 +15,16 @@ import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; import org.springframework.boot.autoconfigure.AutoConfigurations; import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.util.ClassUtils; /** * Off means nothing, and on means exactly one transport (MSG-INT-004). * *

    The starter registered five auto-configurations and none carried a messaging master condition, * so an application acquired the platform by having the starter on its classpath. Worse, provider - * selection was {@code @ConditionalOnClass} while this starter depends on {@code messaging-kafka} - * and {@code messaging-rabbit}, each of which exposes its client library as {@code api} — so both - * conditions are true for **every** adopter, always. Selection by classpath could not select; it - * assembled both and let a {@code @ConditionalOnMissingBean} race decide where messages went. + * selection was {@code @ConditionalOnClass} while this starter depended on two broker adapters, + * each exposing its client library as {@code api} — so both conditions were true for **every** + * adopter, always. Selection by classpath could not select; it assembled both and let a + * {@code @ConditionalOnMissingBean} race decide where messages went. * *

    The cases below are the replacement contract. The off case is deliberately checked on bean * names and live threads rather than on a handful of types: a platform that leaves a scheduler @@ -38,40 +37,41 @@ class MessagingStarterOffContractTest { .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)); @Test - @DisplayName( - "both client libraries really are on this classpath, so the old condition could not select") - void bothClientLibrariesArePresent() { - ClassLoader loader = getClass().getClassLoader(); - - assertThat(ClassUtils.isPresent("org.apache.kafka.clients.producer.Producer", loader)) - .as("messaging-kafka exposes kafka-clients as api") - .isTrue(); - assertThat(ClassUtils.isPresent("com.rabbitmq.client.Channel", loader)) - .as("messaging-rabbit exposes spring-rabbit as api") - .isTrue(); + @DisplayName("present without the switch is on") + void presentWithoutTheSwitchIsOn() { + runner + .withPropertyValues( + "app.messaging.broker=kafka", "spring.kafka.bootstrap-servers=localhost:9092") + .run( + context -> { + assertThat(context).hasNotFailed(); + assertThat(context).hasSingleBean(MessagePublisher.class); + }); } @Test @DisplayName("with the switch off there are no platform beans") void offMeansNoBeans() { - runner.run( - context -> { - assertThat(context).hasNotFailed(); + runner + .withPropertyValues("app.messaging.enabled=false") + .run( + context -> { + assertThat(context).hasNotFailed(); - Set platformBeans = - java.util.Arrays.stream(context.getBeanDefinitionNames()) - .filter( - name -> { - Class type = context.getBeanFactory().getType(name, false); - return type != null - && type.getName().startsWith("dev.caskeleton.messaging."); - }) - .collect(Collectors.toSet()); + Set platformBeans = + java.util.Arrays.stream(context.getBeanDefinitionNames()) + .filter( + name -> { + Class type = context.getBeanFactory().getType(name, false); + return type != null + && type.getName().startsWith("dev.caskeleton.messaging."); + }) + .collect(Collectors.toSet()); - assertThat(platformBeans) - .as("off is a structural fact, not a set of individually-conditioned beans") - .isEmpty(); - }); + assertThat(platformBeans) + .as("off is a structural fact, not a set of individually-conditioned beans") + .isEmpty(); + }); } @Test @@ -79,7 +79,9 @@ class MessagingStarterOffContractTest { void offMeansNoThreads() { Set before = liveThreadNames(); - runner.run(context -> assertThat(context).hasNotFailed()); + runner + .withPropertyValues("app.messaging.enabled=false") + .run(context -> assertThat(context).hasNotFailed()); assertThat(newMessagingThreadsSince(before)) .as("a bean that exists but was never started is acceptable; a running thread is not") @@ -111,11 +113,11 @@ class MessagingStarterOffContractTest { .hasFailed() .getFailure() .rootCause() - .hasMessageContaining("registered: [kafka, rabbit]")); + .hasMessageContaining("registered: [kafka]")); } @Test - @DisplayName("selecting Kafka assembles Kafka and not Rabbit") + @DisplayName("selecting Kafka assembles Kafka") void selectingKafkaAssemblesOnlyKafka() { withAPublisher() .withPropertyValues( @@ -126,48 +128,9 @@ class MessagingStarterOffContractTest { context -> { assertThat(context).hasNotFailed(); assertThat(context).hasSingleBean(KafkaMessagingAutoConfiguration.class); - assertThat(context) - .as("both client libraries are present, so only the property can be selecting") - .doesNotHaveBean(RabbitMessagingAutoConfiguration.class); }); } - @Test - @DisplayName("selecting Rabbit is refused, and the message says the transport is not finished") - void selectingRabbitIsRefused() { - // This test used to run under `withAPublisher()` and assert the context started. The fake - // MessagePublisher tripped @ConditionalOnMissingBean and removed the very bean whose missing - // dependency is the defect — so a configuration that cannot start in any deployment passed as - // "assembles Rabbit and not Kafka". Without the fake, the real behaviour appears. - // - // Rabbit ships validators and a security configurer and no MessagingTransport, because its - // native channel publisher has only test implementations. That is refused at selection now, - // where the message can say so, rather than surfacing as "No qualifying bean of type - // MessagingTransport" from inside the core configuration. - runner - .withPropertyValues("app.messaging.enabled=true", "app.messaging.broker=rabbit") - .run( - context -> { - assertThat(context).hasFailed(); - assertThat(context.getStartupFailure()) - .rootCause() - .hasMessageContaining("app.messaging.broker=rabbit") - .hasMessageContaining("no MessagingTransport") - .hasMessageContaining("kafka"); - }); - } - - @Test - @DisplayName("a fake publisher does not make an unassemblable transport look assemblable") - void aFakePublisherDoesNotHideAnUnassemblableTransport() { - // The guard on the test above. Supplying a MessagePublisher removes the dependency that fails, - // so if the refusal ever moves back to bean wiring this case goes green while the deployment - // stays broken. Selection-time refusal is what makes it fail either way. - withAPublisher() - .withPropertyValues("app.messaging.enabled=true", "app.messaging.broker=rabbit") - .run(context -> assertThat(context).hasFailed()); - } - @Test @DisplayName("MSG-INT-003: a selected transport now assembles a publisher") void aSelectedTransportAssemblesAPublisher() { @@ -279,8 +242,6 @@ class MessagingStarterOffContractTest { name -> { String lower = name.toLowerCase(java.util.Locale.ROOT); return lower.contains("kafka") - || lower.contains("rabbit") - || lower.contains("amqp") || lower.contains("messaging") || lower.contains("outbox"); }) diff --git a/messaging/messaging-spring-boot-starter/src/test/resources/application-valid.yml b/messaging/messaging-spring-boot-starter/src/test/resources/application-valid.yml index 8101b6e..b1894f0 100644 --- a/messaging/messaging-spring-boot-starter/src/test/resources/application-valid.yml +++ b/messaging/messaging-spring-boot-starter/src/test/resources/application-valid.yml @@ -1,16 +1,6 @@ # A configuration the platform accepts. -# -# Every experimental adapter is off and the bridge is off, which is the shipped default: an -# adapter whose contract suite is still being proven must never become load-bearing because a -# configuration defaulted it on. backend: messaging: - experimental: - kafka-share: false - pulsar: false - nats: false - bridge: - spring-cloud-stream: false backpressure: # The per-destination limit stays below the global one; a per-destination limit at or above # the global limit would let one destination consume every slot in the process. diff --git a/messaging/messaging-spring-cloud-stream-bridge/build.gradle b/messaging/messaging-spring-cloud-stream-bridge/build.gradle deleted file mode 100644 index d104567..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/build.gradle +++ /dev/null @@ -1,10 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-transport-spi') - -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/gradle.lockfile b/messaging/messaging-spring-cloud-stream-bridge/gradle.lockfile deleted file mode 100644 index e2c9585..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/gradle.lockfile +++ /dev/null @@ -1,84 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty=compileClasspath,runtimeClasspath diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingCapabilityReport.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingCapabilityReport.java deleted file mode 100644 index fb34a1a..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingCapabilityReport.java +++ /dev/null @@ -1,88 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import java.util.List; -import java.util.Objects; - -/** - * States, per binding, which platform guarantees the bridge does not provide. - * - *

    An explicit report rather than silence. The binder does provide retry and dead-lettering of - * its own, so a binding looks like it has them; what it does not have is the platform's versions — - * bounded attempts under the destination's retry policy, and a dead-letter publish confirmed before - * the source is settled. An operator comparing a bridged binding to a native one needs that - * difference written down, because nothing at runtime will show it. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - * @param publishEvidenceAvailable whether the bridge can report broker publish evidence - * @param platformRetryApplied whether the destination's retry policy is enforced - * @param confirmedDeadLetterApplied whether dead-letter publishes are confirmed before settlement - * @param orderingEnforced whether the destination's ordering scope is enforced - */ -public record BindingCapabilityReport( - DestinationName destination, - String bindingName, - boolean publishEvidenceAvailable, - boolean platformRetryApplied, - boolean confirmedDeadLetterApplied, - boolean orderingEnforced) { - - public BindingCapabilityReport { - Objects.requireNonNull(destination, "destination must not be null"); - if (bindingName == null || bindingName.isBlank()) { - throw new IllegalArgumentException("bindingName must not be blank"); - } - } - - /** - * Returns the report for a binding served through the bridge. - * - *

    Every platform guarantee is false, because the bridge hands the message to the binder and - * the binder is what decides these. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - * @return the report - */ - public static BindingCapabilityReport bridged(DestinationName destination, String bindingName) { - return new BindingCapabilityReport(destination, bindingName, false, false, false, false); - } - - /** - * Returns the guarantees this binding does not carry, as sanitized text. - * - * @return the gaps, empty when nothing is missing - */ - public List gaps() { - List gaps = new java.util.ArrayList<>(); - if (!publishEvidenceAvailable) { - gaps.add( - "publish evidence: the binder reports a send, not a broker confirmation, so an " - + "ambiguous publish is indistinguishable from a confirmed one"); - } - if (!platformRetryApplied) { - gaps.add( - "retry: the binder's own retry runs instead of the destination's retry policy, with its " - + "own attempt budget and backoff"); - } - if (!confirmedDeadLetterApplied) { - gaps.add( - "dead letter: the binder settles the source without waiting for the dead-letter publish " - + "to confirm, so a dead-letter outage loses the message"); - } - if (!orderingEnforced) { - gaps.add("ordering: the binder's concurrency settings decide ordering, not the profile"); - } - return List.copyOf(gaps); - } - - /** - * Reports whether this binding carries every platform guarantee. - * - * @return true when nothing is missing - */ - public boolean isFullyGuaranteed() { - return gaps().isEmpty(); - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingProfileValidator.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingProfileValidator.java deleted file mode 100644 index cb3d159..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingProfileValidator.java +++ /dev/null @@ -1,97 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.util.Objects; -import java.util.regex.Pattern; - -/** - * Checks a binding against its destination before the bridge is wired. - * - *

    Runs on top of {@link StreamBridgePolicyGuard}: the guard decides whether a destination is - * eligible for the bridge at all, and this decides whether a particular binding is configured - * coherently for it. - * - *

    The binder's extended properties are the sharp edge. Stream lets a binding override the - * serializer, the acknowledgement mode, and the concurrency, and each of those silently replaces - * something the destination profile already decided. Rather than merging the two — which produces a - * configuration nobody can read — a conflicting extended property is rejected and the operator is - * told which side to remove. - */ -public final class BindingProfileValidator { - - private static final Pattern BINDING_NAME = Pattern.compile("[a-zA-Z][a-zA-Z0-9-]{0,63}"); - - /** Extended properties that would override a platform decision. */ - private static final java.util.Set CONFLICTING_PROPERTIES = - java.util.Set.of( - "autoBindDlq", - "republishToDlq", - "maxAttempts", - "backOffInitialInterval", - "autoCommitOffset", - "ackMode", - "useNativeEncoding", - "contentType"); - - private final StreamBridgePolicyGuard guard = new StreamBridgePolicyGuard(); - - /** - * Validates one binding. - * - * @param profile the destination profile - * @param bindingName the Stream binding name - * @param extendedProperties the binder extended properties configured for the binding - * @param enabled whether the bridge is switched on - * @return the capability report for the accepted binding - */ - public BindingCapabilityReport validate( - DestinationProfile profile, - String bindingName, - java.util.Map extendedProperties, - boolean enabled) { - Objects.requireNonNull(profile, "profile must not be null"); - Objects.requireNonNull(extendedProperties, "extendedProperties must not be null"); - - guard.validate(profile, enabled); - - if (bindingName == null || !BINDING_NAME.matcher(bindingName).matches()) { - throw new MessagingConfigurationException( - "INVALID_BINDING_NAME", - "binding name must match %s: %s".formatted(BINDING_NAME.pattern(), bindingName)); - } - - for (String property : extendedProperties.keySet()) { - if (CONFLICTING_PROPERTIES.contains(property)) { - throw new MessagingConfigurationException( - "BINDING_OVERRIDES_PLATFORM_POLICY", - "binding %s sets %s, which overrides what the %s destination profile already decides; " - .formatted(bindingName, property, profile.name().value()) - + "remove it from the binding or move the destination to the native adapter"); - } - } - - if (profile.production()) { - throw new MessagingConfigurationException( - "BRIDGE_ON_PRODUCTION_DESTINATION", - "destination %s is marked production; the bridge does not carry the platform's publish " - .formatted(profile.name().value()) - + "evidence, retry, or confirmed dead lettering"); - } - - return BindingCapabilityReport.bridged(profile.name(), bindingName); - } - - /** - * Returns the report for a destination that is not bridged at all. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - * @return a report stating every guarantee holds - */ - public static BindingCapabilityReport nativeAdapter( - DestinationName destination, String bindingName) { - return new BindingCapabilityReport(destination, bindingName, true, true, true, true); - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/MessagingBindingBridge.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/MessagingBindingBridge.java deleted file mode 100644 index 49be29e..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/MessagingBindingBridge.java +++ /dev/null @@ -1,34 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.destination.DestinationName; - -/** - * Connects a logical destination to a Spring Cloud Stream binding. - * - *

    The bridge is an interoperability seam, not a second messaging API. Its whole reason to exist - * is that a service already has Stream bindings and needs to reach the same destinations without a - * rewrite. - * - *

    Binder semantics are never promoted to platform guarantees. Stream's binder has its own retry, - * its own dead-letter, and its own acknowledgement mode, and they look enough like the platform's - * to be mistaken for them — so a destination that actually relies on the platform's versions is - * refused by {@link StreamBridgePolicyGuard} rather than served with the binder's. - */ -public interface MessagingBindingBridge { - - /** - * Binds a Stream output to a logical destination. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - */ - void bindPublisher(DestinationName destination, String bindingName); - - /** - * Binds a Stream input to a logical destination. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - */ - void bindConsumer(DestinationName destination, String bindingName); -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamConsumerBridge.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamConsumerBridge.java deleted file mode 100644 index e0b304f..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamConsumerBridge.java +++ /dev/null @@ -1,87 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import java.util.Map; -import java.util.Objects; -import java.util.concurrent.ConcurrentHashMap; - -/** - * Receives from a Spring Cloud Stream input binding and hands the message to a platform handler. - * - *

    Settlement stays with the binder. The bridge cannot acknowledge, retry, or dead-letter a - * message itself, because Stream's binder already owns the acknowledgement for that binding and two - * things settling one message is worse than either doing it alone. - * - *

    What the bridge does own is the translation and the honesty about it: a handler failure is - * rethrown so the binder's error channel sees it, rather than being converted into a platform - * {@code HandleResult} that nothing downstream would act on. - */ -public final class SpringCloudStreamConsumerBridge { - - private final Map handlers = new ConcurrentHashMap<>(); - private final Map destinations = new ConcurrentHashMap<>(); - - /** - * Registers a handler for a binding. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - * @param handler the handler to invoke - */ - public void register(DestinationName destination, String bindingName, BridgedHandler handler) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(handler, "handler must not be null"); - if (bindingName == null || bindingName.isBlank()) { - throw new IllegalArgumentException("bindingName must not be blank"); - } - handlers.put(bindingName, handler); - destinations.put(bindingName, destination); - } - - /** - * Dispatches one message received on a binding. - * - * @param bindingName the Stream binding name - * @param payload the encoded payload - * @param headers the received headers - * @throws MessagingConfigurationException when nothing is registered for the binding - */ - public void dispatch(String bindingName, byte[] payload, Map headers) { - Objects.requireNonNull(payload, "payload must not be null"); - Objects.requireNonNull(headers, "headers must not be null"); - - BridgedHandler handler = handlers.get(bindingName); - DestinationName destination = destinations.get(bindingName); - if (handler == null || destination == null) { - throw new MessagingConfigurationException( - "NO_BRIDGED_HANDLER", "no handler is registered for Stream binding " + bindingName); - } - // Not caught. The binder's error channel is what retries and dead-letters this binding, and - // swallowing the failure here would acknowledge a message nothing handled. - handler.handle(destination, payload, headers); - } - - /** - * Returns how many bindings have a handler. - * - * @return the registered binding count - */ - public int registeredBindings() { - return handlers.size(); - } - - /** A platform handler reached through a Stream binding. */ - @FunctionalInterface - public interface BridgedHandler { - - /** - * Handles one bridged message. - * - * @param destination the logical destination - * @param payload the encoded payload - * @param headers the received headers - */ - void handle(DestinationName destination, byte[] payload, Map headers); - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamPublisherBridge.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamPublisherBridge.java deleted file mode 100644 index 059144f..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamPublisherBridge.java +++ /dev/null @@ -1,156 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.time.Duration; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.ConcurrentHashMap; - -/** - * Framework-neutral publish evidence bridge for a future Spring Cloud Stream adapter. - * - *

    Despite the historical module/class name, this type does not itself depend on Spring Cloud - * Stream and does not auto-configure {@code StreamBridge}. A product integrating Spring Cloud - * Stream supplies the {@link ChannelSend} seam explicitly. Keeping that limitation in the type - * contract prevents this module from being mistaken for a shipped Spring Cloud Stream integration. - * - *

    Sends through a Spring Cloud Stream output binding and reports what that actually proves. - * - *

    The result is deliberately {@code AMBIGUOUS} rather than {@code CONFIRMED}. A Stream {@code - * send} returns a boolean from the message channel — it says the binder accepted the message, not - * that a broker did. Reporting that as confirmed would put the platform's strongest word on the - * binder's weakest evidence, and a caller reading {@code CONFIRMED} would stop worrying about a - * message that may never have left the process. - * - *

    A caller that needs real publish evidence has to use the native adapter. That is the honest - * trade the bridge exists to make visible. - */ -public final class SpringCloudStreamPublisherBridge implements MessagingBindingBridge { - - private final Map outputBindings = new ConcurrentHashMap<>(); - private final Map inputBindings = new ConcurrentHashMap<>(); - private final ChannelSend send; - - /** - * Creates a publisher bridge. - * - * @param send the Stream channel send operation - */ - public SpringCloudStreamPublisherBridge(ChannelSend send) { - this.send = Objects.requireNonNull(send, "send must not be null"); - } - - @Override - public void bindPublisher(DestinationName destination, String bindingName) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(bindingName, "bindingName must not be null"); - outputBindings.put(destination, bindingName); - } - - @Override - public void bindConsumer(DestinationName destination, String bindingName) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(bindingName, "bindingName must not be null"); - inputBindings.put(destination, bindingName); - } - - /** - * Publishes through the bound output binding. - * - * @param destination the logical destination - * @param payload the encoded payload - * @param headers the headers to carry - * @return the outcome, never stronger than the binder can prove - */ - public PublishResult publish( - DestinationName destination, byte[] payload, Map headers) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(payload, "payload must not be null"); - Objects.requireNonNull(headers, "headers must not be null"); - - String binding = outputBindings.get(destination); - if (binding == null) { - throw new MessagingConfigurationException( - "NO_OUTPUT_BINDING", "no Stream output binding is bound to " + destination.value()); - } - - boolean accepted = send.send(binding, payload, headers); - return accepted ? acceptedByBinder() : rejectedByBinder(); - } - - /** - * Returns the binding bound to a destination for consuming. - * - * @param destination the logical destination - * @return the binding name, when one is bound - */ - public Optional consumerBinding(DestinationName destination) { - return Optional.ofNullable(inputBindings.get(destination)); - } - - /** - * Returns the binding bound to a destination for publishing. - * - * @param destination the logical destination - * @return the binding name, when one is bound - */ - public Optional publisherBinding(DestinationName destination) { - return Optional.ofNullable(outputBindings.get(destination)); - } - - private static PublishResult acceptedByBinder() { - return new PublishResult( - // The binder accepted it; no broker has confirmed anything. AMBIGUOUS is the only honest - // answer, and it is what makes the bridge's weaker guarantee visible to the caller. - PublishCompletion.AMBIGUOUS, - new PublishEvidence( - true, TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of( - dev.caskeleton.messaging.api.error.FailureDescriptor.of( - dev.caskeleton.messaging.api.error.FailureCategory.AMBIGUOUS, - "STREAM_BRIDGE_NO_BROKER_EVIDENCE", - "the Stream binder accepted the message; no broker confirmation is available"))); - } - - private static PublishResult rejectedByBinder() { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of( - dev.caskeleton.messaging.api.error.FailureDescriptor.of( - dev.caskeleton.messaging.api.error.FailureCategory.TRANSIENT_INFRASTRUCTURE, - "STREAM_BRIDGE_SEND_REFUSED", - "the Stream channel refused the message"))); - } - - /** The Stream channel send, isolated so the bridge is testable without a binder. */ - @FunctionalInterface - public interface ChannelSend { - - /** - * Sends one message to a binding. - * - * @param bindingName the Stream binding name - * @param payload the encoded payload - * @param headers the headers to carry - * @return whether the channel accepted the message - */ - boolean send(String bindingName, byte[] payload, Map headers); - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/StreamBridgePolicyGuard.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/StreamBridgePolicyGuard.java deleted file mode 100644 index e676f35..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/StreamBridgePolicyGuard.java +++ /dev/null @@ -1,52 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.util.Objects; - -/** - * Keeps the optional Spring Cloud Stream bridge inside platform policy. - * - *

    The bridge exists for interoperability with existing Spring Cloud Stream bindings, and its - * risk is specific: Stream owns its own binder configuration, so a binding can quietly acquire its - * own serializer, its own error handling, and its own acknowledgement mode — none of which the - * destination profile knows about. - * - *

    So the bridge is only permitted where the platform's guarantees are not the thing being relied - * on: a destination that declares an ordering scope, a retry policy, or a dead letter destination - * must go through the native adapter, where those are actually enforced. - */ -public final class StreamBridgePolicyGuard { - - /** - * Validates that a destination may be served through the bridge. - * - * @param profile the destination profile - * @param enabled whether the bridge is switched on - */ - public void validate(DestinationProfile profile, boolean enabled) { - Objects.requireNonNull(profile, "profile must not be null"); - - if (!enabled) { - throw new MessagingConfigurationException( - "STREAM_BRIDGE_DISABLED", - "the Spring Cloud Stream bridge is optional and disabled unless " - + "backend.messaging.bridge.spring-cloud-stream=true"); - } - if (profile.isOrdered()) { - throw new MessagingConfigurationException( - "STREAM_BRIDGE_ORDERING_UNSUPPORTED", - "an ordered destination must use the native adapter: " + profile.name().value()); - } - if (profile.retry().mode() != dev.caskeleton.messaging.policy.RetryMode.NONE) { - throw new MessagingConfigurationException( - "STREAM_BRIDGE_RETRY_UNSUPPORTED", - "the bridge does not apply the platform retry policy: " + profile.name().value()); - } - if (profile.deadLetter().enabled()) { - throw new MessagingConfigurationException( - "STREAM_BRIDGE_DLQ_UNSUPPORTED", - "the bridge does not apply confirmed dead lettering: " + profile.name().value()); - } - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BindingProfileValidatorTest.java b/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BindingProfileValidatorTest.java deleted file mode 100644 index 1f5f403..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BindingProfileValidatorTest.java +++ /dev/null @@ -1,164 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; -import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.CapabilityTier; -import dev.caskeleton.messaging.policy.ConsumerPolicy; -import dev.caskeleton.messaging.policy.DeadLetterPolicy; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.PayloadPolicy; -import dev.caskeleton.messaging.policy.PhysicalDestination; -import dev.caskeleton.messaging.policy.ProducerPolicy; -import dev.caskeleton.messaging.policy.RetryPolicy; -import dev.caskeleton.messaging.policy.SchemaPolicy; -import dev.caskeleton.messaging.schema.SchemaCompatibility; -import java.util.Map; -import java.util.Set; -import org.junit.jupiter.api.Test; - -class BindingProfileValidatorTest { - - private final BindingProfileValidator validator = new BindingProfileValidator(); - - private static DestinationProfile profile( - OrderingScope ordering, RetryPolicy retry, DeadLetterPolicy deadLetter, boolean production) { - return new DestinationProfile( - new DestinationName("orders.v1"), - "kafka-primary", - DestinationKind.EVENT_STREAM, - PhysicalDestination.kafkaTopic("orders.v1"), - new SchemaPolicy( - ContentType.JSON, - SchemaCompatibility.BACKWARD, - Set.of(new MessageType("order.created"))), - DeliveryGuarantee.AT_LEAST_ONCE, - ordering, - ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, - ProducerPolicy.defaults(), - ConsumerPolicy.defaults("orders"), - retry, - deadLetter, - PayloadPolicy.defaults(), - CapabilityTier.M1, - production, - false, - false); - } - - private static DestinationProfile bridgeable() { - return profile(OrderingScope.NONE, RetryPolicy.none(), DeadLetterPolicy.disabled(), false); - } - - @Test - void aBridgeableDestinationIsAccepted() { - BindingCapabilityReport report = validator.validate(bridgeable(), "orders-out", Map.of(), true); - - assertThat(report.bindingName()).isEqualTo("orders-out"); - } - - @Test - void aDisabledBridgeRefusesEverything() { - assertThatThrownBy(() -> validator.validate(bridgeable(), "orders-out", Map.of(), false)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("disabled"); - } - - @Test - void anOrderedDestinationMustUseTheNativeAdapter() { - assertThatThrownBy( - () -> - validator.validate( - profile( - OrderingScope.KEY, RetryPolicy.none(), DeadLetterPolicy.disabled(), false), - "orders-out", - Map.of(), - true)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("ordered"); - } - - @Test - void aDestinationWithADeadLetterMustUseTheNativeAdapter() { - assertThatThrownBy( - () -> - validator.validate( - profile( - OrderingScope.NONE, - RetryPolicy.none(), - DeadLetterPolicy.to(new DestinationName("orders.v1.dlq")), - false), - "orders-out", - Map.of(), - true)) - .as("the binder settles the source without confirming the dead-letter publish") - .isInstanceOf(MessagingConfigurationException.class); - } - - @Test - void aProductionDestinationIsRefused() { - assertThatThrownBy( - () -> - validator.validate( - profile( - OrderingScope.NONE, RetryPolicy.none(), DeadLetterPolicy.disabled(), true), - "orders-out", - Map.of(), - true)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("production"); - } - - @Test - void aBinderPropertyThatOverridesPlatformPolicyIsRejectedRatherThanMerged() { - assertThatThrownBy( - () -> validator.validate(bridgeable(), "orders-out", Map.of("maxAttempts", "10"), true)) - .as("merging the two produces a configuration nobody can read") - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("maxAttempts"); - } - - @Test - void aHarmlessBinderPropertyIsAllowedThrough() { - assertThatCode( - () -> validator.validate(bridgeable(), "orders-out", Map.of("concurrency", "4"), true)) - .doesNotThrowAnyException(); - } - - @Test - void anInvalidBindingNameIsRejected() { - assertThatThrownBy(() -> validator.validate(bridgeable(), "9-bad name!", Map.of(), true)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("binding name"); - } - - @Test - void aBridgedBindingReportsEveryGuaranteeItDoesNotCarry() { - BindingCapabilityReport report = validator.validate(bridgeable(), "orders-out", Map.of(), true); - - assertThat(report.isFullyGuaranteed()).isFalse(); - assertThat(report.gaps()).hasSize(4); - assertThat(String.join(" ", report.gaps())) - .contains("publish evidence") - .contains("retry") - .contains("dead letter") - .contains("ordering"); - } - - @Test - void aNativeBindingReportsNoGaps() { - assertThat( - BindingProfileValidator.nativeAdapter(new DestinationName("orders.v1"), "orders-out") - .isFullyGuaranteed()) - .isTrue(); - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BridgePublishEvidenceTest.java b/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BridgePublishEvidenceTest.java deleted file mode 100644 index 411f04f..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BridgePublishEvidenceTest.java +++ /dev/null @@ -1,131 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; -import org.junit.jupiter.api.Test; - -class BridgePublishEvidenceTest { - - private static final DestinationName ORDERS = new DestinationName("orders.v1"); - private static final byte[] PAYLOAD = "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8); - - private static SpringCloudStreamPublisherBridge bridge(boolean channelAccepts) { - SpringCloudStreamPublisherBridge bridge = - new SpringCloudStreamPublisherBridge((binding, payload, headers) -> channelAccepts); - bridge.bindPublisher(ORDERS, "orders-out"); - return bridge; - } - - @Test - void anAcceptedSendIsAmbiguousBecauseNoBrokerConfirmedIt() { - PublishResult result = bridge(true).publish(ORDERS, PAYLOAD, Map.of()); - - assertThat(result.completion()) - .as("a channel send says the binder took it, not that a broker did") - .isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.evidence().brokerAccepted()).isFalse(); - assertThat(result.evidence().confirmationLevel()).isEqualTo(ConfirmationLevel.NONE); - } - - @Test - void anAcceptedSendReportsThatTransmissionIsUnknown() { - assertThat(bridge(true).publish(ORDERS, PAYLOAD, Map.of()).evidence().transmission()) - .isEqualTo(TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED); - } - - @Test - void theFailureDescriptorNamesTheMissingEvidence() { - assertThat(bridge(true).publish(ORDERS, PAYLOAD, Map.of()).failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("STREAM_BRIDGE_NO_BROKER_EVIDENCE")); - } - - @Test - void aRefusedChannelSendIsRejectedNotAmbiguous() { - PublishResult result = bridge(false).publish(ORDERS, PAYLOAD, Map.of()); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.NOT_TRANSMITTED); - } - - @Test - void publishingToAnUnboundDestinationFailsLoudly() { - SpringCloudStreamPublisherBridge bridge = - new SpringCloudStreamPublisherBridge((binding, payload, headers) -> true); - - assertThatThrownBy(() -> bridge.publish(ORDERS, PAYLOAD, Map.of())) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("orders.v1"); - } - - @Test - void thePayloadReachesTheBoundBinding() { - List bindings = new ArrayList<>(); - SpringCloudStreamPublisherBridge bridge = - new SpringCloudStreamPublisherBridge( - (binding, payload, headers) -> { - bindings.add(binding); - return true; - }); - bridge.bindPublisher(ORDERS, "orders-out"); - - bridge.publish(ORDERS, PAYLOAD, Map.of()); - - assertThat(bindings).containsExactly("orders-out"); - } - - @Test - void bindingsAreQueryableInBothDirections() { - SpringCloudStreamPublisherBridge bridge = bridge(true); - bridge.bindConsumer(ORDERS, "orders-in"); - - assertThat(bridge.publisherBinding(ORDERS)).hasValue("orders-out"); - assertThat(bridge.consumerBinding(ORDERS)).hasValue("orders-in"); - } - - @Test - void aHandlerFailureIsRethrownSoTheBindersErrorChannelSeesIt() { - SpringCloudStreamConsumerBridge consumer = new SpringCloudStreamConsumerBridge(); - consumer.register( - ORDERS, - "orders-in", - (destination, payload, headers) -> { - throw new IllegalStateException("handler failed"); - }); - - assertThatThrownBy(() -> consumer.dispatch("orders-in", PAYLOAD, Map.of())) - .as("swallowing it would acknowledge a message nothing handled") - .isInstanceOf(IllegalStateException.class); - } - - @Test - void dispatchingToAnUnregisteredBindingFailsLoudly() { - assertThatThrownBy( - () -> new SpringCloudStreamConsumerBridge().dispatch("orders-in", PAYLOAD, Map.of())) - .isInstanceOf(MessagingConfigurationException.class); - } - - @Test - void aRegisteredHandlerReceivesTheDestinationItWasBoundTo() { - List seen = new ArrayList<>(); - SpringCloudStreamConsumerBridge consumer = new SpringCloudStreamConsumerBridge(); - consumer.register( - ORDERS, "orders-in", (destination, payload, headers) -> seen.add(destination)); - - consumer.dispatch("orders-in", PAYLOAD, Map.of()); - - assertThat(seen).containsExactly(ORDERS); - assertThat(consumer.registeredBindings()).isEqualTo(1); - } -} diff --git a/messaging/messaging-testkit/build.gradle b/messaging/messaging-testkit/build.gradle index 559464f..a7492ce 100644 --- a/messaging/messaging-testkit/build.gradle +++ b/messaging/messaging-testkit/build.gradle @@ -1,5 +1,11 @@ plugins { - id 'ca.jmh-benchmarks' + id 'ca.platform-module' + id 'me.champeau.jmh' +} + +jmh { + jmhVersion = libs.versions.jmh.get() + includeTests = true } dependencies { diff --git a/messaging/messaging-testkit/gradle.lockfile b/messaging/messaging-testkit/gradle.lockfile index 98b1f9d..4334c11 100644 --- a/messaging/messaging-testkit/gradle.lockfile +++ b/messaging/messaging-testkit/gradle.lockfile @@ -35,12 +35,12 @@ io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,jmhAnnotation javax.inject:javax.inject:1=annotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs net.bytebuddy:byte-buddy:1.17.8=compileClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -net.sf.jopt-simple:jopt-simple:5.0.4=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath +net.sf.jopt-simple:jopt-simple:5.0.4=jmh,jmhCompileClasspath,jmhRuntimeClasspath net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs org.antlr:antlr4-runtime:4.13.2=checkstyle org.apache.bcel:bcel:6.12.0=spotbugs org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-math3:3.6.1=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath +org.apache.commons:commons-math3:3.6.1=jmh,jmhCompileClasspath,jmhRuntimeClasspath org.apache.commons:commons-text:1.15.0=spotbugs org.apache.commons:commons-text:1.3=checkstyle org.apache.httpcomponents:httpclient:4.5.13=checkstyle @@ -71,13 +71,16 @@ org.junit.platform:junit-platform-launcher:6.0.3=jmhRuntimeClasspath,testRuntime org.junit:junit-bom:6.0.3=compileClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.junit:junit-bom:6.1.0=spotbugs org.mockito:mockito-core:5.20.0=mockitoAgent -org.openjdk.jmh:jmh-core:1.37=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath -org.openjdk.jmh:jmh-generator-annprocess:1.37=jmhAnnotationProcessor +org.openjdk.jmh:jmh-core:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-asm:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-bytecode:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-reflection:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath org.opentest4j:opentest4j:1.3.0=compileClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.ow2.asm:asm-analysis:9.10.1=spotbugs org.ow2.asm:asm-commons:9.10.1=spotbugs org.ow2.asm:asm-tree:9.10.1=spotbugs org.ow2.asm:asm-util:9.10.1=spotbugs +org.ow2.asm:asm:9.0=jmh,jmhCompileClasspath,jmhRuntimeClasspath org.ow2.asm:asm:9.10.1=spotbugs org.pcollections:pcollections:4.0.1=annotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor org.reflections:reflections:0.10.2=checkstyle diff --git a/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java b/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java index 13d7486..42d4271 100644 --- a/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java +++ b/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java @@ -51,9 +51,9 @@ public final class CompatibilityMatrix { * Reports whether a lane has run this adapter against a real broker. * *

    Read from the evidence rather than declared. As a field it was a boolean an author set - * next to the tier, and RabbitMQ carried {@code true} while no fault scenario had ever been - * executed against it — the same shape as the coverage matrix that used to assert its own - * constants. + * next to the tier, and an Experimental adapter carried {@code true} while no fault scenario + * had ever been executed against it — the same shape as the coverage matrix that used to assert + * its own constants. * * @return true when {@link CertifiedEvidence} has live-broker evidence for this adapter */ @@ -78,36 +78,11 @@ public final class CompatibilityMatrix { private static final Map ENTRIES = Map.of( "messaging-kafka", - // 4.1 only. The declared 4.2/4.3 were never run: the fixtures pin apache/kafka:4.1.0 - // and - // the locked client is 4.1.1, so the matrix and the support document agreed with each - // other and with nothing that had executed. A version returns here when a lane runs - // against its image. - new Entry("messaging-kafka", List.of("4.1"), Tier.STABLE, true, true), - "messaging-rabbit", - // Experimental until a lane runs a fault scenario against a RabbitMQ container. The - // adapter passes the shared contract and its broker IT exercises the routable and - // unroutable happy paths, but Stable in this repository means fault evidence, and - // there is none: connection-refused, cut-after-write, confirm-timeout, - // settlement-lost and high-latency have never been executed here. - new Entry("messaging-rabbit", List.of("4.3"), Tier.EXPERIMENTAL, true, false), - "messaging-kafka-share-experimental", - new Entry( - "messaging-kafka-share-experimental", - List.of("4.2", "4.3"), - Tier.EXPERIMENTAL, - false, - false), - "messaging-pulsar-experimental", - new Entry( - "messaging-pulsar-experimental", - List.of("4.0", "4.2"), - Tier.EXPERIMENTAL, - false, - false), - "messaging-nats-experimental", - new Entry( - "messaging-nats-experimental", List.of("2.14"), Tier.EXPERIMENTAL, false, false)); + // 4.1 only. The declared 4.2/4.3 were never run: the fixtures pin apache/kafka:4.1.0 and + // the locked client is 4.1.1, so the matrix and the support document agreed with each + // other and with nothing that had executed. A version returns here when a lane runs + // against its image. + new Entry("messaging-kafka", List.of("4.1"), Tier.STABLE, true, true)); /** * Returns every certified adapter. diff --git a/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/MessagingAdapterHarness.java b/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/MessagingAdapterHarness.java index 4df6fa8..a17f10e 100644 --- a/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/MessagingAdapterHarness.java +++ b/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/MessagingAdapterHarness.java @@ -9,8 +9,8 @@ import java.util.concurrent.CompletionStage; * The seam each broker adapter implements to run the common contract. * *

    The harness exposes outcomes, not internals: what the platform decided, what reached the dead - * letter destination, and what is still unsettled. Kafka and RabbitMQ answer those three questions - * with completely different machinery, and the contract only passes if the answers match. + * letter destination, and what is still unsettled. Each broker answers those three questions with + * its own machinery, and the contract only passes if the answers match. */ public interface MessagingAdapterHarness extends AutoCloseable { diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CertifiedEvidenceTest.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CertifiedEvidenceTest.java index 0ed64f1..a8eacb4 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CertifiedEvidenceTest.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CertifiedEvidenceTest.java @@ -53,8 +53,8 @@ class CertifiedEvidenceTest { assertThat(CertifiedEvidence.knownGaps("messaging-kafka")) .as("a Kafka producer buffers before it learns a connection exists, so this stays unproven") .contains(NetworkFaultScenario.CONNECTION_REFUSED); - assertThat(CertifiedEvidence.knownGaps("messaging-rabbit")) - .as("no lane runs a fault scenario against RabbitMQ, so every scenario is a gap") + assertThat(CertifiedEvidence.knownGaps("messaging-artemis")) + .as("an adapter no lane has run has every scenario as a gap") .containsExactlyElementsOf(NetworkFaultScenario.all()); } diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java index f126796..7cccaf2 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java @@ -32,7 +32,6 @@ class CompatibilityMatrixTest { void noExperimentalAdapterIsEnabledByDefault() { assertThat(CompatibilityMatrix.entries().values()) .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.EXPERIMENTAL) - .isNotEmpty() .allSatisfy(entry -> assertThat(entry.enabledByDefault()).isFalse()); } @@ -41,7 +40,6 @@ class CompatibilityMatrixTest { assertThat(CompatibilityMatrix.entries().values()) .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) .extracting(CompatibilityMatrix.Entry::adapter) - .as("RabbitMQ passes the shared contract, but no fault scenario has been run against it") .containsExactly("messaging-kafka"); } diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java index adb286a..ff0b29b 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java @@ -48,19 +48,16 @@ class CrossBrokerContractSuite { assertThat(matrix.gapsFor("messaging-kafka")) .as("what the Kafka lanes have not exercised against a live broker") .isEqualTo(CertifiedEvidence.knownGaps("messaging-kafka")); - assertThat(matrix.gapsFor("messaging-rabbit")) - .as("what the Rabbit lanes have not exercised against a live broker") - .isEqualTo(CertifiedEvidence.knownGaps("messaging-rabbit")); } @Test void evidenceThatDisagreesWithItsScenarioIsNotCoverage() { BrokerCertificationEvidence contradicting = new BrokerCertificationEvidence( - "messaging-rabbit", - "rabbitmq:4.3-management", + "messaging-kafka", + "apache/kafka:4.1.0", NetworkFaultScenario.CONNECTION_CUT_AFTER_WRITE, - "RabbitBrokerIT#anUnroutablePublishIsRejectedEvenThoughTheExchangeConfirmedIt", + "SomeIT#someScenario", "REJECTED", "repository-local", java.time.Instant.parse("2026-08-15T00:00:00Z")); @@ -105,14 +102,6 @@ class CrossBrokerContractSuite { .isNotEqualTo(Coverage.LIVE_BROKER))); } - @Test - void anExperimentalAdapterIsAllowedGapsButTheyAreRecorded() { - assertThat(matrix.isComplete("messaging-pulsar-experimental")) - .as("the Experimental tier is a statement of fact, not a disclaimer") - .isFalse(); - assertThat(matrix.gapsFor("messaging-pulsar-experimental")).isNotEmpty(); - } - @Test void aFaultThatWasNeverRecordedReadsAsUncoveredRatherThanPassing() { assertThat(matrix.coverageOf("messaging-artemis", NetworkFaultScenario.CONFIRM_TIMEOUT)) diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java index 0b7471a..e97c8ac 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java @@ -135,10 +135,6 @@ class MessagingDocumentationContractTest { private static String brokerLabel(String adapter) { return switch (adapter) { case "messaging-kafka" -> "Kafka"; - case "messaging-rabbit" -> "RabbitMQ"; - case "messaging-pulsar-experimental" -> "Pulsar"; - case "messaging-nats-experimental" -> "NATS"; - case "messaging-kafka-share-experimental" -> "Kafka Share"; default -> adapter; }; } From 4d4cc35e6ccb4b338e4c8b7615492f557b92670d Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Fri, 25 Sep 2026 17:57:07 +0900 Subject: [PATCH 2/7] =?UTF-8?q?refactor:=20=EB=B0=B1=EC=97=94=EB=93=9C=20?= =?UTF-8?q?=EB=A6=AC=ED=8E=99=ED=86=A0=EB=A7=81=20=EC=A7=84=ED=96=89=20?= =?UTF-8?q?=EC=A4=91?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/messaging/configuration-reference.md | 37 +- docs/messaging/security.md | 28 +- .../kafka/KafkaCredentialCallbackHandler.java | 84 ++++ .../kafka/KafkaSecurityConfigurer.java | 172 +++++--- .../kafka/KafkaSecurityConfigurerTest.java | 111 +++-- .../security/BrokerCredentialProfile.java | 21 +- .../security/BrokerSecurityProfile.java | 27 +- .../messaging/security/BrokerTrustStore.java | 30 ++ .../security/CredentialProvider.java | 17 + .../messaging/security/CredentialRuntime.java | 14 + .../security/CredentialRuntimeRegistry.java | 3 +- .../KafkaMessagingAutoConfiguration.java | 5 +- ...ssagingLiveRoundTripQualificationTest.java | 4 +- ...ssagingSecuredBrokerQualificationTest.java | 396 ++++++++++++++++++ .../autoconfigure/BrokerSecuritySettings.java | 31 +- .../MessagingConfigurationCompiler.java | 35 +- .../MessagingCoreAutoConfiguration.java | 38 +- .../autoconfigure/BrokerAccessListTest.java | 103 +++++ .../KafkaProducerSecurityTest.java | 188 ++++++++- 19 files changed, 1205 insertions(+), 139 deletions(-) create mode 100644 messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaCredentialCallbackHandler.java create mode 100644 messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerTrustStore.java create mode 100644 messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java create mode 100644 messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java diff --git a/docs/messaging/configuration-reference.md b/docs/messaging/configuration-reference.md index d4dc00d..0e3c748 100644 --- a/docs/messaging/configuration-reference.md +++ b/docs/messaging/configuration-reference.md @@ -177,6 +177,8 @@ app: consumer: { type: SASL_SCRAM, credential-id: kafka-consumer } # admin은 application runtime에 설정하지 않는다 hostname-verification: true + # 사설 CA로 서명된 TLS 브로커면(tls-enabled: true일 때만): + # trust-store: { location: /etc/kafka/ca.p12, type: PKCS12, password-credential-id: kafka-truststore } access: publishable: [order-events] consumable: [] @@ -191,13 +193,36 @@ app: 연결 시점에 해석하므로, 설정 덤프나 힙 덤프에서 나오는 것은 이름뿐이다. producer와 consumer는 서로 다른 `credential-id`를 써야 하며, 같으면 부팅에 실패한다. +`SASL_SCRAM`·`USERNAME_PASSWORD`의 사용자 이름은 `CredentialProvider.username(credentialId)`가 +준다. credential id를 사용자 이름으로 보내지 않는다. 비밀번호와 사용자 이름은 JAAS 문자열에 들어가지 +않고, 연결이 인증할 때마다 provider에서 읽힌다. 그래서 provider가 만료 시각을 주는 credential은 +재기동 없이 rotation된다(`security.md`). + +`MUTUAL_TLS`는 클라이언트 인증서를 key store로 제출한다. `credential-id`는 key store 비밀번호를 +가리킨다. + +```text +producer: + type: MUTUAL_TLS + credential-id: kafka-producer-tls # key store 비밀번호 + key-store: { location: /etc/kafka/producer.p12, type: PKCS12 } +``` + +`trust-store`를 생략하면 JVM 기본 trust store를 쓴다. 사설 CA 브로커는 `trust-store.location` +(PKCS12·JKS·PEM)과, 잠긴 store면 `password-credential-id`를 준다. + Kafka producer는 이 항목을 실제 클라이언트 설정(`security.protocol`, `sasl.mechanism`, -`sasl.jaas.config`, `ssl.enabled.protocols=TLSv1.3,TLSv1.2`, hostname 검증)으로 옮긴다. 평문으로 -조용히 넘어가지 않도록 다음 경우 부팅에 실패한다: 브로커가 `tls-enabled`나 -`authentication-enabled`를 켰는데 `app.messaging.security.`가 없을 때, security 항목은 -있는데 `CredentialProvider` bean이 없을 때, 같은 브로커에 맞는 항목이 둘 이상일 때. -한계: truststore/keystore 설정 키가 없어 TLS는 JVM 기본 truststore를 쓰고, `MUTUAL_TLS`는 클라이언트 -인증서를 제출할 수 없다(미지원). +`sasl.jaas.config`, `sasl.client.callback.handler.class`, `ssl.truststore.*`, `ssl.keystore.*`, +`ssl.enabled.protocols=TLSv1.3,TLSv1.2`, hostname 검증)으로 옮긴다. 평문으로 조용히 넘어가지 +않도록 다음 경우 부팅에 실패한다: 브로커가 `tls-enabled`나 `authentication-enabled`를 켰는데 +`app.messaging.security.`가 없을 때, security 항목은 있는데 `CredentialProvider` bean이 없을 +때, 같은 브로커에 맞는 항목이 둘 이상일 때, provider가 SASL 사용자 이름을 주지 않을 때, +`MUTUAL_TLS`인데 `key-store`가 없거나 `tls-enabled`가 꺼져 있을 때, `trust-store`를 줬는데 +`tls-enabled`가 꺼져 있을 때, store 파일을 열 수 없을 때. + +`access.publishable`은 강제된다. security 항목이 있는 브로커의 destination은 이 목록에 있어야 +publish된다. 목록 밖 publish는 `REJECTED`(`PUBLISH_FORBIDDEN`)이고, 빈 목록이면 아무것도 publish되지 +않는다. ## Backpressure diff --git a/docs/messaging/security.md b/docs/messaging/security.md index c6dc392..a847205 100644 --- a/docs/messaging/security.md +++ b/docs/messaging/security.md @@ -36,17 +36,30 @@ heap dump나 설정 출력에서 사용 가능한 credential이 나오지 않는 `CredentialIds`는 `bearer `, `sk-`, `-----begin`, `eyJ` 같은 접두사를 거부한다. 참조가 들어갈 자리에 secret 자체를 붙여넣는 가장 흔한 사고를 막는다. +SASL(PLAIN·SCRAM) 사용자 이름도 `CredentialProvider.username(credentialId)`가 준다. credential id는 +저장소 안의 참조이지 broker가 아는 사용자 이름이 아니다. provider가 이름을 주지 않으면 기동에 +실패한다. + +key store·trust store는 경로만 설정에 둔다. 비밀번호는 credential id로 provider가 해석한다 +(`configuration-reference.md` 보안 절). + ## Rotation `CredentialRotationPlan.isDue()`는 만료 **전에** 참이 된다. broker가 연결을 거부하기 시작한 시점에는 이미 publish가 실패하고 consumer가 멈춰 있다. -rotation은 세대 교체다. `DefaultMessagingRuntimeRegistry.install()`이 새 세대를 원자적으로 -게시하고, 이전 세대는 마지막 lease가 닫힐 때까지 열려 있다가 닫힌다. -진행 중인 publish는 시작한 연결에서 confirm을 받는다. +Kafka SASL credential은 producer를 다시 만들지 않고 바뀐다. JAAS 설정에는 login module 이름만 +있고, `KafkaCredentialCallbackHandler`가 연결이 인증할 때마다 `CredentialRuntimeRegistry`에서 +사용자 이름과 비밀번호를 읽는다. registry는 provider가 준 만료 시각의 rotation lead(기본 30분) +안으로 들어오면 새로 가져오므로, 새 연결과 재인증(broker `connections.max.reauth.ms`)이 새 값을 +쓴다. 만료 시각을 주지 않는 provider의 credential은 캐시된 값을 계속 쓴다. -drain deadline이 이 대기를 제한한다. 없으면 lease 하나가 새면 폐기된 credential이 -무기한 열려 있고, rotation이 보안상 무의미해진다. +key store·trust store 비밀번호는 예외다. Kafka가 SSL context를 만들 때 한 번 읽으므로 바꾸려면 +producer를 다시 만들어야 한다(재기동). + +transport 세대 교체(`DefaultMessagingRuntimeRegistry.install()`)는 transport 자체를 바꿀 때 쓴다. +새 세대를 원자적으로 게시하고, 이전 세대는 마지막 lease가 닫힐 때까지 열려 있다가 닫힌다. +drain deadline이 이 대기를 제한한다. ## Header @@ -66,10 +79,13 @@ logical identity가 호출자 제어가 된다. ## ACL -`DestinationAccessValidator`가 broker ACL **이전에** 검사한다. +publisher가 broker ACL **이전에** 검사한다(`PUBLISH_FORBIDDEN`). broker ACL 거부는 애플리케이션 컨텍스트가 없는 연결 수준 오류로 도착하므로 "어느 모듈이 어디에 publish하려 했는가"가 조사 대상이 된다. +publish 가능한 destination은 선언된 destination이다. broker에 `app.messaging.security.` +항목이 있으면 그중 `access.publishable`에 적힌 것만 남는다(빈 목록이면 publish 불가). + ## 관측성 누출 `MessagingRedactor`는 denylist다. diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaCredentialCallbackHandler.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaCredentialCallbackHandler.java new file mode 100644 index 0000000..b503424 --- /dev/null +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaCredentialCallbackHandler.java @@ -0,0 +1,84 @@ +package dev.caskeleton.messaging.kafka; + +import dev.caskeleton.messaging.api.error.MessagingConfigurationException; +import dev.caskeleton.messaging.security.CredentialRuntime; +import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; +import java.time.Instant; +import java.util.Arrays; +import java.util.List; +import java.util.Map; +import javax.security.auth.callback.Callback; +import javax.security.auth.callback.NameCallback; +import javax.security.auth.callback.PasswordCallback; +import javax.security.auth.callback.UnsupportedCallbackException; +import javax.security.auth.login.AppConfigurationEntry; +import org.apache.kafka.common.security.auth.AuthenticateCallbackHandler; + +/** + * Answers the SASL client's user name and password from {@link CredentialRuntimeRegistry}, at each + * authentication. + * + *

    The JAAS entry used to carry both values, fixed when the producer was built: a rotated secret + * never reached a connection until the process restarted, and the user name was the credential id. + * Kafka asks this handler every time a connection authenticates — a new connection, or a + * re-authentication when the broker sets {@code connections.max.reauth.ms} — so each one reads the + * registry's current generation. + * + *

    Kafka instantiates this class itself, so the registry and the credential id arrive through the + * client configuration under {@link #REGISTRY} and {@link #CREDENTIAL_ID}. + */ +public final class KafkaCredentialCallbackHandler implements AuthenticateCallbackHandler { + + /** Client configuration key carrying the {@link CredentialRuntimeRegistry}. */ + public static final String REGISTRY = "caskeleton.messaging.credential.registry"; + + /** Client configuration key carrying the credential id this client authenticates with. */ + public static final String CREDENTIAL_ID = "caskeleton.messaging.credential.id"; + + private CredentialRuntimeRegistry registry; + + private String credentialId; + + /** Created by Kafka through reflection. */ + public KafkaCredentialCallbackHandler() {} + + @Override + public void configure( + Map configs, String saslMechanism, List jaasConfigEntries) { + if (!(configs.get(REGISTRY) instanceof CredentialRuntimeRegistry configured) + || !(configs.get(CREDENTIAL_ID) instanceof String id)) { + throw new MessagingConfigurationException( + "SASL_CALLBACK_NOT_CONFIGURED", + "the Kafka client names this callback handler but does not carry " + + REGISTRY + + " and " + + CREDENTIAL_ID); + } + this.registry = configured; + this.credentialId = id; + } + + @Override + public void handle(Callback[] callbacks) throws UnsupportedCallbackException { + CredentialRuntime credential = registry.resolve(credentialId, Instant.now()); + for (Callback callback : callbacks) { + if (callback instanceof NameCallback name) { + name.setName( + credential + .username() + .orElseThrow(() -> KafkaSecurityConfigurer.usernameMissing(credentialId))); + } else if (callback instanceof PasswordCallback password) { + char[] material = credential.material(); + // PasswordCallback keeps its own copy, so this one is wiped as soon as it is handed over. + password.setPassword(material); + Arrays.fill(material, '\0'); + } else { + // SCRAM asks for extensions and carries on without them when the handler declines. + throw new UnsupportedCallbackException(callback); + } + } + } + + @Override + public void close() {} +} diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurer.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurer.java index e9ca621..391ece3 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurer.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurer.java @@ -1,5 +1,6 @@ package dev.caskeleton.messaging.kafka; +import dev.caskeleton.messaging.api.error.MessagingConfigurationException; import dev.caskeleton.messaging.security.BrokerCredentialProfile; import dev.caskeleton.messaging.security.BrokerSecurityProfile; import dev.caskeleton.messaging.security.BrokerTlsPolicy; @@ -14,17 +15,18 @@ import java.util.Objects; /** * Builds the Kafka client security properties from a validated security profile. * - *

    Nothing here reads an environment variable or a property file. The credential arrives already - * resolved from {@link CredentialRuntimeRegistry}, which is what makes rotation possible: a client - * configured from a value read once at startup holds that value until the process restarts, so the - * rotation the credential store performs never reaches the broker connection. + *

    Nothing here reads an environment variable or a property file. Credentials resolve through + * {@link CredentialRuntimeRegistry}. * - *

    The JAAS config is assembled into a {@code String} because that is the shape Kafka's client - * configuration accepts. The configurer keeps no field reference after returning the map, but the - * returned immutable String contains the credential and cannot be zeroized; its lifetime is the - * Kafka/client configuration lifetime. The source {@link CredentialRuntime} material is still - * zeroizable, but converting it to JAAS necessarily weakens that property and must not be described - * as if the String never existed. + *

    SASL user names and passwords are not written into the configuration at all. The JAAS entry + * names the login module and nothing else, and {@link KafkaCredentialCallbackHandler} answers the + * user name and password from the registry each time a connection authenticates — so a credential + * the store rotates (one whose provider reports an expiry) reaches the next connection without a + * restart, and no immutable {@code String} holds the password for the client's lifetime. + * + *

    Key and trust store passwords are the exception: Kafka reads them once, when the client builds + * its SSL context, and accepts them only as configuration values. They are resolved here and live + * as long as the client does; rotating one means rebuilding the client. */ public final class KafkaSecurityConfigurer { @@ -37,6 +39,9 @@ public final class KafkaSecurityConfigurer { /** Kafka's key for the JAAS configuration. */ public static final String SASL_JAAS_CONFIG = "sasl.jaas.config"; + /** Kafka's key for the SASL client callback handler class. */ + public static final String SASL_CLIENT_CALLBACK_HANDLER = "sasl.client.callback.handler.class"; + /** Kafka's key for the endpoint identification algorithm. */ public static final String ENDPOINT_IDENTIFICATION = "ssl.endpoint.identification.algorithm"; @@ -78,6 +83,9 @@ public final class KafkaSecurityConfigurer { // Validate before resolving. A profile that will be refused should not cause a credential to be // fetched and briefly held in memory for nothing. tlsPolicy.validate(profile, enabledProtocols); + if (!profile.tlsEnabled()) { + requireNoTlsMaterial(profile, credential); + } Map properties = new LinkedHashMap<>(); properties.put(SECURITY_PROTOCOL, securityProtocol(profile, credential)); @@ -87,31 +95,51 @@ public final class KafkaSecurityConfigurer { // Kafka disables hostname verification by setting this to the empty string. The policy has // already refused that combination, so the value here is always the verifying one. properties.put(ENDPOINT_IDENTIFICATION, "https"); + profile + .trustStore() + .ifPresent( + store -> { + properties.put("ssl.truststore.location", store.location()); + properties.put("ssl.truststore.type", store.type()); + store + .passwordCredentialId() + .ifPresent(id -> properties.put("ssl.truststore.password", secret(id, now))); + }); } switch (credential) { - case BrokerCredentialProfile.SaslScram scram -> { - CredentialRuntime resolved = credentials.resolve(scram.credentialId(), now); - properties.put(SASL_MECHANISM, "SCRAM-SHA-512"); - properties.put(SASL_JAAS_CONFIG, scramJaas(scram.credentialId(), resolved)); - } + case BrokerCredentialProfile.SaslScram scram -> + properties.putAll( + sasl( + "SCRAM-SHA-512", + "org.apache.kafka.common.security.scram.ScramLoginModule", + scram.credentialId(), + now)); case BrokerCredentialProfile.OAuth2 oauth -> { // Refused rather than half-configured. Setting the mechanism name without a callback // handler produces a client that authenticates with nothing and fails at connect time, // which is later and harder to attribute than failing here. - throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( + throw new MessagingConfigurationException( "SASL_OAUTH_NOT_CONFIGURED", "OAuth bearer needs a login callback handler and a token endpoint; this platform" + " configures the mechanism name only, so the profile is refused rather than" + " producing a client that cannot authenticate: " + oauth.credentialId()); } - case BrokerCredentialProfile.MutualTls ignored -> properties.put(SASL_MECHANISM, "NONE"); - case BrokerCredentialProfile.UsernamePassword plain -> { - CredentialRuntime resolved = credentials.resolve(plain.credentialId(), now); - properties.put(SASL_MECHANISM, "PLAIN"); - properties.put(SASL_JAAS_CONFIG, plainJaas(plain.credentialId(), resolved)); + case BrokerCredentialProfile.MutualTls tls -> { + String password = secret(tls.credentialId(), now); + properties.put("ssl.keystore.location", tls.keyStoreLocation()); + properties.put("ssl.keystore.type", tls.keyStoreType()); + properties.put("ssl.keystore.password", password); + properties.put("ssl.key.password", password); } + case BrokerCredentialProfile.UsernamePassword plain -> + properties.putAll( + sasl( + "PLAIN", + "org.apache.kafka.common.security.plain.PlainLoginModule", + plain.credentialId(), + now)); case BrokerCredentialProfile.Nkey ignored -> throw new IllegalArgumentException( "NKey credentials are a NATS concept, not a Kafka one"); @@ -119,6 +147,62 @@ public final class KafkaSecurityConfigurer { return Map.copyOf(properties); } + /** + * The SASL properties: a JAAS entry naming only the login module, and the handler that answers. + * + *

    The credential is resolved once here so that a provider with no user name, or no secret, + * fails startup rather than the first connection. + */ + private Map sasl( + String mechanism, String loginModule, String credentialId, Instant now) { + if (credentials.resolve(credentialId, now).username().isEmpty()) { + throw usernameMissing(credentialId); + } + return Map.of( + SASL_MECHANISM, + mechanism, + SASL_JAAS_CONFIG, + loginModule + " required;", + SASL_CLIENT_CALLBACK_HANDLER, + KafkaCredentialCallbackHandler.class, + KafkaCredentialCallbackHandler.REGISTRY, + credentials, + KafkaCredentialCallbackHandler.CREDENTIAL_ID, + credentialId); + } + + private String secret(String credentialId, Instant now) { + CredentialRuntime resolved = credentials.resolve(credentialId, now); + return new String(resolved.material()); + } + + /** TLS material on a plaintext connection is a configuration that cannot mean what it says. */ + private static void requireNoTlsMaterial( + BrokerSecurityProfile profile, BrokerCredentialProfile credential) { + if (profile.trustStore().isPresent()) { + throw new MessagingConfigurationException( + "TRUST_STORE_REQUIRES_TLS", + "broker %s configures a trust store but not TLS; nothing would ever read it" + .formatted(profile.broker())); + } + if (credential instanceof BrokerCredentialProfile.MutualTls) { + throw new MessagingConfigurationException( + "MUTUAL_TLS_REQUIRES_TLS", + "broker %s authenticates with a client certificate but does not enable TLS, and a" + .formatted(profile.broker()) + + " certificate is only presented inside a TLS handshake"); + } + } + + static MessagingConfigurationException usernameMissing(String credentialId) { + return new MessagingConfigurationException( + "SASL_USERNAME_MISSING", + "credential " + + credentialId + + " authenticates with a user name and password, and CredentialProvider.username" + + " returned none; the credential id is a reference, not the broker's user name"); + } + private static String securityProtocol( BrokerSecurityProfile profile, BrokerCredentialProfile credential) { boolean sasl = !(credential instanceof BrokerCredentialProfile.MutualTls); @@ -127,50 +211,4 @@ public final class KafkaSecurityConfigurer { } return sasl ? "SASL_PLAINTEXT" : "PLAINTEXT"; } - - private static String scramJaas(String credentialId, CredentialRuntime resolved) { - return jaas("org.apache.kafka.common.security.scram.ScramLoginModule", credentialId, resolved); - } - - private static String plainJaas(String credentialId, CredentialRuntime resolved) { - return jaas("org.apache.kafka.common.security.plain.PlainLoginModule", credentialId, resolved); - } - - /** - * Builds a JAAS entry with both values escaped. - * - *

    The password was interpolated into a quoted string unescaped. A perfectly ordinary generated - * secret containing a quote or a backslash then produced a JAAS config that either fails to parse - * — an authentication error that names nothing useful — or, with a quote and a semicolon, - * terminates the option early and injects another one. A secret is attacker-influenced in exactly - * one sense that matters here: nobody reads it before it is used. - */ - private static String jaas(String loginModule, String credentialId, CredentialRuntime resolved) { - return loginModule - + " required username=\"" - + escapeJaas(credentialId) - + "\" password=\"" - + escapeJaas(new String(resolved.material())) - + "\";"; - } - - /** - * Escapes a JAAS quoted-string value. - * - *

    Backslash first, then quote — the other order double-escapes the backslash it just added. - * Control characters are refused rather than escaped: JAAS has no escape for them, and a secret - * containing one cannot be expressed in this format at all. - */ - private static String escapeJaas(String value) { - for (int index = 0; index < value.length(); index++) { - char character = value.charAt(index); - if (character < 0x20 || character == 0x7F) { - throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( - "JAAS_VALUE_UNREPRESENTABLE", - "a credential containing a control character cannot be expressed in a JAAS entry;" - + " configure the client with a callback handler instead"); - } - } - return value.replace("\\", "\\\\").replace("\"", "\\\""); - } } diff --git a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurerTest.java b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurerTest.java index e0aea61..5ce3ac2 100644 --- a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurerTest.java +++ b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurerTest.java @@ -2,80 +2,106 @@ package dev.caskeleton.messaging.kafka; import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.assertThatThrownBy; +import static org.assertj.core.api.Assertions.catchThrowable; import dev.caskeleton.messaging.api.error.MessagingConfigurationException; import dev.caskeleton.messaging.security.BrokerCredentialProfile; import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import dev.caskeleton.messaging.security.BrokerTlsPolicy; +import dev.caskeleton.messaging.security.BrokerTrustStore; import dev.caskeleton.messaging.security.CredentialProvider; import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; +import dev.caskeleton.messaging.security.DestinationAccessPolicy; import java.time.Duration; import java.time.Instant; +import java.util.List; import java.util.Map; import java.util.Optional; -import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; /** - * What reaches a JAAS entry, and what is refused outright. + * What reaches the Kafka client configuration, and what is refused outright. * - *

    The password was interpolated into a quoted string with no escaping. A generated secret - * containing a quote or a backslash produced a config that fails to parse — an authentication error - * naming nothing useful — and one containing a quote and a semicolon terminated the option early - * and injected another. + *

    The JAAS entry used to carry the credential id as the user name and the resolved secret as an + * immutable string, fixed for the life of the producer. It now names the login module and nothing + * else: the user name and password are answered by {@link KafkaCredentialCallbackHandler} at each + * authentication. */ class KafkaSecurityConfigurerTest { private static final Instant NOW = Instant.parse("2026-08-14T00:00:00Z"); - @Test - @DisplayName("a secret containing a quote and a backslash is escaped, not injected") - void aSecretWithQuotesIsEscaped() { - Map properties = configure("pa\"ss\\word;"); + private static final BrokerCredentialProfile SCRAM = + new BrokerCredentialProfile.SaslScram("kafka-app"); - String jaas = String.valueOf(properties.get("sasl.jaas.config")); - assertThat(jaas) - .as("an unescaped quote closes the option and everything after it becomes new options") - .contains("password=\"pa\\\"ss\\\\word;\";"); - assertThat(jaas) - .as("the raw form would close the password option and make the rest new options") - .doesNotContain("password=\"pa\""); + @Test + void theJaasEntryCarriesNeitherTheUserNameNorTheSecret() { + Map properties = configure(registry(Optional.of("svc-orders")), tls(SCRAM)); + + assertThat(properties.get("sasl.jaas.config")) + .isEqualTo("org.apache.kafka.common.security.scram.ScramLoginModule required;"); } @Test - @DisplayName("a secret containing a control character is refused") - void aSecretWithAControlCharacterIsRefused() { - assertThatThrownBy(() -> configure("pass\nword")) + void aSaslCredentialWhoseProviderNamesNoUserIsRefused() { + // given + CredentialRuntimeRegistry anonymous = registry(Optional.empty()); + + // when + Throwable thrown = catchThrowable(() -> configure(anonymous, tls(SCRAM))); + + // then + assertThat(thrown) .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("control character"); + .extracting(failure -> ((MessagingConfigurationException) failure).failure().code()) + .isEqualTo("SASL_USERNAME_MISSING"); + } + + @Test + void aTrustStoreOnABrokerWithoutTlsIsRefused() { + // given + BrokerSecurityProfile plaintext = + new BrokerSecurityProfile( + "kafka-primary", + false, + false, + true, + SCRAM, + SCRAM, + Optional.empty(), + DestinationAccessPolicy.denyAll(), + Optional.of(new BrokerTrustStore("/etc/kafka/ca.p12", "PKCS12", Optional.empty()))); + + // when + Throwable thrown = + catchThrowable(() -> configure(registry(Optional.of("svc-orders")), plaintext)); + + // then + assertThat(thrown) + .isInstanceOf(MessagingConfigurationException.class) + .extracting(failure -> ((MessagingConfigurationException) failure).failure().code()) + .isEqualTo("TRUST_STORE_REQUIRES_TLS"); } @Test - @DisplayName("an OAuth profile is refused rather than half-configured") void anOAuthProfileIsRefused() { - CredentialRuntimeRegistry credentials = registry("secret"); + CredentialRuntimeRegistry credentials = registry(Optional.of("svc-orders")); BrokerCredentialProfile oauth = new BrokerCredentialProfile.OAuth2("kafka-oauth"); - assertThatThrownBy( - () -> - new KafkaSecurityConfigurer( - credentials, new dev.caskeleton.messaging.security.BrokerTlsPolicy()) - .configure(profile(oauth), oauth, java.util.List.of("TLSv1.3"), NOW)) + // when & then + assertThatThrownBy(() -> configure(credentials, tls(oauth))) .as("naming the mechanism without a callback handler fails at connect time instead") .isInstanceOf(MessagingConfigurationException.class) .hasMessageContaining("callback handler"); } - private static Map configure(String secret) { - return new KafkaSecurityConfigurer( - registry(secret), new dev.caskeleton.messaging.security.BrokerTlsPolicy()) - .configure( - profile(new BrokerCredentialProfile.SaslScram("kafka-app")), - new BrokerCredentialProfile.SaslScram("kafka-app"), - java.util.List.of("TLSv1.3"), - NOW); + private static Map configure( + CredentialRuntimeRegistry credentials, BrokerSecurityProfile profile) { + return new KafkaSecurityConfigurer(credentials, new BrokerTlsPolicy()) + .configure(profile, profile.producer(), List.of("TLSv1.3"), NOW); } - private static BrokerSecurityProfile profile(BrokerCredentialProfile credential) { + private static BrokerSecurityProfile tls(BrokerCredentialProfile credential) { return new BrokerSecurityProfile( "kafka-primary", true, @@ -84,21 +110,26 @@ class KafkaSecurityConfigurerTest { credential, credential, Optional.empty(), - dev.caskeleton.messaging.security.DestinationAccessPolicy.denyAll()); + DestinationAccessPolicy.denyAll()); } - private static CredentialRuntimeRegistry registry(String secret) { + private static CredentialRuntimeRegistry registry(Optional username) { return new CredentialRuntimeRegistry( new CredentialProvider() { @Override public char[] resolve(String credentialId) { - return secret.toCharArray(); + return "secret".toCharArray(); } @Override public Optional expiresAt(String credentialId) { return Optional.of(NOW.plus(Duration.ofHours(1))); } + + @Override + public Optional username(String credentialId) { + return username; + } }, Duration.ofMinutes(5)); } diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerCredentialProfile.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerCredentialProfile.java index 6bd1f5b..87a4df6 100644 --- a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerCredentialProfile.java +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerCredentialProfile.java @@ -44,13 +44,28 @@ public sealed interface BrokerCredentialProfile } /** - * Mutual TLS authentication. + * Mutual TLS authentication: the client certificate in a key store, unlocked by the credential. * - * @param credentialId the credential id + *

    The key store was not part of the profile, so a {@code MUTUAL_TLS} credential configured a + * TLS connection with no client certificate — one every broker that asks for a certificate + * refuses. The path is not a secret and lives here; the password is, and resolves through the + * credential id like every other secret. + * + * @param credentialId the credential whose material unlocks the key store + * @param keyStoreLocation the key store holding the client certificate and private key + * @param keyStoreType the key store format, such as {@code PKCS12} */ - record MutualTls(String credentialId) implements BrokerCredentialProfile { + record MutualTls(String credentialId, String keyStoreLocation, String keyStoreType) + implements BrokerCredentialProfile { public MutualTls { CredentialIds.require(credentialId); + if (keyStoreLocation == null || keyStoreLocation.isBlank()) { + throw new IllegalArgumentException( + "mutual TLS presents a client certificate, and without a key store there is none"); + } + if (keyStoreType == null || keyStoreType.isBlank()) { + throw new IllegalArgumentException("keyStoreType must not be blank"); + } } } diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerSecurityProfile.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerSecurityProfile.java index ab59d9a..40d386f 100644 --- a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerSecurityProfile.java +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerSecurityProfile.java @@ -18,6 +18,7 @@ import java.util.Optional; * @param consumer the consumer credential * @param admin the admin credential, absent in an application runtime * @param access the destination access policy + * @param trustStore the certificates the connection trusts; empty means the JVM default trust store */ public record BrokerSecurityProfile( String broker, @@ -27,15 +28,39 @@ public record BrokerSecurityProfile( BrokerCredentialProfile producer, BrokerCredentialProfile consumer, Optional admin, - DestinationAccessPolicy access) { + DestinationAccessPolicy access, + Optional trustStore) { public BrokerSecurityProfile { Objects.requireNonNull(producer, "producer credential must not be null"); Objects.requireNonNull(consumer, "consumer credential must not be null"); Objects.requireNonNull(admin, "admin credential must not be null"); Objects.requireNonNull(access, "access policy must not be null"); + Objects.requireNonNull(trustStore, "trust store must not be null"); if (broker == null || broker.isBlank()) { throw new IllegalArgumentException("broker must not be blank"); } } + + /** A profile that trusts what the JVM's default trust store trusts. */ + public BrokerSecurityProfile( + String broker, + boolean production, + boolean tlsEnabled, + boolean hostnameVerification, + BrokerCredentialProfile producer, + BrokerCredentialProfile consumer, + Optional admin, + DestinationAccessPolicy access) { + this( + broker, + production, + tlsEnabled, + hostnameVerification, + producer, + consumer, + admin, + access, + Optional.empty()); + } } diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerTrustStore.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerTrustStore.java new file mode 100644 index 0000000..b5b7db6 --- /dev/null +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerTrustStore.java @@ -0,0 +1,30 @@ +package dev.caskeleton.messaging.security; + +import java.util.Objects; +import java.util.Optional; + +/** + * The certificates a broker connection trusts, when the JVM's default trust store is not enough. + * + *

    A broker signed by a private CA could not be reached at all: there was no setting for a trust + * store, so the client verified the broker against the JVM defaults and refused it. The path is not + * a secret; the store password, when the format has one, resolves through a credential id. + * + * @param location the trust store file + * @param type the trust store format, such as {@code PKCS12} or {@code PEM} + * @param passwordCredentialId the credential whose material unlocks the store, when it is locked + */ +public record BrokerTrustStore( + String location, String type, Optional passwordCredentialId) { + + public BrokerTrustStore { + Objects.requireNonNull(passwordCredentialId, "passwordCredentialId must not be null"); + if (location == null || location.isBlank()) { + throw new IllegalArgumentException("a trust store needs a location"); + } + if (type == null || type.isBlank()) { + throw new IllegalArgumentException("a trust store needs a type"); + } + passwordCredentialId.ifPresent(CredentialIds::require); + } +} diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialProvider.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialProvider.java index 60836f6..24df053 100644 --- a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialProvider.java +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialProvider.java @@ -26,4 +26,21 @@ public interface CredentialProvider { * @return the expiry instant when known */ Optional expiresAt(String credentialId); + + /** + * Returns the user name the secret authenticates, for mechanisms that send one. + * + *

    SASL PLAIN and SCRAM send a user name and a password. The credential id is a reference into + * the store, not the principal the broker knows, and sending it as the user name authenticated as + * whoever happened to share the reference's spelling. A store that rotates by alternating between + * two users rotates the name together with the secret, so the name is resolved alongside it. + * + *

    Empty by default. A SASL credential whose provider names no user is refused at startup. + * + * @param credentialId the identifier + * @return the user name, when this credential has one + */ + default Optional username(String credentialId) { + return Optional.empty(); + } } diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntime.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntime.java index aefc5bc..5801728 100644 --- a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntime.java +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntime.java @@ -26,12 +26,14 @@ public final class CredentialRuntime { private final Instant resolvedAt; private final Optional expiresAt; private final Duration rotationLead; + private final Optional username; private char[] material; /** * Creates a runtime entry. * * @param credentialId the credential reference + * @param username the user name the secret authenticates, when the mechanism sends one * @param material the resolved secret, taken by reference and cleared by {@link #clear()} * @param resolvedAt when the material was resolved * @param expiresAt when it stops being valid, when known @@ -39,11 +41,13 @@ public final class CredentialRuntime { */ public CredentialRuntime( String credentialId, + Optional username, char[] material, Instant resolvedAt, Optional expiresAt, Duration rotationLead) { CredentialIds.require(credentialId); + Objects.requireNonNull(username, "username must not be null"); Objects.requireNonNull(material, "material must not be null"); Objects.requireNonNull(resolvedAt, "resolvedAt must not be null"); Objects.requireNonNull(expiresAt, "expiresAt must not be null"); @@ -55,6 +59,7 @@ public final class CredentialRuntime { throw new IllegalArgumentException("credential material must not be empty"); } this.credentialId = credentialId; + this.username = username; this.material = material; this.resolvedAt = resolvedAt; this.expiresAt = expiresAt; @@ -70,6 +75,15 @@ public final class CredentialRuntime { return credentialId; } + /** + * Returns the user name this secret authenticates, when the mechanism sends one. + * + * @return the user name + */ + public Optional username() { + return username; + } + /** * Returns a copy of the material. * diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntimeRegistry.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntimeRegistry.java index ef7e898..35a1e77 100644 --- a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntimeRegistry.java +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntimeRegistry.java @@ -134,6 +134,7 @@ public final class CredentialRuntimeRegistry { private CredentialRuntime fetch(String credentialId, Instant now) { char[] material = provider.resolve(credentialId); Optional expiresAt = provider.expiresAt(credentialId); - return new CredentialRuntime(credentialId, material, now, expiresAt, rotationLead); + return new CredentialRuntime( + credentialId, provider.username(credentialId), material, now, expiresAt, rotationLead); } } diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java index bec88a6..e4eea40 100644 --- a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java @@ -245,8 +245,9 @@ public class KafkaMessagingAutoConfiguration { @ConditionalOnMissingBean(dev.caskeleton.messaging.transport.MessagingTransport.class) public dev.caskeleton.messaging.kafka.KafkaMessagingTransport messagingKafkaTransport( org.apache.kafka.clients.producer.Producer producer) { - // Generation 1: the credential generation a rotation increments. Starting elsewhere would make - // the first rotation look like it had already happened. + // Generation 1 for the life of the producer. A SASL credential rotation does not replace the + // producer: KafkaCredentialCallbackHandler reads the rotated secret at the next authentication, + // so there is no second transport generation to install. return new dev.caskeleton.messaging.kafka.KafkaMessagingTransport("kafka", 1L, producer); } diff --git a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java index 8fdc0a2..4edfe2b 100644 --- a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java +++ b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java @@ -169,7 +169,7 @@ class MessagingLiveRoundTripQualificationTest { * same constant: a round trip that read a different topic from the one the transport wrote would * pass or fail for reasons unrelated to the platform. */ - private static dev.caskeleton.messaging.policy.DestinationProfile profile( + static dev.caskeleton.messaging.policy.DestinationProfile profile( DestinationName name, DestinationName deadLetter) { return new dev.caskeleton.messaging.policy.DestinationProfile( name, @@ -198,7 +198,7 @@ class MessagingLiveRoundTripQualificationTest { false); } - private static MessageEnvelope envelope() { + static MessageEnvelope envelope() { return new MessageEnvelope<>( MessageId.newId(), new MessageType("order.created"), diff --git a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java new file mode 100644 index 0000000..5d34659 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java @@ -0,0 +1,396 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.publish.MessagePublisher; +import dev.caskeleton.messaging.api.publish.PublishCompletion; +import dev.caskeleton.messaging.api.publish.PublishOptions; +import dev.caskeleton.messaging.api.publish.PublishResult; +import dev.caskeleton.messaging.security.CredentialProvider; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Path; +import java.time.Instant; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.UUID; +import java.util.concurrent.TimeUnit; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.condition.EnabledIf; +import org.junit.jupiter.api.io.TempDir; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; +import org.testcontainers.DockerClientFactory; +import org.testcontainers.kafka.KafkaContainer; +import org.testcontainers.utility.MountableFile; + +/** + * A publish reaches a broker that trusts nobody by default (Review Focus 5, M-2e/NEW-4b, M-2d). + * + *

    Both brokers present a certificate signed by a CA generated for this test, which no JVM trusts + * — so a publish is confirmed only if the configured trust store reached the client. The first + * broker then requires SASL PLAIN, and accepts only the user name the credential provider returns; + * the second requires a client certificate, which only the configured key store can present. + * + *

    No secret is written in this file: the store and broker passwords are generated per run. + */ +@EnabledIf(value = "dockerAvailable", disabledReason = "Docker not available for the live broker") +class MessagingSecuredBrokerQualificationTest { + + private static final String IMAGE = "apache/kafka:4.1.0"; + + private static final String BROKER_USER = "svc-orders"; + + private static final String BROKER_STORE = "/etc/kafka/secrets/broker.p12"; + + private static final String BROKER_TRUST = "/etc/kafka/secrets/truststore.p12"; + + static boolean dockerAvailable() { + return DockerClientFactory.instance().isDockerAvailable(); + } + + @Test + void aPublishIsConfirmedByAPrivateCaBrokerThatRequiresSaslAsTheProviderUser( + @TempDir Path directory) { + // given + PrivateCa ca = new PrivateCa(directory); + String userPassword = UUID.randomUUID().toString(); + try (KafkaContainer kafka = + broker(ca, "SASL_SSL") + .withEnv("KAFKA_SASL_ENABLED_MECHANISMS", "PLAIN") + .withEnv( + "KAFKA_LISTENER_NAME_PLAINTEXT_PLAIN_SASL_JAAS_CONFIG", + "org.apache.kafka.common.security.plain.PlainLoginModule required user_" + + BROKER_USER + + "=\"" + + userPassword + + "\";")) { + kafka.start(); + ApplicationContextRunner runner = + platform(kafka, ca, Map.of("orders-producer", userPassword, "orders-consumer", "unused")) + .withPropertyValues( + "app.messaging.security.kafka.producer.type=USERNAME_PASSWORD", + "app.messaging.security.kafka.producer.credential-id=orders-producer", + "app.messaging.security.kafka.consumer.type=USERNAME_PASSWORD", + "app.messaging.security.kafka.consumer.credential-id=orders-consumer"); + + // when + List results = new ArrayList<>(); + runner.run(context -> results.add(publish(context.getBean(MessagePublisher.class)))); + + // then + assertThat(results) + .extracting(PublishResult::completion) + .containsExactly(PublishCompletion.CONFIRMED); + } + } + + @Test + void aPublishIsConfirmedByAPrivateCaBrokerThatRequiresAClientCertificate( + @TempDir Path directory) { + // given + PrivateCa ca = new PrivateCa(directory); + Path clientStore = ca.issue("orders-producer", "dns:orders-producer"); + try (KafkaContainer kafka = + broker(ca, "SSL") + .withEnv("KAFKA_SSL_CLIENT_AUTH", "required") + .withEnv("KAFKA_SSL_TRUSTSTORE_LOCATION", BROKER_TRUST) + .withEnv("KAFKA_SSL_TRUSTSTORE_TYPE", "PKCS12") + .withEnv("KAFKA_SSL_TRUSTSTORE_PASSWORD", ca.password) + .withCopyToContainer(MountableFile.forHostPath(ca.trustStore, 0644), BROKER_TRUST)) { + kafka.start(); + ApplicationContextRunner runner = + platform( + kafka, + ca, + Map.of("orders-producer-tls", ca.password, "orders-consumer-tls", ca.password)) + .withPropertyValues( + "app.messaging.security.kafka.producer.type=MUTUAL_TLS", + "app.messaging.security.kafka.producer.credential-id=orders-producer-tls", + "app.messaging.security.kafka.producer.key-store.location=" + clientStore, + "app.messaging.security.kafka.consumer.type=MUTUAL_TLS", + "app.messaging.security.kafka.consumer.credential-id=orders-consumer-tls", + "app.messaging.security.kafka.consumer.key-store.location=" + clientStore); + + // when + List results = new ArrayList<>(); + runner.run(context -> results.add(publish(context.getBean(MessagePublisher.class)))); + + // then + assertThat(results) + .extracting(PublishResult::completion) + .containsExactly(PublishCompletion.CONFIRMED); + } + } + + /** + * The Testcontainers broker with its external listener switched to a secured protocol. + * + *

    The container names its host-facing listener {@code PLAINTEXT} and advertises it by that + * name; only the protocol that name maps to changes here. + */ + @SuppressWarnings("resource") + private static KafkaContainer broker(PrivateCa ca, String protocol) { + Path brokerStore = ca.issue("broker", ca.brokerSubjectAlternativeNames()); + return new KafkaContainer(IMAGE) + .withEnv( + "KAFKA_LISTENER_SECURITY_PROTOCOL_MAP", + "BROKER:PLAINTEXT,PLAINTEXT:" + protocol + ",CONTROLLER:PLAINTEXT") + .withEnv("KAFKA_SSL_KEYSTORE_LOCATION", BROKER_STORE) + .withEnv("KAFKA_SSL_KEYSTORE_TYPE", "PKCS12") + .withEnv("KAFKA_SSL_KEYSTORE_PASSWORD", ca.password) + .withEnv("KAFKA_SSL_KEY_PASSWORD", ca.password) + .withCopyToContainer(MountableFile.forHostPath(brokerStore, 0644), BROKER_STORE); + } + + /** The assembled platform, pointed at the broker through the private CA's trust store. */ + private static ApplicationContextRunner platform( + KafkaContainer kafka, PrivateCa ca, Map secrets) { + Map all = new java.util.HashMap<>(secrets); + all.put("kafka-trust", ca.password); + return new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "spring.kafka.bootstrap-servers=" + kafka.getBootstrapServers(), + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=" + kafka.getBootstrapServers(), + "app.messaging.brokers.kafka.tls-enabled=true", + "app.messaging.brokers.kafka.authentication-enabled=true", + "app.messaging.security.kafka.trust-store.location=" + ca.trustStore, + "app.messaging.security.kafka.trust-store.password-credential-id=kafka-trust", + "app.messaging.security.kafka.access.publishable[0]=order-events") + .withBean(CredentialProvider.class, () -> provider(all)) + .withBean( + MessageContracts.class, + () -> + new MessageContracts( + Map.of( + new dev.caskeleton.messaging.schema.MessageContractKey( + new MessageType("order.created"), new SchemaVersion(1)), + String.class))) + .withBean( + "ordersProfile", + dev.caskeleton.messaging.policy.DestinationProfile.class, + () -> + MessagingLiveRoundTripQualificationTest.profile( + new DestinationName("order-events"), new DestinationName("order-events-dlq"))) + .withBean( + "ordersDlqProfile", + dev.caskeleton.messaging.policy.DestinationProfile.class, + () -> + MessagingLiveRoundTripQualificationTest.profile( + new DestinationName("order-events-dlq"), null)); + } + + private static PublishResult publish(MessagePublisher publisher) { + return publisher + .publish( + new MessageDestination<>( + new DestinationName("order-events"), + new MessageType("order.created"), + String.class), + MessagingLiveRoundTripQualificationTest.envelope(), + PublishOptions.defaults()) + .toCompletableFuture() + .join(); + } + + private static CredentialProvider provider(Map secrets) { + return new CredentialProvider() { + @Override + public char[] resolve(String credentialId) { + return secrets.get(credentialId).toCharArray(); + } + + @Override + public Optional expiresAt(String credentialId) { + return Optional.empty(); + } + + @Override + public Optional username(String credentialId) { + return Optional.of(BROKER_USER); + } + }; + } + + /** + * A certificate authority made for one test run with the JDK's {@code keytool}. + * + *

    Every store shares one generated password, which is also what the credential provider hands + * the client for the trust store and the client key store. + */ + private static final class PrivateCa { + + private final Path directory; + + private final String password = UUID.randomUUID().toString(); + + private final Path authority; + + private final Path certificate; + + private final Path trustStore; + + private PrivateCa(Path directory) { + this.directory = directory; + this.authority = directory.resolve("ca.p12"); + this.certificate = directory.resolve("ca.pem"); + this.trustStore = directory.resolve("truststore.p12"); + keytool( + "-genkeypair", + "-alias", + "ca", + "-keyalg", + "RSA", + "-keysize", + "2048", + "-validity", + "2", + "-dname", + "CN=messaging-test-ca", + "-ext", + "bc:c", + "-keystore", + authority.toString(), + "-storetype", + "PKCS12", + "-storepass", + password); + keytool( + "-exportcert", + "-rfc", + "-alias", + "ca", + "-keystore", + authority.toString(), + "-storepass", + password, + "-file", + certificate.toString()); + keytool( + "-importcert", + "-noprompt", + "-alias", + "ca", + "-file", + certificate.toString(), + "-keystore", + trustStore.toString(), + "-storetype", + "PKCS12", + "-storepass", + password); + } + + /** The names the client may reach the broker by, so hostname verification can pass. */ + private String brokerSubjectAlternativeNames() { + String host = DockerClientFactory.instance().dockerHostIpAddress(); + String dockerHost = + host.chars().allMatch(c -> Character.isDigit(c) || c == '.') + ? "ip:" + host + : "dns:" + host; + return "dns:localhost,ip:127.0.0.1," + dockerHost; + } + + /** Issues a key store holding a CA-signed certificate for {@code name}. */ + private Path issue(String name, String subjectAlternativeNames) { + Path store = directory.resolve(name + ".p12"); + Path request = directory.resolve(name + ".csr"); + Path signed = directory.resolve(name + ".pem"); + keytool( + "-genkeypair", + "-alias", + name, + "-keyalg", + "RSA", + "-keysize", + "2048", + "-validity", + "2", + "-dname", + "CN=" + name, + "-keystore", + store.toString(), + "-storetype", + "PKCS12", + "-storepass", + password); + keytool( + "-certreq", + "-alias", + name, + "-keystore", + store.toString(), + "-storepass", + password, + "-file", + request.toString()); + keytool( + "-gencert", + "-rfc", + "-alias", + "ca", + "-keystore", + authority.toString(), + "-storepass", + password, + "-infile", + request.toString(), + "-outfile", + signed.toString(), + "-validity", + "2", + "-ext", + "SAN=" + subjectAlternativeNames); + keytool( + "-importcert", + "-noprompt", + "-alias", + "ca", + "-file", + certificate.toString(), + "-keystore", + store.toString(), + "-storepass", + password); + keytool( + "-importcert", + "-alias", + name, + "-file", + signed.toString(), + "-keystore", + store.toString(), + "-storepass", + password); + return store; + } + + private static void keytool(String... arguments) { + List command = new ArrayList<>(); + command.add(Path.of(System.getProperty("java.home"), "bin", "keytool").toString()); + command.addAll(List.of(arguments)); + try { + Process process = new ProcessBuilder(command).redirectErrorStream(true).start(); + String output = new String(process.getInputStream().readAllBytes(), StandardCharsets.UTF_8); + if (!process.waitFor(60, TimeUnit.SECONDS) || process.exitValue() != 0) { + throw new IllegalStateException("keytool " + arguments[0] + " failed: " + output); + } + } catch (IOException failure) { + throw new IllegalStateException("keytool could not run", failure); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + throw new IllegalStateException("keytool was interrupted", interrupted); + } + } + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSecuritySettings.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSecuritySettings.java index 0b629b1..ddc56f8 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSecuritySettings.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSecuritySettings.java @@ -8,7 +8,9 @@ import org.springframework.boot.context.properties.bind.DefaultValue; * *

    No field here holds a secret, and that is a structural rule rather than a convention: the * platform stores a credential id and resolves the material through a {@code CredentialProvider} at - * connect time, so a configuration dump or a heap dump yields a name and nothing usable. + * connect time, so a configuration dump or a heap dump yields a name and nothing usable. Key and + * trust store passwords follow the same rule — the stores are named by path, their passwords by + * credential id. * *

    The admin credential is bindable and its presence on a production broker is a startup failure. * Leaving it unbindable would have been the safer-looking choice and the wrong one: an operator who @@ -19,6 +21,7 @@ import org.springframework.boot.context.properties.bind.DefaultValue; * @param consumer the consumer credential * @param admin the admin credential, which an application runtime must not hold in production * @param hostnameVerification whether the server certificate hostname is verified + * @param trustStore the certificates the connection trusts; absent means the JVM default store * @param access which destinations each role may touch */ public record BrokerSecuritySettings( @@ -26,6 +29,7 @@ public record BrokerSecuritySettings( Credential consumer, Credential admin, @DefaultValue("true") boolean hostnameVerification, + TrustStore trustStore, @DefaultValue Access access) { /** @@ -33,12 +37,35 @@ public record BrokerSecuritySettings( * * @param type the authentication mechanism * @param credentialId the identifier the credential provider resolves + * @param keyStore the client certificate, for {@code MUTUAL_TLS} and nothing else */ - public record Credential(String type, String credentialId) {} + public record Credential(String type, String credentialId, KeyStore keyStore) {} + + /** + * A client certificate store, unlocked by the credential it belongs to. + * + * @param location the key store file + * @param type the key store format + */ + public record KeyStore(String location, @DefaultValue("PKCS12") String type) {} + + /** + * The certificates the connection trusts. + * + * @param location the trust store file + * @param type the trust store format + * @param passwordCredentialId the credential that unlocks the store, when it is locked + */ + public record TrustStore( + String location, @DefaultValue("PKCS12") String type, String passwordCredentialId) {} /** * Which destinations each role may touch. * + *

    {@code publishable} is enforced: once a broker has a security entry, the platform publishes + * only to the declared destinations of that broker that this list names. An empty list therefore + * publishes nothing. + * * @param publishable destinations the producer credential may publish to * @param consumable destinations the consumer credential may consume * @param administrable destinations the admin credential may operate on diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java index 88fa8cb..876b390 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java @@ -14,6 +14,7 @@ import dev.caskeleton.messaging.policy.RetryPolicy; import dev.caskeleton.messaging.policy.SchemaPolicy; import dev.caskeleton.messaging.security.BrokerCredentialProfile; import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import dev.caskeleton.messaging.security.BrokerTrustStore; import dev.caskeleton.messaging.security.DestinationAccessPolicy; import java.util.ArrayList; import java.util.LinkedHashSet; @@ -235,7 +236,22 @@ public final class MessagingConfigurationCompiler { new DestinationAccessPolicy( destinationNames(security.access().publishable()), destinationNames(security.access().consumable()), - destinationNames(security.access().administrable()))); + destinationNames(security.access().administrable())), + Optional.ofNullable(security.trustStore()).map(store -> trustStore(name, store))); + } + + private static BrokerTrustStore trustStore( + String broker, BrokerSecuritySettings.TrustStore store) { + if (blankToNull(store.location()) == null) { + throw configurationError( + key("security", broker, "trust-store.location"), + "a trust store entry needs the file it names; without one the JVM default store is used," + + " which is what leaving the entry out already says"); + } + return new BrokerTrustStore( + store.location(), + store.type(), + Optional.ofNullable(blankToNull(store.passwordCredentialId()))); } private static BrokerCredentialProfile credential( @@ -252,10 +268,25 @@ public final class MessagingConfigurationCompiler { } String type = credential.type() == null ? "" : credential.type().toUpperCase(Locale.ROOT); String id = credential.credentialId(); + BrokerSecuritySettings.KeyStore keyStore = credential.keyStore(); + boolean hasKeyStore = keyStore != null && blankToNull(keyStore.location()) != null; + if ("MUTUAL_TLS".equals(type) && !hasKeyStore) { + throw configurationError( + key("security", broker, role + ".key-store"), + "MUTUAL_TLS presents a client certificate, and without a key store there is none to" + + " present; the broker refuses the handshake"); + } + if (!"MUTUAL_TLS".equals(type) && keyStore != null) { + throw configurationError( + key("security", broker, role + ".key-store"), + "only a MUTUAL_TLS credential presents a key store; a %s credential would ignore it" + .formatted(credential.type())); + } return switch (type) { case "SASL_SCRAM" -> new BrokerCredentialProfile.SaslScram(id); case "OAUTH2" -> new BrokerCredentialProfile.OAuth2(id); - case "MUTUAL_TLS" -> new BrokerCredentialProfile.MutualTls(id); + case "MUTUAL_TLS" -> + new BrokerCredentialProfile.MutualTls(id, keyStore.location(), keyStore.type()); case "USERNAME_PASSWORD" -> new BrokerCredentialProfile.UsernamePassword(id); case "NKEY" -> new BrokerCredentialProfile.Nkey(id); default -> diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java index a61db17..0f1c771 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java @@ -408,14 +408,48 @@ public class MessagingCoreAutoConfiguration { * Declaring a destination profile is already an act of configuration, so a message to one nobody * declared is a typo or a module reaching past its contract — which is what this check is for. * + *

    A broker with a security entry narrows that further to its {@code access.publishable} list. + * The list was compiled and then read by nothing, so a destination the security entry left out + * was published to anyway. + * * @param destinations the validated registry + * @param compiled the security profiles compiled from configuration + * @param declared the security profiles the application declared as beans * @return the access policy */ @Bean @ConditionalOnMissingBean public dev.caskeleton.messaging.security.DestinationAccessPolicy messagingDestinationAccess( - ValidatedDestinationRegistry destinations) { - return dev.caskeleton.messaging.runtime.DeclaredDestinationAccess.of(destinations.all()); + ValidatedDestinationRegistry destinations, + CompiledMessagingConfiguration compiled, + ObjectProvider declared) { + java.util.Map byBroker = + new java.util.HashMap<>(); + java.util.stream.Stream.concat(compiled.security().stream(), declared.orderedStream()) + .forEach( + profile -> { + if (byBroker.putIfAbsent(profile.broker(), profile.access()) != null) { + throw new IllegalStateException( + "broker '" + + profile.broker() + + "' has more than one security profile; its access list is ambiguous"); + } + }); + java.util.Map< + dev.caskeleton.messaging.api.destination.DestinationName, + dev.caskeleton.messaging.policy.DestinationProfile> + permitted = new java.util.HashMap<>(); + destinations + .all() + .forEach( + (name, profile) -> { + dev.caskeleton.messaging.security.DestinationAccessPolicy acl = + byBroker.get(profile.broker()); + if (acl == null || acl.mayPublish(name)) { + permitted.put(name, profile); + } + }); + return dev.caskeleton.messaging.runtime.DeclaredDestinationAccess.of(permitted); } /** diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java new file mode 100644 index 0000000..fef555d --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java @@ -0,0 +1,103 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.ContentType; +import dev.caskeleton.messaging.api.MessageEnvelope; +import dev.caskeleton.messaging.api.MessageId; +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.ProducerId; +import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.error.FailureDescriptor; +import dev.caskeleton.messaging.api.header.MessageHeaders; +import dev.caskeleton.messaging.api.publish.MessagePublisher; +import dev.caskeleton.messaging.api.publish.PublishOptions; +import dev.caskeleton.messaging.api.publish.PublishResult; +import java.time.Instant; +import java.util.Optional; +import org.junit.jupiter.api.Test; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; + +/** + * The broker's {@code access.publishable} list is enforced by the publish path, not only compiled. + * + *

    The list was bound, validated and turned into a {@code DestinationAccessPolicy} that nothing + * read: the publisher's access check was built from the declared destinations alone, so a + * destination the security entry left out was published to anyway. + */ +class BrokerAccessListTest { + + private final ApplicationContextRunner runner = + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withUserConfiguration(KafkaProducerSecurityTest.Credentials.class) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "spring.kafka.bootstrap-servers=localhost:9092", + "app.messaging.brokers.kafka-primary.type=kafka", + "app.messaging.brokers.kafka-primary.bootstrap-servers[0]=localhost:9092", + "app.messaging.security.kafka-primary.producer.type=SASL_SCRAM", + "app.messaging.security.kafka-primary.producer.credential-id=kafka-producer", + "app.messaging.security.kafka-primary.consumer.type=SASL_SCRAM", + "app.messaging.security.kafka-primary.consumer.credential-id=kafka-consumer", + "app.messaging.security.kafka-primary.access.publishable[0]=order-events") + .withPropertyValues(destination("order-events")) + .withPropertyValues(destination("audit-events")); + + @Test + void aDeclaredDestinationTheSecurityEntryDoesNotListIsRejectedAsForbidden() { + runner.run( + context -> { + // given + MessagePublisher publisher = context.getBean(MessagePublisher.class); + + // when + PublishResult result = publish(publisher, "audit-events"); + + // then + assertThat(result.failure().map(FailureDescriptor::code)).contains("PUBLISH_FORBIDDEN"); + }); + } + + private static PublishResult publish(MessagePublisher publisher, String destination) { + return publisher + .publish( + new MessageDestination<>( + new DestinationName(destination), new MessageType("order.created"), String.class), + new MessageEnvelope<>( + MessageId.newId(), + new MessageType("order.created"), + new SchemaVersion(1), + Instant.parse("2026-09-25T00:00:00Z"), + Optional.empty(), + new ProducerId("orders"), + Optional.empty(), + Optional.empty(), + ContentType.JSON, + Optional.empty(), + Optional.empty(), + Optional.empty(), + TraceContext.none(), + MessageHeaders.empty(), + "{}"), + PublishOptions.defaults()) + .toCompletableFuture() + .join(); + } + + private static String[] destination(String name) { + String prefix = "app.messaging.destinations." + name + "."; + return new String[] { + prefix + "broker=kafka-primary", + prefix + "kind=EVENT_STREAM", + prefix + "physical.topic=" + name, + prefix + "schema.message-types[0]=order.created", + prefix + "guarantees.delivery=AT_MOST_ONCE", + }; + } +} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java index ac651f8..2efd948 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java @@ -6,8 +6,14 @@ import dev.caskeleton.messaging.kafka.KafkaSecurityConfigurer; import dev.caskeleton.messaging.security.BrokerSecurityProfile; import dev.caskeleton.messaging.security.CredentialProvider; import java.time.Instant; +import java.util.List; import java.util.Map; import java.util.Optional; +import java.util.concurrent.atomic.AtomicReference; +import javax.security.auth.callback.Callback; +import javax.security.auth.callback.NameCallback; +import javax.security.auth.callback.PasswordCallback; +import org.apache.kafka.common.security.auth.AuthenticateCallbackHandler; import org.junit.jupiter.api.Test; import org.springframework.boot.autoconfigure.AutoConfigurations; import org.springframework.boot.test.context.assertj.AssertableApplicationContext; @@ -44,6 +50,16 @@ class KafkaProducerSecurityTest { "app.messaging.security.kafka-primary.consumer.credential-id=kafka-consumer", }; + private static final String[] MUTUAL_TLS_CREDENTIALS = { + "app.messaging.brokers.kafka-primary.tls-enabled=true", + "app.messaging.security.kafka-primary.producer.type=MUTUAL_TLS", + "app.messaging.security.kafka-primary.producer.credential-id=kafka-producer-tls", + "app.messaging.security.kafka-primary.producer.key-store.location=/etc/kafka/producer.p12", + "app.messaging.security.kafka-primary.consumer.type=MUTUAL_TLS", + "app.messaging.security.kafka-primary.consumer.credential-id=kafka-consumer-tls", + "app.messaging.security.kafka-primary.consumer.key-store.location=/etc/kafka/consumer.p12", + }; + @Test void aTlsBrokerWithScramCredentialsConfiguresTheProducerForSaslSsl() { runner @@ -81,16 +97,107 @@ class KafkaProducerSecurityTest { } @Test - void theProducerLogsInWithTheProducerCredentialResolvedFromTheProvider() { + void theProducerAuthenticatesAsTheUsernameTheProviderReturnsNotAsTheCredentialId() { runner .withUserConfiguration(Credentials.class) .withPropertyValues(SCRAM_CREDENTIALS) .withPropertyValues("app.messaging.brokers.kafka-primary.tls-enabled=true") + .run( + context -> { + // given + Map config = producerConfig(context); + + // when + Answer answer = authenticate(config); + + // then + assertThat(answer).isEqualTo(new Answer(Credentials.USERNAME, Credentials.SECRET)); + }); + } + + @Test + void aRotatedPasswordReachesTheNextAuthenticationWithoutRebuildingTheProducer() { + runner + .withUserConfiguration(RotatingCredentials.class) + .withPropertyValues(SCRAM_CREDENTIALS) + .withPropertyValues("app.messaging.brokers.kafka-primary.tls-enabled=true") + .run( + context -> { + // given + Map config = producerConfig(context); + RotatingCredentials.CURRENT.set("rotated-secret"); + + // when + Answer answer = authenticate(config); + + // then + assertThat(answer.password()).isEqualTo("rotated-secret"); + }); + } + + @Test + void aPrivateCaTrustStoreReachesTheProducer() { + runner + .withUserConfiguration(Credentials.class, StoreFilesAbsent.class) + .withPropertyValues(SCRAM_CREDENTIALS) + .withPropertyValues( + "app.messaging.brokers.kafka-primary.tls-enabled=true", + "app.messaging.security.kafka-primary.trust-store.location=/etc/kafka/private-ca.p12", + "app.messaging.security.kafka-primary.trust-store.password-credential-id=kafka-trust") .run( context -> - assertThat(String.valueOf(producerConfig(context).get("sasl.jaas.config"))) - .startsWith("org.apache.kafka.common.security.scram.ScramLoginModule required") - .contains("username=\"kafka-producer\"", "password=\"" + Credentials.SECRET)); + assertThat(producerConfig(context)) + .extractingByKeys( + "ssl.truststore.location", "ssl.truststore.type", "ssl.truststore.password") + .containsExactly("/etc/kafka/private-ca.p12", "PKCS12", Credentials.SECRET)); + } + + @Test + void mutualTlsPresentsTheProducerKeyStoreUnlockedByItsCredential() { + runner + .withUserConfiguration(Credentials.class, StoreFilesAbsent.class) + .withPropertyValues(MUTUAL_TLS_CREDENTIALS) + .run( + context -> + assertThat(producerConfig(context)) + .extractingByKeys( + "security.protocol", + "ssl.keystore.location", + "ssl.keystore.type", + "ssl.keystore.password") + .containsExactly( + "SSL", "/etc/kafka/producer.p12", "PKCS12", Credentials.SECRET)); + } + + @Test + void mutualTlsWithoutAKeyStoreFailsStartupNamingTheKey() { + runner + .withUserConfiguration(Credentials.class) + .withPropertyValues(MUTUAL_TLS_CREDENTIALS) + .withPropertyValues("app.messaging.security.kafka-primary.producer.key-store.location=") + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .as("a client certificate nobody configured cannot be presented") + .hasMessageContaining( + "app.messaging.security.kafka-primary.producer.key-store: MUTUAL_TLS")); + } + + @Test + void mutualTlsOnABrokerWithoutTlsFailsStartup() { + runner + .withUserConfiguration(Credentials.class) + .withPropertyValues(MUTUAL_TLS_CREDENTIALS) + .withPropertyValues("app.messaging.brokers.kafka-primary.tls-enabled=false") + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .as("a certificate only authenticates inside a TLS handshake") + .hasMessageContaining("authenticates with a client certificate but does not")); } @Test @@ -127,12 +234,34 @@ class KafkaProducerSecurityTest { context.getBeanProvider(KafkaSecurityConfigurer.class)); } - /** A credential source that answers every id with the same test secret. */ + /** What a SASL client is told when it asks for a name and a password. */ + private record Answer(String name, String password) {} + + /** + * Authenticates the way Kafka's SASL client does: instantiate the configured handler class, + * configure it with the client's own configuration, then ask it for a name and a password. + */ + private static Answer authenticate(Map config) throws Exception { + assertThat(config).containsKey("sasl.client.callback.handler.class"); + Class type = (Class) config.get("sasl.client.callback.handler.class"); + AuthenticateCallbackHandler handler = + (AuthenticateCallbackHandler) type.getDeclaredConstructor().newInstance(); + handler.configure(config, String.valueOf(config.get("sasl.mechanism")), List.of()); + NameCallback name = new NameCallback("name"); + PasswordCallback password = new PasswordCallback("password", false); + handler.handle(new Callback[] {name, password}); + handler.close(); + return new Answer(name.getName(), new String(password.getPassword())); + } + + /** A credential source that answers every id with the same test principal and secret. */ @Configuration(proxyBeanMethods = false) static class Credentials { static final String SECRET = "test-only-secret"; + static final String USERNAME = "svc-orders"; + @Bean CredentialProvider credentialProvider() { return new CredentialProvider() { @@ -145,6 +274,55 @@ class KafkaProducerSecurityTest { public Optional expiresAt(String credentialId) { return Optional.empty(); } + + @Override + public Optional username(String credentialId) { + return Optional.of(USERNAME); + } + }; + } + } + + /** + * Stands in for the producer when the store paths in a test name no real file. + * + *

    A real producer opens its key and trust stores while it is constructed, so a path that does + * not exist fails startup — which is the behaviour a deployment wants, and not what these tests + * are about. They read the configuration the producer would have been built from. + */ + @Configuration(proxyBeanMethods = false) + static class StoreFilesAbsent { + + @Bean + org.apache.kafka.clients.producer.Producer messagingKafkaProducer() { + return new org.apache.kafka.clients.producer.MockProducer<>(); + } + } + + /** A credential store whose secret has already expired, so every resolution reads it again. */ + @Configuration(proxyBeanMethods = false) + static class RotatingCredentials { + + static final AtomicReference CURRENT = new AtomicReference<>("first-secret"); + + @Bean + CredentialProvider credentialProvider() { + CURRENT.set("first-secret"); + return new CredentialProvider() { + @Override + public char[] resolve(String credentialId) { + return CURRENT.get().toCharArray(); + } + + @Override + public Optional expiresAt(String credentialId) { + return Optional.of(Instant.EPOCH); + } + + @Override + public Optional username(String credentialId) { + return Optional.of(Credentials.USERNAME); + } }; } } From b5e3c037773212104d40b24b7060a738ca26eb3c Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Mon, 28 Sep 2026 14:46:30 +0900 Subject: [PATCH 3/7] =?UTF-8?q?review-followup=20Wave=202:=20=EC=A0=95?= =?UTF-8?q?=ED=95=A9=EC=84=B1=20=EC=88=98=EC=A0=95=20=ED=86=B5=ED=95=A9?= =?UTF-8?q?=EA=B3=BC=20=EA=B2=80=EC=A6=9D,=20=ED=98=84=ED=99=A9=ED=91=9C?= =?UTF-8?q?=C2=B7=EC=9E=AC=EA=B0=9C=20=ED=94=84=EB=A1=AC=ED=94=84=ED=8A=B8?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 13개 작업 묶음(P1–P13)의 W2 커밋을 합치고 전체 게이트로 검증했다. - 묶음별 정합성 수정(notification dispatch·recovery, object storage staged publication, fileserver 복구·정리, gRPC 플랫폼, Redis 분류·lease TTL, JPA outbox claim 순서·durable operation HTTP, web 요청 id·category 필수, graphql 에러 mapper, Mongo profile 바인딩, build 소스 보존 게이트·compose smoke info 확인) - 통합 때 드러난 3건 수정: outbox claim 테스트 fixture의 가짜 envelope hash, sample scan이 새 operation controller를 등록하던 문제, SpotBugs aux classpath 파일이 truncate되지 않아 로컬에서 ".jar.jar"로 실패하던 문제 - 현황표: 묶음 결과 반영, 소유자 결정 21–27, 후속 발견 결함 47행, 요약 갱신 - docs/capability-status.md 「제거된 코드」에 85행 추가 - 묶음 결과 파일·에이전트 지시·스크립트를 docs로 옮기고 desktop 재개 프롬프트 추가 검증: check, ci verifyDependencyLocks, mainCheck, architecture/public path/env keys/config contract, bootstrap·sample integration/system, build-logic·build-tools test, optional-platforms ci, qualificationBootJar 통과. integrationCheck는 이번 작업 전에도 실패하던 lane(Redis topology 모드 인자, httpclient failure injection, AWS 보호 lane)만 실패. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/messaging/configuration-reference.md | 6 + docs/messaging/outbox-inbox.md | 18 +- docs/messaging/retry-dlq-redrive.md | 9 + .../messaging/inbox/InboxPostgresIT.java | 2 +- .../messaging/inbox/InboxCleanupJob.java | 4 +- .../messaging/inbox/JdbcInboxRepository.java | 14 - .../inbox/IdempotentConsumerTest.java | 9 +- .../messaging/inbox/InboxOperationsTest.java | 23 +- .../kafka/KafkaConsumerRegistrar.java | 155 +++++++++-- .../kafka/KafkaQuarantinePublisher.java | 17 +- .../kafka/KafkaConsumerRegistrarTest.java | 255 +++++++++++++++--- .../messaging/outbox/OutboxPostgresIT.java | 36 ++- .../outbox/JdbcOutboxRepository.java | 40 ++- .../messaging/outbox/OutboxCleanupJob.java | 8 +- .../messaging/outbox/OutboxRelay.java | 58 +++- .../messaging/outbox/OutboxRelayWorker.java | 11 +- ...5__messaging_outbox_ordering_key_index.sql | 9 + .../outbox/OutboxOperationsTest.java | 25 +- .../messaging/outbox/OutboxRelayTest.java | 56 +++- .../outbox/OutboxRelayWorkerTest.java | 21 +- .../messaging/policy/InFlightLimiter.java | 15 +- .../policy/MessagingAdmissionController.java | 19 +- .../reliability/InboxRepository.java | 10 +- .../reliability/OutboxRepository.java | 16 +- .../runtime/DefaultMessagePublisher.java | 21 +- .../runtime/DefaultMessagePublisherTest.java | 83 ++++++ .../KafkaMessagingAutoConfiguration.java | 69 +++-- ...ssagingLiveRoundTripQualificationTest.java | 3 +- ...ssagingSecuredBrokerQualificationTest.java | 1 - .../DefaultReactiveMessagePublisher.java | 6 +- ...MessagingReliabilityAutoConfiguration.java | 27 ++ .../autoconfigure/BrokerAccessListTest.java | 1 - .../KafkaProducerConfigTest.java | 96 +++++++ .../KafkaProducerSecurityTest.java | 2 - .../MessagingAutoConfigurationTest.java | 4 +- .../MessagingConfigurationBindingTest.java | 4 +- .../MessagingOutboxRelayLifecycleTest.java | 54 +++- .../MessagingStarterOffContractTest.java | 15 +- .../autoconfigure/ReactiveFacadeTest.java | 28 ++ 39 files changed, 1024 insertions(+), 226 deletions(-) create mode 100644 messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql create mode 100644 messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java diff --git a/docs/messaging/configuration-reference.md b/docs/messaging/configuration-reference.md index 0e3c748..2dd4be8 100644 --- a/docs/messaging/configuration-reference.md +++ b/docs/messaging/configuration-reference.md @@ -166,6 +166,12 @@ app: `kafka`뿐이다. 위 예시가 `production: false`인 것은 이 페이지가 그대로 실행되는 fixture이기 때문이며, 실 배포는 셋 다 참이다. +Kafka producer는 이 항목 하나로 만든다. `bootstrap-servers`, `acks`, `enable-idempotence`, +`max-in-flight-requests-per-connection`, `delivery-timeout`이 그대로 client 설정이 된다. +`spring.kafka.bootstrap-servers`는 읽지 않는다. type이 `kafka`인 항목이 정확히 하나가 아니면 기동에 +실패한다. `send()`가 metadata를 기다리며 호출 스레드를 막는 시간(`max.block.ms`)은 기본 publish +timeout(5s)이다. + ## 보안 ```yaml diff --git a/docs/messaging/outbox-inbox.md b/docs/messaging/outbox-inbox.md index d4b68ab..fede443 100644 --- a/docs/messaging/outbox-inbox.md +++ b/docs/messaging/outbox-inbox.md @@ -21,7 +21,7 @@ Flyway version `1000..1999`를 예약해서 애플리케이션의 일반적인 classpath:db/migration/messaging ``` -현재 순서는 `V1001..V1004`(outbox/admin)와 `V1101`(inbox)이다. starter 테스트가 두 JDBC +현재 순서는 `V1001..V1005`(outbox/admin)와 `V1101`(inbox)이다. starter 테스트가 두 JDBC 모듈을 실제 combined classpath로 올려 version 중복과 예약 대역 이탈을 검사한다. 개별 DDL의 의미와 전환 순서는 [migration-guide.md](migration-guide.md)를 따른다. @@ -46,6 +46,11 @@ leaseBatch(100, 30s) -- lease로 다중 relay 인스턴스 안전 → REJECTED → markFailed ``` +row 하나의 실패는 그 row에서 끝난다. envelope로 만들 수 없는 row(예약 header `msg.*`를 담은 row +등)는 `FAILED`가 되고, publisher가 예외를 던지면 결과를 모르므로 `AMBIGUOUS`가 된다. 같은 batch의 +나머지 row는 그대로 publish된다. pass 전체가 실패하면(DB 연결 등) WARN 로그를 남기고 +`messaging.outbox.relay.failed.passes` 카운터를 올린다. + ### 핵심 규칙: ambiguous는 같은 messageId로 재시도 새 id를 발급하면 "전달됐을 수도 있는 메시지"가 "확실히 두 번째인 메시지"가 되어 @@ -61,8 +66,16 @@ status IN ('PENDING','AMBIGUOUS','IN_FLIGHT') AND (lease_expires_at IS NULL OR lease_expires_at <= now) AND next_attempt_at <= now AND attempts < maxAttempts + AND (ordering_key IS NULL + OR 같은 destination·ordering_key의 더 오래된 미완료 row가 없음) ``` +`ordering_key`가 있는 row는 같은 key의 더 오래된 row가 `PENDING`·`AMBIGUOUS`·`IN_FLIGHT`인 동안 +claim되지 않는다. 이 조건이 없으면 두 relay가 같은 key의 두 row를 나눠 가져가거나, AMBIGUOUS로 +재시도되는 row보다 뒤의 row가 먼저 나간다(`OutboxPostgresIT#aNewerRowWaitsWhileAnOlderRowWithTheSameOrderingKeyIsUnfinished`). +`V1005`의 partial index가 이 확인을 받친다. 더 오래된 row가 `FAILED`·`EXHAUSTED`가 되면 뒤의 row는 +진행한다. + `IN_FLIGHT`가 목록에 있는 것이 핵심이다. relay가 publish 도중 죽으면 row는 `IN_FLIGHT`로 남는데, 이를 제외하면 그 메시지는 **영원히** 발행되지 않는다 — outbox가 막으려던 바로 그 실패다. 대신 lease가 만료됐을 때만 회수하므로, 살아 있는 relay가 들고 있는 row는 회수되지 않는다. @@ -107,6 +120,9 @@ message_id만으로 중복 제거하면 같은 event를 소비하는 두 번째 broker의 최대 redelivery window보다 **길어야** 한다. row를 먼저 지우면 늦게 도착한 redelivery가 두 번 처리된다. +outbox·inbox cleanup job은 한 번에 최대 1,000행씩 지우고, 지운 수가 1,000보다 적으면 멈춘다. +repository port에는 개수 제한 없는 삭제 메서드가 없다. + ## CDC 대안 relay는 polling 하나뿐이다. Debezium Event Router 매핑(`DebeziumOutboxEventRouter` 등)은 조립하는 diff --git a/docs/messaging/retry-dlq-redrive.md b/docs/messaging/retry-dlq-redrive.md index 39355e6..9d64b63 100644 --- a/docs/messaging/retry-dlq-redrive.md +++ b/docs/messaging/retry-dlq-redrive.md @@ -49,6 +49,15 @@ full jitter는 `[0, delay]` 균등 분포다. jitter가 없으면 같은 초에 pause-and-seek는 메시지가 로그의 자기 자리를 떠나지 않는다. partition을 멈추고, 기다리고, 같은 offset으로 seek해 재전달한다. 뒤의 메시지도 함께 기다리며 이것이 의도된 동작이다. +`KafkaConsumerRegistrar`가 handler 실패(예외, settle 없이 끝남, timeout)를 다루는 방식: + +- 재전달은 destination `retry`의 backoff 뒤에 한다. 최소 1초. 0초 재시도는 없다. +- 같은 offset의 실패 횟수가 `retry.maxAttempts`에 닿으면 `KafkaQuarantinePublisher`에 쓰고, + 쓰기가 확인되면 offset을 넘긴다. quarantine이 없으면 backoff로 계속 재시도한다(메시지를 버리지 + 않는다). 그래서 `production` destination은 quarantine 없이 만들 수 없다(`KAFKA_QUARANTINE_REQUIRED`). +- handler timeout이 지나도 handler가 끝날 때까지 partition slot을 잡고 있는다. `CompletableFuture`를 + cancel해도 handler는 멈추지 않으므로, slot을 먼저 풀면 같은 offset이 두 번 동시에 돈다. + ## RabbitMQ: delayed retry queue core broker에 per-message delay가 없으므로 **TTL + DLX**로 구현한다. diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java b/messaging/messaging-inbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java index 367a6f5..899707c 100644 --- a/messaging/messaging-inbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java +++ b/messaging/messaging-inbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java @@ -151,7 +151,7 @@ class InboxPostgresIT { MessageId purged = MessageId.newId(); inTransaction(() -> repository.reserve(purged, "order-projection", NOW)); - assertThat(repository.purgeProcessedBefore(NOW.plusSeconds(1))).isEqualTo(1); + assertThat(repository.purgeProcessedBefore(NOW.plusSeconds(1), 100)).isEqualTo(1); } /** Reserves and writes inside one Spring transaction, exactly as an idempotent consumer must. */ diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java b/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java index fbeda2e..6baceae 100644 --- a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java +++ b/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java @@ -53,10 +53,10 @@ public final class InboxCleanupJob { int removed = 0; for (int batch = 0; batch < maxBatches; batch++) { - int deleted = inbox.purgeProcessedBefore(cutoff); + int deleted = inbox.purgeProcessedBefore(cutoff, DEFAULT_BATCH_SIZE); removed += deleted; // A short batch means the backlog is drained; continuing would just re-scan an empty range. - if (deleted == 0) { + if (deleted < DEFAULT_BATCH_SIZE) { break; } } diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java b/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java index 1ac930c..d941d73 100644 --- a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java +++ b/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java @@ -167,18 +167,4 @@ public final class JdbcInboxRepository implements InboxRepository { "INBOX_PURGE_FAILED", "a bounded inbox purge failed", exception); } } - - @Override - public int purgeProcessedBefore(Instant processedBefore) { - Objects.requireNonNull(processedBefore, "processedBefore must not be null"); - try (Connection connection = dataSource.getConnection(); - PreparedStatement statement = - connection.prepareStatement("DELETE FROM messaging_inbox WHERE processed_at < ?")) { - statement.setTimestamp(1, Timestamp.from(processedBefore)); - return statement.executeUpdate(); - } catch (SQLException exception) { - throw new MessagingConfigurationException( - "INBOX_PURGE_FAILED", "could not purge the inbox", exception); - } - } } diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java b/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java index 013fe84..b4331a6 100644 --- a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java +++ b/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java @@ -82,7 +82,7 @@ class IdempotentConsumerTest { void retentionPruningRemovesOldRows() { consumer.runOnce(MessageId.newId(), "order-projection", NOW, () -> "done"); - assertThat(inbox.purgeProcessedBefore(NOW.plusSeconds(1))).isEqualTo(1); + assertThat(inbox.purgeProcessedBefore(NOW.plusSeconds(1), 100)).isEqualTo(1); assertThat(inbox.size()).isZero(); } } @@ -104,12 +104,7 @@ final class InMemoryInbox implements InboxRepository { @Override public int purgeProcessedBefore(Instant processedBefore, int limit) { - return Math.min(purgeProcessedBefore(processedBefore), limit); - } - - @Override - public int purgeProcessedBefore(Instant processedBefore) { - int removed = reserved.size(); + int removed = Math.min(reserved.size(), limit); reserved.clear(); return removed; } diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java b/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java index 17409b1..b822834 100644 --- a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java +++ b/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java @@ -42,15 +42,13 @@ class InboxOperationsTest { return reserved.contains(messageId.value() + "|" + consumerId); } - @Override - public int purgeProcessedBefore(Instant processedBefore, int limit) { - return Math.min(purgeProcessedBefore(processedBefore), limit); - } + private final List limits = new ArrayList<>(); @Override - public int purgeProcessedBefore(Instant processedBefore) { + public int purgeProcessedBefore(Instant processedBefore, int limit) { cutoffs.add(processedBefore); - return pass < deletions.size() ? deletions.get(pass++) : 0; + limits.add(limit); + return Math.min(pass < deletions.size() ? deletions.get(pass++) : 0, limit); } } @@ -91,14 +89,17 @@ class InboxOperationsTest { } @Test - void cleanupDeletesInBoundedBatches() { + void cleanupAsksForBoundedBatchesAndStopsAtTheFirstShortOne() { + // given InMemoryInbox inbox = new InMemoryInbox(List.of(1000, 500)); - int removed = - new InboxCleanupJob(inbox, policy(Duration.ofDays(7), Duration.ofDays(1)), 10).runOnce(NOW); + // when + new InboxCleanupJob(inbox, policy(Duration.ofDays(7), Duration.ofDays(1)), 10).runOnce(NOW); - assertThat(removed).isEqualTo(1500); - assertThat(inbox.cutoffs).hasSize(3); + // then + assertThat(inbox.limits) + .as("DEFAULT_BATCH_SIZE was declared and never passed to the repository") + .containsExactly(InboxCleanupJob.DEFAULT_BATCH_SIZE, InboxCleanupJob.DEFAULT_BATCH_SIZE); } @Test diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java index c285b5c..9bfec58 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java @@ -66,6 +66,35 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati private final KafkaQuarantinePublisher quarantine; + private final java.time.InstantSource clock; + + /** The shortest wait before a failed delivery is read again, whatever the policy computes. */ + private static final Duration MIN_RETRY_DELAY = Duration.ofSeconds(1); + + private static final dev.caskeleton.messaging.policy.BackoffCalculator BACKOFF = + new dev.caskeleton.messaging.policy.BackoffCalculator(); + + /** A delivery that failed, by where it sits in the log. */ + private record FailedOffset(TopicPartition partition, long offset) {} + + /** + * How many times each offset has failed without being settled. + * + *

    Pause-and-seek re-reads the same record, so the envelope's own attempt header never moves; + * the count has to live here. Without it a poison message was retried with no delay and no limit, + * and its partition never advanced again. + */ + private final Map failures = new ConcurrentHashMap<>(); + + /** + * Partitions paused because a record could not be dispatched — the ceiling was reached, the + * runtime was draining, or the pool refused. Poll thread only. + * + *

    They were paused and never resumed: an ordered partition handed two records by one poll ran + * the first and then stopped reading for good. + */ + private final Set saturated = new java.util.HashSet<>(); + private final Map committed = new ConcurrentHashMap<>(); /** @@ -127,8 +156,38 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati GracefulShutdownCoordinator shutdown, Duration pollTimeout, KafkaQuarantinePublisher quarantine) { + this( + spec, + consumer, + handlerPool, + shutdown, + pollTimeout, + quarantine, + java.time.InstantSource.system()); + } + + /** A consumer runtime with the clock retry delays are measured on, for tests. */ + KafkaConsumerRegistrar( + TransportConsumerSpec spec, + Consumer consumer, + Executor handlerPool, + GracefulShutdownCoordinator shutdown, + Duration pollTimeout, + KafkaQuarantinePublisher quarantine, + java.time.InstantSource clock) { + this.clock = Objects.requireNonNull(clock, "clock must not be null"); this.quarantine = Objects.requireNonNull(quarantine, "quarantine must not be null"); this.spec = Objects.requireNonNull(spec, "spec must not be null"); + if (spec.profile().production() && quarantine == KafkaQuarantinePublisher.unavailable()) { + // Without somewhere to park a message that keeps failing, its partition retries it forever + // and nothing behind it is ever read. + throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( + "KAFKA_QUARANTINE_REQUIRED", + "production destination " + + spec.profile().name().value() + + " needs a KafkaQuarantinePublisher; without one a message that exhausts its" + + " attempts stops its partition for good"); + } this.consumer = Objects.requireNonNull(consumer, "consumer must not be null"); this.handlerPool = Objects.requireNonNull(handlerPool, "handlerPool must not be null"); this.shutdown = Objects.requireNonNull(shutdown, "shutdown must not be null"); @@ -169,6 +228,8 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati coordinator.forget(partition); retries.forget(partition); committed.remove(partition); + saturated.remove(partition); + failures.keySet().removeIf(failed -> failed.partition().equals(partition)); }); } @@ -199,6 +260,7 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati syncEpochs(); applySettlements(); applyDueResumes(now); + resumeSaturated(); commitContiguous(); if (!shutdown.isAcceptingWork()) { @@ -251,6 +313,7 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati // Only this partition pauses. Its earliest unsubmitted offset is where it resumes. consumer.pause(Set.of(partition)); consumer.seek(partition, seekBackTo); + saturated.add(partition); } } return dispatched; @@ -283,7 +346,7 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati if (quarantine.quarantine(record, undecodable)) { settlement.acknowledgeAfterQuarantine(); } else { - settlement.requeueAfterFailure(); + settlement.requeueAfterFailure(record, undecodable); } coordinator.release(partition); shutdown.endWork(); @@ -308,37 +371,36 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati handler = spec.sink().apply(delivery).toCompletableFuture(); handler.get(handlerTimeout.toMillis(), java.util.concurrent.TimeUnit.MILLISECONDS); } catch (java.util.concurrent.TimeoutException timedOut) { - // A timed-out handler may still be applying side effects. Cancel it before releasing - // the partition permit; otherwise the same offset can be dispatched again while the - // first handler is still running. + // A timed-out handler may still be applying side effects, and cancelling the + // CompletableFuture does not stop it — it only completes the stage this worker holds, + // which released the permit and let the same offset run again beside the first. So + // the slot stays taken until the handler really finishes, and only then is a delivery + // it left unsettled requeued. java.util.concurrent.CompletableFuture timedOutHandler = handler; - settlement.requeueAfterFailure(); - if (timedOutHandler != null - && !timedOutHandler.cancel(true) - && !timedOutHandler.isDone()) { - releaseImmediately = false; - java.util.concurrent.CompletableFuture releaseAfterCompletion = - timedOutHandler.whenComplete( - (ignored, failure) -> { - coordinator.release(partition); - shutdown.endWork(); - }); - // Keep the continuation strongly referenced through this worker frame. The - // completion action owns the delayed permit/work-count release; its own result is - // intentionally not joined here. - Objects.requireNonNull(releaseAfterCompletion); - } + releaseImmediately = false; + java.util.concurrent.CompletableFuture releaseAfterCompletion = + timedOutHandler.whenComplete( + (ignored, failure) -> { + settlement.requeueAfterFailure( + record, failure == null ? timedOut : failure); + coordinator.release(partition); + shutdown.endWork(); + }); + // Keep the continuation strongly referenced through this worker frame. The + // completion action owns the delayed permit/work-count release; its own result is + // intentionally not joined here. + Objects.requireNonNull(releaseAfterCompletion); } catch (InterruptedException interrupted) { Thread.currentThread().interrupt(); if (handler != null) { handler.cancel(true); } - settlement.requeueAfterFailure(); + settlement.requeueAfterFailure(record, interrupted); } catch (java.util.concurrent.ExecutionException | RuntimeException failure) { // A handler failure is retried, not parked. PARKED completed the offset without any // quarantine or dead-letter write, so a transient database outage silently discarded // every message it touched. - settlement.requeueAfterFailure(); + settlement.requeueAfterFailure(record, failure); } finally { if (releaseImmediately) { coordinator.release(partition); @@ -365,7 +427,10 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati continue; } switch (command.kind()) { - case COMPLETE, PARKED -> offsets.completed(command.partition(), command.offset()); + case COMPLETE, PARKED -> { + offsets.completed(command.partition(), command.offset()); + failures.remove(new FailedOffset(command.partition(), command.offset())); + } case PAUSE_AND_SEEK -> { coordinator.pause(command.partition()); consumer.pause(Set.of(command.partition())); @@ -391,6 +456,22 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati current.forEach( partition -> epochs.computeIfAbsent(partition, key -> epochSequence.incrementAndGet())); epochs.keySet().removeIf(partition -> !current.contains(partition)); + saturated.retainAll(current); + } + + /** Resumes a partition paused for lack of a slot once it has one and nothing else holds it. */ + private void resumeSaturated() { + int ceiling = spec.profile().consumer().maxInFlightPerOrderingUnit(); + saturated.removeIf( + partition -> { + if (coordinator.isPaused(partition) + || retries.isPaused(partition) + || coordinator.inFlight(partition) >= ceiling) { + return false; + } + consumer.resume(Set.of(partition)); + return true; + }); } private void applyDueResumes(Instant now) { @@ -603,16 +684,36 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati } } - /** Used by the worker when the handler failed or timed out and never settled. */ - private void requeueAfterFailure() { + /** + * Used by the worker when the handler failed or timed out and never settled. + * + *

    Retried after the destination's backoff, never sooner than {@code MIN_RETRY_DELAY}; once + * the attempt budget is spent the record is written to quarantine and committed past. When + * there is no quarantine it keeps being retried — the offset may not advance past a message + * that exists nowhere else — which is why a production destination must have one. + */ + private void requeueAfterFailure(ConsumerRecord record, Throwable reason) { + if (terminal.get() != null) { + // The handler settled before it failed or before its late completion; that answer stands. + return; + } + FailedOffset key = new FailedOffset(partition, offset); + int attempt = failures.merge(key, 1, Integer::sum); + dev.caskeleton.messaging.policy.RetryPolicy retry = spec.profile().retry(); + if (attempt >= retry.maxAttempts() && quarantine.quarantine(record, reason)) { + failures.remove(key); + acknowledgeAfterQuarantine(); + return; + } if (terminal.compareAndSet(null, "requeue")) { - enqueueRequeue(Duration.ZERO); + Duration backoff = BACKOFF.delayFor(retry, attempt); + enqueueRequeue(backoff.compareTo(MIN_RETRY_DELAY) < 0 ? MIN_RETRY_DELAY : backoff); } } private dev.caskeleton.messaging.api.settlement.SettlementResult enqueueRequeue( Duration delay) { - retries.pauseUntil(partition, offset, delay, Instant.now()); + retries.pauseUntil(partition, offset, delay, clock.instant()); settlements.enqueue( new KafkaSettlementCommand( partition, offset, KafkaSettlementCommand.Kind.PAUSE_AND_SEEK, epoch)); diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java index bdb4365..20489c9 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java @@ -24,8 +24,21 @@ public interface KafkaQuarantinePublisher { */ boolean quarantine(ConsumerRecord record, Throwable reason); - /** The default: nothing is configured, so nothing may be committed past. */ + /** + * The default: nothing is configured, so nothing may be committed past. + * + *

    One instance, so a consumer can tell it apart from a configured quarantine and refuse it on + * a production destination. + */ static KafkaQuarantinePublisher unavailable() { - return (record, reason) -> false; + return Unavailable.INSTANCE; + } + + /** Holds the single "not configured" quarantine. */ + final class Unavailable { + + private static final KafkaQuarantinePublisher INSTANCE = (record, reason) -> false; + + private Unavailable() {} } } diff --git a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java index 941abf3..db4e105 100644 --- a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java +++ b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java @@ -4,6 +4,9 @@ import static org.assertj.core.api.Assertions.assertThat; import dev.caskeleton.messaging.api.MessageId; import dev.caskeleton.messaging.api.header.ReservedHeaders; +import dev.caskeleton.messaging.policy.OrderingImpact; +import dev.caskeleton.messaging.policy.RetryMode; +import dev.caskeleton.messaging.policy.RetryPolicy; import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; import dev.caskeleton.messaging.transport.TransportConsumerSpec; import dev.caskeleton.messaging.transport.TransportDelivery; @@ -154,51 +157,165 @@ class KafkaConsumerRegistrarTest { } @Test - void handlerTimeoutCancelsTheOutstandingHandlerBeforeThePartitionPermitIsReleased() { - CompletableFuture handler = new CompletableFuture<>(); - dev.caskeleton.messaging.policy.DestinationProfile base = KafkaContractHarness.profile(); - dev.caskeleton.messaging.policy.DestinationProfile fastTimeout = - new dev.caskeleton.messaging.policy.DestinationProfile( - base.name(), - base.broker(), - base.kind(), - base.physical(), - base.schema(), - base.deliveryGuarantee(), - base.orderingScope(), - base.externalSideEffectGuarantee(), - base.producer(), - new dev.caskeleton.messaging.policy.ConsumerPolicy( - base.consumer().group(), - base.consumer().concurrency(), - base.consumer().maxInFlightPerOrderingUnit(), - base.consumer().prefetch(), - Duration.ofMillis(5), - base.consumer().manualSettlement()), - base.retry(), - base.deadLetter(), - base.payload(), - base.tier(), - base.production(), - base.keyResolverConfigured(), - base.topologyAutoCreate()); - + void aTimedOutHandlerStillRunningIsNotDispatchedASecondTime() { + // given + Instant start = Instant.now(); + CompletableFuture stillRunning = new CompletableFuture<>(); KafkaConsumerRegistrar registrar = - new KafkaConsumerRegistrar( - new TransportConsumerSpec(fastTimeout, delivery -> handler), - consumer, - handlers, - new GracefulShutdownCoordinator(Duration.ofSeconds(30)), - Duration.ofMillis(1)); + registrar( + profile(Duration.ofMillis(5), RetryPolicy.none(), false), + delivery -> { + handled.add(delivery); + return stillRunning; + }, + KafkaQuarantinePublisher.unavailable(), + () -> start); + assign(); + addRecord(10); + registrar.pollOnce(start); + handlers.runAll(); + // What the broker serves again if the consumer seeks back to the timed-out offset. + addRecord(10); + // when + registrar.pollOnce(start.plusSeconds(60)); + handlers.runAll(); + + // then + assertThat(handled) + .as("cancelling a CompletableFuture does not stop the handler behind it") + .hasSize(1); + } + + @Test + void aTimedOutHandlerIsRedeliveredOnceItFinishesWithoutSettling() { + // given + CompletableFuture stillRunning = new CompletableFuture<>(); + KafkaConsumerRegistrar registrar = + registrar( + profile(Duration.ofMillis(5), RetryPolicy.none(), false), + delivery -> { + handled.add(delivery); + return stillRunning; + }, + KafkaQuarantinePublisher.unavailable(), + () -> NOW); + assign(); + addRecord(10); + registrar.pollOnce(NOW); + handlers.runAll(); + stillRunning.complete(null); + addRecord(10); + + // when + registrar.pollOnce(NOW.plusSeconds(60)); + handlers.runAll(); + + // then + assertThat(handled).hasSize(2); + } + + @Test + void aFailedHandlerWaitsOutItsBackoffBeforeThePartitionIsReadAgain() { + // given + Instant start = Instant.now(); + RetryPolicy threeAttempts = + new RetryPolicy( + RetryMode.PAUSE_PARTITION, + 3, + Duration.ofSeconds(1), + Duration.ofSeconds(10), + 2.0, + false, + OrderingImpact.PRESERVE, + java.util.Set.of(), + java.util.Set.of()); + KafkaConsumerRegistrar registrar = + registrar( + profile(Duration.ofSeconds(1), threeAttempts, false), + this::fail, + KafkaQuarantinePublisher.unavailable(), + () -> start); + assign(); + addRecord(10); + registrar.pollOnce(start); + handlers.runAll(); + + // when + registrar.pollOnce(start.plusMillis(500)); + + // then + assertThat(consumer.paused()) + .as("a failure retried with no delay re-reads the same record as fast as the loop polls") + .contains(PARTITION); + } + + @Test + void aHandlerThatSpendsItsAttemptBudgetIsQuarantinedAndCommittedPast() { + // given + List quarantined = new ArrayList<>(); + KafkaConsumerRegistrar registrar = + registrar( + KafkaContractHarness.profile(), + this::fail, + (record, reason) -> quarantined.add(record.offset()), + () -> NOW); assign(); addRecord(10); registrar.pollOnce(NOW); handlers.runAll(); - assertThat(handler) - .as("a timed-out handler must be cancelled before the same partition can be redelivered") - .isCancelled(); + // when + registrar.pollOnce(NOW); + + // then + assertThat(registrar.committedOffset(PARTITION)) + .as("a poison message retried forever stops its partition for good") + .hasValue(11L); + } + + @Test + void aProductionDestinationWithoutAQuarantineIsRefused() { + // given + dev.caskeleton.messaging.policy.DestinationProfile production = + profile(Duration.ofSeconds(30), RetryPolicy.none(), true); + + // when + Throwable thrown = + org.assertj.core.api.Assertions.catchThrowable( + () -> registrar(production, this::settleImmediately)); + + // then + assertThat(thrown) + .isInstanceOf(dev.caskeleton.messaging.api.error.MessagingConfigurationException.class) + .extracting( + failure -> + ((dev.caskeleton.messaging.api.error.MessagingConfigurationException) failure) + .failure() + .code()) + .isEqualTo("KAFKA_QUARANTINE_REQUIRED"); + } + + @Test + void aPartitionPausedAtItsCeilingResumesOnceItsDeliveryFinishes() { + // given + KafkaConsumerRegistrar registrar = registrar(this::settleImmediately); + assign(); + addRecord(10); + addRecord(11); + registrar.pollOnce(NOW); + handlers.runAll(); + // The second record was returned by the first poll and handed back with a seek. + addRecord(11); + + // when + registrar.pollOnce(NOW); + handlers.runAll(); + + // then + assertThat(handled) + .as("pausing a full partition is backpressure only if something resumes it") + .hasSize(2); } @Test @@ -495,6 +612,68 @@ class KafkaConsumerRegistrarTest { Duration.ofMillis(1)); } + private KafkaConsumerRegistrar registrar( + dev.caskeleton.messaging.policy.DestinationProfile profile, + java.util.function.Function> sink) { + return new KafkaConsumerRegistrar( + new TransportConsumerSpec(profile, sink), + consumer, + handlers, + new GracefulShutdownCoordinator(Duration.ofSeconds(30)), + Duration.ofMillis(1)); + } + + private KafkaConsumerRegistrar registrar( + dev.caskeleton.messaging.policy.DestinationProfile profile, + java.util.function.Function> sink, + KafkaQuarantinePublisher quarantine, + java.time.InstantSource clock) { + return new KafkaConsumerRegistrar( + new TransportConsumerSpec(profile, sink), + consumer, + handlers, + new GracefulShutdownCoordinator(Duration.ofSeconds(30)), + Duration.ofMillis(1), + quarantine, + clock); + } + + /** The harness profile with the three settings these tests vary. */ + private static dev.caskeleton.messaging.policy.DestinationProfile profile( + Duration handlerTimeout, RetryPolicy retry, boolean production) { + dev.caskeleton.messaging.policy.DestinationProfile base = KafkaContractHarness.profile(); + return new dev.caskeleton.messaging.policy.DestinationProfile( + base.name(), + base.broker(), + base.kind(), + base.physical(), + base.schema(), + base.deliveryGuarantee(), + base.orderingScope(), + base.externalSideEffectGuarantee(), + base.producer(), + new dev.caskeleton.messaging.policy.ConsumerPolicy( + base.consumer().group(), + base.consumer().concurrency(), + base.consumer().maxInFlightPerOrderingUnit(), + base.consumer().prefetch(), + handlerTimeout, + base.consumer().manualSettlement()), + retry, + base.deadLetter(), + base.payload(), + base.tier(), + production, + base.keyResolverConfigured(), + base.topologyAutoCreate()); + } + + /** A handler that fails every delivery it is given. */ + private CompletionStage fail(TransportDelivery delivery) { + handled.add(delivery); + throw new IllegalStateException("the downstream this handler writes to is down"); + } + /** Acknowledges inside the handler, as a successful M1 delivery does. */ private CompletionStage settleImmediately(TransportDelivery delivery) { handled.add(delivery); diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java b/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java index 0e2d870..66fba44 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java +++ b/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java @@ -205,7 +205,7 @@ class OutboxPostgresIT { appendInTransaction(pending); repository.markPublishedUnfenced(published.messageId(), NOW); - int removed = repository.purgePublishedBefore(NOW.plusSeconds(1)); + int removed = repository.purgePublishedBefore(NOW.plusSeconds(1), 100); assertThat(removed).isEqualTo(1); assertThat(repository.find(pending.messageId())).isPresent(); @@ -217,7 +217,35 @@ class OutboxPostgresIT { OutboxCanonicalMetadata.none().withRoutingKeys("acct-1", "acct-1")); } + @Test + void aNewerRowWaitsWhileAnOlderRowWithTheSameOrderingKeyIsUnfinished() { + // given + OutboxRecord older = record(ordered("customer-42"), NOW.minusSeconds(60)); + OutboxRecord newer = record(ordered("customer-42"), NOW.minusSeconds(30)); + OutboxRecord otherKey = record(ordered("customer-7"), NOW.minusSeconds(30)); + appendInTransaction(older); + appendInTransaction(newer); + appendInTransaction(otherKey); + + // when + List claimed = repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW); + + // then + assertThat(claimed) + .extracting(OutboxLease::messageId) + .as("a second relay, or a retry of the older row, would put the newer one first") + .containsExactlyInAnyOrder(older.messageId(), otherKey.messageId()); + } + + private static OutboxCanonicalMetadata ordered(String key) { + return OutboxCanonicalMetadata.none().withRoutingKeys(key, key); + } + private static OutboxRecord record(OutboxCanonicalMetadata metadata) { + return record(metadata, NOW.minusSeconds(60)); + } + + private static OutboxRecord record(OutboxCanonicalMetadata metadata, Instant createdAt) { return new OutboxRecord( MessageId.newId(), new DestinationName("order-events"), @@ -227,7 +255,7 @@ class OutboxPostgresIT { "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), Map.of(), metadata, - NOW.minusSeconds(60), + createdAt, OutboxStatus.PENDING, 0, Optional.empty(), @@ -361,8 +389,8 @@ class OutboxPostgresIT { @Test void twoRelaysClaimingConcurrentlyGetDisjointRowsAndDistinctTokens() { - OutboxRecord one = record(); - OutboxRecord two = record(); + OutboxRecord one = record(OutboxCanonicalMetadata.none().withRoutingKeys("acct-1", "acct-1")); + OutboxRecord two = record(OutboxCanonicalMetadata.none().withRoutingKeys("acct-2", "acct-2")); appendInTransaction(one); appendInTransaction(two); diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java index 44a986a..13b87a3 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java +++ b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java @@ -113,17 +113,29 @@ public final class JdbcOutboxRepository implements OutboxRepository { private static final String CLAIM = """ WITH claimable AS ( - SELECT message_id - FROM messaging_outbox - WHERE status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT') - AND (lease_expires_at IS NULL OR lease_expires_at <= ?) + SELECT m.message_id + FROM messaging_outbox m + WHERE m.status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT') + AND (m.lease_expires_at IS NULL OR m.lease_expires_at <= ?) -- The retry clock lives in the row, not in the relay's memory. Without these two -- predicates an AMBIGUOUS row became claimable again on the very next pass, so a -- broker outage meant the whole backlog was republished every poll interval and the -- configured attempt budget was a number nothing consulted. - AND (next_attempt_at IS NULL OR next_attempt_at <= ?) - AND attempts < ? - ORDER BY created_at + AND (m.next_attempt_at IS NULL OR m.next_attempt_at <= ?) + AND m.attempts < ? + -- A row with an ordering key waits while an older row with the same key is still + -- unfinished. SKIP LOCKED let a second relay take the newer row while the first still + -- held the older one, and an AMBIGUOUS older row was retried after newer ones had + -- already gone out. md5 matches the partial index, which cannot hold a key that long. + AND (m.ordering_key IS NULL OR NOT EXISTS ( + SELECT 1 + FROM messaging_outbox older + WHERE older.destination = m.destination + AND md5(older.ordering_key) = md5(m.ordering_key) + AND older.ordering_key = m.ordering_key + AND older.status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT') + AND (older.created_at, older.message_id) < (m.created_at, m.message_id))) + ORDER BY m.created_at, m.message_id LIMIT ? FOR UPDATE SKIP LOCKED ) @@ -514,20 +526,6 @@ public final class JdbcOutboxRepository implements OutboxRepository { }); } - @Override - public int purgePublishedBefore(Instant publishedBefore) { - Objects.requireNonNull(publishedBefore, "publishedBefore must not be null"); - return withConnection( - connection -> { - try (PreparedStatement statement = - connection.prepareStatement( - "DELETE FROM messaging_outbox WHERE status = 'PUBLISHED' AND published_at < ?")) { - statement.setTimestamp(1, Timestamp.from(publishedBefore)); - return statement.executeUpdate(); - } - }); - } - private void update(String sql, MessageId messageId, Instant now, String failureCode) { withConnection( connection -> { diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java index 2bed4fc..5d067bd 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java +++ b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java @@ -15,6 +15,9 @@ import java.util.Objects; */ public final class OutboxCleanupJob { + /** How many rows one delete statement removes before yielding. */ + public static final int DEFAULT_BATCH_SIZE = 1_000; + private final OutboxRepository outbox; private final OutboxProperties properties; private final int maxBatches; @@ -47,9 +50,10 @@ public final class OutboxCleanupJob { int removed = 0; for (int batch = 0; batch < maxBatches; batch++) { - int deleted = outbox.purgePublishedBefore(cutoff); + int deleted = outbox.purgePublishedBefore(cutoff, DEFAULT_BATCH_SIZE); removed += deleted; - if (deleted == 0) { + // A short batch means the backlog is drained; asking again would re-scan an empty range. + if (deleted < DEFAULT_BATCH_SIZE) { break; } } diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java index 64decb8..52350c5 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java +++ b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java @@ -1,9 +1,16 @@ package dev.caskeleton.messaging.outbox; +import dev.caskeleton.messaging.api.MessageEnvelope; import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.error.FailureCategory; +import dev.caskeleton.messaging.api.error.FailureDescriptor; +import dev.caskeleton.messaging.api.error.MessagingException; import dev.caskeleton.messaging.api.publish.MessagePublisher; +import dev.caskeleton.messaging.api.publish.PublishCompletion; +import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; +import dev.caskeleton.messaging.api.publish.RoutingOutcome; import dev.caskeleton.messaging.reliability.OutboxLease; import dev.caskeleton.messaging.reliability.OutboxRecord; import dev.caskeleton.messaging.reliability.OutboxRepository; @@ -13,6 +20,7 @@ import java.time.Duration; import java.time.Instant; import java.util.List; import java.util.Objects; +import java.util.Optional; /** * Publishes outbox rows, treating an unknown outcome as retryable rather than final. @@ -165,7 +173,22 @@ public final class OutboxRelay { for (OutboxLease lease : batch) { OutboxRecord record = lease.record(); - PublishResult result = publish(record); + MessageEnvelope envelope; + try { + envelope = envelopeFactory.toEnvelope(record); + } catch (RuntimeException unpublishable) { + // A row that cannot become an envelope never will. Thrown out of the pass, it took the + // rest of the batch with it: those rows stayed IN_FLIGHT until their lease expired, and the + // same row was claimed and thrown again on the next pass, forever. + if (repository.markFailed(lease, failureCode(unpublishable), now) + == OutboxTransitionResult.APPLIED) { + failed++; + } else { + stale++; + } + continue; + } + PublishResult result = publish(record, envelope); switch (result.completion()) { case CONFIRMED -> { if (repository.markPublished(lease, now) == OutboxTransitionResult.APPLIED) { @@ -220,12 +243,35 @@ public final class OutboxRelay { return new OutboxRelayReport(batch.size(), published, ambiguous, failed, stale, exhausted); } - private PublishResult publish(OutboxRecord record) { + private PublishResult publish(OutboxRecord record, MessageEnvelope envelope) { MessageDestination destination = new MessageDestination<>(record.destination(), record.messageType(), EncodedMessage.class); - return publisher - .publish(destination, envelopeFactory.toEnvelope(record), PublishOptions.defaults()) - .toCompletableFuture() - .join(); + try { + return publisher + .publish(destination, envelope, PublishOptions.defaults()) + .toCompletableFuture() + .join(); + } catch (RuntimeException unknown) { + // An exception says nothing about whether the broker took the message, so the row is retried + // under the same id like any other unknown outcome — not failed, not left IN_FLIGHT. + return new PublishResult( + PublishCompletion.AMBIGUOUS, + PublishEvidence.ambiguous(), + RoutingOutcome.UNKNOWN, + Optional.empty(), + 1, + Duration.ZERO, + Optional.of( + FailureDescriptor.of( + FailureCategory.AMBIGUOUS, + "PUBLISH_OUTCOME_UNKNOWN", + "the publisher threw " + unknown.getClass().getSimpleName()))); + } + } + + private static String failureCode(RuntimeException failure) { + return failure instanceof MessagingException known + ? known.failure().code() + : "OUTBOX_ROW_UNPUBLISHABLE"; } } diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java index 5a3ac4c..5de58c6 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java +++ b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java @@ -34,6 +34,8 @@ public final class OutboxRelayWorker implements AutoCloseable { /** How long {@link #close()} waits for a running pass when no deadline is given. */ public static final Duration DEFAULT_DRAIN_DEADLINE = Duration.ofSeconds(30); + private static final System.Logger LOG = System.getLogger(OutboxRelayWorker.class.getName()); + private final OutboxRelay relay; private final OutboxRetryScheduler scheduler; @@ -184,9 +186,14 @@ public final class OutboxRelayWorker implements AutoCloseable { } catch (RuntimeException passFailed) { // A failed pass must not stop the loop: the scheduled task's own exception would cancel every // future pass, turning one broker error into a relay that never runs again. The failure is - // counted and the next pass backs off as if nothing was published, which is true. - failedPasses.incrementAndGet(); + // counted and the next pass backs off as if nothing was published, which is true. It is also + // logged: counted silently, a relay failing every pass looked exactly like an idle one. + long failures = failedPasses.incrementAndGet(); unproductive = unproductivePasses.incrementAndGet(); + LOG.log( + System.Logger.Level.WARNING, + "outbox relay pass failed (" + failures + " failed so far); retrying with backoff", + passFailed); } completedPasses.incrementAndGet(); schedule(scheduler.backoff(unproductive)); diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql b/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql new file mode 100644 index 0000000..f62af01 --- /dev/null +++ b/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql @@ -0,0 +1,9 @@ +-- The claim now holds back a row while an older, unfinished row shares its destination and +-- ordering key. That check runs once per candidate row, so it needs an index or every claim scans +-- the backlog once per row it considers. +-- +-- md5 of the key rather than the key: ordering_key is VARCHAR(1024), and a long multi-byte key +-- would exceed the btree entry limit and fail the business transaction that appends the row. +CREATE INDEX ix_messaging_outbox_unfinished_ordering_key + ON messaging_outbox (destination, md5(ordering_key), created_at) + WHERE ordering_key IS NOT NULL AND status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT'); diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java b/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java index f7df5cb..7eb3f0a 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java +++ b/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java @@ -96,15 +96,13 @@ class OutboxOperationsTest { throw new UnsupportedOperationException(); } - @Override - public int purgePublishedBefore(Instant publishedBefore, int limit) { - return Math.min(purgePublishedBefore(publishedBefore), limit); - } + private final List limits = new ArrayList<>(); @Override - public int purgePublishedBefore(Instant publishedBefore) { + public int purgePublishedBefore(Instant publishedBefore, int limit) { cutoffs.add(publishedBefore); - return pass < deletions.size() ? deletions.get(pass++) : 0; + limits.add(limit); + return Math.min(pass < deletions.size() ? deletions.get(pass++) : 0, limit); } } @@ -130,13 +128,20 @@ class OutboxOperationsTest { } @Test - void cleanupDeletesInBoundedBatchesRatherThanOneLongStatement() { + void cleanupAsksForBoundedBatchesAndStopsAtTheFirstShortOne() { + // given RecordingRepository repository = new RecordingRepository(List.of(1000, 1000, 250)); - int removed = new OutboxCleanupJob(repository, OutboxProperties.defaults(), 10).runOnce(NOW); + // when + new OutboxCleanupJob(repository, OutboxProperties.defaults(), 10).runOnce(NOW); - assertThat(removed).isEqualTo(2250); - assertThat(repository.cutoffs).hasSize(4); + // then + assertThat(repository.limits) + .as("one DELETE over the whole backlog holds its locks for as long as the backlog is big") + .containsExactly( + OutboxCleanupJob.DEFAULT_BATCH_SIZE, + OutboxCleanupJob.DEFAULT_BATCH_SIZE, + OutboxCleanupJob.DEFAULT_BATCH_SIZE); } @Test diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java b/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java index 3a8256a..8749e0b 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java +++ b/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java @@ -86,6 +86,49 @@ class OutboxRelayTest { .isEqualTo(OutboxStatus.PUBLISHED); } + @Test + void aRowThatCannotBecomeAnEnvelopeFailsWhileTheRestOfItsBatchPublishes() { + // given + OutboxRecord poison = OutboxFixtures.withHeaders(Map.of("msg.id", "forged")); + OutboxRecord healthy = OutboxFixtures.pending(); + repository.append(poison); + repository.append(healthy); + + // when + relay(RecordingOutboxPublisher.confirming()).runOnce(NOW); + + // then + assertThat(List.of(poison, healthy)) + .extracting(row -> repository.find(row.messageId()).orElseThrow().status()) + .as("one bad row threw out of the pass and held the whole batch until its lease expired") + .containsExactly(OutboxStatus.FAILED, OutboxStatus.PUBLISHED); + } + + @Test + void aPublisherThatThrowsLeavesTheRowAmbiguousRatherThanStrandingTheBatch() { + // given + OutboxRecord first = OutboxFixtures.pending(); + repository.append(first); + MessagePublisher throwing = + new MessagePublisher() { + @Override + public CompletionStage publish( + MessageDestination destination, + MessageEnvelope message, + PublishOptions options) { + throw new IllegalStateException("the broker connection is gone"); + } + }; + + // when + relay(throwing).runOnce(NOW); + + // then + assertThat(repository.find(first.messageId()).orElseThrow().status()) + .as("an exception says nothing about whether the broker took the message") + .isEqualTo(OutboxStatus.AMBIGUOUS); + } + @Test void anAmbiguousPublishStaysRetryableRatherThanFailing() { OutboxRecord record = OutboxFixtures.pending(); @@ -263,6 +306,10 @@ final class OutboxFixtures { private OutboxFixtures() {} static OutboxRecord pending() { + return withHeaders(Map.of()); + } + + static OutboxRecord withHeaders(Map headers) { return new OutboxRecord( MessageId.newId(), new DestinationName("order-events"), @@ -270,7 +317,7 @@ final class OutboxFixtures { new SchemaVersion(1), ContentType.JSON, "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), - Map.of(), + headers, dev.caskeleton.messaging.reliability.OutboxCanonicalMetadata.none(), Instant.parse("2026-08-10T09:14:00Z"), OutboxStatus.PENDING, @@ -455,14 +502,9 @@ final class InMemoryOutboxRepository implements OutboxRepository { @Override public int purgePublishedBefore(Instant publishedBefore, int limit) { - return Math.min(purgePublishedBefore(publishedBefore), limit); - } - - @Override - public int purgePublishedBefore(Instant publishedBefore) { int before = rows.size(); rows.values().removeIf(record -> record.status() == OutboxStatus.PUBLISHED); - return before - rows.size(); + return Math.min(before - rows.size(), limit); } private void transition(MessageId messageId, OutboxStatus status, Optional failureCode) { diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java b/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java index cadc679..9f71564 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java +++ b/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java @@ -79,8 +79,7 @@ class OutboxRelayWorkerTest { @Test @DisplayName("a pass that throws does not stop every pass after it") void aFailedPassDoesNotEndTheLoop() throws InterruptedException { - repository.append(OutboxFixtures.pending()); - OutboxRelayWorker worker = worker(new ThrowingPublisher()); + OutboxRelayWorker worker = worker(unreachableDatabase(), new ThrowingPublisher()); worker.start(); try { @@ -99,9 +98,25 @@ class OutboxRelayWorkerTest { } private OutboxRelayWorker worker(MessagePublisher publisher) { + return worker(repository, publisher); + } + + /** An outbox whose every call fails, as it does while its database is down. */ + private static dev.caskeleton.messaging.reliability.OutboxRepository unreachableDatabase() { + return (dev.caskeleton.messaging.reliability.OutboxRepository) + java.lang.reflect.Proxy.newProxyInstance( + OutboxRelayWorkerTest.class.getClassLoader(), + new Class[] {dev.caskeleton.messaging.reliability.OutboxRepository.class}, + (proxy, method, arguments) -> { + throw new IllegalStateException("the outbox database is unreachable"); + }); + } + + private static OutboxRelayWorker worker( + dev.caskeleton.messaging.reliability.OutboxRepository outbox, MessagePublisher publisher) { OutboxRelay relay = new OutboxRelay( - repository, + outbox, publisher, new OutboxEnvelopeFactory(new ProducerId("order-api")), 100, diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/InFlightLimiter.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/InFlightLimiter.java index 30ef288..e15b7e6 100644 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/InFlightLimiter.java +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/InFlightLimiter.java @@ -50,7 +50,20 @@ public final class InFlightLimiter { * @throws InterruptedException when the waiting thread is interrupted */ public boolean tryAcquire() throws InterruptedException { - return permits.tryAcquire(maxWait.toNanos(), TimeUnit.NANOSECONDS); + return tryAcquire(maxWait); + } + + /** + * Tries to acquire one permit, waiting no longer than the configured bound or the given budget. + * + * @param budget what the caller has left to spend on the wait + * @return true when a permit was acquired + * @throws InterruptedException when the waiting thread is interrupted + */ + public boolean tryAcquire(Duration budget) throws InterruptedException { + Objects.requireNonNull(budget, "budget must not be null"); + long wait = Math.max(0, Math.min(maxWait.toNanos(), budget.toNanos())); + return permits.tryAcquire(wait, TimeUnit.NANOSECONDS); } /** Returns one permit to the pool. */ diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/MessagingAdmissionController.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/MessagingAdmissionController.java index 4b2cc98..118f998 100644 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/MessagingAdmissionController.java +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/MessagingAdmissionController.java @@ -73,7 +73,24 @@ public final class MessagingAdmissionController { * @throws MessageBackpressureException when shutting down or no permit became available */ public void admit(String destination, int payloadBytes) { + admit(destination, payloadBytes, java.time.Duration.ofNanos(Long.MAX_VALUE)); + } + + /** + * Admits one publish, waiting for a permit no longer than the caller has left. + * + *

    The limiter's own bound was the only one: a publish with a 100ms deadline could block its + * caller for the limiter's full wait before the deadline was even consulted. + * + * @param destination the logical destination name + * @param payloadBytes the encoded payload size + * @param budget the most this call may wait for a permit + * @throws MessageTooLargeException when the payload exceeds the destination limit + * @throws MessageBackpressureException when shutting down or no permit became available + */ + public void admit(String destination, int payloadBytes, java.time.Duration budget) { Objects.requireNonNull(destination, "destination must not be null"); + Objects.requireNonNull(budget, "budget must not be null"); payloadGuard.checkPayload(destination, payloadBytes); if (!acceptingNewWork) { @@ -91,7 +108,7 @@ public final class MessagingAdmissionController { boolean acquired; try { - acquired = limiter.tryAcquire(); + acquired = limiter.tryAcquire(budget); } catch (InterruptedException interrupted) { // The destination slot was taken a moment ago and no publish will use it, so it goes back // here: a slot leaked per interruption shrinks the destination's ceiling until it is zero. diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRepository.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRepository.java index a438805..55846b4 100644 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRepository.java +++ b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRepository.java @@ -34,19 +34,11 @@ public interface InboxRepository { boolean isProcessed(MessageId messageId, String consumerId); /** - * Deletes records older than a cutoff. + * Deletes at most {@code limit} processed rows older than the cutoff. * *

    Retention must outlive the broker's maximum redelivery window, otherwise a late redelivery * arrives after its inbox row was pruned and is processed a second time. * - * @param processedBefore the retention cutoff - * @return how many rows were removed - */ - int purgeProcessedBefore(Instant processedBefore); - - /** - * Deletes at most {@code limit} processed rows older than the cutoff. - * * @return how many rows were deleted; fewer than {@code limit} means the sweep is finished */ int purgeProcessedBefore(Instant processedBefore, int limit); diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java index 6552a3b..8197a2c 100644 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java +++ b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java @@ -94,22 +94,12 @@ public interface OutboxRepository { */ Optional find(MessageId messageId); - /** - * Deletes records published before a cutoff. - * - * @param publishedBefore the retention cutoff - * @return how many rows were removed - */ - int purgePublishedBefore(Instant publishedBefore); - /** * Deletes at most {@code limit} published rows older than the cutoff. * - *

    The unbounded version deletes everything before the cutoff in one statement. On a table that - * has been accumulating published rows since the last sweep that is a single long transaction - * holding locks and generating WAL in proportion to the backlog, which shows up as the relay and - * the business writes stalling behind retention. The cleanup jobs describe themselves as bounded - * by batch size; this is the parameter that makes that true. + *

    There is no unbounded version. Deleting everything before the cutoff in one statement is a + * single long transaction holding locks and generating WAL in proportion to the backlog, which + * shows up as the relay and the business writes stalling behind retention. * * @return how many rows were deleted; fewer than {@code limit} means the sweep is finished */ diff --git a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java index fe59a3e..46c7546 100644 --- a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java +++ b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java @@ -224,7 +224,16 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM startedAt); } - admission.admit(destination.name().value(), encoded.payload().bytes().length); + // Refusals are results, not exceptions. They were thrown out of publish() on the caller's + // thread, so a CompletionStage caller — the outbox relay, the reactive facade — saw neither a + // REJECTED result nor a failed stage. The permit wait is bounded by what is left of the + // caller's deadline, not by the limiter's own bound. + try { + admission.admit(destination.name().value(), encoded.payload().bytes().length, remaining); + } catch (dev.caskeleton.messaging.api.error.MessageTooLargeException + | dev.caskeleton.messaging.api.error.MessageBackpressureException refused) { + return refused(refused.failure(), startedAt); + } MessagingRuntimeLease lease = null; try { lease = runtimes.acquire(profile.broker()); @@ -316,6 +325,12 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM } private CompletionStage rejected(String code, String reason, long startedAt) { + return refused( + new FailureDescriptor(FailureCategory.CONFIGURATION, code, false, reason, Optional.empty()), + startedAt); + } + + private CompletionStage refused(FailureDescriptor failure, long startedAt) { return CompletableFuture.completedFuture( new PublishResult( PublishCompletion.REJECTED, @@ -324,9 +339,7 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM Optional.empty(), 1, elapsedSince(startedAt), - Optional.of( - new FailureDescriptor( - FailureCategory.CONFIGURATION, code, false, reason, Optional.empty())))); + Optional.of(failure))); } private PublishResult ambiguousResult(String code, String reason, long startedAt) { diff --git a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java index 5302ed6..4ac4854 100644 --- a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java +++ b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java @@ -312,6 +312,89 @@ class DefaultMessagePublisherTest { assertThat(admission.inFlight()).isZero(); } + @Test + void anOversizedPayloadIsRejectedAsNotRetryableInsteadOfThrown() { + // given + DefaultMessagePublisher publisher = + new DefaultMessagePublisher( + registry(), + allowOrders(), + new SingleCodecRegistry(), + new MessagingAdmissionController( + new PayloadLimitGuard(new PayloadPolicy(4, 4)), + new InFlightLimiter(4, Duration.ofSeconds(1))), + new FixedRuntimeRegistry(transport), + transport); + + // when + CompletableFuture result = + CompletableFuture.supplyAsync( + () -> + publisher + .publish( + new MessageDestination<>( + ORDERS, new MessageType("order.created"), String.class), + envelope(), + PublishOptions.defaults()) + .toCompletableFuture() + .join()); + + // then + assertThat(result) + .succeedsWithin(Duration.ofSeconds(5)) + .extracting( + PublishResult::completion, outcome -> outcome.failure().orElseThrow().retryable()) + .containsExactly(PublishCompletion.REJECTED, false); + } + + @Test + void aSaturatedPermitPoolRejectsAsRetryableWithinTheCallersDeadline() { + // given + DestinationName audit = new DestinationName("audit-events"); + DefaultMessagePublisher publisher = + new DefaultMessagePublisher( + DestinationProfileRegistry.of( + Map.of( + ORDERS, + RuntimeFixtures.profile(ORDERS), + audit, + RuntimeFixtures.profile(audit))), + new DestinationAccessPolicy( + java.util.Set.of(ORDERS, audit), java.util.Set.of(), java.util.Set.of()), + new SingleCodecRegistry(), + new MessagingAdmissionController( + new PayloadLimitGuard(PayloadPolicy.defaults()), + new InFlightLimiter(1, Duration.ofMinutes(1))), + new FixedRuntimeRegistry(transport), + transport); + transport.stall(); + publisher.publish( + new MessageDestination<>(ORDERS, new MessageType("order.created"), String.class), + envelope(), + withTimeout(Duration.ofMinutes(1))); + + // when + CompletableFuture second = + CompletableFuture.supplyAsync( + () -> + publisher + .publish( + new MessageDestination<>( + audit, new MessageType("order.created"), String.class), + envelope(), + withTimeout(Duration.ofMillis(100))) + .toCompletableFuture() + .join()); + + // then + assertThat(second) + .as("the permit wait was the limiter's own minute, not what was left of 100ms") + .succeedsWithin(Duration.ofSeconds(5)) + .extracting( + PublishResult::completion, outcome -> outcome.failure().orElseThrow().retryable()) + .containsExactly(PublishCompletion.REJECTED, true); + } + @Test @DisplayName("an unavailable runtime releases the permit it took") void anUnavailableRuntimeReleasesThePermit() { diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java index e4eea40..49a1ccb 100644 --- a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java @@ -121,15 +121,14 @@ public class KafkaMessagingAutoConfiguration { /** * The producer this transport publishes through (MSG-INT-003). * - *

    Built from Spring Boot's own {@code spring.kafka.bootstrap-servers} rather than from a - * second description on {@code app.messaging.*}. One broker described twice is the defect this - * repository has already paid for in {@code app.jpa-platform.datasource.*}, and the - * operator-facing property for "where is Kafka" is the one they already know. + *

    Built from the one {@code app.messaging.brokers.} entry of type {@code kafka}. It used + * to read {@code spring.kafka.bootstrap-servers} while the compiler required and validated the + * entry's own {@code bootstrap-servers}, so one broker was described twice and only one of the + * two descriptions reached the client. * *

    Byte-array serializers on purpose: the platform's codec registry decides the wire format, * and a serializer configured here would be a second encoder disagreeing with it. * - * @param environment the resolved environment * @param compiled the broker and security profiles compiled from {@code app.messaging} * @param declaredSecurity the security profiles the application declared as beans * @param securityConfigurer the Kafka security configurer, present only with a credential source @@ -138,45 +137,75 @@ public class KafkaMessagingAutoConfiguration { @Bean(destroyMethod = "close") @ConditionalOnMissingBean public org.apache.kafka.clients.producer.Producer messagingKafkaProducer( - org.springframework.core.env.Environment environment, CompiledMessagingConfiguration compiled, ObjectProvider declaredSecurity, ObjectProvider securityConfigurer) { return new org.apache.kafka.clients.producer.KafkaProducer<>( - producerConfig(environment, compiled, declaredSecurity, securityConfigurer)); + producerConfig(compiled, declaredSecurity, securityConfigurer)); } /** The producer's client configuration; package-visible so a test can read what it holds. */ static java.util.Map producerConfig( - org.springframework.core.env.Environment environment, CompiledMessagingConfiguration compiled, ObjectProvider declaredSecurity, ObjectProvider securityConfigurer) { - String bootstrapServers = environment.getProperty("spring.kafka.bootstrap-servers", "").trim(); - if (bootstrapServers.isEmpty()) { - throw new IllegalStateException( - "app.messaging.broker=kafka needs spring.kafka.bootstrap-servers; a producer with no " - + "broker address defaults to localhost:9092 and fails at the first publish rather " - + "than at startup"); - } + BrokerSettings broker = kafkaBroker(compiled); java.util.Map config = new java.util.HashMap<>(); config.put( org.apache.kafka.clients.producer.ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, - bootstrapServers); + String.join(",", broker.bootstrapServers())); config.put( org.apache.kafka.clients.producer.ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, org.apache.kafka.common.serialization.ByteArraySerializer.class); config.put( org.apache.kafka.clients.producer.ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, org.apache.kafka.common.serialization.ByteArraySerializer.class); - // Durability is not a default worth inheriting: acks=1 loses an accepted publish to a leader - // failover, which is exactly the outcome an outbox exists to prevent. - config.put(org.apache.kafka.clients.producer.ProducerConfig.ACKS_CONFIG, "all"); - config.put(org.apache.kafka.clients.producer.ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true); + // The entry's delivery settings, which KafkaProfileValidator checks (a stable broker must keep + // acks=all and idempotence) and which used to be replaced here by hard-coded values. + config.put(org.apache.kafka.clients.producer.ProducerConfig.ACKS_CONFIG, broker.acks()); + config.put( + org.apache.kafka.clients.producer.ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, + broker.enableIdempotence()); + config.put( + org.apache.kafka.clients.producer.ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, + broker.maxInFlightRequestsPerConnection()); + // Kafka refuses delivery.timeout.ms below linger.ms + request.timeout.ms (5ms + 30s by + // default), which made the documented 30s a startup failure. The request timeout follows a + // short delivery deadline down, and records are not held back to fill a batch. + long delivery = broker.deliveryTimeout().toMillis(); + config.put(org.apache.kafka.clients.producer.ProducerConfig.LINGER_MS_CONFIG, 0); + config.put( + org.apache.kafka.clients.producer.ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, + Math.toIntExact(Math.min(30_000L, delivery))); + config.put( + org.apache.kafka.clients.producer.ProducerConfig.DELIVERY_TIMEOUT_MS_CONFIG, + Math.toIntExact(delivery)); + // send() blocks the caller while it waits for metadata or buffer space, for Kafka's default + // 60s — before the publish deadline is even applied. Bounded by the default deadline instead. + config.put( + org.apache.kafka.clients.producer.ProducerConfig.MAX_BLOCK_MS_CONFIG, + dev.caskeleton.messaging.api.publish.PublishOptions.defaults().timeout().toMillis()); config.putAll(producerSecurity(compiled, declaredSecurity, securityConfigurer)); return config; } + /** The one Kafka broker entry the producer connects to. */ + private static BrokerSettings kafkaBroker(CompiledMessagingConfiguration compiled) { + java.util.List kafka = + compiled.brokers().values().stream() + .filter(broker -> BrokerSettings.KAFKA.equalsIgnoreCase(broker.type())) + .toList(); + if (kafka.size() != 1) { + throw new IllegalStateException( + "app.messaging.broker=kafka needs exactly one app.messaging.brokers. entry with" + + " type kafka and its bootstrap-servers; found " + + kafka.size() + + ". A producer with no broker address defaults to localhost:9092 and fails at the" + + " first publish rather than at startup"); + } + return kafka.getFirst(); + } + /** * The producer's security properties, from the one security profile of the Kafka broker. * diff --git a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java index 4edfe2b..65c4340 100644 --- a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java +++ b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java @@ -77,7 +77,8 @@ class MessagingLiveRoundTripQualificationTest { .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=" + KAFKA.getBootstrapServers()) + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=" + KAFKA.getBootstrapServers()) // Both, because ValidatedDestinationRegistry refuses a profile whose dead-letter target is // not itself registered — a good invariant, and one a fixture declaring only the happy path // walks straight into. diff --git a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java index 5d34659..5e823a3 100644 --- a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java +++ b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java @@ -159,7 +159,6 @@ class MessagingSecuredBrokerQualificationTest { .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=" + kafka.getBootstrapServers(), "app.messaging.brokers.kafka.type=kafka", "app.messaging.brokers.kafka.bootstrap-servers[0]=" + kafka.getBootstrapServers(), "app.messaging.brokers.kafka.tls-enabled=true", diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java index a17c0e3..5c3a25b 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java @@ -34,6 +34,10 @@ public final class DefaultReactiveMessagePublisher implements ReactiveMessagePub Objects.requireNonNull(destination, "destination must not be null"); Objects.requireNonNull(message, "message must not be null"); Objects.requireNonNull(options, "options must not be null"); - return Mono.fromCompletionStage(() -> delegate.publish(destination, message, options)); + // publish() is not purely asynchronous: admission may wait for a permit and the Kafka producer + // may wait for metadata before the stage exists. Subscribing on a Netty event loop would pin + // that loop for the wait, so the call runs on the bounded elastic pool instead. + return Mono.fromCompletionStage(() -> delegate.publish(destination, message, options)) + .subscribeOn(reactor.core.scheduler.Schedulers.boundedElastic()); } } diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java index 72350cc..0fa0d2d 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java @@ -109,6 +109,33 @@ public class MessagingReliabilityAutoConfiguration { return new OutboxRelayWorker(relay, scheduler); } + /** + * Publishes the relay's failed-pass count when a meter registry exists. + * + *

    {@code failedPasses()} was counted and read by nothing, so a relay failing every pass looked + * on every dashboard exactly like one with nothing to do. + * + * @param worker the relay worker + * @param registries the meter registry, when the application has one + * @return the registration, run once the context has the registry + */ + @Bean + @ConditionalOnBean(OutboxRelayWorker.class) + public org.springframework.beans.factory.InitializingBean outboxRelayMeters( + OutboxRelayWorker worker, + org.springframework.beans.factory.ObjectProvider + registries) { + return () -> + registries.ifAvailable( + registry -> + io.micrometer.core.instrument.FunctionCounter.builder( + "messaging.outbox.relay.failed.passes", + worker, + OutboxRelayWorker::failedPasses) + .description("Outbox relay passes that ended in an exception") + .register(registry)); + } + /** * Binds the relay worker to the context lifecycle. * diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java index fef555d..936c710 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java @@ -38,7 +38,6 @@ class BrokerAccessListTest { .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=localhost:9092", "app.messaging.brokers.kafka-primary.type=kafka", "app.messaging.brokers.kafka-primary.bootstrap-servers[0]=localhost:9092", "app.messaging.security.kafka-primary.producer.type=SASL_SCRAM", diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java new file mode 100644 index 0000000..643a088 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java @@ -0,0 +1,96 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.publish.PublishOptions; +import dev.caskeleton.messaging.kafka.KafkaSecurityConfigurer; +import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import java.util.Map; +import org.junit.jupiter.api.Test; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.assertj.AssertableApplicationContext; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; + +/** + * Where the producer connects and how it delivers come from the one broker entry. + * + *

    The producer read {@code spring.kafka.bootstrap-servers} while the compiler required and + * validated {@code app.messaging.brokers..bootstrap-servers}, so a broker was described twice + * and only one description was used. The entry's delivery settings were validated and then replaced + * by hard-coded values. + */ +class KafkaProducerConfigTest { + + private final ApplicationContextRunner runner = + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "app.messaging.brokers.kafka-primary.type=kafka", + // Resolvable names: the client refuses an address it cannot look up at construction. + "app.messaging.brokers.kafka-primary.bootstrap-servers[0]=localhost:9093", + "app.messaging.brokers.kafka-primary.bootstrap-servers[1]=127.0.0.1:9094"); + + @Test + void theProducerConnectsToTheBrokerEntryAddressesNotToSpringKafka() { + runner + .withPropertyValues("spring.kafka.bootstrap-servers=elsewhere:9092") + .run( + context -> + assertThat(producerConfig(context).get("bootstrap.servers")) + .isEqualTo("localhost:9093,127.0.0.1:9094")); + } + + @Test + void theProducerAppliesTheBrokerEntryDeliverySettings() { + runner + .withPropertyValues( + "app.messaging.brokers.kafka-primary.enable-idempotence=false", + "app.messaging.brokers.kafka-primary.acks=1", + "app.messaging.brokers.kafka-primary.max-in-flight-requests-per-connection=3", + "app.messaging.brokers.kafka-primary.delivery-timeout=45s") + .run( + context -> + assertThat(producerConfig(context)) + .extractingByKeys( + "enable.idempotence", + "acks", + "max.in.flight.requests.per.connection", + "delivery.timeout.ms") + .containsExactly(false, "1", 3, 45_000)); + } + + @Test + void theProducerStopsWaitingForMetadataWithinTheDefaultPublishDeadline() { + runner.run( + context -> + assertThat(producerConfig(context).get("max.block.ms")) + .as("the caller's thread waited up to Kafka's own 60s before any deadline applied") + .isEqualTo(PublishOptions.defaults().timeout().toMillis())); + } + + @Test + void aBrokerWithoutAnEntryFailsStartupNamingTheKey() { + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "spring.kafka.bootstrap-servers=localhost:9092") + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .rootCause() + .hasMessageContaining("app.messaging.brokers. entry with type kafka")); + } + + private static Map producerConfig(AssertableApplicationContext context) { + return KafkaMessagingAutoConfiguration.producerConfig( + context.getBean(CompiledMessagingConfiguration.class), + context.getBeanProvider(BrokerSecurityProfile.class), + context.getBeanProvider(KafkaSecurityConfigurer.class)); + } +} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java index 2efd948..98c4bde 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java @@ -39,7 +39,6 @@ class KafkaProducerSecurityTest { .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=localhost:9092", "app.messaging.brokers.kafka-primary.type=kafka", "app.messaging.brokers.kafka-primary.bootstrap-servers[0]=localhost:9092"); @@ -228,7 +227,6 @@ class KafkaProducerSecurityTest { private static Map producerConfig(AssertableApplicationContext context) { return KafkaMessagingAutoConfiguration.producerConfig( - context.getEnvironment(), context.getBean(CompiledMessagingConfiguration.class), context.getBeanProvider(BrokerSecurityProfile.class), context.getBeanProvider(KafkaSecurityConfigurer.class)); diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java index 6c82922..0d4708c 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java @@ -314,7 +314,9 @@ class MessagingAutoConfigurationTest { org.springframework.boot.autoconfigure.AutoConfigurations.of( MessagingCoreAutoConfiguration.class, KafkaMessagingAutoConfiguration.class)) .withUserConfiguration(PublisherConfiguration.class, AutoCommittingKafkaProfile.class) - .withPropertyValues("spring.kafka.bootstrap-servers=localhost:9092") + .withPropertyValues( + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=localhost:9092") .run( context -> org.assertj.core.api.Assertions.assertThat(context) diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java index 73f8a08..c3239aa 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java @@ -258,9 +258,7 @@ class MessagingConfigurationBindingTest { runner .withConfiguration(AutoConfigurations.of(KafkaMessagingAutoConfiguration.class)) .withPropertyValues(minimal()) - .withPropertyValues( - "spring.kafka.bootstrap-servers=localhost:9092", - "app.messaging.brokers.kafka-primary.production=true") + .withPropertyValues("app.messaging.brokers.kafka-primary.production=true") .run( context -> assertThat(context) diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java index 6527539..5c9e3b0 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java @@ -73,6 +73,55 @@ class MessagingOutboxRelayLifecycleTest { .isFalse(); } + @Test + void aRelayPassThatFailsIsCountedOnTheMeterRegistry() { + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingReliabilityAutoConfiguration.class)) + .withBean( + io.micrometer.core.instrument.MeterRegistry.class, + io.micrometer.core.instrument.simple.SimpleMeterRegistry::new) + .withBean( + OutboxRepository.class, + () -> + (OutboxRepository) + java.lang.reflect.Proxy.newProxyInstance( + getClass().getClassLoader(), + new Class[] {OutboxRepository.class}, + (proxy, method, arguments) -> { + throw new IllegalStateException("the outbox database is unreachable"); + })) + .withBean( + OutboxEnvelopeFactory.class, + () -> new OutboxEnvelopeFactory(new ProducerId("order-api"))) + .withBean(MessagePublisher.class, () -> new RelayDependencies().messagePublisher()) + .run( + context -> { + // given + io.micrometer.core.instrument.MeterRegistry registry = + context.getBean(io.micrometer.core.instrument.MeterRegistry.class); + + // when + double failed = awaitFailedPasses(registry); + + // then + assertThat(failed) + .as("a pass that failed was swallowed with no log and a counter nothing read") + .isPositive(); + }); + } + + /** Reads the failed-pass counter until the relay's first pass has had time to fail. */ + private static double awaitFailedPasses(io.micrometer.core.instrument.MeterRegistry registry) + throws InterruptedException { + long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(5); + double failed = 0; + while (failed == 0 && System.nanoTime() < deadline) { + Thread.sleep(10); + failed = registry.get("messaging.outbox.relay.failed.passes").functionCounter().count(); + } + return failed; + } + /** The two beans only the application can supply, plus the publisher the platform needs. */ @Configuration(proxyBeanMethods = false) static class RelayDependencies { @@ -170,11 +219,6 @@ class MessagingOutboxRelayLifecycleTest { throw new UnsupportedOperationException("not used by this test"); } - @Override - public int purgePublishedBefore(Instant publishedBefore) { - throw new UnsupportedOperationException("not used by this test"); - } - @Override public int purgePublishedBefore(Instant publishedBefore, int limit) { throw new UnsupportedOperationException("not used by this test"); diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java index 4da59b9..e611c07 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java @@ -41,7 +41,9 @@ class MessagingStarterOffContractTest { void presentWithoutTheSwitchIsOn() { runner .withPropertyValues( - "app.messaging.broker=kafka", "spring.kafka.bootstrap-servers=localhost:9092") + "app.messaging.broker=kafka", + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=localhost:9092") .run( context -> { assertThat(context).hasNotFailed(); @@ -123,7 +125,8 @@ class MessagingStarterOffContractTest { .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=localhost:9092") + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=localhost:9092") .run( context -> { assertThat(context).hasNotFailed(); @@ -138,7 +141,8 @@ class MessagingStarterOffContractTest { .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=localhost:9092") + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=localhost:9092") .run( context -> { assertThat(context) @@ -165,7 +169,8 @@ class MessagingStarterOffContractTest { .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=localhost:9092") + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=localhost:9092") .run( context -> { assertThat(context) @@ -203,7 +208,7 @@ class MessagingStarterOffContractTest { .hasFailed() .getFailure() .rootCause() - .hasMessageContaining("spring.kafka.bootstrap-servers")); + .hasMessageContaining("app.messaging.brokers.")); } /** diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java index 21931ea..91e82fe 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java @@ -91,6 +91,34 @@ class ReactiveFacadeTest { .verify(); } + @Test + void theBlockingPublishRunsOffTheSubscribingThread() { + // given + java.util.concurrent.atomic.AtomicReference publishedOn = + new java.util.concurrent.atomic.AtomicReference<>(); + MessagePublisher recording = + new MessagePublisher() { + @Override + public CompletionStage publish( + MessageDestination destination, + MessageEnvelope message, + PublishOptions options) { + publishedOn.set(Thread.currentThread()); + return CompletableFuture.completedFuture(confirmed()); + } + }; + + // when + new DefaultReactiveMessagePublisher(recording) + .publish(ORDERS, envelope(), PublishOptions.defaults()) + .block(); + + // then + assertThat(publishedOn.get()) + .as("admission and the producer's metadata wait block; an event loop must not pay for it") + .isNotSameAs(Thread.currentThread()); + } + private MessagePublisher counting() { return new MessagePublisher() { @Override From 6397f52dc88ab22f69ca3faa9f5c5cac7eb36ca8 Mon Sep 17 00:00:00 2001 From: donghyeon-ka Date: Tue, 29 Sep 2026 12:30:01 +0900 Subject: [PATCH 4/7] =?UTF-8?q?review-followup=20Wave=203=20=EB=9D=BC?= =?UTF-8?q?=EC=9A=B4=EB=93=9C=202(P18):=20outbox=20canonical=20=ED=86=B5?= =?UTF-8?q?=EC=9D=BC,=20composition=20=EB=AA=A8=EB=93=88,=20legacy=20Kafka?= =?UTF-8?q?=20seam=20=EC=82=AD=EC=A0=9C?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 결정 7: `LegacyOutboxAppendPort`·legacy 저장 모델·relay·Kafka R0 seam 삭제, 템플릿 JSON encoder(`JsonIntegrationEventEncoder`), durable operation·sample 이벤트가 `OutboxAppendPort` + encoder로, persistence-jpa V19(pre-canonical 미발행 row 거부) - 결정 3: outbox relay·멱등성 executor 조립을 `composition:outbox-relay`·`composition:idempotency`로, sample이 복사한 bootstrap 인프라를 `composition:service-runtime`으로(sample `SampleIdempotencyConfig` 복사본 삭제) - persistence-jpa outbox·멱등성 패키지 feature 기준 이동, messaging outbox·inbox leaf를 reliability / reliability-jdbc-postgresql로 - messaging outbox header를 jsonb로(P18-new-1 — `\`로 끝나는 값이 다음 header를 삼키던 결함) - 통합(충돌 22건): P25 A-10c(전략 선택 설정 삭제)를 service-runtime `DomainContextConfig`에 적용, P25 startup 예외 통합을 composition 설정 테스트에, P25 `CommandUseCase.handle` 이름을 PG 통합 테스트에, P15 `EventJson`은 호출자 0이 되어 삭제 - root CLAUDE.md의 legacy seam 문장 수정(P18-new-2) 검증: ./gradlew check architectureCheck verifyDependencyLocks verifyPublicPathSnapshot :app-bootstrap:verifyEnvKeys configContractCheck :app-bootstrap:integrationTest :sample-portfolio:integrationTest :sample-portfolio:systemTest persistence-jpa jpaPlatformContractTest·postgresql{Idempotency,OutboxAppendTransactional,OutboxConcurrentRelay,OutboxRowLifecycle}IntegrationTest :messaging:messaging-reliability-jdbc-postgresql:integrationTest --continue 통과(700 task) Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/messaging/configuration-reference.md | 20 +- docs/messaging/outbox-inbox.md | 6 +- docs/messaging/support-matrix.md | 6 +- messaging/CLAUDE.md | 41 +-- .../build.gradle | 33 --- .../gradle.lockfile | 111 -------- .../reliability/OutboxRepository.java | 43 +-- .../build.gradle | 12 +- .../gradle.lockfile | 0 .../messaging/inbox/InboxPostgresIT.java | 0 .../AdminOperationJournalPostgresIT.java | 0 .../messaging/outbox/OutboxPostgresIT.java | 107 +++++--- .../messaging/inbox/JdbcInboxRepository.java | 0 .../outbox/JdbcAdminOperationJournal.java | 0 .../outbox/JdbcOutboxRepository.java | 254 +++--------------- .../messaging/V1001__messaging_outbox.sql | 0 .../V1002__messaging_outbox_lease_fencing.sql | 0 ...003__messaging_admin_operation_journal.sql | 0 ...4__messaging_outbox_canonical_metadata.sql | 0 ...5__messaging_outbox_ordering_key_index.sql | 0 .../messaging/V1101__messaging_inbox.sql | 0 .../JdbcInboxTransactionRequirementTest.java | 0 .../JdbcOutboxTransactionRequirementTest.java | 0 messaging/messaging-reliability/build.gradle | 19 ++ .../messaging-reliability/gradle.lockfile | 89 ++++++ .../messaging/inbox/IdempotentConsumer.java | 0 .../messaging/inbox/InboxCleanupJob.java | 0 .../messaging/inbox/InboxOutcome.java | 0 .../messaging/inbox/InboxRetentionPolicy.java | 0 .../inbox/TransactionalInboxHandler.java | 0 .../messaging/outbox/OutboxCleanupJob.java | 0 .../outbox/OutboxEnvelopeFactory.java | 0 .../messaging/outbox/OutboxProperties.java | 0 .../messaging/outbox/OutboxRelay.java | 2 +- .../messaging/outbox/OutboxRelayReport.java | 0 .../messaging/outbox/OutboxRelayWorker.java | 0 .../outbox/OutboxRetryScheduler.java | 0 .../inbox/IdempotentConsumerTest.java | 0 .../messaging/inbox/InboxOperationsTest.java | 0 .../outbox/OutboxEnvelopeFactoryTest.java | 0 .../outbox/OutboxOperationsTest.java | 19 +- .../messaging/outbox/OutboxRelayTest.java | 46 ++-- .../outbox/OutboxRelayWorkerTest.java | 0 .../build.gradle | 5 +- .../MessagingOutboxRelayLifecycleTest.java | 17 -- 45 files changed, 298 insertions(+), 532 deletions(-) delete mode 100644 messaging/messaging-inbox-jdbc-postgresql/build.gradle delete mode 100644 messaging/messaging-outbox-jdbc-postgresql/gradle.lockfile rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/build.gradle (69%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/gradle.lockfile (100%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/integrationTest/java/dev/caskeleton/messaging/outbox/AdminOperationJournalPostgresIT.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java (86%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/main/java/dev/caskeleton/messaging/outbox/JdbcAdminOperationJournal.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java (71%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/main/resources/db/migration/messaging/V1001__messaging_outbox.sql (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/main/resources/db/migration/messaging/V1002__messaging_outbox_lease_fencing.sql (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/main/resources/db/migration/messaging/V1003__messaging_admin_operation_journal.sql (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/main/resources/db/migration/messaging/V1004__messaging_outbox_canonical_metadata.sql (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql (100%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/main/resources/db/migration/messaging/V1101__messaging_inbox.sql (100%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/test/java/dev/caskeleton/messaging/inbox/JdbcInboxTransactionRequirementTest.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability-jdbc-postgresql}/src/test/java/dev/caskeleton/messaging/outbox/JdbcOutboxTransactionRequirementTest.java (100%) create mode 100644 messaging/messaging-reliability/build.gradle create mode 100644 messaging/messaging-reliability/gradle.lockfile rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/inbox/IdempotentConsumer.java (100%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java (100%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/inbox/InboxOutcome.java (100%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/inbox/InboxRetentionPolicy.java (100%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactory.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/outbox/OutboxProperties.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java (99%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayReport.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/main/java/dev/caskeleton/messaging/outbox/OutboxRetryScheduler.java (100%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability}/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java (100%) rename messaging/{messaging-inbox-jdbc-postgresql => messaging-reliability}/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/test/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactoryTest.java (100%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java (91%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java (95%) rename messaging/{messaging-outbox-jdbc-postgresql => messaging-reliability}/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java (100%) diff --git a/docs/messaging/configuration-reference.md b/docs/messaging/configuration-reference.md index 2dd4be8..0f87d73 100644 --- a/docs/messaging/configuration-reference.md +++ b/docs/messaging/configuration-reference.md @@ -21,20 +21,16 @@ Application의 canonical integration event를 platform publish pipeline으로 producing-service identity다. 값이 없으면 `IntegrationEventPublishPort` bridge 자체를 만들지 않는다. `spring.application.name`이나 -현재 process 이름으로 추론하지 않는다. 기존 legacy `OutboxEvent`/realtime 경로는 별도 cutover가 -끝날 때까지 `app.messaging.broker` 경로를 유지한다. +현재 process 이름으로 추론하지 않는다. 모든 발행(outbox relay 포함)이 이 bridge를 지나므로 messaging +모듈이 켜진 상태에서 값이 없으면 기동이 거부된다. -## Outbox canonical transport-only cutover +## Outbox -`APP_OUTBOX_CANONICAL_TRANSPORT_ENABLED` / `ca-skeleton.outbox.canonical-transport-enabled`은 -기존 `outbox_event` writer/claim/status authority를 유지한 채 canonical row의 **transport만** platform -publish path로 보내는 compatibility gate다. 기본값은 `false`이며 두 번째 relay를 만들지 않는다. - -`true`일 때는 `OutboxAppendPort`가 `ValidatedIntegrationEvent`의 exact envelope bytes와 canonical -metadata를 기존 outbox row에 저장하고, claim된 canonical row는 `IntegrationEventPublishPort`로 간다. -legacy row는 계속 `MessageBroker`를 사용한다. 따라서 mixed-row compatibility 기간에는 relay가 켜져 -있다면 `app.messaging.broker`도 계속 필요하며, canonical path를 위해 `IntegrationEventPublishPort`도 -추가로 필요하다. legacy backlog가 0이라는 별도 증거 없이 broker 요구를 제거하지 않는다. +outbox는 canonical 하나다(2026-09-28). use case가 `IntegrationEventEncoderPort`로 만든 +`ValidatedIntegrationEvent`를 `OutboxAppendPort`로 append하고, relay는 claim한 row의 exact envelope +bytes를 `IntegrationEventPublishPort`로 보낸다. 예전의 `ca-skeleton.outbox.canonical-transport-enabled` +gate와 legacy `MessageBroker → KafkaSender` 경로는 삭제됐다. 발행할 contract의 logical destination마다 +아래 destination profile을 선언한다(템플릿의 durable operation event는 `durable-operations`). ## Destination profile diff --git a/docs/messaging/outbox-inbox.md b/docs/messaging/outbox-inbox.md index fede443..d71c30f 100644 --- a/docs/messaging/outbox-inbox.md +++ b/docs/messaging/outbox-inbox.md @@ -21,8 +21,8 @@ Flyway version `1000..1999`를 예약해서 애플리케이션의 일반적인 classpath:db/migration/messaging ``` -현재 순서는 `V1001..V1005`(outbox/admin)와 `V1101`(inbox)이다. starter 테스트가 두 JDBC -모듈을 실제 combined classpath로 올려 version 중복과 예약 대역 이탈을 검사한다. 개별 DDL의 +현재 순서는 `V1001..V1005`(outbox/admin)와 `V1101`(inbox)이고, 모두 +`messaging-reliability-jdbc-postgresql`에 있다. starter 테스트가 그 모듈을 실제 combined classpath로 올려 version 중복과 예약 대역 이탈을 검사한다. 개별 DDL의 의미와 전환 순서는 [migration-guide.md](migration-guide.md)를 따른다. ## Outbox @@ -39,7 +39,7 @@ COMMIT; ### relay ```text -leaseBatch(100, 30s) -- lease로 다중 relay 인스턴스 안전 +claimBatch(owner, 100, 30s, now, maxAttempts) -- owner + fencing token으로 다중 relay 인스턴스 안전 → publish (messageId 그대로) → CONFIRMED → markPublished → AMBIGUOUS → markAmbiguous (같은 messageId로 재시도 가능) diff --git a/docs/messaging/support-matrix.md b/docs/messaging/support-matrix.md index 84c9647..bd3164c 100644 --- a/docs/messaging/support-matrix.md +++ b/docs/messaging/support-matrix.md @@ -17,8 +17,10 @@ > `BrokerFailureMatrix.shipped()` 하드코딩이 아니라 레인이 낸 증거(`BrokerCertificationEvidence`)에서 > 나온다. 증거가 없는 조합은 `NOT_COVERED`다 (MSG-014). -> **모듈 이름과 런타임 편입.** `messaging-outbox-jdbc-postgresql` / `messaging-inbox-jdbc-postgresql`은 -> 이전에 `-jpa`로 불렸다. 구현은 Spring JDBC이고 SQL은 PostgreSQL 전용(`?::jsonb`, +> **모듈 이름과 런타임 편입.** 관계형 신뢰성 구현은 `messaging-reliability-jdbc-postgresql`, +> 그 port만 보는 relay·consumer 로직은 `messaging-reliability`다(2026-09-28에 +> `messaging-outbox-jdbc-postgresql` / `messaging-inbox-jdbc-postgresql`을 이렇게 나눴다). 그 전에는 +> `-jpa`로 불렸다. 구현은 Spring JDBC이고 SQL은 PostgreSQL 전용(`jsonb`, > `FOR UPDATE SKIP LOCKED`, `ON CONFLICT`, `TIMESTAMPTZ`)이므로, 그 이름은 쓰지 않는 기술을 > 광고하고 vendor 중립 port(`messaging-reliability-api`)의 위치를 가렸다 (MSG-023). > diff --git a/messaging/CLAUDE.md b/messaging/CLAUDE.md index 6b2ad64..57e6932 100644 --- a/messaging/CLAUDE.md +++ b/messaging/CLAUDE.md @@ -16,16 +16,19 @@ leaf의 정확한 id는 registry에서 읽는다. family는 **디렉터리 이 | --- | --- | --- | | `*-api` (`core-api`, `schema-api`, `policy`, `reliability-api`, `admin-api`, `transport-spi`) | 값 타입, 포트, 정책 계약 | Java stdlib만. Spring/broker SDK/logging framework 금지 | | `schema-*` (json/avro/protobuf), `cloudevents` | 인코딩 구현 | 해당 codec 라이브러리만. Spring 금지 | -| `runtime-core`, `security`, `observability`, `admin-runtime` | framework-free 실행 로직 | Micrometer는 `observability`에서만. 나머지는 stdlib | +| `runtime-core`, `security`, `observability`, `admin-runtime`, `reliability` | framework-free 실행 로직 | Micrometer는 `observability`에서만. 나머지는 stdlib | | `kafka` | broker adapter | 해당 broker client. Spring은 client가 곧 Spring인 경우만 | -| `outbox-jdbc-postgresql`, `inbox-jdbc-postgresql` | 관계형 신뢰성 adapter | Spring JDBC / Spring TX. **JPA 아님** — 이름이 곧 계약이다 | +| `reliability-jdbc-postgresql` | 관계형 신뢰성 adapter (outbox·inbox repository, admin journal) | Spring JDBC / Spring TX. **JPA 아님** — 이름이 곧 계약이다 | | `spring-boot-starter`, `spring-boot-starter-kafka` | 조립 경계 | Spring Boot autoconfigure. 여기서만 | | `testkit` | 테스트 지원 | Testcontainers, JUnit. production leaf가 의존하지 않는다 | -`outbox`/`inbox` leaf가 `jdbc-postgresql`로 명명된 이유: 구현은 Spring JDBC이고 SQL은 -`?::jsonb` · `FOR UPDATE SKIP LOCKED` · `ON CONFLICT` · `TIMESTAMPTZ`로 PostgreSQL 전용이다. -`jpa`라는 이름은 사용하지 않는 기술을 광고하고, vendor 중립 port가 어디 있는지도 숨겼다. vendor -중립 계약은 `messaging-reliability-api`에 있다. +신뢰성 기능은 세 leaf다. `messaging-reliability-api`가 vendor 중립 port(`OutboxRepository`, +`InboxRepository`)를, `messaging-reliability`가 그 port만 보는 relay·consumer 로직(stdlib)을, +`messaging-reliability-jdbc-postgresql`이 port의 관계형 구현과 migration(`V1001..V1005`, `V1101`)을 +갖는다. 예전에는 `outbox-jdbc-postgresql` 하나가 relay 로직과 PG repository와 admin journal을 함께 +가졌고 inbox도 같은 모양이었다(S7-#5b). 구현 leaf가 `jdbc-postgresql`로 명명된 이유: 구현은 Spring +JDBC이고 SQL은 `jsonb` · `FOR UPDATE SKIP LOCKED` · `ON CONFLICT` · `TIMESTAMPTZ`로 PostgreSQL +전용이다. `jpa`라는 이름은 사용하지 않는 기술을 광고하고, vendor 중립 port가 어디 있는지도 숨겼다. ## Public surface 정책 @@ -57,7 +60,7 @@ RabbitMQ·Pulsar·NATS·Kafka Share·Spring Cloud Stream bridge leaf와 Debezium 새 leaf를 application에 채택할 때는 starter 또는 composition dependency와 architecture edge policy, qualification/support 상태를 함께 변경한다. -## application이 이 family에 도달하는 경로 (MSG-015, canonical bridge 구현 / legacy cutover 잔존) +## application이 이 family에 도달하는 경로 (MSG-015, cutover 완료) 리뷰(`docs/reviews/2026-08-14-messaging-module-code-review.md` §6.1·§6.3)가 정한 `application-owned port → platform anti-corruption bridge → platform` 경로는 이제 존재한다. @@ -72,21 +75,21 @@ lease/transport/outcome normalization/observation은 platform 중앙 publish pat package는 `messaging-core-api`와 `messaging-schema-api`만 보며 concrete runtime/transport/Kafka 타입을 직접 보지 않는다. -다만 **MSG-015의 전체 cutover가 끝난 것은 아니다.** legacy `OutboxEvent`와 realtime publisher는 -schema/order/tenant 등 canonical metadata가 부족해 아직 `MessageBroker → KafkaSender` R0 경로를 -사용한다. 그 metadata를 추측해 platform envelope를 만드는 것은 금지한다. 따라서 legacy outbox -storage/relay migration을 별도 change set으로 완료한 뒤에만 R0 seam을 제거한다. +**MSG-015 cutover는 끝났다(2026-09-28, 소유자 결정 7).** application의 outbox append는 canonical +`OutboxAppendPort` 하나이고, relay는 claim한 row의 validated event를 이 bridge로만 보낸다. legacy +`OutboxEvent`·`LegacyOutboxAppendPort`와 `MessageBroker → KafkaSender` R0 경로(TLS/SASL을 적용하지 +않던 두 번째 producer)는 삭제했다. 남은 Kafka producer는 platform의 것 하나이고, broker security +profile(`KafkaSecurityConfigurer`)을 적용한다. canonical bridge의 producer identity는 `app.messaging.producer-id`로 명시한다. 값이 없으면 bridge bean을 -만들지 않으며 `spring.application.name`/host/pod 이름에서 identity를 발명하지 않는다. +만들지 않으며 `spring.application.name`/host/pod 이름에서 identity를 발명하지 않는다. bridge가 없으면 +relay가 발행할 곳이 없으므로 messaging 모듈이 켜진 상태에서는 기동이 거부된다. -`ca-skeleton.outbox.canonical-transport-enabled=true`는 bridge 존재만으로 충분하지 않다. Stable -composition은 애플리케이션 계약 catalog/schema source를 임의로 만들지 않으므로 adopter가 -`IntegrationEventEncoderPort`를 명시적으로 제공해야 한다. 템플릿은 구현체를 주지 않는다(후보였던 -R1 JSON Schema encoder는 2026-09-25에 삭제). `OutboxCanonicalTransportRequirementValidator`는 -publisher 또는 encoder 중 하나라도 -없으면 startup을 거부한다. 즉 canonical flag가 켜졌는데 정본 encoder가 없는 상태로 성공 기동하는 -경로는 없다. +`IntegrationEventEncoderPort`는 템플릿이 제공한다: `adapter:outbound:support`의 +`JsonIntegrationEventEncoder`가 envelope v1(`shared-contract`의 `contracts/messaging/envelope/v1.schema.json`) +안에 payload record를 결정적 JSON으로 싣는다. 계약 catalog는 애플리케이션이 bean으로 등록한 +`IntegrationEventContractContribution`뿐이고, 선언한 payload schema hash가 classpath의 schema 파일과 +다르면 기동이 실패한다. payload를 JSON Schema로 매번 검증하려는 서비스는 자기 encoder bean을 두면 된다. ## Stable 승격에 필요한 증거 diff --git a/messaging/messaging-inbox-jdbc-postgresql/build.gradle b/messaging/messaging-inbox-jdbc-postgresql/build.gradle deleted file mode 100644 index bf339f0..0000000 --- a/messaging/messaging-inbox-jdbc-postgresql/build.gradle +++ /dev/null @@ -1,33 +0,0 @@ -plugins { - id 'ca.auxiliary-source-set' - id 'ca.platform-module' -} - -auxiliarySourceSets { - sourceSet('integrationTest') { - compilesAgainst 'main' - } -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-reliability-api') - - implementation 'org.springframework:spring-jdbc' - implementation 'org.springframework:spring-tx' - - // Live-database certification. The reliability patterns are claims about transaction - // boundaries and uniqueness constraints, and only a real database can settle them. - testImplementation project(':messaging:messaging-testkit') - integrationTestImplementation 'org.testcontainers:testcontainers-postgresql' - integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' - integrationTestImplementation 'org.postgresql:postgresql' -} - -strictTestLanes { - lane('integrationTest') { - integration() - sourceSet = 'integrationTest' - description = 'Runs inbox transaction and uniqueness contracts against real PostgreSQL.' - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/gradle.lockfile b/messaging/messaging-outbox-jdbc-postgresql/gradle.lockfile deleted file mode 100644 index 608e6f7..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/gradle.lockfile +++ /dev/null @@ -1,111 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.fasterxml.jackson.core:jackson-annotations:2.21=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.github.docker-java:docker-java-api:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.docker-java:docker-java-transport-zerodep:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.docker-java:docker-java-transport:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-codec:commons-codec:1.19.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.20.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -commons-io:commons-io:2.21.0=spotbugs -commons-logging:commons-logging:1.3.6=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -net.java.dev.jna:jna:5.18.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-compress:1.28.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.apache.commons:commons-lang3:3.20.0=checkstyle,integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,testCompileClasspath -org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.checkerframework:checker-qual:3.55.1=integrationTestRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jetbrains:annotations:17.0.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,integrationTestAnnotationProcessor,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath -org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -org.postgresql:postgresql:42.7.13=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.reflections:reflections:0.10.2=checkstyle -org.rnorth.duct-tape:duct-tape:1.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.slf4j:slf4j-api:2.0.18=integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.springframework:spring-beans:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-core:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-jdbc:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-tx:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.testcontainers:testcontainers-database-commons:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers-jdbc:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers-junit-jupiter:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers-postgresql:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty= diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java index 8197a2c..6765abf 100644 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java +++ b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java @@ -22,38 +22,23 @@ public interface OutboxRepository { void append(OutboxRecord record); /** - * Leases a batch of publishable records. + * Claims a batch and returns the leases, each carrying its owner and fencing token, skipping rows + * that are waiting for their backoff or out of attempts. * - *

    Leasing rather than simply selecting is what makes multiple relay instances safe: a record - * claimed by one relay is invisible to the others until its lease expires, so the same message is - * not published concurrently by two processes. - * - * @param batchSize how many records to claim - * @param leaseDuration how long the claim holds - * @param now the current instant - * @return the leased records, oldest first - */ - List leaseBatch(int batchSize, Duration leaseDuration, Instant now); - - /** - * Claims a batch and returns the leases, each carrying its owner and fencing token. - * - *

    The token is what a terminal write is checked against. {@link #leaseBatch} returns records - * without one, so its callers cannot prove a write belongs to their claim; it remains for - * inspection paths and is deprecated for the relay's use. - * - * @param owner this relay's stable identity - */ - List claimBatch(String owner, int batchSize, Duration leaseDuration, Instant now); - - /** - * Claims a batch, skipping rows that are waiting for their backoff or out of attempts. + *

    Claiming rather than simply selecting is what makes multiple relay instances safe: a record + * claimed by one relay is invisible to the others until its lease expires, and the token is what + * a terminal write is checked against, so a relay that stalled past its lease writes nothing. * *

    Eligibility belongs in the row. A relay that decides in memory forgets on restart, and every * replica decides for itself — so the same message is retried by as many workers as there are * replicas, as often as they poll. * + * @param owner this relay's stable identity + * @param batchSize how many records to claim + * @param leaseDuration how long the claim holds + * @param now the current instant * @param maxAttempts the attempt budget; rows at or above it are not claimed + * @return the leases, oldest first */ List claimBatch( String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts); @@ -65,10 +50,10 @@ public interface OutboxRepository { */ OutboxTransitionResult markPublished(OutboxLease lease, Instant now); - /** Records an ambiguous outcome, only if this lease still owns the row. */ - OutboxTransitionResult markAmbiguous(OutboxLease lease, String failureCode, Instant now); - - /** Records an ambiguous outcome and when this row may next be claimed. */ + /** + * Records an ambiguous outcome and when this row may next be claimed, only if this lease still + * owns the row. + */ OutboxTransitionResult markAmbiguous( OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt); diff --git a/messaging/messaging-outbox-jdbc-postgresql/build.gradle b/messaging/messaging-reliability-jdbc-postgresql/build.gradle similarity index 69% rename from messaging/messaging-outbox-jdbc-postgresql/build.gradle rename to messaging/messaging-reliability-jdbc-postgresql/build.gradle index c933afe..e6c57bb 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/build.gradle +++ b/messaging/messaging-reliability-jdbc-postgresql/build.gradle @@ -9,11 +9,13 @@ auxiliarySourceSets { } } +// The relational implementations of the reliability ports: the outbox repository, the inbox +// repository and the destructive-operation journal. Spring JDBC for the transaction join, and SQL +// that is PostgreSQL's (`jsonb`, `FOR UPDATE SKIP LOCKED`, `ON CONFLICT`, `TIMESTAMPTZ`) — the name +// is the contract. The relay and consumer logic that uses them is messaging-reliability. dependencies { api project(':messaging:messaging-core-api') api project(':messaging:messaging-reliability-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-observability') // The destructive-operation journal lives here because it needs exactly what the outbox needs: // one relational database every replica can see, and a migration lane that already exists. The @@ -26,9 +28,7 @@ dependencies { // Live-database certification. The reliability patterns are claims about transaction // boundaries and uniqueness constraints, and only a real database can settle them. testImplementation project(':messaging:messaging-testkit') - // Test-only. The relay is also driven through the real DefaultMessagePublisher: a fake publisher - // accepted the pre-encoded rows the real one re-encoded and rejected. - testImplementation project(':messaging:messaging-runtime-core') + integrationTestImplementation project(':messaging:messaging-testkit') integrationTestImplementation 'org.testcontainers:testcontainers-postgresql' integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' integrationTestImplementation 'org.postgresql:postgresql' @@ -38,6 +38,6 @@ strictTestLanes { lane('integrationTest') { integration() sourceSet = 'integrationTest' - description = 'Runs outbox and admin-journal contracts against real PostgreSQL.' + description = 'Runs outbox, inbox and admin-journal contracts against real PostgreSQL.' } } diff --git a/messaging/messaging-inbox-jdbc-postgresql/gradle.lockfile b/messaging/messaging-reliability-jdbc-postgresql/gradle.lockfile similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/gradle.lockfile rename to messaging/messaging-reliability-jdbc-postgresql/gradle.lockfile diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java rename to messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/AdminOperationJournalPostgresIT.java b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/AdminOperationJournalPostgresIT.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/AdminOperationJournalPostgresIT.java rename to messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/AdminOperationJournalPostgresIT.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java similarity index 86% rename from messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java rename to messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java index 66fba44..f3f1d2a 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java +++ b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java @@ -71,6 +71,9 @@ class OutboxPostgresIT { try (Connection connection = dataSource.getConnection(); Statement statement = connection.createStatement()) { statement.execute("DROP TABLE IF EXISTS messaging_outbox"); + // The inbox migration ships in the same module now, so "every migration" creates its table + // too. + statement.execute("DROP TABLE IF EXISTS messaging_inbox"); // Every migration in order, not just V1: a test that applies part of the schema certifies a // database nobody deploys. for (String migration : migrations()) { @@ -90,7 +93,7 @@ class OutboxPostgresIT { OutboxRecord record = record(); appendInTransaction(record); - List leased = repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + List leased = claim("relay-a", NOW).stream().map(OutboxLease::record).toList(); assertThat(leased) .singleElement() @@ -109,8 +112,8 @@ class OutboxPostgresIT { void aLeasedRowIsInvisibleToASecondRelayInstance() { appendInTransaction(record()); - List first = repository.leaseBatch(10, Duration.ofSeconds(30), NOW); - List second = repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + List first = claim("relay-a", NOW); + List second = claim("relay-b", NOW); assertThat(first).hasSize(1); assertThat(second).as("SKIP LOCKED plus the lease keeps two relays off the same row").isEmpty(); @@ -119,10 +122,9 @@ class OutboxPostgresIT { @Test void anExpiredLeaseBecomesClaimableAgain() { appendInTransaction(record()); - repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + claim("relay-a", NOW); - List reclaimed = - repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(31)); + List reclaimed = claim("relay-b", NOW.plusSeconds(31)); assertThat(reclaimed).hasSize(1); } @@ -131,18 +133,16 @@ class OutboxPostgresIT { void anAmbiguousRowStaysClaimableUnderItsOriginalIdentity() { OutboxRecord record = record(); appendInTransaction(record); - // Claimed rather than leased: a terminal write is checked against the lease token, so - // `markAmbiguous` takes the lease that owns the row. The identity-keyed overload this test used - // to call is gone, and with it the possibility of one relay writing an outcome onto a row - // another relay currently owns. - OutboxLease lease = - repository.claimBatch("ambiguous-relay", 10, Duration.ofSeconds(30), NOW).getFirst(); + // A terminal write is checked against the lease token, so `markAmbiguous` takes the lease that + // owns the row; no identity-keyed write exists for one relay to put an outcome on a row another + // relay currently owns. + OutboxLease lease = claim("ambiguous-relay", NOW).getFirst(); - repository.markAmbiguous(lease, "CONFIRM_TIMEOUT", NOW); + repository.markAmbiguous(lease, "CONFIRM_TIMEOUT", NOW, NOW); assertThat(repository.find(record.messageId()).orElseThrow().status()) .isEqualTo(OutboxStatus.AMBIGUOUS); - assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW)) + assertThat(claim("relay-b", NOW)) .singleElement() .satisfies(retry -> assertThat(retry.messageId()).isEqualTo(record.messageId())); } @@ -151,24 +151,24 @@ class OutboxPostgresIT { void aPublishedRowIsNotClaimedAgain() { OutboxRecord record = record(); appendInTransaction(record); - repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + OutboxLease lease = claim("relay-a", NOW).getFirst(); - repository.markPublishedUnfenced(record.messageId(), NOW); + repository.markPublished(lease, NOW); assertThat(repository.find(record.messageId()).orElseThrow().status()) .isEqualTo(OutboxStatus.PUBLISHED); - assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(60))).isEmpty(); + assertThat(claim("relay-b", NOW.plusSeconds(60))).isEmpty(); } @Test void aFailedRowIsNotClaimedAgainWithoutIntervention() { OutboxRecord record = record(); appendInTransaction(record); - repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + OutboxLease lease = claim("relay-a", NOW).getFirst(); - repository.markFailedUnfenced(record.messageId(), "INVALID_TOPIC", NOW); + repository.markFailed(lease, "INVALID_TOPIC", NOW); - assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(60))).isEmpty(); + assertThat(claim("relay-b", NOW.plusSeconds(60))).isEmpty(); } @Test @@ -202,8 +202,8 @@ class OutboxPostgresIT { OutboxRecord published = record(); OutboxRecord pending = record(); appendInTransaction(published); + repository.markPublished(claim("relay-a", NOW).getFirst(), NOW); appendInTransaction(pending); - repository.markPublishedUnfenced(published.messageId(), NOW); int removed = repository.purgePublishedBefore(NOW.plusSeconds(1), 100); @@ -211,6 +211,11 @@ class OutboxPostgresIT { assertThat(repository.find(pending.messageId())).isPresent(); } + /** Claims with an unlimited attempt budget, as a relay does on a row's first attempts. */ + private List claim(String owner, Instant now) { + return repository.claimBatch(owner, 10, Duration.ofSeconds(30), now, Integer.MAX_VALUE); + } + private static OutboxRecord record() { return record( // The routing key is a column now, not a reserved header smuggled through the header map. @@ -228,7 +233,8 @@ class OutboxPostgresIT { appendInTransaction(otherKey); // when - List claimed = repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW); + List claimed = + repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); // then assertThat(claimed) @@ -352,12 +358,13 @@ class OutboxPostgresIT { // Relay A claims and stalls. java.util.List first = - repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW); + repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); assertThat(first).hasSize(1); // The lease expires and relay B takes the row and publishes it. java.util.List second = - repository.claimBatch("relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31)); + repository.claimBatch( + "relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31), Integer.MAX_VALUE); assertThat(second).hasSize(1); assertThat(second.get(0).token()) .as("every claim increments the token, which is what makes the older one detectable") @@ -366,7 +373,9 @@ class OutboxPostgresIT { .isEqualTo(dev.caskeleton.messaging.reliability.OutboxTransitionResult.APPLIED); // Relay A finally wakes up and tries to record its own outcome. - assertThat(repository.markAmbiguous(first.get(0), "CONFIRM_TIMEOUT", NOW.plusSeconds(40))) + assertThat( + repository.markAmbiguous( + first.get(0), "CONFIRM_TIMEOUT", NOW.plusSeconds(40), NOW.plusSeconds(40))) .as("this write used to succeed and made a published message claimable again") .isEqualTo(dev.caskeleton.messaging.reliability.OutboxTransitionResult.STALE_LEASE); assertThat(repository.find(record.messageId()).orElseThrow().status()) @@ -378,7 +387,7 @@ class OutboxPostgresIT { OutboxRecord record = record(); appendInTransaction(record); dev.caskeleton.messaging.reliability.OutboxLease lease = - repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW).get(0); + repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE).get(0); assertThat(repository.markPublished(lease, NOW)) .isEqualTo(dev.caskeleton.messaging.reliability.OutboxTransitionResult.APPLIED); @@ -395,9 +404,9 @@ class OutboxPostgresIT { appendInTransaction(two); java.util.List a = - repository.claimBatch("relay-a", 1, Duration.ofSeconds(30), NOW); + repository.claimBatch("relay-a", 1, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); java.util.List b = - repository.claimBatch("relay-b", 1, Duration.ofSeconds(30), NOW); + repository.claimBatch("relay-b", 1, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); assertThat(a).hasSize(1); assertThat(b).hasSize(1); @@ -410,9 +419,15 @@ class OutboxPostgresIT { appendInTransaction(record); long firstToken = - repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW).get(0).token(); + repository + .claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE) + .get(0) + .token(); dev.caskeleton.messaging.reliability.OutboxLease reclaimed = - repository.claimBatch("relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31)).get(0); + repository + .claimBatch( + "relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31), Integer.MAX_VALUE) + .get(0); assertThat(reclaimed.messageId()).isEqualTo(record.messageId()); assertThat(reclaimed.token()).isEqualTo(firstToken + 1); @@ -589,6 +604,38 @@ class OutboxPostgresIT { .containsEntry("x-note", "line one\nline\ttwo"); } + /** + * The hand-rolled reader took a quote preceded by a backslash for an escaped quote, so a value + * ending in an escaped backslash ran on into the next header and both came back wrong. + */ + @Test + void aHeaderValueEndingInABackslashRoundTripsWithoutSwallowingTheNextHeader() { + // given + OutboxRecord record = + new OutboxRecord( + MessageId.newId(), + new DestinationName("order-events"), + new MessageType("order.created"), + new SchemaVersion(1), + ContentType.JSON, + "{}".getBytes(StandardCharsets.UTF_8), + Map.of("x-path", "C:\\exports\\", "x-next", "after"), + OutboxCanonicalMetadata.none(), + NOW.minusSeconds(60), + OutboxStatus.PENDING, + 0, + Optional.empty(), + Optional.empty()); + appendInTransaction(record); + + // when + Map headers = repository.find(record.messageId()).orElseThrow().headers(); + + // then + assertThat(headers) + .containsExactlyInAnyOrderEntriesOf(Map.of("x-path", "C:\\exports\\", "x-next", "after")); + } + private void appendInTransaction(OutboxRecord record) { transactions.executeWithoutResult(status -> repository.append(record)); } diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java b/messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java rename to messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcAdminOperationJournal.java b/messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcAdminOperationJournal.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcAdminOperationJournal.java rename to messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcAdminOperationJournal.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java b/messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java similarity index 71% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java rename to messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java index 13b87a3..49eec64 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java +++ b/messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java @@ -17,6 +17,7 @@ import dev.caskeleton.messaging.reliability.OutboxRecord; import dev.caskeleton.messaging.reliability.OutboxRepository; import dev.caskeleton.messaging.reliability.OutboxStatus; import dev.caskeleton.messaging.reliability.OutboxTransitionResult; +import java.sql.Array; import java.sql.Connection; import java.sql.PreparedStatement; import java.sql.ResultSet; @@ -41,7 +42,7 @@ import org.springframework.transaction.support.TransactionSynchronizationManager * already inside, which is the entire mechanism. An outbox row written on a separate connection * commits independently of the business change and reopens the window the pattern exists to close. * - *

    Leasing uses {@code SELECT ... FOR UPDATE SKIP LOCKED}. Two relay instances polling the same + *

    Claiming uses {@code SELECT ... FOR UPDATE SKIP LOCKED}. Two relay instances polling the same * table would otherwise both claim the same rows and publish them twice; {@code SKIP LOCKED} lets * the second instance step over what the first already holds instead of blocking behind it. */ @@ -55,7 +56,7 @@ public final class JdbcOutboxRepository implements OutboxRepository { producer, occurred_at, correlation_id, causation_id, tenant, partition_key, ordering_key, schema_subject, schema_uri, traceparent, tracestate, baggage) - VALUES (?, ?, ?, ?, ?, ?, ?::jsonb, ?, ?, ?, + VALUES (?, ?, ?, ?, ?, ?, jsonb_object(?::text[]), ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) @@ -72,38 +73,15 @@ public final class JdbcOutboxRepository implements OutboxRepository { private static final String COLUMNS = """ message_id, destination, message_type, schema_version, content_type, - payload, headers, created_at, status, attempts, lease_expires_at, last_failure_code, + payload, + ARRAY(SELECT h.key FROM jsonb_each_text(headers) h ORDER BY h.key) AS header_keys, + ARRAY(SELECT h.value FROM jsonb_each_text(headers) h ORDER BY h.key) AS header_values, + created_at, status, attempts, lease_expires_at, last_failure_code, producer, occurred_at, correlation_id, causation_id, tenant, partition_key, ordering_key, schema_subject, schema_uri, traceparent, tracestate, baggage\ """; - private static final String LEASE = - """ - WITH claimable AS ( - SELECT message_id - FROM messaging_outbox - -- IN_FLIGHT is claimable too, but only once its lease has expired. A relay that dies - -- mid-publish leaves rows in that state; excluding them would strand those messages - -- forever, which is the one failure mode the outbox exists to prevent. - WHERE status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT') - AND (lease_expires_at IS NULL OR lease_expires_at <= ?) - ORDER BY created_at - LIMIT ? - FOR UPDATE SKIP LOCKED - ) - UPDATE messaging_outbox o - SET status = 'IN_FLIGHT', lease_expires_at = ? - FROM claimable c - WHERE o.message_id = c.message_id - RETURNING o.message_id, o.destination, o.message_type, o.schema_version, o.content_type, - o.payload, o.headers, o.created_at, o.status, o.attempts, - o.lease_expires_at, o.last_failure_code, - o.producer, o.occurred_at, o.correlation_id, o.causation_id, o.tenant, - o.partition_key, o.ordering_key, o.schema_subject, o.schema_uri, - o.traceparent, o.tracestate, o.baggage - """; - /** * The claim that records who holds the row and increments its fencing token. * @@ -147,7 +125,11 @@ public final class JdbcOutboxRepository implements OutboxRepository { FROM claimable c WHERE o.message_id = c.message_id RETURNING o.message_id, o.destination, o.message_type, o.schema_version, o.content_type, - o.payload, o.headers, o.created_at, o.status, o.attempts, + o.payload, + ARRAY(SELECT h.key FROM jsonb_each_text(o.headers) h ORDER BY h.key) AS header_keys, + ARRAY(SELECT h.value FROM jsonb_each_text(o.headers) h ORDER BY h.key) + AS header_values, + o.created_at, o.status, o.attempts, o.lease_expires_at, o.last_failure_code, o.lease_owner, o.lease_token, o.producer, o.occurred_at, o.correlation_id, o.causation_id, o.tenant, o.partition_key, o.ordering_key, o.schema_subject, o.schema_uri, @@ -187,7 +169,7 @@ public final class JdbcOutboxRepository implements OutboxRepository { statement.setInt(4, record.schemaVersion().value()); statement.setString(5, record.contentType().value()); statement.setBytes(6, record.payload()); - statement.setString(7, toJson(record.headers())); + statement.setArray(7, connection.createArrayOf("text", flatten(record.headers()))); statement.setTimestamp(8, Timestamp.from(record.createdAt())); statement.setString(9, record.status().name()); statement.setInt(10, record.attempts()); @@ -258,33 +240,6 @@ public final class JdbcOutboxRepository implements OutboxRepository { } } - @Override - public List leaseBatch(int batchSize, Duration leaseDuration, Instant now) { - Objects.requireNonNull(leaseDuration, "leaseDuration must not be null"); - Objects.requireNonNull(now, "now must not be null"); - return withConnection( - connection -> { - List leased = new ArrayList<>(); - try (PreparedStatement statement = connection.prepareStatement(LEASE)) { - statement.setTimestamp(1, Timestamp.from(now)); - statement.setInt(2, batchSize); - statement.setTimestamp(3, Timestamp.from(now.plus(leaseDuration))); - try (ResultSet results = statement.executeQuery()) { - while (results.next()) { - leased.add(read(results)); - } - } - } - return List.copyOf(leased); - }); - } - - @Override - public List claimBatch( - String owner, int batchSize, Duration leaseDuration, Instant now) { - return claimBatch(owner, batchSize, leaseDuration, now, Integer.MAX_VALUE); - } - @Override public List claimBatch( String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts) { @@ -328,11 +283,6 @@ public final class JdbcOutboxRepository implements OutboxRepository { null); } - @Override - public OutboxTransitionResult markAmbiguous(OutboxLease lease, String failureCode, Instant now) { - return markAmbiguous(lease, failureCode, now, now); - } - @Override public OutboxTransitionResult markAmbiguous( OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt) { @@ -435,49 +385,6 @@ public final class JdbcOutboxRepository implements OutboxRepository { }); } - public void markPublishedUnfenced(MessageId messageId, Instant now) { - update( - "UPDATE messaging_outbox SET status = 'PUBLISHED', published_at = ?, " - + "lease_expires_at = NULL, attempts = attempts + 1 WHERE message_id = ?", - messageId, - now, - null); - } - - public void markAmbiguousUnfenced(MessageId messageId, String failureCode, Instant now) { - // The lease is released rather than held: an ambiguous row must become claimable again so the - // relay retries it under the same message id. - update( - "UPDATE messaging_outbox SET status = 'AMBIGUOUS', lease_expires_at = NULL, " - + "last_failure_code = ?, attempts = attempts + 1 WHERE message_id = ?", - messageId, - now, - failureCode); - } - - public void markFailedUnfenced(MessageId messageId, String failureCode, Instant now) { - update( - "UPDATE messaging_outbox SET status = 'FAILED', lease_expires_at = NULL, " - + "last_failure_code = ?, attempts = attempts + 1 WHERE message_id = ?", - messageId, - now, - failureCode); - } - - public void releaseLeaseUnfenced(MessageId messageId) { - withConnection( - connection -> { - try (PreparedStatement statement = - connection.prepareStatement( - "UPDATE messaging_outbox SET status = 'PENDING', lease_expires_at = NULL " - + "WHERE message_id = ? AND status = 'IN_FLIGHT'")) { - statement.setObject(1, messageId.value()); - statement.executeUpdate(); - } - return null; - }); - } - @Override public Optional find(MessageId messageId) { return withConnection( @@ -526,22 +433,6 @@ public final class JdbcOutboxRepository implements OutboxRepository { }); } - private void update(String sql, MessageId messageId, Instant now, String failureCode) { - withConnection( - connection -> { - try (PreparedStatement statement = connection.prepareStatement(sql)) { - if (failureCode == null) { - statement.setTimestamp(1, Timestamp.from(now)); - } else { - statement.setString(1, failureCode); - } - statement.setObject(2, messageId.value()); - statement.executeUpdate(); - } - return null; - }); - } - private static OutboxRecord read(ResultSet results) throws SQLException { Timestamp lease = results.getTimestamp("lease_expires_at"); return new OutboxRecord( @@ -551,7 +442,7 @@ public final class JdbcOutboxRepository implements OutboxRepository { new SchemaVersion(results.getInt("schema_version")), new ContentType(results.getString("content_type")), results.getBytes("payload"), - fromJson(results.getString("headers")), + headers(results.getArray("header_keys"), results.getArray("header_values")), readMetadata(results), results.getTimestamp("created_at").toInstant(), OutboxStatus.valueOf(results.getString("status")), @@ -597,119 +488,36 @@ public final class JdbcOutboxRepository implements OutboxRepository { } /** - * Serialises headers as a flat JSON object. + * The headers as one flat key, value, key, value array for {@code jsonb_object(text[])}. * - *

    Hand-rolled rather than pulled from a JSON library so this module keeps no codec dependency: - * outbox headers are always flat string pairs, validated by {@code MessageHeaders} before they - * ever reach here. + *

    PostgreSQL builds and escapes the JSON. The hand-rolled writer and reader this replaces got + * the escaping right one way and wrong the other: the reader took a quote after an escaped + * backslash for an escaped quote, so a value ending in a backslash ran on into the next header. */ - private static String toJson(Map headers) { - StringBuilder json = new StringBuilder("{"); - boolean first = true; + private static String[] flatten(Map headers) { + String[] flat = new String[headers.size() * 2]; + int index = 0; for (Map.Entry entry : headers.entrySet()) { - if (!first) { - json.append(','); - } - first = false; - json.append('"') - .append(escape(entry.getKey())) - .append("\":\"") - .append(escape(entry.getValue())) - .append('"'); + flat[index++] = entry.getKey(); + flat[index++] = entry.getValue(); } - return json.append('}').toString(); + return flat; } - private static Map fromJson(String json) { + /** Rebuilds the headers from the aligned key and value arrays the reads select. */ + private static Map headers(Array keys, Array values) throws SQLException { Map headers = new LinkedHashMap<>(); - if (json == null || json.length() <= 2) { + if (keys == null || values == null) { return headers; } - String body = json.substring(1, json.length() - 1); - int index = 0; - while (index < body.length()) { - int keyStart = body.indexOf('"', index); - if (keyStart < 0) { - break; - } - int keyEnd = findClosingQuote(body, keyStart + 1); - int valueStart = body.indexOf('"', body.indexOf(':', keyEnd)); - int valueEnd = findClosingQuote(body, valueStart + 1); - headers.put( - unescape(body.substring(keyStart + 1, keyEnd)), - unescape(body.substring(valueStart + 1, valueEnd))); - index = valueEnd + 1; + String[] names = (String[]) keys.getArray(); + String[] texts = (String[]) values.getArray(); + for (int index = 0; index < names.length; index++) { + headers.put(names[index], texts[index]); } return headers; } - private static int findClosingQuote(String text, int from) { - for (int index = from; index < text.length(); index++) { - if (text.charAt(index) == '"' && text.charAt(index - 1) != '\\') { - return index; - } - } - return text.length(); - } - - /** - * JSON string escaping, including the control characters. - * - *

    Only backslash and quote were escaped. A header value containing a newline or a NUL — which - * the value objects now refuse, but which older rows and other writers may still carry — produced - * a document PostgreSQL rejects as invalid JSONB, so the append failed at the database with an - * error naming neither the header nor the row. Control characters are the part of the JSON - * grammar that is easy to forget and impossible to get away with. - */ - private static String escape(String value) { - StringBuilder escaped = new StringBuilder(value.length() + 8); - for (int index = 0; index < value.length(); index++) { - char character = value.charAt(index); - switch (character) { - case '\\' -> escaped.append("\\\\"); - case '"' -> escaped.append("\\\""); - case '\n' -> escaped.append("\\n"); - case '\r' -> escaped.append("\\r"); - case '\t' -> escaped.append("\\t"); - case '\b' -> escaped.append("\\b"); - case '\f' -> escaped.append("\\f"); - default -> { - if (character < 0x20) { - escaped.append("\\u%04x".formatted((int) character)); - } else { - escaped.append(character); - } - } - } - } - return escaped.toString(); - } - - private static String unescape(String value) { - StringBuilder plain = new StringBuilder(value.length()); - for (int index = 0; index < value.length(); index++) { - char character = value.charAt(index); - if (character != '\\' || index == value.length() - 1) { - plain.append(character); - continue; - } - char next = value.charAt(++index); - switch (next) { - case 'n' -> plain.append('\n'); - case 'r' -> plain.append('\r'); - case 't' -> plain.append('\t'); - case 'b' -> plain.append('\b'); - case 'f' -> plain.append('\f'); - case 'u' -> { - plain.append((char) Integer.parseInt(value.substring(index + 1, index + 5), 16)); - index += 4; - } - default -> plain.append(next); - } - } - return plain.toString(); - } - /** A unit of work against a borrowed connection. */ @FunctionalInterface private interface ConnectionCallback { diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1001__messaging_outbox.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1001__messaging_outbox.sql similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1001__messaging_outbox.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1001__messaging_outbox.sql diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1002__messaging_outbox_lease_fencing.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1002__messaging_outbox_lease_fencing.sql similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1002__messaging_outbox_lease_fencing.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1002__messaging_outbox_lease_fencing.sql diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1003__messaging_admin_operation_journal.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1003__messaging_admin_operation_journal.sql similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1003__messaging_admin_operation_journal.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1003__messaging_admin_operation_journal.sql diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1004__messaging_outbox_canonical_metadata.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1004__messaging_outbox_canonical_metadata.sql similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1004__messaging_outbox_canonical_metadata.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1004__messaging_outbox_canonical_metadata.sql diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1101__messaging_inbox.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1101__messaging_inbox.sql similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1101__messaging_inbox.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1101__messaging_inbox.sql diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/JdbcInboxTransactionRequirementTest.java b/messaging/messaging-reliability-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/JdbcInboxTransactionRequirementTest.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/JdbcInboxTransactionRequirementTest.java rename to messaging/messaging-reliability-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/JdbcInboxTransactionRequirementTest.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/JdbcOutboxTransactionRequirementTest.java b/messaging/messaging-reliability-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/JdbcOutboxTransactionRequirementTest.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/JdbcOutboxTransactionRequirementTest.java rename to messaging/messaging-reliability-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/JdbcOutboxTransactionRequirementTest.java diff --git a/messaging/messaging-reliability/build.gradle b/messaging/messaging-reliability/build.gradle new file mode 100644 index 0000000..9c08eb5 --- /dev/null +++ b/messaging/messaging-reliability/build.gradle @@ -0,0 +1,19 @@ +plugins { + id 'ca.platform-module' +} + +// The outbox relay and the inbox consumer logic, framework-free. What they store and claim goes +// through the ports in messaging-reliability-api; the relational implementation of those ports is +// messaging-reliability-jdbc-postgresql. Split so a leaf's name says what it depends on: this one +// is Java stdlib plus the messaging APIs, the other is Spring JDBC and PostgreSQL SQL. +dependencies { + api project(':messaging:messaging-core-api') + api project(':messaging:messaging-reliability-api') + // The relay hands encoded rows to the publisher as EncodedMessage. + api project(':messaging:messaging-schema-api') + + // Test-only. The relay is also driven through the real DefaultMessagePublisher: a fake publisher + // accepted the pre-encoded rows the real one re-encoded and rejected. + testImplementation project(':messaging:messaging-runtime-core') + testImplementation project(':messaging:messaging-policy') +} diff --git a/messaging/messaging-reliability/gradle.lockfile b/messaging/messaging-reliability/gradle.lockfile new file mode 100644 index 0000000..58cc2a9 --- /dev/null +++ b/messaging/messaging-reliability/gradle.lockfile @@ -0,0 +1,89 @@ +# This is a Gradle generated file for dependency locking. +# Manual edits can break the build and are not advised. +# This file is expected to be part of source control. +com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor +com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor +com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs +com.github.spotbugs:spotbugs:4.10.2=spotbugs +com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs +com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor +com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor +com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor +com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs +com.google.code.gson:gson:2.13.2=spotbugs +com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor +com.google.errorprone:error_prone_annotations:2.41.0=spotbugs +com.google.errorprone:error_prone_annotations:2.47.0=checkstyle +com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor +com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor +com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor +com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor +com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor +com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor +com.google.guava:guava:33.6.0-jre=checkstyle +com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor +com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor +com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor +com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins +com.puppycrawl.tools:checkstyle:13.5.0=checkstyle +commons-beanutils:commons-beanutils:1.11.0=checkstyle +commons-collections:commons-collections:3.2.2=checkstyle +commons-io:commons-io:2.21.0=spotbugs +info.picocli:picocli:4.7.7=checkstyle +io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor +io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor +io.micrometer:micrometer-commons:1.16.7=testCompileClasspath,testRuntimeClasspath +io.micrometer:micrometer-core:1.16.7=testCompileClasspath,testRuntimeClasspath +io.micrometer:micrometer-observation:1.16.7=testCompileClasspath,testRuntimeClasspath +javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor +jaxen:jaxen:2.0.6=spotbugs +net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath +net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs +org.antlr:antlr4-runtime:4.13.2=checkstyle +org.apache.bcel:bcel:6.12.0=spotbugs +org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs +org.apache.commons:commons-text:1.15.0=spotbugs +org.apache.commons:commons-text:1.3=checkstyle +org.apache.httpcomponents:httpclient:4.5.13=checkstyle +org.apache.httpcomponents:httpcore:4.4.16=checkstyle +org.apache.logging.log4j:log4j-api:2.25.5=spotbugs +org.apache.logging.log4j:log4j-core:2.25.5=spotbugs +org.apache.maven.doxia:doxia-core:1.12.0=checkstyle +org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle +org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle +org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle +org.apache.xbean:xbean-reflect:3.7=checkstyle +org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath +org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath +org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle +org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle +org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle +org.codehaus.plexus:plexus-utils:3.3.0=checkstyle +org.dom4j:dom4j:2.2.0=spotbugs +org.hdrhistogram:HdrHistogram:2.2.2=testRuntimeClasspath +org.javassist:javassist:3.28.0-GA=checkstyle +org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath +org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath +org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath +org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath +org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath +org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath +org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath +org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath +org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath +org.junit:junit-bom:6.1.0=spotbugs +org.latencyutils:LatencyUtils:2.0.3=testRuntimeClasspath +org.mockito:mockito-core:5.20.0=mockitoAgent +org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath +org.ow2.asm:asm-analysis:9.10.1=spotbugs +org.ow2.asm:asm-commons:9.10.1=spotbugs +org.ow2.asm:asm-tree:9.10.1=spotbugs +org.ow2.asm:asm-util:9.10.1=spotbugs +org.ow2.asm:asm:9.10.1=spotbugs +org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor +org.reflections:reflections:0.10.2=checkstyle +org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j +org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j +org.slf4j:slf4j-simple:2.0.18=checkstyle +org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs +empty=compileClasspath,runtimeClasspath diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/IdempotentConsumer.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/IdempotentConsumer.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/IdempotentConsumer.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/IdempotentConsumer.java diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxOutcome.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxOutcome.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxOutcome.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxOutcome.java diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxRetentionPolicy.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxRetentionPolicy.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxRetentionPolicy.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxRetentionPolicy.java diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactory.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactory.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactory.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactory.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxProperties.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxProperties.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxProperties.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxProperties.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java similarity index 99% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java index 52350c5..566f93c 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java +++ b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java @@ -158,7 +158,7 @@ public final class OutboxRelay { */ public OutboxRelayReport runOnce(Instant now) { Objects.requireNonNull(now, "now must not be null"); - // claimBatch, not leaseBatch: the claim carries an owner and a fencing token, and every write + // The claim carries an owner and a fencing token, and every write // below is conditioned on them. Without that, a relay that stalled past its lease could record // AMBIGUOUS over the PUBLISHED another relay had already written, making a confirmed message // claimable again. diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayReport.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayReport.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayReport.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayReport.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRetryScheduler.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRetryScheduler.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRetryScheduler.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRetryScheduler.java diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactoryTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactoryTest.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactoryTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactoryTest.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java similarity index 91% rename from messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java index 7eb3f0a..2a52e0e 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java @@ -35,17 +35,6 @@ class OutboxOperationsTest { throw new UnsupportedOperationException(); } - @Override - public List leaseBatch(int batchSize, Duration leaseDuration, Instant now) { - throw new UnsupportedOperationException(); - } - - @Override - public java.util.List claimBatch( - String owner, int batchSize, Duration leaseDuration, Instant now) { - return java.util.List.of(); - } - @Override public java.util.List claimBatch( String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts) { @@ -58,7 +47,7 @@ class OutboxOperationsTest { String failureCode, Instant now, Instant nextAttemptAt) { - return markAmbiguous(lease, failureCode, now); + throw new UnsupportedOperationException(); } @Override @@ -73,12 +62,6 @@ class OutboxOperationsTest { throw new UnsupportedOperationException(); } - @Override - public dev.caskeleton.messaging.reliability.OutboxTransitionResult markAmbiguous( - dev.caskeleton.messaging.reliability.OutboxLease lease, String failureCode, Instant now) { - throw new UnsupportedOperationException(); - } - @Override public dev.caskeleton.messaging.reliability.OutboxTransitionResult markFailed( dev.caskeleton.messaging.reliability.OutboxLease lease, String failureCode, Instant now) { diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java similarity index 95% rename from messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java index 8749e0b..fdc8591 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java @@ -199,15 +199,18 @@ class OutboxRelayTest { assertThat(repository.find(record.messageId()).orElseThrow().status()) .isEqualTo(OutboxStatus.FAILED); - assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW)).isEmpty(); + assertThat(repository.claimBatch("relay", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE)) + .isEmpty(); } @Test void aLeasedRowIsInvisibleToAnotherRelayInstance() { repository.append(OutboxFixtures.pending()); - List first = repository.leaseBatch(10, Duration.ofSeconds(30), NOW); - List second = repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + List first = + repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); + List second = + repository.claimBatch("relay-b", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); assertThat(first).hasSize(1); assertThat(second).isEmpty(); @@ -216,10 +219,11 @@ class OutboxRelayTest { @Test void anExpiredLeaseBecomesClaimableAgain() { repository.append(OutboxFixtures.pending()); - repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); - List reclaimed = - repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(31)); + List reclaimed = + repository.claimBatch( + "relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31), Integer.MAX_VALUE); assertThat(reclaimed).hasSize(1); } @@ -352,8 +356,7 @@ final class InMemoryOutboxRepository implements OutboxRepository { * thing it stands in for hides exactly the defects it was written to catch, so this one follows * the same three-state rule. */ - @Override - public List leaseBatch(int batchSize, Duration leaseDuration, Instant now) { + private List lease(int batchSize, Duration leaseDuration, Instant now) { List leased = new ArrayList<>(); for (OutboxRecord record : List.copyOf(rows.values())) { if (leased.size() >= batchSize) { @@ -376,11 +379,10 @@ final class InMemoryOutboxRepository implements OutboxRepository { return List.copyOf(leased); } - @Override - public List claimBatch( + private List claimIgnoringBudget( String owner, int batchSize, Duration leaseDuration, Instant now) { List claimed = new ArrayList<>(); - for (OutboxRecord record : leaseBatch(batchSize, leaseDuration, now)) { + for (OutboxRecord record : lease(batchSize, leaseDuration, now)) { long token = tokens.merge(record.messageId(), 1L, Long::sum); owners.put(record.messageId(), owner); claimed.add(new OutboxLease(record, owner, token, now.plus(leaseDuration))); @@ -408,7 +410,7 @@ final class InMemoryOutboxRepository implements OutboxRepository { public List claimBatch( String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts) { List claimed = new ArrayList<>(); - for (OutboxLease lease : claimBatch(owner, batchSize, leaseDuration, now)) { + for (OutboxLease lease : claimIgnoringBudget(owner, batchSize, leaseDuration, now)) { Instant nextAttempt = nextAttempts.get(lease.messageId()); boolean waiting = nextAttempt != null && now.isBefore(nextAttempt); if (waiting || lease.record().attempts() >= maxAttempts) { @@ -424,11 +426,13 @@ final class InMemoryOutboxRepository implements OutboxRepository { @Override public OutboxTransitionResult markAmbiguous( OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt) { - OutboxTransitionResult result = markAmbiguous(lease, failureCode, now); - if (result == OutboxTransitionResult.APPLIED) { - nextAttempts.put(lease.messageId(), nextAttemptAt); + if (!owns(lease)) { + return OutboxTransitionResult.STALE_LEASE; } - return result; + markAmbiguousUnfenced(lease.messageId(), failureCode); + owners.remove(lease.messageId()); + nextAttempts.put(lease.messageId(), nextAttemptAt); + return OutboxTransitionResult.APPLIED; } @Override @@ -452,16 +456,6 @@ final class InMemoryOutboxRepository implements OutboxRepository { return OutboxTransitionResult.APPLIED; } - @Override - public OutboxTransitionResult markAmbiguous(OutboxLease lease, String failureCode, Instant now) { - if (!owns(lease)) { - return OutboxTransitionResult.STALE_LEASE; - } - markAmbiguousUnfenced(lease.messageId(), failureCode); - owners.remove(lease.messageId()); - return OutboxTransitionResult.APPLIED; - } - @Override public OutboxTransitionResult markFailed(OutboxLease lease, String failureCode, Instant now) { if (!owns(lease)) { diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java diff --git a/messaging/messaging-spring-boot-starter/build.gradle b/messaging/messaging-spring-boot-starter/build.gradle index acae04e..88f4717 100644 --- a/messaging/messaging-spring-boot-starter/build.gradle +++ b/messaging/messaging-spring-boot-starter/build.gradle @@ -31,8 +31,9 @@ dependencies { // which is why MessagingCoreAutoConfiguration consumed a MessagePublisher that nothing in // reach could produce (MSG-INT-003). implementation project(':messaging:messaging-runtime-core') - implementation project(':messaging:messaging-outbox-jdbc-postgresql') - implementation project(':messaging:messaging-inbox-jdbc-postgresql') + // The relay and consumer logic, and the PostgreSQL stores behind their ports. + implementation project(':messaging:messaging-reliability') + implementation project(':messaging:messaging-reliability-jdbc-postgresql') implementation project(':messaging:messaging-admin-api') implementation project(':messaging:messaging-admin-runtime') diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java index 5c9e3b0..db68ef1 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java @@ -171,28 +171,11 @@ class MessagingOutboxRelayLifecycleTest { throw new UnsupportedOperationException("not used by this test"); } - @Override - public List leaseBatch(int batchSize, Duration leaseDuration, Instant now) { - throw new UnsupportedOperationException("not used by this test"); - } - - @Override - public List claimBatch( - String owner, int batchSize, Duration leaseDuration, Instant now) { - throw new UnsupportedOperationException("not used by this test"); - } - @Override public OutboxTransitionResult markPublished(OutboxLease lease, Instant now) { throw new UnsupportedOperationException("not used by this test"); } - @Override - public OutboxTransitionResult markAmbiguous( - OutboxLease lease, String failureCode, Instant now) { - throw new UnsupportedOperationException("not used by this test"); - } - @Override public OutboxTransitionResult markAmbiguous( OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt) { From 763392c8b220eafce823567c47cd63ed5e403fb6 Mon Sep 17 00:00:00 2001 From: donghyeon-ka Date: Tue, 29 Sep 2026 13:29:46 +0900 Subject: [PATCH 5/7] =?UTF-8?q?review-followup=20Wave=203=20=EB=9D=BC?= =?UTF-8?q?=EC=9A=B4=EB=93=9C=203(P21):=20messaging=20=ED=94=8C=EB=9E=AB?= =?UTF-8?q?=ED=8F=BC=20=E2=80=94=20Kafka=20=EC=86=8C=EB=B9=84(MessageHandl?= =?UTF-8?q?er=20=ED=95=98=EB=82=98),=20API=20=EC=A0=95=EB=A6=AC?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 결정 10: consume 계약을 `MessageHandler` 하나로, starter-kafka가 consumer 조립(handler bean이 있으면 켜짐, `app.messaging.enabled=false`로 끔). 오프셋은 handler Success(inbox면 트랜잭션 커밋) 뒤 또는 DLQ 쓰기 확인 뒤에만 커밋, auto-commit 금지, `read_committed`, 재시도 소진 → DLQ(없으면 멈춤) — Kafka+PG로 왕복·중복 제거·DLQ 검증 - 미사용 API 타입·Blocking/Reactive/Batch facade·reactor api 노출 삭제, `PublishResult` 정적 factory, `MessagingCapabilities` → EnumSet, API leaf 병합(core-api+schema-api, admin-api+admin-runtime), starter god config 분해, Kafka broker 설정은 starter-kafka로, 등급 용어 정리 - 결함 수정: Retry 응답이 시도 횟수를 세지 않아 partition 정지(P21-new-1), deliveryAttempt 항상 1(P21-new-2) - 통합: app-bootstrap `MessagingPublicSurfaceContractTest` 기대값을 P21 변경에 맞춤, starter-kafka에 `api kafka-clients` 검증: ./gradlew check architectureCheck verifyDependencyLocks :app-bootstrap:verifyEnvKeys :app-bootstrap:integrationTest :sample-portfolio:integrationTest :messaging:messaging-spring-boot-starter:integrationTest :messaging:messaging-reliability-jdbc-postgresql:integrationTest mainCheck --continue 통과(812 task) Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/messaging/outbox-inbox.md | 18 + messaging/CLAUDE.md | 8 +- messaging/messaging-admin-api/build.gradle | 8 - messaging/messaging-admin-api/gradle.lockfile | 84 --- .../messaging-admin-runtime/build.gradle | 12 - messaging/messaging-admin/build.gradle | 14 + .../gradle.lockfile | 0 .../messaging/admin/AdminApproval.java | 0 .../admin/AdminOperationJournal.java | 0 .../messaging/admin/AdminOperationLease.java | 0 .../messaging/admin/AdminOperationRecord.java | 0 .../messaging/admin/AdminOperationState.java | 0 .../messaging/admin/ApprovalGrant.java | 0 .../messaging/admin/ApprovalVerifier.java | 0 .../messaging/admin/ApprovedRedrivePlan.java | 0 .../messaging/admin/ApprovedReplayPlan.java | 0 .../messaging/admin/DestinationTopology.java | 0 .../messaging/admin/DestructiveOperation.java | 0 .../admin/DestructiveOperationGuard.java | 0 .../messaging/admin/HmacApprovalVerifier.java | 0 .../messaging/admin/PlanDigest.java | 0 .../messaging/admin/RedrivePlan.java | 0 .../messaging/admin/RedriveRequest.java | 0 .../messaging/admin/RedriveResult.java | 0 .../messaging/admin/ReplayPlan.java | 0 .../messaging/admin/ReplayRequest.java | 0 .../messaging/admin/ReplayResult.java | 0 .../messaging/admin/TopologyIssue.java | 0 .../admin/TopologyManagementMode.java | 0 .../messaging/admin/TopologyManifest.java | 0 .../admin/TopologyValidationReport.java | 0 .../messaging/admin/VerifiedApproval.java | 0 .../runtime/BrokerTopologyInspector.java | 0 .../runtime/CompositeTopologyValidator.java | 0 .../runtime/DefaultMessagingAdminService.java | 0 .../runtime/DestructiveMessagingAdmin.java | 0 .../InMemoryAdminOperationJournal.java | 0 .../admin/runtime/MessagingAdminService.java | 0 .../admin/runtime/RedriveReport.java | 0 .../admin/runtime/RedriveService.java | 0 .../messaging/admin/runtime/ReplayReport.java | 0 .../admin/runtime/ReplayService.java | 0 .../runtime/TopologyValidationRuntime.java | 0 .../admin/runtime/TopologyValidator.java | 0 .../admin/DestructiveOperationGuardTest.java | 0 .../runtime/AdminOperationJournalTest.java | 0 .../admin/runtime/ApprovalForgeryTest.java | 0 .../runtime/ApprovedPlanExecutionTest.java | 0 .../admin/runtime/RedriveResumptionTest.java | 0 .../admin/runtime/ReplayServiceTest.java | 0 .../TopologyValidationRuntimeTest.java | 0 .../admin/runtime/TopologyValidatorTest.java | 0 messaging/messaging-cloudevents/build.gradle | 1 - .../api/delivery/BatchDeliveryMetadata.java | 46 -- .../api/delivery/BatchMessageDelivery.java | 29 - .../api/delivery/BatchMessageHandler.java | 24 - .../api/delivery/MessageHandler.java | 21 +- .../api/delivery/PauseResumeController.java | 33 - .../api/delivery/ProcessingGuarantee.java | 11 - .../api/destination/CapabilityRegistry.java | 15 - .../destination/MessagingCapabilities.java | 82 ++- .../error/MessageAuthenticationException.java | 49 -- .../MessageBrokerUnavailableException.java | 49 -- .../api/error/MessageConsumerException.java | 49 -- .../api/error/MessageDeadLetterException.java | 54 -- .../error/MessageHandlerTimeoutException.java | 54 -- .../error/MessageHeaderRejectedException.java | 49 -- .../MessagePublishAmbiguousException.java | 54 -- .../MessagePublishRejectedException.java | 53 -- .../api/error/MessageRedriveException.java | 49 -- .../error/MessageRetryExhaustedException.java | 49 -- .../api/error/MessageRoutingException.java | 49 -- .../api/error/MessageSettlementException.java | 49 -- .../MessageSettlementUnknownException.java | 53 -- .../api/publish/BatchMessagePublisher.java | 25 - .../api/publish/BatchPublishItemResult.java | 27 - .../api/publish/BatchPublishOptions.java | 42 -- .../api/publish/BatchPublishResult.java | 44 -- .../api/publish/BlockingMessagePublisher.java | 30 - .../api/publish/DelayedMessagePublisher.java | 28 - .../messaging/api/publish/PublishResult.java | 64 ++ .../api/settlement/ManualMessageHandler.java | 25 - .../api/settlement/SettlementController.java | 50 -- .../messaging/schema/BoundedByteSink.java | 0 .../messaging/schema/EncodedMessage.java | 0 .../schema/EncodedMessagePublisher.java | 0 .../messaging/schema/MessageCodec.java | 0 .../schema/MessageCodecRegistry.java | 0 .../messaging/schema/MessageContractKey.java | 0 .../schema/RawBytesMessageCodec.java | 0 .../messaging/schema/SchemaCompatibility.java | 0 .../schema/SchemaCompatibilityValidator.java | 0 .../messaging/schema/SchemaReference.java | 0 .../messaging/schema/SchemaRegistry.java | 0 .../DestinationCapabilityTest.java | 21 +- .../api/publish/PublishResultTest.java | 44 ++ .../messaging/schema/BoundedByteSinkTest.java | 0 .../schema/RawBytesMessageCodecTest.java | 0 .../SchemaCompatibilityValidatorTest.java | 0 messaging/messaging-kafka/build.gradle | 7 +- .../kafka/KafkaBatchConsumerRegistrar.java | 146 ----- .../kafka/KafkaConsumerRegistrar.java | 72 ++- .../kafka/KafkaMessagingTransport.java | 22 +- .../kafka/KafkaPublishFailureClassifier.java | 21 +- .../messaging/kafka/KafkaPublishMapper.java | 11 +- .../kafka/KafkaTopologyInspector.java | 45 -- .../kafka/KafkaConsumerRegistrarTest.java | 73 +++ .../messaging/kafka/KafkaContractHarness.java | 4 +- messaging/messaging-policy/build.gradle | 1 - .../caskeleton/messaging/policy/ApiTier.java | 30 + .../messaging/policy/CapabilityTier.java | 21 - .../policy/DefaultRetryDecisionEngine.java | 13 +- .../messaging/policy/DestinationProfile.java | 2 +- .../policy/DestinationProfileValidator.java | 2 +- .../DestinationProfileValidatorTest.java | 4 +- .../policy/RetryDecisionEngineTest.java | 9 +- .../reliability/IdempotentMessageHandler.java | 33 - .../messaging/reliability/InboxRecord.java | 27 - .../messaging/reliability/InboxResult.java | 45 -- .../reliability/ReliableMessagePublisher.java | 27 - .../TransactionalMessageAction.java | 29 - .../build.gradle | 2 +- .../gradle.lockfile | 5 + messaging/messaging-reliability/build.gradle | 1 - .../messaging/inbox/InboxMessageHandler.java | 129 ++++ .../inbox/TransactionalInboxHandler.java | 104 ---- .../messaging/outbox/OutboxRelay.java | 20 +- .../inbox/InboxMessageHandlerTest.java | 179 ++++++ .../messaging/inbox/InboxOperationsTest.java | 10 - .../messaging/outbox/OutboxRelayTest.java | 4 +- messaging/messaging-runtime-core/build.gradle | 1 - .../runtime/DefaultDeliveryProcessor.java | 272 ++++++-- .../runtime/DefaultMessagePublisher.java | 30 +- .../runtime/DefaultDeliveryProcessorTest.java | 341 ++++++++-- .../messaging/runtime/RuntimeFixtures.java | 4 +- messaging/messaging-schema-api/build.gradle | 7 - .../messaging-schema-api/gradle.lockfile | 84 --- messaging/messaging-schema-avro/build.gradle | 1 - messaging/messaging-schema-json/build.gradle | 1 - .../messaging-schema-protobuf/build.gradle | 1 - .../build.gradle | 2 + .../gradle.lockfile | 2 - .../autoconfigure/KafkaBrokerCatalog.java | 37 ++ .../autoconfigure/KafkaBrokerSettings.java | 86 +++ .../KafkaConsumerConfiguration.java | 165 +++++ .../autoconfigure/KafkaConsumerLifecycle.java | 69 ++ .../KafkaDeadLetterQuarantine.java | 65 ++ .../KafkaMessagingAutoConfiguration.java | 105 ++-- .../KafkaPolledRegistration.java | 169 +++++ .../build.gradle | 17 +- .../gradle.lockfile | 12 +- ...essagingKafkaConsumeQualificationTest.java | 460 ++++++++++++++ ...ssagingLiveRoundTripQualificationTest.java | 2 +- .../autoconfigure/BrokerSettings.java | 30 +- .../DefaultBatchMessagePublisher.java | 262 -------- .../DefaultBlockingMessagePublisher.java | 65 -- .../DefaultReactiveMessagePublisher.java | 43 -- .../autoconfigure/DestinationSettings.java | 4 +- .../autoconfigure/MessageConsumers.java | 46 ++ .../MessagingConfigurationCompiler.java | 21 +- .../MessagingConfigurationKeyValidator.java | 33 +- .../MessagingConsumeConfiguration.java | 151 +++++ .../MessagingCoreAutoConfiguration.java | 587 +----------------- .../MessagingObservabilityConfiguration.java | 77 +++ .../MessagingProviderSelection.java | 10 + .../MessagingPublishConfiguration.java | 304 +++++++++ ...MessagingReliabilityAutoConfiguration.java | 16 - .../MessagingValidationConfiguration.java | 194 ++++++ .../autoconfigure/PublishResults.java | 71 --- .../ReactiveMessagePublisher.java | 32 - .../autoconfigure/BatchPublisherTest.java | 328 ---------- .../autoconfigure/BlockingFacadeTest.java | 148 ----- .../KafkaProducerConfigTest.java | 1 + .../KafkaProducerSecurityTest.java | 1 + .../MessagingAutoConfigurationTest.java | 7 +- .../MessagingConfigurationBindingTest.java | 27 +- .../MessagingConsumeConfigurationTest.java | 117 ++++ .../autoconfigure/MessagingEndpointTest.java | 4 +- .../autoconfigure/ReactiveFacadeTest.java | 162 ----- messaging/messaging-testkit/build.gradle | 1 - .../testkit/CompatibilityMatrix.java | 20 +- .../testkit/CompatibilityMatrixTest.java | 20 +- .../testkit/CrossBrokerContractSuite.java | 4 +- .../MessagingDocumentationContractTest.java | 4 +- .../MessagingPublicSurfaceContractTest.java | 6 +- .../messaging-transport-spi/build.gradle | 1 - .../transport/MessagingLifecycle.java | 67 -- .../transport/MessagingLifecycleTest.java | 59 -- 188 files changed, 3405 insertions(+), 4015 deletions(-) delete mode 100644 messaging/messaging-admin-api/build.gradle delete mode 100644 messaging/messaging-admin-api/gradle.lockfile delete mode 100644 messaging/messaging-admin-runtime/build.gradle create mode 100644 messaging/messaging-admin/build.gradle rename messaging/{messaging-admin-runtime => messaging-admin}/gradle.lockfile (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/AdminApproval.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/AdminOperationJournal.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/AdminOperationLease.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/AdminOperationRecord.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/AdminOperationState.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/ApprovalGrant.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/ApprovalVerifier.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/ApprovedRedrivePlan.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/ApprovedReplayPlan.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/DestinationTopology.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperation.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperationGuard.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/HmacApprovalVerifier.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/PlanDigest.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/RedrivePlan.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/RedriveRequest.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/RedriveResult.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/ReplayPlan.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/ReplayRequest.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/ReplayResult.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/TopologyIssue.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/TopologyManagementMode.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/TopologyManifest.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/TopologyValidationReport.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/VerifiedApproval.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/BrokerTopologyInspector.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/CompositeTopologyValidator.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/DefaultMessagingAdminService.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/DestructiveMessagingAdmin.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/InMemoryAdminOperationJournal.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/MessagingAdminService.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveReport.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveService.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayReport.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayService.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntime.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidator.java (100%) rename messaging/{messaging-admin-api => messaging-admin}/src/test/java/dev/caskeleton/messaging/admin/DestructiveOperationGuardTest.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/test/java/dev/caskeleton/messaging/admin/runtime/AdminOperationJournalTest.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovalForgeryTest.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovedPlanExecutionTest.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/test/java/dev/caskeleton/messaging/admin/runtime/RedriveResumptionTest.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/test/java/dev/caskeleton/messaging/admin/runtime/ReplayServiceTest.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntimeTest.java (100%) rename messaging/{messaging-admin-runtime => messaging-admin}/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidatorTest.java (100%) delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchDeliveryMetadata.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageDelivery.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageHandler.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/PauseResumeController.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/ProcessingGuarantee.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/CapabilityRegistry.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageAuthenticationException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageBrokerUnavailableException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageConsumerException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageDeadLetterException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHandlerTimeoutException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHeaderRejectedException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishAmbiguousException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishRejectedException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRedriveException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRetryExhaustedException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRoutingException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementUnknownException.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchMessagePublisher.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishItemResult.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishOptions.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishResult.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BlockingMessagePublisher.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/DelayedMessagePublisher.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/ManualMessageHandler.java delete mode 100644 messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/SettlementController.java rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/BoundedByteSink.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/EncodedMessage.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/EncodedMessagePublisher.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/MessageCodec.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/MessageCodecRegistry.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/MessageContractKey.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/RawBytesMessageCodec.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibility.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidator.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/SchemaReference.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/main/java/dev/caskeleton/messaging/schema/SchemaRegistry.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/test/java/dev/caskeleton/messaging/schema/BoundedByteSinkTest.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/test/java/dev/caskeleton/messaging/schema/RawBytesMessageCodecTest.java (100%) rename messaging/{messaging-schema-api => messaging-core-api}/src/test/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidatorTest.java (100%) delete mode 100644 messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaBatchConsumerRegistrar.java delete mode 100644 messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaTopologyInspector.java create mode 100644 messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/ApiTier.java delete mode 100644 messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/CapabilityTier.java delete mode 100644 messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/IdempotentMessageHandler.java delete mode 100644 messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRecord.java delete mode 100644 messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxResult.java delete mode 100644 messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/ReliableMessagePublisher.java delete mode 100644 messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/TransactionalMessageAction.java create mode 100644 messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxMessageHandler.java delete mode 100644 messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java create mode 100644 messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxMessageHandlerTest.java delete mode 100644 messaging/messaging-schema-api/build.gradle delete mode 100644 messaging/messaging-schema-api/gradle.lockfile create mode 100644 messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerCatalog.java create mode 100644 messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerSettings.java create mode 100644 messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerConfiguration.java create mode 100644 messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerLifecycle.java create mode 100644 messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java create mode 100644 messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaPolledRegistration.java create mode 100644 messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingKafkaConsumeQualificationTest.java delete mode 100644 messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBatchMessagePublisher.java delete mode 100644 messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBlockingMessagePublisher.java delete mode 100644 messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java create mode 100644 messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessageConsumers.java create mode 100644 messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfiguration.java create mode 100644 messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingObservabilityConfiguration.java create mode 100644 messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPublishConfiguration.java create mode 100644 messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingValidationConfiguration.java delete mode 100644 messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/PublishResults.java delete mode 100644 messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/ReactiveMessagePublisher.java delete mode 100644 messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BatchPublisherTest.java delete mode 100644 messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BlockingFacadeTest.java create mode 100644 messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfigurationTest.java delete mode 100644 messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java delete mode 100644 messaging/messaging-transport-spi/src/main/java/dev/caskeleton/messaging/transport/MessagingLifecycle.java delete mode 100644 messaging/messaging-transport-spi/src/test/java/dev/caskeleton/messaging/transport/MessagingLifecycleTest.java diff --git a/docs/messaging/outbox-inbox.md b/docs/messaging/outbox-inbox.md index d71c30f..bf68398 100644 --- a/docs/messaging/outbox-inbox.md +++ b/docs/messaging/outbox-inbox.md @@ -108,6 +108,24 @@ transactions.inTransaction(() -> { 별도 transaction으로 예약하면 Inbox가 닫으려던 바로 그 창이 다시 열린다. +### 소비와 연결 (소유자 결정 10) + +consume 계약은 `MessageHandler` 하나다. handler를 bean으로 등록하면 `messaging-spring-boot-starter-kafka`가 +그 handler가 가리키는 destination마다 Kafka consumer를 연다(`app.messaging.enabled=false`면 없다). + +- destination profile이 `guarantees.external-side-effect: INBOX_TRANSACTIONAL`이면 platform이 handler를 + `InboxMessageHandler`로 감싼다. 예약과 handler의 쓰기가 같은 transaction에서 commit되고, 이미 예약된 + message는 handler를 부르지 않고 `Success`로 settle한다. 이 경우 애플리케이션이 `IdempotentConsumer` + bean(inbox repository + transaction runner)을 둬야 하며, 없으면 기동이 실패한다. +- inbox의 consumer 식별자와 Kafka `group.id`는 destination의 `consumer.group`이다. 없으면 기동이 실패한다. +- offset은 handler가 `Success`로 답하고 (inbox destination이면) 그 transaction이 commit된 뒤에만, 또는 + DLQ 토픽 기록이 확인된 뒤에만 넘어간다. auto-commit은 쓰지 않는다. +- `Retry`·예외·timeout은 `retry.max-attempts`까지 partition을 멈추고 다시 읽으며, 다 쓰면 DLQ 토픽에 + 원본 record를 남기고 넘어간다. DLQ가 없는 destination(at-most-once)은 넘어가지 않고 멈춘다. +- `Reject`는 at-most-once destination에서만 버리고, 그 밖에서는 dead letter로 보낸다. + +실제 왕복·중복 제거·DLQ는 `MessagingKafkaConsumeQualificationTest`(Kafka + PostgreSQL Testcontainers)가 확인한다. + ### 복합 키 `PRIMARY KEY (message_id, consumer_id)`. diff --git a/messaging/CLAUDE.md b/messaging/CLAUDE.md index 57e6932..14c58e8 100644 --- a/messaging/CLAUDE.md +++ b/messaging/CLAUDE.md @@ -14,9 +14,9 @@ leaf의 정확한 id는 registry에서 읽는다. family는 **디렉터리 이 | Family | 역할 | 허용 framework | | --- | --- | --- | -| `*-api` (`core-api`, `schema-api`, `policy`, `reliability-api`, `admin-api`, `transport-spi`) | 값 타입, 포트, 정책 계약 | Java stdlib만. Spring/broker SDK/logging framework 금지 | +| `*-api` (`core-api` — `api`·`schema` 패키지, `policy`, `reliability-api`, `transport-spi`) | 값 타입, 포트, 정책 계약 | Java stdlib만. Spring/broker SDK/logging framework 금지 | | `schema-*` (json/avro/protobuf), `cloudevents` | 인코딩 구현 | 해당 codec 라이브러리만. Spring 금지 | -| `runtime-core`, `security`, `observability`, `admin-runtime`, `reliability` | framework-free 실행 로직 | Micrometer는 `observability`에서만. 나머지는 stdlib | +| `runtime-core`, `security`, `observability`, `admin`(계약 `admin` + 실행 `admin.runtime`), `reliability` | framework-free 실행 로직 | Micrometer는 `observability`에서만. 나머지는 stdlib | | `kafka` | broker adapter | 해당 broker client. Spring은 client가 곧 Spring인 경우만 | | `reliability-jdbc-postgresql` | 관계형 신뢰성 adapter (outbox·inbox repository, admin journal) | Spring JDBC / Spring TX. **JPA 아님** — 이름이 곧 계약이다 | | `spring-boot-starter`, `spring-boot-starter-kafka` | 조립 경계 | Spring Boot autoconfigure. 여기서만 | @@ -35,7 +35,7 @@ JDBC이고 SQL은 `jsonb` · `FOR UPDATE SKIP LOCKED` · `ON CONFLICT` · `TIMES - **vendor 타입을 public signature에 노출하면 그 의존성은 `api`로 선언한다.** 숨긴다고 타입이 숨겨지지 않는다 — consumer가 이 모듈이 문서화한 API에 대해 컴파일하지 못하게 만들 뿐이다. (`kafka-clients`, `cloudevents-api`, `micrometer-core`, `avro`, - `protobuf-java`, `reactor-core`가 이 규칙으로 `api`다.) + `protobuf-java`가 이 규칙으로 `api`다.) - 반대로 **노출하지 않는 것은 `implementation`으로 낮춘다.** starter가 노출하지 않는 하위 모듈을 `implementation`으로 내린 것이, JSON codec이 `ObjectMapper`를 private으로만 쓰고 Jackson을 `implementation`으로 둔 것이 그 예다. @@ -72,7 +72,7 @@ qualification/support 상태를 함께 변경한다. bridge는 exact canonical envelope bytes를 다시 serialize하지 않고, destination/access/admission/runtime lease/transport/outcome normalization/observation은 platform 중앙 publish path를 그대로 탄다. bridge -package는 `messaging-core-api`와 `messaging-schema-api`만 보며 concrete runtime/transport/Kafka 타입을 +package는 `messaging-core-api`만 보며 concrete runtime/transport/Kafka 타입을 직접 보지 않는다. **MSG-015 cutover는 끝났다(2026-09-28, 소유자 결정 7).** application의 outbox append는 canonical diff --git a/messaging/messaging-admin-api/build.gradle b/messaging/messaging-admin-api/build.gradle deleted file mode 100644 index 55c8be5..0000000 --- a/messaging/messaging-admin-api/build.gradle +++ /dev/null @@ -1,8 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-policy') -} diff --git a/messaging/messaging-admin-api/gradle.lockfile b/messaging/messaging-admin-api/gradle.lockfile deleted file mode 100644 index e2c9585..0000000 --- a/messaging/messaging-admin-api/gradle.lockfile +++ /dev/null @@ -1,84 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty=compileClasspath,runtimeClasspath diff --git a/messaging/messaging-admin-runtime/build.gradle b/messaging/messaging-admin-runtime/build.gradle deleted file mode 100644 index feb5781..0000000 --- a/messaging/messaging-admin-runtime/build.gradle +++ /dev/null @@ -1,12 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-admin-api') - api project(':messaging:messaging-transport-spi') - api project(':messaging:messaging-security') - api project(':messaging:messaging-observability') -} diff --git a/messaging/messaging-admin/build.gradle b/messaging/messaging-admin/build.gradle new file mode 100644 index 0000000..a6b6ce5 --- /dev/null +++ b/messaging/messaging-admin/build.gradle @@ -0,0 +1,14 @@ +plugins { + id 'ca.platform-module' +} + +// The admin plane: its contract (package `admin`) and the stdlib services that execute it +// (package `admin.runtime`). They were two leaves, and every consumer of the runtime took the +// contract too, so the split added a registry entry without separating anything (S7-#12). +dependencies { + api project(':messaging:messaging-core-api') + + // ReplayService records a MessagingAuditEvent internally; no public signature names it, so + // Micrometer does not reach the journal adapters that implement the contract. + implementation project(':messaging:messaging-observability') +} diff --git a/messaging/messaging-admin-runtime/gradle.lockfile b/messaging/messaging-admin/gradle.lockfile similarity index 100% rename from messaging/messaging-admin-runtime/gradle.lockfile rename to messaging/messaging-admin/gradle.lockfile diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminApproval.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminApproval.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminApproval.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminApproval.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationJournal.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationJournal.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationJournal.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationJournal.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationLease.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationLease.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationLease.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationLease.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationRecord.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationRecord.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationRecord.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationRecord.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationState.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationState.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationState.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationState.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovalGrant.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovalGrant.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovalGrant.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovalGrant.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovalVerifier.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovalVerifier.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovalVerifier.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovalVerifier.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovedRedrivePlan.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovedRedrivePlan.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovedRedrivePlan.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovedRedrivePlan.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovedReplayPlan.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovedReplayPlan.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovedReplayPlan.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovedReplayPlan.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestinationTopology.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestinationTopology.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestinationTopology.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestinationTopology.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperation.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperation.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperation.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperation.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperationGuard.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperationGuard.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperationGuard.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperationGuard.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/HmacApprovalVerifier.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/HmacApprovalVerifier.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/HmacApprovalVerifier.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/HmacApprovalVerifier.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/PlanDigest.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/PlanDigest.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/PlanDigest.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/PlanDigest.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedrivePlan.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedrivePlan.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedrivePlan.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedrivePlan.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedriveRequest.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedriveRequest.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedriveRequest.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedriveRequest.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedriveResult.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedriveResult.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedriveResult.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedriveResult.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayPlan.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayPlan.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayPlan.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayPlan.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayRequest.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayRequest.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayRequest.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayRequest.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayResult.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayResult.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayResult.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayResult.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyIssue.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyIssue.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyIssue.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyIssue.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyManagementMode.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyManagementMode.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyManagementMode.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyManagementMode.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyManifest.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyManifest.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyManifest.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyManifest.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyValidationReport.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyValidationReport.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyValidationReport.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyValidationReport.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/VerifiedApproval.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/VerifiedApproval.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/VerifiedApproval.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/VerifiedApproval.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/BrokerTopologyInspector.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/BrokerTopologyInspector.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/BrokerTopologyInspector.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/BrokerTopologyInspector.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/CompositeTopologyValidator.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/CompositeTopologyValidator.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/CompositeTopologyValidator.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/CompositeTopologyValidator.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/DefaultMessagingAdminService.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/DefaultMessagingAdminService.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/DefaultMessagingAdminService.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/DefaultMessagingAdminService.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/DestructiveMessagingAdmin.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/DestructiveMessagingAdmin.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/DestructiveMessagingAdmin.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/DestructiveMessagingAdmin.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/InMemoryAdminOperationJournal.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/InMemoryAdminOperationJournal.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/InMemoryAdminOperationJournal.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/InMemoryAdminOperationJournal.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/MessagingAdminService.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/MessagingAdminService.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/MessagingAdminService.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/MessagingAdminService.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveReport.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveReport.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveReport.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveReport.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveService.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveService.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveService.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveService.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayReport.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayReport.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayReport.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayReport.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayService.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayService.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayService.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayService.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntime.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntime.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntime.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntime.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidator.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidator.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidator.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidator.java diff --git a/messaging/messaging-admin-api/src/test/java/dev/caskeleton/messaging/admin/DestructiveOperationGuardTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/DestructiveOperationGuardTest.java similarity index 100% rename from messaging/messaging-admin-api/src/test/java/dev/caskeleton/messaging/admin/DestructiveOperationGuardTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/DestructiveOperationGuardTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/AdminOperationJournalTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/AdminOperationJournalTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/AdminOperationJournalTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/AdminOperationJournalTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovalForgeryTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovalForgeryTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovalForgeryTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovalForgeryTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovedPlanExecutionTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovedPlanExecutionTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovedPlanExecutionTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovedPlanExecutionTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/RedriveResumptionTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/RedriveResumptionTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/RedriveResumptionTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/RedriveResumptionTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ReplayServiceTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ReplayServiceTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ReplayServiceTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ReplayServiceTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntimeTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntimeTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntimeTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntimeTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidatorTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidatorTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidatorTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidatorTest.java diff --git a/messaging/messaging-cloudevents/build.gradle b/messaging/messaging-cloudevents/build.gradle index b65cf72..80f04d9 100644 --- a/messaging/messaging-cloudevents/build.gradle +++ b/messaging/messaging-cloudevents/build.gradle @@ -4,7 +4,6 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') // api, because CloudEventMapper's public signatures return io.cloudevents.CloudEvent. // diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchDeliveryMetadata.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchDeliveryMetadata.java deleted file mode 100644 index 61b9ae7..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchDeliveryMetadata.java +++ /dev/null @@ -1,46 +0,0 @@ -package dev.caskeleton.messaging.api.delivery; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import java.time.Instant; -import java.util.Objects; -import java.util.Optional; - -/** - * Batch-wide facts about one delivered batch. - * - *

    {@code orderingUnit} is what makes a batch safe to hand to an ordered destination. A batch - * drawn from two partitions cannot be settled or retried as a unit without reordering one of them, - * so an ordered destination requires the batch to name exactly one ordering unit and the runtime - * rejects a batch that spans more. - * - * @param destination the logical destination - * @param size the number of deliveries in the batch - * @param orderingUnit the single partition or queue the batch was drawn from, when it has one - * @param settlableAsBatch whether the broker can settle the whole batch in one operation - * @param receivedAt when the consumer assembled the batch - */ -public record BatchDeliveryMetadata( - DestinationName destination, - int size, - Optional orderingUnit, - boolean settlableAsBatch, - Instant receivedAt) { - - public BatchDeliveryMetadata { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(orderingUnit, "orderingUnit must not be null"); - Objects.requireNonNull(receivedAt, "receivedAt must not be null"); - if (size < 1) { - throw new IllegalArgumentException("a delivered batch has at least one delivery"); - } - } - - /** - * Reports whether this batch may be handed to a destination with strict ordering. - * - * @return true when the batch came from exactly one ordering unit - */ - public boolean isSafeForOrderedDestination() { - return orderingUnit.isPresent(); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageDelivery.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageDelivery.java deleted file mode 100644 index 54c035c..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageDelivery.java +++ /dev/null @@ -1,29 +0,0 @@ -package dev.caskeleton.messaging.api.delivery; - -import java.util.List; -import java.util.Objects; - -/** - * A batch of decoded messages handed to a batch handler. - * - * @param the payload type - * @param deliveries the individual deliveries, in broker order - * @param metadata batch-wide facts - */ -public record BatchMessageDelivery( - List> deliveries, BatchDeliveryMetadata metadata) { - - public BatchMessageDelivery { - Objects.requireNonNull(deliveries, "deliveries must not be null"); - Objects.requireNonNull(metadata, "metadata must not be null"); - deliveries = List.copyOf(deliveries); - if (deliveries.isEmpty()) { - throw new IllegalArgumentException("a delivered batch is never empty"); - } - if (deliveries.size() != metadata.size()) { - throw new IllegalArgumentException( - "metadata size %d does not match %d deliveries" - .formatted(metadata.size(), deliveries.size())); - } - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageHandler.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageHandler.java deleted file mode 100644 index 655e920..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageHandler.java +++ /dev/null @@ -1,24 +0,0 @@ -package dev.caskeleton.messaging.api.delivery; - -import java.util.concurrent.CompletionStage; - -/** - * The M2 batch consume entry point. - * - *

    The handler returns one {@link HandleResult} for the whole batch. On a broker that settles - * batches atomically the runtime applies that result once; on a broker that settles per message it - * applies the same result to each delivery. Either way the handler is not asked to reason about - * which settlement mode it is running under. - * - * @param the payload type - */ -public interface BatchMessageHandler { - - /** - * Handles one delivered batch. - * - * @param batch the batch to handle - * @return a stage completing with the outcome for the whole batch - */ - CompletionStage handle(BatchMessageDelivery batch); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/MessageHandler.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/MessageHandler.java index 4fc54b1..0abdc89 100644 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/MessageHandler.java +++ b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/MessageHandler.java @@ -1,18 +1,33 @@ package dev.caskeleton.messaging.api.delivery; +import dev.caskeleton.messaging.api.destination.MessageDestination; import java.util.concurrent.CompletionStage; /** - * The M1 typed handler implemented by ordinary business code. + * The one consume contract, implemented by ordinary business code. * *

    Handlers state an outcome and never touch broker acknowledgement APIs. Duplicate delivery is a - * normal condition, not an error: implementations are expected to be idempotent, or to sit behind - * the Inbox. + * normal condition, not an error: implementations are expected to be idempotent, or to consume a + * destination whose profile declares {@code INBOX_TRANSACTIONAL}, where the platform runs the + * handler inside the inbox reservation's transaction. + * + *

    Registering an implementation as a bean is what turns consumption on: the broker starter opens + * a consumer for every destination at least one handler names. There is no second, batch or manual + * settlement contract; the platform settles, after the handler has answered. * * @param the payload type */ public interface MessageHandler { + /** + * The destination, message type and payload type this handler consumes. + * + *

    A destination may carry several message types; each type has at most one handler. + * + * @return what this handler consumes + */ + MessageDestination destination(); + /** * Handles one delivery. * diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/PauseResumeController.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/PauseResumeController.java deleted file mode 100644 index 5c86238..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/PauseResumeController.java +++ /dev/null @@ -1,33 +0,0 @@ -package dev.caskeleton.messaging.api.delivery; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import java.util.concurrent.CompletionStage; - -/** - * The M2 consumer flow-control entry point. - * - *

    Pausing stops new deliveries; it does not abandon the ones already in flight. A paused - * consumer stays a member of its group and keeps its assignment, which is the point: leaving the - * group to stop consuming would trigger a rebalance and hand the work to another instance that is - * just as overloaded. - */ -public interface PauseResumeController { - - /** - * Stops new deliveries for a destination scope. - * - * @param destination the logical destination - * @param scope the partition, queue, or {@code "*"} for every assigned unit - * @return a stage completing once no further deliveries will be dispatched - */ - CompletionStage pause(DestinationName destination, String scope); - - /** - * Resumes deliveries for a destination scope. - * - * @param destination the logical destination - * @param scope the partition, queue, or {@code "*"} for every assigned unit - * @return a stage completing once deliveries may flow again - */ - CompletionStage resume(DestinationName destination, String scope); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/ProcessingGuarantee.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/ProcessingGuarantee.java deleted file mode 100644 index f6f537d..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/ProcessingGuarantee.java +++ /dev/null @@ -1,11 +0,0 @@ -package dev.caskeleton.messaging.api.delivery; - -/** How duplicate processing is neutralised once a message has been delivered. */ -public enum ProcessingGuarantee { - - /** The handler suppresses duplicate effects using the message id or a business key. */ - APPLICATION_IDEMPOTENT, - - /** Atomicity holds only inside the transaction scope the broker itself defines. */ - BROKER_TRANSACTIONAL -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/CapabilityRegistry.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/CapabilityRegistry.java deleted file mode 100644 index b4628f5..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/CapabilityRegistry.java +++ /dev/null @@ -1,15 +0,0 @@ -package dev.caskeleton.messaging.api.destination; - -/** Resolves the capability snapshot for a logical destination. */ -public interface CapabilityRegistry { - - /** - * Returns the capabilities of a destination. - * - * @param destination the logical destination - * @return the capability snapshot - * @throws dev.caskeleton.messaging.api.error.MessagingConfigurationException when the destination - * is not registered - */ - DestinationCapabilities capabilities(DestinationName destination); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/MessagingCapabilities.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/MessagingCapabilities.java index fce026c..44cc6c5 100644 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/MessagingCapabilities.java +++ b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/MessagingCapabilities.java @@ -1,5 +1,10 @@ package dev.caskeleton.messaging.api.destination; +import java.util.Collections; +import java.util.EnumSet; +import java.util.Objects; +import java.util.Set; + /** * What a broker adapter can actually prove or do. * @@ -8,32 +13,35 @@ package dev.caskeleton.messaging.api.destination; * capability exception — rather than quietly degrading, because a silently weakened guarantee is * indistinguishable from a working one until the incident. * - * @param brokerAcknowledgement the adapter can prove the broker accepted a publish - * @param replicationOrPersistenceEvidence the adapter can prove replication or persistence - * @param perMessageSettlement individual messages can be settled - * @param batchSettlement batches can be settled as a unit - * @param orderedStream the destination preserves order inside an ordering unit - * @param keyedOrdering order is preserved per key - * @param replay historical messages can be re-read - * @param delayedDelivery delivery can be scheduled for a future instant - * @param brokerTransaction the broker offers a transaction scope - * @param deduplicatedPublish the broker suppresses duplicate publishes of a stable id - * @param nativeDeadLetter the broker provides dead lettering itself - * @param topologyManagement topology can be inspected or created through the adapter + *

    A set of named capabilities rather than a row of positional booleans. The record used to take + * twelve booleans in a fixed order, and an adapter declared itself with {@code new + * MessagingCapabilities(true, true, …, false, true)} — a line where swapping two arguments changes + * a guarantee and still compiles. Only the capabilities the platform reads are named: a flag + * nothing consults is a promise nobody checks. + * + * @param supported the capabilities the adapter can prove */ -public record MessagingCapabilities( - boolean brokerAcknowledgement, - boolean replicationOrPersistenceEvidence, - boolean perMessageSettlement, - boolean batchSettlement, - boolean orderedStream, - boolean keyedOrdering, - boolean replay, - boolean delayedDelivery, - boolean brokerTransaction, - boolean deduplicatedPublish, - boolean nativeDeadLetter, - boolean topologyManagement) { +public record MessagingCapabilities(Set supported) { + + /** One capability the platform consults before relying on it. */ + public enum Capability { + + /** The destination preserves order inside an ordering unit; required by pause-and-retry. */ + ORDERED_STREAM, + + /** Delivery can be scheduled for a future instant; required by broker-delayed retry. */ + DELAYED_DELIVERY, + + /** The broker suppresses duplicate publishes of a stable id; required by publish dedup. */ + DEDUPLICATED_PUBLISH + } + + public MessagingCapabilities { + Objects.requireNonNull(supported, "supported must not be null"); + supported = + Collections.unmodifiableSet( + supported.isEmpty() ? EnumSet.noneOf(Capability.class) : EnumSet.copyOf(supported)); + } /** * Returns a capability set with nothing enabled. @@ -41,7 +49,27 @@ public record MessagingCapabilities( * @return the empty capability set */ public static MessagingCapabilities none() { - return new MessagingCapabilities( - false, false, false, false, false, false, false, false, false, false, false, false); + return new MessagingCapabilities(EnumSet.noneOf(Capability.class)); + } + + /** + * Returns a capability set holding exactly the named capabilities. + * + * @param first the first capability + * @param rest the remaining capabilities + * @return the capability set + */ + public static MessagingCapabilities of(Capability first, Capability... rest) { + return new MessagingCapabilities(EnumSet.of(first, rest)); + } + + /** + * Reports whether the adapter can prove a capability. + * + * @param capability the capability to look up + * @return true when it is supported + */ + public boolean supports(Capability capability) { + return supported.contains(Objects.requireNonNull(capability, "capability must not be null")); } } diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageAuthenticationException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageAuthenticationException.java deleted file mode 100644 index c7bc1e8..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageAuthenticationException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** Broker authentication failed. */ -public class MessageAuthenticationException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.AUTHENTICATION; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageAuthenticationException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageAuthenticationException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageAuthenticationException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageBrokerUnavailableException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageBrokerUnavailableException.java deleted file mode 100644 index 65a9640..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageBrokerUnavailableException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** The broker could not be reached. */ -public class MessageBrokerUnavailableException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.TRANSIENT_INFRASTRUCTURE; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageBrokerUnavailableException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageBrokerUnavailableException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageBrokerUnavailableException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageConsumerException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageConsumerException.java deleted file mode 100644 index 3b3b095..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageConsumerException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** The consumer runtime failed while handling a delivery. */ -public class MessageConsumerException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PROCESSING_TRANSIENT; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageConsumerException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageConsumerException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageConsumerException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageDeadLetterException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageDeadLetterException.java deleted file mode 100644 index 210acd7..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageDeadLetterException.java +++ /dev/null @@ -1,54 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** - * Publishing to the dead letter destination failed. - * - *

    The source message stays unsettled. Acknowledging a source whose DLQ publish failed would - * destroy the only remaining copy. - */ -public class MessageDeadLetterException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.TRANSIENT_INFRASTRUCTURE; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageDeadLetterException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageDeadLetterException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageDeadLetterException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHandlerTimeoutException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHandlerTimeoutException.java deleted file mode 100644 index 9632b7f..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHandlerTimeoutException.java +++ /dev/null @@ -1,54 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** - * The handler exceeded its timeout. - * - *

    Whether the handler committed a side effect before the deadline is unknown, so the delivery is - * recorded as possibly duplicated when it is retried. - */ -public class MessageHandlerTimeoutException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PROCESSING_TRANSIENT; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageHandlerTimeoutException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageHandlerTimeoutException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageHandlerTimeoutException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHeaderRejectedException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHeaderRejectedException.java deleted file mode 100644 index 7730c35..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHeaderRejectedException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** A header was reserved, secret, or exceeded a limit. */ -public class MessageHeaderRejectedException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PERMANENT_BUSINESS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageHeaderRejectedException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageHeaderRejectedException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageHeaderRejectedException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishAmbiguousException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishAmbiguousException.java deleted file mode 100644 index 26af714..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishAmbiguousException.java +++ /dev/null @@ -1,54 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** - * The publish outcome could not be determined. - * - *

    The broker may hold the message. Republishing is permitted only under the same logical message - * id, so that broker deduplication or a downstream Inbox can collapse the duplicate. - */ -public class MessagePublishAmbiguousException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.AMBIGUOUS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessagePublishAmbiguousException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessagePublishAmbiguousException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessagePublishAmbiguousException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishRejectedException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishRejectedException.java deleted file mode 100644 index 6f913b1..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishRejectedException.java +++ /dev/null @@ -1,53 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** - * The broker definitively refused the publish. - * - *

    The message is known not to be stored, so abandoning this attempt is safe. - */ -public class MessagePublishRejectedException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PERMANENT_BUSINESS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessagePublishRejectedException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessagePublishRejectedException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessagePublishRejectedException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRedriveException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRedriveException.java deleted file mode 100644 index 9d4b657..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRedriveException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** A redrive operation failed. */ -public class MessageRedriveException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.TRANSIENT_INFRASTRUCTURE; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageRedriveException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageRedriveException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageRedriveException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRetryExhaustedException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRetryExhaustedException.java deleted file mode 100644 index 2ffea59..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRetryExhaustedException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** The retry policy's attempt budget was exhausted. */ -public class MessageRetryExhaustedException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PERMANENT_BUSINESS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageRetryExhaustedException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageRetryExhaustedException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageRetryExhaustedException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRoutingException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRoutingException.java deleted file mode 100644 index c5ea68f..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRoutingException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** The broker accepted the publish but could not route it. */ -public class MessageRoutingException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PERMANENT_BUSINESS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageRoutingException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageRoutingException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageRoutingException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementException.java deleted file mode 100644 index 63cd72b..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** Settlement was refused by the broker. */ -public class MessageSettlementException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.TRANSIENT_INFRASTRUCTURE; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageSettlementException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageSettlementException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageSettlementException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementUnknownException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementUnknownException.java deleted file mode 100644 index 27072f1..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementUnknownException.java +++ /dev/null @@ -1,53 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** - * Settlement was sent but not confirmed. - * - *

    Redelivery is possible. Callers must not treat this as a successful settlement. - */ -public class MessageSettlementUnknownException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.AMBIGUOUS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageSettlementUnknownException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageSettlementUnknownException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageSettlementUnknownException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchMessagePublisher.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchMessagePublisher.java deleted file mode 100644 index 6a56824..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchMessagePublisher.java +++ /dev/null @@ -1,25 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import java.util.List; -import java.util.concurrent.CompletionStage; - -/** - * The M2 batch publish entry point. - * - *

    A batch is a throughput optimisation, not a transaction. Entries succeed, fail, and go - * ambiguous independently, and {@link BatchPublishResult} keeps every entry's outcome against its - * submission index so the caller can act per entry. Nothing here rolls back: an adapter that - * reported a confirmed entry cannot un-publish it because a later entry was rejected. - */ -public interface BatchMessagePublisher { - - /** - * Publishes a batch of independent requests. - * - * @param requests the entries to publish, in submission order - * @param options the batch-wide options - * @return a stage completing with one outcome per submitted entry - */ - CompletionStage publish( - List> requests, BatchPublishOptions options); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishItemResult.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishItemResult.java deleted file mode 100644 index a275061..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishItemResult.java +++ /dev/null @@ -1,27 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import dev.caskeleton.messaging.api.MessageId; -import java.util.Objects; - -/** - * One item's outcome inside a batch publish. - * - *

    Kept per index rather than collapsed into a batch-level success flag. A batch is not a - * transaction: some items confirm, some are rejected, and some are ambiguous, and the caller has to - * be able to act differently on each. A single boolean would force it to re-submit everything, - * which duplicates the items that already landed. - * - * @param index the item's position in the submitted list - * @param messageId the item's logical identity - * @param result the item's outcome and evidence - */ -public record BatchPublishItemResult(int index, MessageId messageId, PublishResult result) { - - public BatchPublishItemResult { - Objects.requireNonNull(messageId, "messageId must not be null"); - Objects.requireNonNull(result, "result must not be null"); - if (index < 0) { - throw new IllegalArgumentException("index must not be negative"); - } - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishOptions.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishOptions.java deleted file mode 100644 index 8361438..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishOptions.java +++ /dev/null @@ -1,42 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import java.time.Duration; -import java.util.Objects; - -/** - * Options covering a whole batch publish. - * - *

    There is no retry setting. A batch is not a transaction, and retrying the batch would resubmit - * entries that already confirmed, so the platform never retries one on the caller's behalf; the - * caller resubmits the individual entries it knows are duplicate-safe under their original {@code - * messageId}. - * - * @param timeout the deadline for the whole batch - * @param maxBatchSize the largest number of entries accepted in one call - * @param stopOnFirstRejection whether to skip remaining entries once one is rejected - */ -public record BatchPublishOptions( - Duration timeout, int maxBatchSize, boolean stopOnFirstRejection) { - - private static final Duration DEFAULT_TIMEOUT = Duration.ofSeconds(30); - private static final int DEFAULT_MAX_BATCH_SIZE = 500; - - public BatchPublishOptions { - Objects.requireNonNull(timeout, "timeout must not be null"); - if (timeout.isNegative() || timeout.isZero()) { - throw new IllegalArgumentException("batch timeout must be positive"); - } - if (maxBatchSize < 1) { - throw new IllegalArgumentException("maxBatchSize must be at least 1"); - } - } - - /** - * Returns the defaults: thirty seconds, 500 entries, best-effort completion of every entry. - * - * @return the default options - */ - public static BatchPublishOptions defaults() { - return new BatchPublishOptions(DEFAULT_TIMEOUT, DEFAULT_MAX_BATCH_SIZE, false); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishResult.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishResult.java deleted file mode 100644 index 707bee0..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishResult.java +++ /dev/null @@ -1,44 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import java.time.Duration; -import java.util.List; -import java.util.Objects; - -/** - * The outcome of one batch publish. - * - *

    There is no automatic batch retry. Re-submitting a whole batch because part of it failed - * duplicates the part that succeeded; the caller re-submits only the items it decides are - * duplicate-safe, under their original message ids. - * - * @param items each item's outcome, in submission order - * @param elapsed how long the batch took - */ -public record BatchPublishResult(List items, Duration elapsed) { - - public BatchPublishResult { - Objects.requireNonNull(items, "items must not be null"); - Objects.requireNonNull(elapsed, "elapsed must not be null"); - items = List.copyOf(items); - } - - /** - * Returns the items with a given completion. - * - * @param completion the completion to select - * @return the matching items - */ - public List withCompletion(PublishCompletion completion) { - return items.stream().filter(item -> item.result().completion() == completion).toList(); - } - - /** - * Reports whether every item confirmed. - * - * @return true when nothing was rejected or ambiguous - */ - public boolean allConfirmed() { - return items.stream() - .allMatch(item -> item.result().completion() == PublishCompletion.CONFIRMED); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BlockingMessagePublisher.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BlockingMessagePublisher.java deleted file mode 100644 index 5b81bf8..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BlockingMessagePublisher.java +++ /dev/null @@ -1,30 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; - -/** - * The blocking facade over the asynchronous publisher. - * - *

    Offered because most business code is written against a blocking data access layer, and - * forcing a {@code CompletionStage} through it produces worse code than a blocking call does. It is - * a facade rather than a second implementation: the core stays asynchronous, and this only converts - * the lifecycle. - * - *

    It respects the publish deadline rather than waiting indefinitely, so a stalled broker - * surfaces as a timeout with ambiguous evidence instead of an exhausted thread pool. - */ -public interface BlockingMessagePublisher { - - /** - * Publishes one message and waits for its outcome. - * - * @param the payload type - * @param destination the logical destination - * @param message the envelope to publish - * @param options per-call options, including the deadline - * @return the outcome and its evidence - */ - PublishResult publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/DelayedMessagePublisher.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/DelayedMessagePublisher.java deleted file mode 100644 index 9b9749a..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/DelayedMessagePublisher.java +++ /dev/null @@ -1,28 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import java.time.Instant; -import java.util.concurrent.CompletionStage; - -/** - * The M2 scheduled-delivery entry point. - * - *

    Only destinations whose runtime declares {@code delayedDelivery} accept these calls. A plain - * Kafka topic does not: emulating a delay by holding the message in the producer would make the - * publisher's own liveness the schedule, so the platform rejects the call instead of pretending. - */ -public interface DelayedMessagePublisher { - - /** - * Publishes a message the broker must not deliver before the given instant. - * - * @param the payload type - * @param destination the logical destination - * @param message the envelope to publish - * @param deliverAt the earliest delivery time - * @return a stage completing with the publish outcome and its evidence - */ - CompletionStage publish( - MessageDestination destination, MessageEnvelope message, Instant deliverAt); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/PublishResult.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/PublishResult.java index 7449c3a..3582dd7 100644 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/PublishResult.java +++ b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/PublishResult.java @@ -100,6 +100,70 @@ public record PublishResult( } } + /** + * A publish the broker confirmed. + * + * @param level the confirmation the broker supplied + * @param routing whether the broker routed it; a broker with no routing step reports {@link + * RoutingOutcome#NOT_APPLICABLE} + * @param position the broker coordinate, when the broker reported one + * @param elapsed wall-clock time spent + * @return the confirmed result + */ + public static PublishResult confirmed( + ConfirmationLevel level, + RoutingOutcome routing, + Optional position, + Duration elapsed) { + return new PublishResult( + PublishCompletion.CONFIRMED, + PublishEvidence.confirmed(level), + routing, + position, + 1, + elapsed, + Optional.empty()); + } + + /** + * A publish refused before anything left the process. + * + *

    Routing is {@link RoutingOutcome#NOT_APPLICABLE}: nothing reached a broker that could have + * routed it. + * + * @param elapsed wall-clock time spent + * @param failure why it was refused + * @return the rejected result + */ + public static PublishResult rejected(Duration elapsed, FailureDescriptor failure) { + return new PublishResult( + PublishCompletion.REJECTED, + PublishEvidence.notTransmitted(), + RoutingOutcome.NOT_APPLICABLE, + Optional.empty(), + 1, + elapsed, + Optional.of(failure)); + } + + /** + * A publish whose fate is unknown: the broker may or may not hold the message. + * + * @param elapsed wall-clock time spent + * @param failure what made the outcome unknowable + * @return the ambiguous result + */ + public static PublishResult ambiguous(Duration elapsed, FailureDescriptor failure) { + return new PublishResult( + PublishCompletion.AMBIGUOUS, + PublishEvidence.ambiguous(), + RoutingOutcome.UNKNOWN, + Optional.empty(), + 1, + elapsed, + Optional.of(failure)); + } + /** * Reports whether the broker may be holding this message. * diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/ManualMessageHandler.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/ManualMessageHandler.java deleted file mode 100644 index 69f1620..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/ManualMessageHandler.java +++ /dev/null @@ -1,25 +0,0 @@ -package dev.caskeleton.messaging.api.settlement; - -import dev.caskeleton.messaging.api.delivery.MessageDelivery; -import java.util.concurrent.CompletionStage; - -/** - * The M2 handler that settles its own deliveries. - * - *

    Available only to modules granted the advanced capability. A handler that returns without - * calling a terminal settlement method is a defect the runtime detects and reports rather than - * silently leaving the message in flight. - * - * @param the payload type - */ -public interface ManualMessageHandler { - - /** - * Handles one delivery, settling it through the supplied controller. - * - * @param delivery the decoded message and its metadata - * @param settlement the settlement handle for this delivery - * @return a stage completing when handling has finished - */ - CompletionStage handle(MessageDelivery delivery, SettlementController settlement); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/SettlementController.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/SettlementController.java deleted file mode 100644 index e17e6b6..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/SettlementController.java +++ /dev/null @@ -1,50 +0,0 @@ -package dev.caskeleton.messaging.api.settlement; - -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import java.time.Duration; -import java.util.concurrent.CompletionStage; - -/** - * The M2 manual settlement handle. - * - *

    No method here accepts a broker channel, delivery tag, or offset. Manual settlement changes - * when the platform settles, never who settles, so the same DLQ-before-ack - * ordering and payload limits still apply. - * - *

    Exactly one terminal call is permitted per delivery; a second call fails rather than racing - * the first. - */ -public interface SettlementController { - - /** - * Acknowledges the message. - * - * @return a stage completing with the settlement outcome - */ - CompletionStage ack(); - - /** - * Schedules the message for another attempt. - * - * @param delay how long to wait before redelivery - * @return a stage completing with the settlement outcome - */ - CompletionStage retry(Duration delay); - - /** - * Routes the message to the dead letter destination, settling the source only after that publish - * is confirmed. - * - * @param failure the sanitized failure description - * @return a stage completing with the settlement outcome - */ - CompletionStage deadLetter(FailureDescriptor failure); - - /** - * Discards the message without dead lettering. - * - * @param failure the sanitized failure description - * @return a stage completing with the settlement outcome - */ - CompletionStage reject(FailureDescriptor failure); -} diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/BoundedByteSink.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/BoundedByteSink.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/BoundedByteSink.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/BoundedByteSink.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessage.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessage.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessage.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessage.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessagePublisher.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessagePublisher.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessagePublisher.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessagePublisher.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodec.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodec.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodec.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodec.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodecRegistry.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodecRegistry.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodecRegistry.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodecRegistry.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageContractKey.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageContractKey.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageContractKey.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageContractKey.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/RawBytesMessageCodec.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/RawBytesMessageCodec.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/RawBytesMessageCodec.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/RawBytesMessageCodec.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibility.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibility.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibility.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibility.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidator.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidator.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidator.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidator.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaReference.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaReference.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaReference.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaReference.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaRegistry.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaRegistry.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaRegistry.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaRegistry.java diff --git a/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/destination/DestinationCapabilityTest.java b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/destination/DestinationCapabilityTest.java index eaf9e32..da9c270 100644 --- a/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/destination/DestinationCapabilityTest.java +++ b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/destination/DestinationCapabilityTest.java @@ -31,9 +31,24 @@ class DestinationCapabilityTest { void emptyCapabilitySetAdvertisesNothing() { MessagingCapabilities capabilities = MessagingCapabilities.none(); - assertThat(capabilities.brokerAcknowledgement()).isFalse(); - assertThat(capabilities.replay()).isFalse(); - assertThat(capabilities.brokerTransaction()).isFalse(); + assertThat(capabilities.supported()).isEmpty(); + } + + @Test + @org.junit.jupiter.api.DisplayName("이름 붙인 capability만 지원하고 넘긴 집합을 나중에 바꿔도 영향이 없다") + void aCapabilitySetSupportsExactlyWhatWasNamedAndIgnoresLaterChangesToTheInput() { + // given + java.util.Set input = + java.util.EnumSet.of(MessagingCapabilities.Capability.ORDERED_STREAM); + MessagingCapabilities capabilities = new MessagingCapabilities(input); + + // when + input.add(MessagingCapabilities.Capability.DEDUPLICATED_PUBLISH); + + // then + assertThat(capabilities.supports(MessagingCapabilities.Capability.ORDERED_STREAM)).isTrue(); + assertThat(capabilities.supports(MessagingCapabilities.Capability.DEDUPLICATED_PUBLISH)) + .isFalse(); } @Test diff --git a/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/publish/PublishResultTest.java b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/publish/PublishResultTest.java index 2e1add5..122ede7 100644 --- a/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/publish/PublishResultTest.java +++ b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/publish/PublishResultTest.java @@ -7,6 +7,7 @@ import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import java.time.Duration; import java.util.Optional; +import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; class PublishResultTest { @@ -239,4 +240,47 @@ class PublishResultTest { Optional.empty())) .isInstanceOf(IllegalArgumentException.class); } + + @Test + @DisplayName("거부 factory는 전송되지 않았고 라우팅 대상이 아닌 결과를 만든다") + void theRejectedFactoryReportsNothingWasTransmittedOrRouted() { + // when + PublishResult result = PublishResult.rejected(Duration.ofMillis(3), AMBIGUOUS_FAILURE); + + // then + assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); + assertThat(result.evidence()).isEqualTo(PublishEvidence.notTransmitted()); + assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.NOT_APPLICABLE); + assertThat(result.failure()).contains(AMBIGUOUS_FAILURE); + } + + @Test + @DisplayName("모호 factory는 브로커가 메시지를 가졌을 수 있다고 보고한다") + void theAmbiguousFactoryReportsTheBrokerMayHoldTheMessage() { + // when + PublishResult result = PublishResult.ambiguous(Duration.ofMillis(3), AMBIGUOUS_FAILURE); + + // then + assertThat(result.mayHaveBeenStored()).isTrue(); + assertThat(result.evidence()).isEqualTo(PublishEvidence.ambiguous()); + assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.UNKNOWN); + } + + @Test + @DisplayName("확인 factory는 브로커 수락과 위치를 담고 실패 설명은 없다") + void theConfirmedFactoryCarriesAcceptanceAndPositionWithoutAFailure() { + // when + PublishResult result = + PublishResult.confirmed( + ConfirmationLevel.BROKER_ACK, + RoutingOutcome.NOT_APPLICABLE, + Optional.of(POSITION), + Duration.ofMillis(3)); + + // then + assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); + assertThat(result.evidence().brokerAccepted()).isTrue(); + assertThat(result.position()).contains(POSITION); + assertThat(result.failure()).isEmpty(); + } } diff --git a/messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/BoundedByteSinkTest.java b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/BoundedByteSinkTest.java similarity index 100% rename from messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/BoundedByteSinkTest.java rename to messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/BoundedByteSinkTest.java diff --git a/messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/RawBytesMessageCodecTest.java b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/RawBytesMessageCodecTest.java similarity index 100% rename from messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/RawBytesMessageCodecTest.java rename to messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/RawBytesMessageCodecTest.java diff --git a/messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidatorTest.java b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidatorTest.java similarity index 100% rename from messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidatorTest.java rename to messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidatorTest.java diff --git a/messaging/messaging-kafka/build.gradle b/messaging/messaging-kafka/build.gradle index 75b48b2..8f28164 100644 --- a/messaging/messaging-kafka/build.gradle +++ b/messaging/messaging-kafka/build.gradle @@ -21,12 +21,11 @@ dependencies { // `org.apache.kafka` producer API directly, and a declared library that no source names is a // resolution, a lock entry and a transitive surface bought for nothing. api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') api project(':messaging:messaging-policy') api project(':messaging:messaging-transport-spi') api project(':messaging:messaging-observability') api project(':messaging:messaging-security') - api project(':messaging:messaging-admin-api') + api project(':messaging:messaging-admin') // api: the public constructors and mappers take and return Producer, ProducerRecord and // RecordMetadata. Hiding the dependency does not hide the types — it only stops a consumer @@ -38,10 +37,6 @@ dependencies { testImplementation project(':messaging:messaging-testkit') testFixturesImplementation project(':messaging:messaging-testkit') - // Test-only. The topology validator is broker-neutral, so it lives in the admin runtime; the - // live-broker suite is here because only this module has a Kafka container to describe. - testImplementation project(':messaging:messaging-admin-runtime') - // Live-broker certification. The deterministic suite proves the platform's semantics; these // prove the client actually behaves that way against the version the matrix claims. integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaBatchConsumerRegistrar.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaBatchConsumerRegistrar.java deleted file mode 100644 index 6ba2dc3..0000000 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaBatchConsumerRegistrar.java +++ /dev/null @@ -1,146 +0,0 @@ -package dev.caskeleton.messaging.kafka; - -import dev.caskeleton.messaging.api.delivery.BatchDeliveryMetadata; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.time.Instant; -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; -import org.apache.kafka.clients.consumer.ConsumerRecord; -import org.apache.kafka.common.TopicPartition; - -/** - * Groups polled records into batches a batch handler can safely settle. - * - *

    Batches never span partitions. A poll returns records from every assigned partition at once, - * and handing that mixture to a handler makes the batch unsettleable as a unit: committing the - * highest offset per partition after a partial failure would acknowledge records the handler never - * finished. Splitting by partition first makes each batch exactly one ordering unit, which is also - * what lets an ordered destination use batching at all. - * - *

    Batch order within a partition is preserved as Kafka returned it. Re-ordering inside a batch - * would break ordering just as thoroughly as mixing partitions would, and less visibly. - */ -public final class KafkaBatchConsumerRegistrar { - - private final DestinationProfile profile; - private final int maxBatchSize; - - /** - * Creates a batch grouper for a destination. - * - * @param profile the validated destination profile - * @param maxBatchSize the largest batch handed to a handler - */ - public KafkaBatchConsumerRegistrar(DestinationProfile profile, int maxBatchSize) { - this.profile = Objects.requireNonNull(profile, "profile must not be null"); - if (maxBatchSize < 1) { - throw new IllegalArgumentException("maxBatchSize must be at least 1"); - } - this.maxBatchSize = maxBatchSize; - } - - /** - * Splits polled records into per-partition batches. - * - * @param records the records returned by one poll - * @param now the current instant - * @return the batches, each drawn from exactly one partition - */ - public List group(List> records, Instant now) { - Objects.requireNonNull(records, "records must not be null"); - Objects.requireNonNull(now, "now must not be null"); - - Map>> byPartition = new LinkedHashMap<>(); - for (ConsumerRecord record : records) { - byPartition - .computeIfAbsent( - new TopicPartition(record.topic(), record.partition()), key -> new ArrayList<>()) - .add(record); - } - - List batches = new ArrayList<>(); - for (Map.Entry>> entry : - byPartition.entrySet()) { - List> partitionRecords = entry.getValue(); - for (int start = 0; start < partitionRecords.size(); start += maxBatchSize) { - List> slice = - List.copyOf( - partitionRecords.subList( - start, Math.min(start + maxBatchSize, partitionRecords.size()))); - batches.add( - new PartitionBatch(entry.getKey(), slice, metadataFor(entry.getKey(), slice, now))); - } - } - return List.copyOf(batches); - } - - /** - * Refuses batch consumption on a destination whose ordering it cannot honour. - * - *

    Destination-wide ordering and batching are incompatible on Kafka: ordering across the whole - * destination requires a single partition, and a single-partition destination gains nothing from - * batching while losing the ability to recover a partial failure. - * - * @throws MessagingConfigurationException when the destination cannot be consumed in batches - */ - public void requireBatchable() { - if (profile.orderingScope() == OrderingScope.DESTINATION) { - throw new MessagingConfigurationException( - "BATCH_ON_DESTINATION_ORDERED", - "destination %s declares destination-wide ordering, which batching cannot preserve " - .formatted(profile.name().value()) - + "across a partial batch failure"); - } - } - - private BatchDeliveryMetadata metadataFor( - TopicPartition partition, List> slice, Instant now) { - return new BatchDeliveryMetadata( - profile.name(), - slice.size(), - Optional.of(partition.toString()), - // Kafka commits an offset per partition, so a single-partition batch is settleable as one. - true, - now); - } - - /** - * One batch drawn from a single partition. - * - * @param partition the partition the batch came from - * @param records the records, in broker order - * @param metadata the batch-wide metadata handed to the handler - */ - public record PartitionBatch( - TopicPartition partition, - List> records, - BatchDeliveryMetadata metadata) { - - public PartitionBatch { - Objects.requireNonNull(partition, "partition must not be null"); - Objects.requireNonNull(metadata, "metadata must not be null"); - records = List.copyOf(records); - if (records.isEmpty()) { - throw new IllegalArgumentException("a batch is never empty"); - } - } - - /** - * Returns the offset to commit once the whole batch succeeds. - * - *

    The last record's offset plus one, because Kafka commits the next offset to read. - * Committing the last record's own offset would replay it on every restart. - * - * @return the offset to commit - */ - public long commitOffset() { - return records.get(records.size() - 1).offset() + 1; - } - } -} diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java index 9bfec58..4b85f2f 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java @@ -333,7 +333,8 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati try { handlerPool.execute( () -> { - QueuedSettlement settlement = new QueuedSettlement(partition, record.offset(), epoch); + QueuedSettlement settlement = + new QueuedSettlement(partition, record, record.offset(), epoch); dev.caskeleton.messaging.api.MessageEnvelope< dev.caskeleton.messaging.schema.EncodedMessage> envelope; @@ -346,7 +347,7 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati if (quarantine.quarantine(record, undecodable)) { settlement.acknowledgeAfterQuarantine(); } else { - settlement.requeueAfterFailure(record, undecodable); + settlement.requeueAfterFailure(undecodable); } coordinator.release(partition); shutdown.endWork(); @@ -355,7 +356,13 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati java.util.concurrent.CompletableFuture handler = null; boolean releaseImmediately = true; try { - int attempt = retryMetadataMapper.attemptOf(envelope); + // Pause-and-seek re-reads the same record, so its retry header never moves; the + // count this runtime keeps for the offset is what tells the handler which attempt it + // is. A retry-topic hop carries its count in the header instead. + int attempt = + Math.max( + retryMetadataMapper.attemptOf(envelope), + failures.getOrDefault(new FailedOffset(partition, record.offset()), 0) + 1); TransportDelivery delivery = new TransportDelivery( envelope, @@ -381,8 +388,7 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati java.util.concurrent.CompletableFuture releaseAfterCompletion = timedOutHandler.whenComplete( (ignored, failure) -> { - settlement.requeueAfterFailure( - record, failure == null ? timedOut : failure); + settlement.requeueAfterFailure(failure == null ? timedOut : failure); coordinator.release(partition); shutdown.endWork(); }); @@ -395,12 +401,12 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati if (handler != null) { handler.cancel(true); } - settlement.requeueAfterFailure(record, interrupted); + settlement.requeueAfterFailure(interrupted); } catch (java.util.concurrent.ExecutionException | RuntimeException failure) { // A handler failure is retried, not parked. PARKED completed the offset without any // quarantine or dead-letter write, so a transient database outage silently discarded // every message it touched. - settlement.requeueAfterFailure(record, failure); + settlement.requeueAfterFailure(failure); } finally { if (releaseImmediately) { coordinator.release(partition); @@ -634,13 +640,16 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati implements dev.caskeleton.messaging.transport.TransportSettlement { private final TopicPartition partition; + private final ConsumerRecord record; private final long offset; private final long epoch; private final java.util.concurrent.atomic.AtomicReference terminal = new java.util.concurrent.atomic.AtomicReference<>(); - private QueuedSettlement(TopicPartition partition, long offset, long epoch) { + private QueuedSettlement( + TopicPartition partition, ConsumerRecord record, long offset, long epoch) { this.partition = partition; + this.record = record; this.offset = offset; this.epoch = epoch; } @@ -657,10 +666,23 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati }); } + /** + * A requeue the handler asked for, counted against the same attempt budget as a failure. + * + *

    It used to pause and seek with no count, so a handler that kept answering {@code Retry} + * re-read its message forever and its partition never advanced — the retry policy's {@code + * maxAttempts} applied to exceptions only. + */ @Override public CompletionStage requeue( Duration delay) { - return terminalOnce("requeue", () -> enqueueRequeue(delay)); + return terminalOnce( + "requeue", + () -> + retryOrQuarantine( + new IllegalStateException( + "the handler asked for another attempt and the retry budget is spent"), + delay)); } @Override @@ -692,23 +714,41 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati * there is no quarantine it keeps being retried — the offset may not advance past a message * that exists nowhere else — which is why a production destination must have one. */ - private void requeueAfterFailure(ConsumerRecord record, Throwable reason) { - if (terminal.get() != null) { + private void requeueAfterFailure(Throwable reason) { + if (!terminal.compareAndSet(null, "requeue")) { // The handler settled before it failed or before its late completion; that answer stands. return; } + retryOrQuarantine(reason, Duration.ZERO); + } + + /** + * Counts one more attempt, then either parks the record or pauses its partition for a retry. + * + *

    The caller has already claimed the terminal slot. Once the attempt budget is spent the + * record is written to quarantine and committed past; when there is no quarantine it keeps + * being retried — the offset may not advance past a message that exists nowhere else. + */ + private dev.caskeleton.messaging.api.settlement.SettlementResult retryOrQuarantine( + Throwable reason, Duration requested) { FailedOffset key = new FailedOffset(partition, offset); int attempt = failures.merge(key, 1, Integer::sum); dev.caskeleton.messaging.policy.RetryPolicy retry = spec.profile().retry(); if (attempt >= retry.maxAttempts() && quarantine.quarantine(record, reason)) { failures.remove(key); - acknowledgeAfterQuarantine(); - return; + settlements.enqueue( + new KafkaSettlementCommand( + partition, offset, KafkaSettlementCommand.Kind.PARKED, epoch)); + return dev.caskeleton.messaging.api.settlement.SettlementResult.settled(); } - if (terminal.compareAndSet(null, "requeue")) { - Duration backoff = BACKOFF.delayFor(retry, attempt); - enqueueRequeue(backoff.compareTo(MIN_RETRY_DELAY) < 0 ? MIN_RETRY_DELAY : backoff); + Duration backoff = BACKOFF.delayFor(retry, attempt); + if (backoff.compareTo(MIN_RETRY_DELAY) < 0) { + backoff = MIN_RETRY_DELAY; } + if (requested != null && backoff.compareTo(requested) < 0) { + backoff = requested; + } + return enqueueRequeue(backoff); } private dev.caskeleton.messaging.api.settlement.SettlementResult enqueueRequeue( diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaMessagingTransport.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaMessagingTransport.java index 6f1a369..e80cff4 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaMessagingTransport.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaMessagingTransport.java @@ -6,10 +6,7 @@ import dev.caskeleton.messaging.api.destination.MessagingCapabilities; import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; import dev.caskeleton.messaging.transport.MessagingTransport; import dev.caskeleton.messaging.transport.TransportConsumerRegistration; import dev.caskeleton.messaging.transport.TransportConsumerSpec; @@ -17,7 +14,6 @@ import dev.caskeleton.messaging.transport.TransportPublishRequest; import dev.caskeleton.messaging.transport.TransportPublishResult; import java.time.Duration; import java.util.Objects; -import java.util.Optional; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionStage; import java.util.concurrent.atomic.AtomicBoolean; @@ -41,7 +37,7 @@ import org.apache.kafka.clients.producer.ProducerRecord; public final class KafkaMessagingTransport implements MessagingTransport { /** - * What this adapter can prove, with deduplicated publish set to false. + * What this adapter can prove, with deduplicated publish absent. * *

    It was {@code true}, justified by producer idempotence. Producer idempotence deduplicates * sequence retries within one producer session: the producer id is reassigned on @@ -55,13 +51,10 @@ public final class KafkaMessagingTransport implements MessagingTransport { * build. False makes that request a startup failure instead, which is the whole reason the flag * exists. * - *

    The order of these flags is the record's: brokerAcknowledgement, replication evidence, - * per-message settlement, batch settlement, ordered stream, keyed ordering, replay, delayed - * delivery, broker transaction, deduplicated publish, native dead letter, topology management. + *

    Delayed delivery is absent too: Kafka has no broker-side schedule. */ private static final MessagingCapabilities CAPABILITIES = - new MessagingCapabilities( - true, true, true, true, true, true, true, false, true, false, false, true); + MessagingCapabilities.of(MessagingCapabilities.Capability.ORDERED_STREAM); private final String brokerName; private final long generation; @@ -199,14 +192,9 @@ public final class KafkaMessagingTransport implements MessagingTransport { private static TransportPublishResult rejectedLocally(String code, String message) { return new TransportPublishResult( - new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, + PublishResult.rejected( Duration.ZERO, - Optional.of(FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message)))); + FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message))); } private static CompletionStage completed(TransportPublishResult result) { diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java index 0d33a58..3436871 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java @@ -2,10 +2,7 @@ package dev.caskeleton.messaging.kafka; import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; import java.time.Duration; import java.util.Optional; import org.apache.kafka.common.errors.AuthenticationException; @@ -40,23 +37,9 @@ public final class KafkaPublishFailureClassifier { public PublishResult classify(Throwable error, Duration elapsed) { Throwable cause = unwrap(error); if (isDefinitiveRejection(cause)) { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - elapsed, - Optional.of(descriptor(cause, rejectionCategory(cause), false))); + return PublishResult.rejected(elapsed, descriptor(cause, rejectionCategory(cause), false)); } - return new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsed, - Optional.of(descriptor(cause, FailureCategory.AMBIGUOUS, false))); + return PublishResult.ambiguous(elapsed, descriptor(cause, FailureCategory.AMBIGUOUS, false)); } /** diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishMapper.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishMapper.java index 39f98ad..7c74f13 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishMapper.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishMapper.java @@ -4,8 +4,6 @@ import dev.caskeleton.messaging.api.MessageEnvelope; import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; import dev.caskeleton.messaging.api.error.MessagingConfigurationException; import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishResult; import dev.caskeleton.messaging.api.publish.RoutingOutcome; import dev.caskeleton.messaging.policy.DestinationProfile; @@ -99,13 +97,10 @@ public final class KafkaPublishMapper { ? ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK : ConfirmationLevel.BROKER_ACK; - return new PublishResult( - PublishCompletion.CONFIRMED, - PublishEvidence.confirmed(level), + return PublishResult.confirmed( + level, RoutingOutcome.NOT_APPLICABLE, Optional.of(new KafkaPosition(metadata.topic(), metadata.partition(), metadata.offset())), - 1, - elapsed, - Optional.empty()); + elapsed); } } diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaTopologyInspector.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaTopologyInspector.java deleted file mode 100644 index 195cf29..0000000 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaTopologyInspector.java +++ /dev/null @@ -1,45 +0,0 @@ -package dev.caskeleton.messaging.kafka; - -import java.util.List; -import java.util.Map; - -/** - * Read-only inspection of Kafka topology. - * - *

    Describe and validate only. Creation, deletion, and configuration changes belong to the Admin - * Plane with its own credential: an application runtime that could alter topology is one bad - * deployment away from deleting a topic it merely meant to read. - */ -public interface KafkaTopologyInspector { - - /** - * Returns the partition count for a topic. - * - * @param topic the topic - * @return the number of partitions - */ - int partitionCount(String topic); - - /** - * Returns the replication factor for a topic. - * - * @param topic the topic - * @return the replication factor - */ - short replicationFactor(String topic); - - /** - * Returns the effective topic configuration. - * - * @param topic the topic - * @return the configuration entries - */ - Map configuration(String topic); - - /** - * Returns the topics that exist. - * - * @return the topic names - */ - List topics(); -} diff --git a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java index db4e105..f5580a5 100644 --- a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java +++ b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java @@ -274,6 +274,72 @@ class KafkaConsumerRegistrarTest { .hasValue(11L); } + @Test + @DisplayName("handler가 요청한 requeue도 시도 횟수를 세어 예산을 쓰면 quarantine 후 commit된다") + void aDeliveryItsHandlerKeepsRequeueingSpendsTheBudgetAndIsQuarantined() { + // given + List quarantined = new ArrayList<>(); + KafkaConsumerRegistrar registrar = + registrar( + KafkaContractHarness.profile(), + this::requeue, + (record, reason) -> quarantined.add(record.offset()), + () -> NOW); + assign(); + addRecord(10); + registrar.pollOnce(NOW); + handlers.runAll(); + + // when + registrar.pollOnce(NOW); + + // then + assertThat(quarantined) + .as("a Retry answer that is never counted is a poison message retried forever") + .containsExactly(10L); + assertThat(registrar.committedOffset(PARTITION)).hasValue(11L); + } + + @Test + @DisplayName("다시 읽힌 delivery는 몇 번째 시도인지 metadata로 handler에 알려진다") + void aReReadDeliveryTellsTheHandlerWhichAttemptItIs() { + // given + Instant start = NOW; + RetryPolicy threeAttempts = + new RetryPolicy( + RetryMode.PAUSE_PARTITION, + 3, + Duration.ofSeconds(1), + Duration.ofSeconds(1), + 1.0, + false, + OrderingImpact.PRESERVE, + java.util.Set.of(), + java.util.Set.of()); + KafkaConsumerRegistrar registrar = + registrar( + profile(Duration.ofSeconds(1), threeAttempts, false), + this::requeue, + KafkaQuarantinePublisher.unavailable(), + () -> start); + assign(); + addRecord(10); + registrar.pollOnce(start); + handlers.runAll(); + registrar.pollOnce(start); + addRecord(10); + + // when + registrar.pollOnce(start.plusSeconds(2)); + handlers.runAll(); + + // then + assertThat(handled) + .extracting(delivery -> delivery.metadata().deliveryAttempt()) + .containsExactly(1, 2); + assertThat(handled.get(1).metadata().redelivered()).isTrue(); + } + @Test void aProductionDestinationWithoutAQuarantineIsRefused() { // given @@ -674,6 +740,13 @@ class KafkaConsumerRegistrarTest { throw new IllegalStateException("the downstream this handler writes to is down"); } + /** Asks for another attempt through the settlement, as the processor does for a Retry answer. */ + private CompletionStage requeue(TransportDelivery delivery) { + handled.add(delivery); + delivery.settlement().requeue(Duration.ZERO).toCompletableFuture().join(); + return CompletableFuture.completedFuture(null); + } + /** Acknowledges inside the handler, as a successful M1 delivery does. */ private CompletionStage settleImmediately(TransportDelivery delivery) { handled.add(delivery); diff --git a/messaging/messaging-kafka/src/testFixtures/java/dev/caskeleton/messaging/kafka/KafkaContractHarness.java b/messaging/messaging-kafka/src/testFixtures/java/dev/caskeleton/messaging/kafka/KafkaContractHarness.java index fb2e3ae..7d05669 100644 --- a/messaging/messaging-kafka/src/testFixtures/java/dev/caskeleton/messaging/kafka/KafkaContractHarness.java +++ b/messaging/messaging-kafka/src/testFixtures/java/dev/caskeleton/messaging/kafka/KafkaContractHarness.java @@ -22,7 +22,7 @@ import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.ConsumerPolicy; import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; import dev.caskeleton.messaging.policy.DeadLetterPolicy; @@ -263,7 +263,7 @@ final class KafkaContractHarness implements MessagingAdapterHarness { RetryPolicy.none(), DeadLetterPolicy.to(DEAD_LETTER), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, false, false, false); diff --git a/messaging/messaging-policy/build.gradle b/messaging/messaging-policy/build.gradle index f2ca78e..fac7b20 100644 --- a/messaging/messaging-policy/build.gradle +++ b/messaging/messaging-policy/build.gradle @@ -4,5 +4,4 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') } diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/ApiTier.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/ApiTier.java new file mode 100644 index 0000000..c48fb24 --- /dev/null +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/ApiTier.java @@ -0,0 +1,30 @@ +package dev.caskeleton.messaging.policy; + +/** + * The API tier a destination is approved for. + * + *

    "Tier" names this one thing in the platform: which API surface a destination may use. How far + * an adapter or capability has been proven is a "grade" ({@code CompatibilityMatrix.SupportGrade}), + * the same word the gRPC family uses for its capability and evidence grades. It was {@code + * CapabilityTier}, which put a third meaning on "capability" beside {@code MessagingCapabilities} + * and the gRPC capability grade (S7-#14). + * + *

    Recorded on the profile so that the tier boundary is enforced by configuration review rather + * than by whichever handler happens to be wired in. An M1 destination that silently accepted a + * manual settlement handler would move the acknowledge-after-success ordering out of the platform + * and into application code. + */ +public enum ApiTier { + + /** Typed publish and handler with platform-owned settlement. */ + M1, + + /** + * Surfaces beyond M1. None ships: the batch, manual-settlement and pause/resume contracts had no + * implementation and were removed (M-9). + */ + M2, + + /** Broker-native capabilities exposed as typed interfaces. */ + M3 +} diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/CapabilityTier.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/CapabilityTier.java deleted file mode 100644 index 3760b8a..0000000 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/CapabilityTier.java +++ /dev/null @@ -1,21 +0,0 @@ -package dev.caskeleton.messaging.policy; - -/** - * The API tier a destination is approved for. - * - *

    Recorded on the profile so that the tier boundary is enforced by configuration review rather - * than by whichever handler happens to be wired in. An M1 destination that silently accepted a - * manual settlement handler would move the acknowledge-after-success ordering out of the platform - * and into application code. - */ -public enum CapabilityTier { - - /** Typed publish and handler with platform-owned settlement. */ - M1, - - /** Batch, manual settlement, pause and resume, delayed delivery, replay request. */ - M2, - - /** Broker-native capabilities exposed as typed interfaces. */ - M3 -} diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DefaultRetryDecisionEngine.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DefaultRetryDecisionEngine.java index 943d498..5708bd5 100644 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DefaultRetryDecisionEngine.java +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DefaultRetryDecisionEngine.java @@ -46,7 +46,11 @@ public final class DefaultRetryDecisionEngine implements RetryDecisionEngine { if (policy.orderingImpact() == OrderingImpact.PRESERVE && profile.isOrdered() - && context.capabilities().orderedStream()) { + && context + .capabilities() + .supports( + dev.caskeleton.messaging.api.destination.MessagingCapabilities.Capability + .ORDERED_STREAM)) { return new RetryDecision.PauseAndRetry(delay); } if (policy.mode() == RetryMode.PAUSE_PARTITION) { @@ -61,7 +65,12 @@ public final class DefaultRetryDecisionEngine implements RetryDecisionEngine { if (policy.mode() == RetryMode.INLINE || policy.mode() == RetryMode.BLOCKING) { return new RetryDecision.RetryInline(delay); } - if (policy.mode() == RetryMode.BROKER_DELAYED && context.capabilities().delayedDelivery()) { + if (policy.mode() == RetryMode.BROKER_DELAYED + && context + .capabilities() + .supports( + dev.caskeleton.messaging.api.destination.MessagingCapabilities.Capability + .DELAYED_DELIVERY)) { return new RetryDecision.PublishToRetryDestination( policy.retryDestination().orElse(profile.name()), delay); } diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfile.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfile.java index 69b3938..11da9cb 100644 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfile.java +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfile.java @@ -47,7 +47,7 @@ public record DestinationProfile( RetryPolicy retry, DeadLetterPolicy deadLetter, PayloadPolicy payload, - CapabilityTier tier, + ApiTier tier, boolean production, boolean keyResolverConfigured, boolean topologyAutoCreate) { diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfileValidator.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfileValidator.java index 4757611..b7bebba 100644 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfileValidator.java +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfileValidator.java @@ -62,7 +62,7 @@ public final class DestinationProfileValidator { throw new IllegalArgumentException( "ordering scope KEY requires a key resolver for " + profile.name().value()); } - if (profile.tier() == CapabilityTier.M1 && profile.consumer().manualSettlement()) { + if (profile.tier() == ApiTier.M1 && profile.consumer().manualSettlement()) { throw new IllegalArgumentException( "an M1 destination cannot configure manual settlement: " + profile.name().value()); } diff --git a/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/DestinationProfileValidatorTest.java b/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/DestinationProfileValidatorTest.java index f2c16a9..7beb6be 100644 --- a/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/DestinationProfileValidatorTest.java +++ b/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/DestinationProfileValidatorTest.java @@ -188,7 +188,7 @@ final class DestinationProfileFixtures { RetryPolicy.none(), DeadLetterPolicy.to(new DestinationName("order-events-dlq")), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, false, false, false); @@ -298,7 +298,7 @@ final class DestinationProfileFixtures { base.retry(), base.deadLetter(), base.payload(), - CapabilityTier.M1, + ApiTier.M1, base.production(), base.keyResolverConfigured(), base.topologyAutoCreate()); diff --git a/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/RetryDecisionEngineTest.java b/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/RetryDecisionEngineTest.java index 53fc29c..6a8dc24 100644 --- a/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/RetryDecisionEngineTest.java +++ b/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/RetryDecisionEngineTest.java @@ -131,12 +131,11 @@ final class RetryContextFixtures { private RetryContextFixtures() {} private static final MessagingCapabilities ORDERED_KAFKA = - new MessagingCapabilities( - true, true, true, true, true, true, true, false, true, true, false, true); + MessagingCapabilities.of( + MessagingCapabilities.Capability.ORDERED_STREAM, + MessagingCapabilities.Capability.DEDUPLICATED_PUBLISH); - private static final MessagingCapabilities PLAIN_QUEUE = - new MessagingCapabilities( - true, true, true, false, false, false, false, false, false, false, true, true); + private static final MessagingCapabilities PLAIN_QUEUE = MessagingCapabilities.none(); static DeliveryMetadata delivery(int attempt) { return new DeliveryMetadata( diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/IdempotentMessageHandler.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/IdempotentMessageHandler.java deleted file mode 100644 index 478e6ff..0000000 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/IdempotentMessageHandler.java +++ /dev/null @@ -1,33 +0,0 @@ -package dev.caskeleton.messaging.reliability; - -import dev.caskeleton.messaging.api.delivery.HandleResult; -import dev.caskeleton.messaging.api.delivery.MessageDelivery; -import java.util.concurrent.CompletionStage; - -/** - * Runs a side effect at most once per consumer, whatever the broker redelivers. - * - *

    {@code consumerName} is a parameter rather than a property of the message because the same - * message legitimately reaches several independent consumers, and each of them is entitled to apply - * its own effect once. Keying the inbox on the message id alone would let whichever consumer ran - * first suppress all the others. - * - *

    A duplicate is not a failure. The handler returns success for a message it has already - * applied, so the broker settles it and stops redelivering; treating it as an error would park a - * message whose work is already done. - * - * @param the payload type - */ -public interface IdempotentMessageHandler { - - /** - * Applies an action once for a given consumer. - * - * @param consumerName the consumer identity the reservation is keyed on - * @param delivery the message being handled - * @param action the effect to apply inside the reservation's transaction - * @return a stage completing with the handler outcome - */ - CompletionStage handleOnce( - String consumerName, MessageDelivery delivery, TransactionalMessageAction action); -} diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRecord.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRecord.java deleted file mode 100644 index bfa16dc..0000000 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRecord.java +++ /dev/null @@ -1,27 +0,0 @@ -package dev.caskeleton.messaging.reliability; - -import dev.caskeleton.messaging.api.MessageId; -import java.time.Instant; -import java.util.Objects; - -/** - * One row of the consumer inbox. - * - *

    Keyed by message id and consumer id, because two independent consumers of the same - * event must each process it once — deduplicating on the message alone would let the first consumer - * suppress the second. - * - * @param messageId the logical message identity - * @param consumerId the consumer's stable identity - * @param processedAt when the side effect committed - */ -public record InboxRecord(MessageId messageId, String consumerId, Instant processedAt) { - - public InboxRecord { - Objects.requireNonNull(messageId, "messageId must not be null"); - Objects.requireNonNull(processedAt, "processedAt must not be null"); - if (consumerId == null || consumerId.isBlank()) { - throw new IllegalArgumentException("consumerId must not be blank"); - } - } -} diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxResult.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxResult.java deleted file mode 100644 index 76d0bef..0000000 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxResult.java +++ /dev/null @@ -1,45 +0,0 @@ -package dev.caskeleton.messaging.reliability; - -/** - * What the inbox decided about one delivery. - * - *

    Three outcomes, not two. Collapsing {@link #ALREADY_APPLIED} and {@link #CLAIMED_ELSEWHERE} - * into a single "duplicate" would settle a message whose effect is still only half-written by - * another instance: if that instance then rolls back, the effect is lost and the broker will never - * redeliver, because this instance already acknowledged it. - */ -public enum InboxResult { - - /** No reservation existed; the effect ran inside this transaction. */ - APPLIED(true), - - /** - * A committed reservation already exists, so the effect ran before. - * - *

    Settle the message: the work is done and redelivering it achieves nothing. - */ - ALREADY_APPLIED(true), - - /** - * Another instance holds an uncommitted reservation for this message. - * - *

    Do not settle. The other transaction may still roll back, and this delivery is the - * only remaining copy that could re-apply the effect. - */ - CLAIMED_ELSEWHERE(false); - - private final boolean safeToSettle; - - InboxResult(boolean safeToSettle) { - this.safeToSettle = safeToSettle; - } - - /** - * Reports whether the source message may be settled on this outcome. - * - * @return true when settling cannot lose the effect - */ - public boolean isSafeToSettle() { - return safeToSettle; - } -} diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/ReliableMessagePublisher.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/ReliableMessagePublisher.java deleted file mode 100644 index 9eac6c7..0000000 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/ReliableMessagePublisher.java +++ /dev/null @@ -1,27 +0,0 @@ -package dev.caskeleton.messaging.reliability; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; - -/** - * Enrols a message in the caller's database transaction instead of publishing it. - * - *

    The return type is {@code void}, and that is the contract. There is no publish outcome to - * report yet: the row is written inside the caller's transaction, so if the transaction rolls back - * the message never existed, and if it commits the relay will publish it later. Handing back a - * {@code PublishResult} here would be a lie about work that has not happened. - * - *

    This is the answer to the dual-write problem. Writing to the database and publishing to the - * broker in the same method cannot be made atomic; writing both to the database can. - */ -public interface ReliableMessagePublisher { - - /** - * Stages a message for publication when the current transaction commits. - * - * @param the payload type - * @param destination the logical destination - * @param message the envelope to publish, carrying the identity the relay will preserve - */ - void addToOutbox(MessageDestination destination, MessageEnvelope message); -} diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/TransactionalMessageAction.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/TransactionalMessageAction.java deleted file mode 100644 index 5b5b121..0000000 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/TransactionalMessageAction.java +++ /dev/null @@ -1,29 +0,0 @@ -package dev.caskeleton.messaging.reliability; - -import dev.caskeleton.messaging.api.delivery.MessageDelivery; - -/** - * The business work that runs inside the same transaction as the inbox reservation. - * - *

    Sharing one transaction is the entire mechanism. If the effect committed separately from the - * "I have handled this message" marker, a crash between the two would either replay the effect or - * suppress a message that was never handled — and which of those you get would depend on the order - * the two commits happened to be written in. - * - *

    Implementations must not settle the message, publish, or start their own transaction. The - * runtime owns the transaction boundary precisely so that the action cannot accidentally commit - * half of it. - * - * @param the payload type - */ -@FunctionalInterface -public interface TransactionalMessageAction { - - /** - * Applies the side effect for one delivery. - * - * @param delivery the message being handled - * @throws Exception when the effect fails, rolling back both it and the inbox reservation - */ - void apply(MessageDelivery delivery) throws Exception; -} diff --git a/messaging/messaging-reliability-jdbc-postgresql/build.gradle b/messaging/messaging-reliability-jdbc-postgresql/build.gradle index e6c57bb..cfa5217 100644 --- a/messaging/messaging-reliability-jdbc-postgresql/build.gradle +++ b/messaging/messaging-reliability-jdbc-postgresql/build.gradle @@ -20,7 +20,7 @@ dependencies { // The destructive-operation journal lives here because it needs exactly what the outbox needs: // one relational database every replica can see, and a migration lane that already exists. The // contract it implements belongs to the admin API. - api project(':messaging:messaging-admin-api') + api project(':messaging:messaging-admin') implementation 'org.springframework:spring-jdbc' implementation 'org.springframework:spring-tx' diff --git a/messaging/messaging-reliability-jdbc-postgresql/gradle.lockfile b/messaging/messaging-reliability-jdbc-postgresql/gradle.lockfile index 865fc31..2eeab5a 100644 --- a/messaging/messaging-reliability-jdbc-postgresql/gradle.lockfile +++ b/messaging/messaging-reliability-jdbc-postgresql/gradle.lockfile @@ -39,6 +39,9 @@ commons-logging:commons-logging:1.3.6=compileClasspath,integrationTestCompileCla info.picocli:picocli:4.7.7=checkstyle io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor +io.micrometer:micrometer-commons:1.16.7=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath +io.micrometer:micrometer-core:1.16.7=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath +io.micrometer:micrometer-observation:1.16.7=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath @@ -67,6 +70,7 @@ org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle org.codehaus.plexus:plexus-utils:3.3.0=checkstyle org.dom4j:dom4j:2.2.0=spotbugs +org.hdrhistogram:HdrHistogram:2.2.2=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath org.javassist:javassist:3.28.0-GA=checkstyle org.jetbrains:annotations:17.0.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,integrationTestAnnotationProcessor,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath @@ -79,6 +83,7 @@ org.junit.platform:junit-platform-engine:6.0.3=integrationTestRuntimeClasspath,t org.junit.platform:junit-platform-launcher:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.junit:junit-bom:6.1.0=spotbugs +org.latencyutils:LatencyUtils:2.0.3=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath org.mockito:mockito-core:5.20.0=mockitoAgent org.opentest4j:opentest4j:1.3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.ow2.asm:asm-analysis:9.10.1=spotbugs diff --git a/messaging/messaging-reliability/build.gradle b/messaging/messaging-reliability/build.gradle index 9c08eb5..d2f6210 100644 --- a/messaging/messaging-reliability/build.gradle +++ b/messaging/messaging-reliability/build.gradle @@ -10,7 +10,6 @@ dependencies { api project(':messaging:messaging-core-api') api project(':messaging:messaging-reliability-api') // The relay hands encoded rows to the publisher as EncodedMessage. - api project(':messaging:messaging-schema-api') // Test-only. The relay is also driven through the real DefaultMessagePublisher: a fake publisher // accepted the pre-encoded rows the real one re-encoded and rejected. diff --git a/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxMessageHandler.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxMessageHandler.java new file mode 100644 index 0000000..23c230f --- /dev/null +++ b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxMessageHandler.java @@ -0,0 +1,129 @@ +package dev.caskeleton.messaging.inbox; + +import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import java.time.Instant; +import java.util.Objects; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionException; +import java.util.concurrent.CompletionStage; +import java.util.function.Supplier; + +/** + * Runs a {@link MessageHandler} at most once per message and consumer, inside the inbox + * reservation's transaction. + * + *

    It is a {@code MessageHandler} itself, so the consume path has one contract. It replaced an + * {@code IdempotentMessageHandler} with a different signature that nothing called (S7-#2): the + * platform wraps a handler in this when the destination's profile declares {@code + * INBOX_TRANSACTIONAL}, and the handler does not change. + * + *

    The reservation and the handler's side effect commit together or not at all: + * + *

      + *
    • the message was reserved before (a redelivery) — the handler is not called and the result + * is {@code Success}, so the broker settles a message whose work is already done; + *
    • the handler answers {@code Success} — the transaction commits with the reservation; + *
    • the handler answers anything else, throws, or its stage fails — the transaction rolls back, + * the reservation with it, and the answer (or the failure) goes to the platform, which + * retries or dead-letters as it would without an inbox. + *
    + * + *

    The handler's stage is awaited on the calling thread, inside the transaction. A handler that + * hands its write to another thread writes outside the transaction, and the inbox cannot protect + * that write. + * + * @param the payload type + */ +public final class InboxMessageHandler implements MessageHandler { + + private final MessageHandler delegate; + private final IdempotentConsumer inbox; + private final String consumerName; + private final Supplier clock; + + /** + * Wraps a handler. + * + * @param delegate the business handler + * @param inbox the reservation-and-effect runner + * @param consumerName the consumer identity the reservation is keyed on + * @param clock supplies the reservation instant + */ + public InboxMessageHandler( + MessageHandler delegate, + IdempotentConsumer inbox, + String consumerName, + Supplier clock) { + this.delegate = Objects.requireNonNull(delegate, "delegate must not be null"); + this.inbox = Objects.requireNonNull(inbox, "inbox must not be null"); + this.clock = Objects.requireNonNull(clock, "clock must not be null"); + if (consumerName == null || consumerName.isBlank()) { + throw new IllegalArgumentException("consumerName must not be blank"); + } + this.consumerName = consumerName; + } + + @Override + public MessageDestination destination() { + return delegate.destination(); + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + Objects.requireNonNull(delivery, "delivery must not be null"); + try { + InboxOutcome outcome = + inbox.runOnce( + delivery.message().messageId(), + consumerName, + clock.get(), + () -> { + HandleResult result = answerOf(delivery); + if (!(result instanceof HandleResult.Success)) { + // Thrown, not returned: the runner rolls back only on a throw, and a reservation + // that commits without its effect suppresses the redelivery that would apply it. + throw new NotAppliedException(result); + } + return result; + }); + return CompletableFuture.completedFuture(outcome.result().orElseGet(HandleResult::success)); + } catch (NotAppliedException notApplied) { + return CompletableFuture.completedFuture(notApplied.result); + } catch (RuntimeException failed) { + return CompletableFuture.failedFuture(failed); + } + } + + private HandleResult answerOf(MessageDelivery delivery) { + CompletionStage stage = delegate.handle(delivery); + if (stage == null) { + throw new IllegalStateException("the handler returned no stage"); + } + try { + HandleResult result = stage.toCompletableFuture().join(); + if (result == null) { + throw new IllegalStateException("the handler returned no result"); + } + return result; + } catch (CompletionException failed) { + throw failed.getCause() instanceof RuntimeException cause ? cause : failed; + } + } + + /** Carries a non-success answer out of the transaction so it rolls back. */ + private static final class NotAppliedException extends RuntimeException { + + private static final long serialVersionUID = 1L; + + @SuppressWarnings("serial") + private final transient HandleResult result; + + private NotAppliedException(HandleResult result) { + super(null, null, false, false); + this.result = result; + } + } +} diff --git a/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java deleted file mode 100644 index 7195d4e..0000000 --- a/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java +++ /dev/null @@ -1,104 +0,0 @@ -package dev.caskeleton.messaging.inbox; - -import dev.caskeleton.messaging.api.delivery.HandleResult; -import dev.caskeleton.messaging.api.delivery.MessageDelivery; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.reliability.IdempotentMessageHandler; -import dev.caskeleton.messaging.reliability.InboxResult; -import dev.caskeleton.messaging.reliability.TransactionalMessageAction; -import java.time.Instant; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.function.Supplier; - -/** - * The JDBC-backed {@link IdempotentMessageHandler}. - * - *

    Reservation and effect commit together, in the runner's single transaction. Everything else - * about this class follows from that: it cannot settle the message (settlement is not - * transactional), it cannot publish (the publish would survive a rollback), and it cannot catch and - * swallow the action's exception (the rollback is how the reservation is undone). - * - *

    An already-reserved message returns success. Under at-least-once delivery a duplicate is the - * normal case, not an error; reporting failure would send a message whose work is complete to the - * dead-letter queue. - * - * @param the payload type - */ -public final class TransactionalInboxHandler implements IdempotentMessageHandler { - - private final IdempotentConsumer consumer; - private final Supplier clock; - - /** - * Creates a handler. - * - * @param consumer the reservation-and-effect runner - * @param clock supplies the current instant - */ - public TransactionalInboxHandler(IdempotentConsumer consumer, Supplier clock) { - this.consumer = Objects.requireNonNull(consumer, "consumer must not be null"); - this.clock = Objects.requireNonNull(clock, "clock must not be null"); - } - - @Override - public CompletionStage handleOnce( - String consumerName, MessageDelivery delivery, TransactionalMessageAction action) { - Objects.requireNonNull(delivery, "delivery must not be null"); - Objects.requireNonNull(action, "action must not be null"); - if (consumerName == null || consumerName.isBlank()) { - throw new IllegalArgumentException("consumerName must not be blank"); - } - - try { - InboxOutcome outcome = - consumer.runOnce( - delivery.message().messageId(), - consumerName, - clock.get(), - () -> { - try { - action.apply(delivery); - } catch (Exception failure) { - // Wrapped, not swallowed: the transaction runner has to see a throw to roll the - // reservation back along with the effect. - throw new ActionFailedException(failure); - } - return InboxResult.APPLIED; - }); - - return CompletableFuture.completedFuture( - outcome.processed() ? HandleResult.success() : duplicateIsSuccess()); - } catch (ActionFailedException failure) { - return CompletableFuture.completedFuture(retryable(failure.getCause())); - } - } - - private static HandleResult duplicateIsSuccess() { - // The effect already ran in an earlier delivery. Settling is correct; redelivering is not. - return HandleResult.success(); - } - - private static HandleResult retryable(Throwable cause) { - return new HandleResult.Retry( - new FailureDescriptor( - FailureCategory.TRANSIENT_INFRASTRUCTURE, - "INBOX_ACTION_FAILED", - true, - "the transactional action failed and its reservation was rolled back", - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName()))); - } - - /** Carries the action's checked exception out through the transaction runner. */ - private static final class ActionFailedException extends RuntimeException { - - private static final long serialVersionUID = 1L; - - private ActionFailedException(Throwable cause) { - super(cause); - } - } -} diff --git a/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java index 566f93c..ae4c3bb 100644 --- a/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java +++ b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java @@ -6,11 +6,8 @@ import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.error.MessagingException; import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; import dev.caskeleton.messaging.reliability.OutboxLease; import dev.caskeleton.messaging.reliability.OutboxRecord; import dev.caskeleton.messaging.reliability.OutboxRepository; @@ -20,7 +17,6 @@ import java.time.Duration; import java.time.Instant; import java.util.List; import java.util.Objects; -import java.util.Optional; /** * Publishes outbox rows, treating an unknown outcome as retryable rather than final. @@ -254,18 +250,12 @@ public final class OutboxRelay { } catch (RuntimeException unknown) { // An exception says nothing about whether the broker took the message, so the row is retried // under the same id like any other unknown outcome — not failed, not left IN_FLIGHT. - return new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, + return PublishResult.ambiguous( Duration.ZERO, - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "PUBLISH_OUTCOME_UNKNOWN", - "the publisher threw " + unknown.getClass().getSimpleName()))); + FailureDescriptor.of( + FailureCategory.AMBIGUOUS, + "PUBLISH_OUTCOME_UNKNOWN", + "the publisher threw " + unknown.getClass().getSimpleName())); } } diff --git a/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxMessageHandlerTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxMessageHandlerTest.java new file mode 100644 index 0000000..2b67205 --- /dev/null +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxMessageHandlerTest.java @@ -0,0 +1,179 @@ +package dev.caskeleton.messaging.inbox; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.ContentType; +import dev.caskeleton.messaging.api.MessageEnvelope; +import dev.caskeleton.messaging.api.MessageId; +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.ProducerId; +import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.delivery.DeliveryContext; +import dev.caskeleton.messaging.api.delivery.DeliveryMetadata; +import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.error.FailureCategory; +import dev.caskeleton.messaging.api.error.FailureDescriptor; +import dev.caskeleton.messaging.api.header.MessageHeaders; +import dev.caskeleton.messaging.reliability.InboxRepository; +import java.time.Instant; +import java.util.ArrayList; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Optional; +import java.util.Set; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionStage; +import java.util.function.Function; +import java.util.function.Supplier; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; + +/** The inbox is a {@code MessageHandler} wrapping the business handler in one transaction. */ +class InboxMessageHandlerTest { + + private static final Instant NOW = Instant.parse("2026-08-14T00:00:00Z"); + private static final DestinationName ORDERS = new DestinationName("order-events"); + + private final RollbackInbox inbox = new RollbackInbox(); + private final List effects = new ArrayList<>(); + + @Test + @DisplayName("같은 메시지가 다시 배달되면 handler를 부르지 않고 Success로 답한다") + void aRedeliveredMessageIsAnsweredWithSuccessWithoutCallingTheHandler() { + // given + MessageDelivery delivery = delivery(MessageId.newId()); + MessageHandler handler = wrap(ignored -> HandleResult.success()); + handler.handle(delivery).toCompletableFuture().join(); + + // when + HandleResult redelivered = handler.handle(delivery).toCompletableFuture().join(); + + // then + assertThat(redelivered).isInstanceOf(HandleResult.Success.class); + assertThat(effects).hasSize(1); + } + + @Test + @DisplayName("handler가 Retry로 답하면 예약이 롤백되어 다음 배달에서 다시 처리된다") + void aRetryAnswerRollsTheReservationBackSoTheNextDeliveryIsProcessed() { + // given + MessageDelivery delivery = delivery(MessageId.newId()); + List answers = + new ArrayList<>(List.of(new HandleResult.Retry(failure()), HandleResult.success())); + MessageHandler handler = wrap(ignored -> answers.removeFirst()); + HandleResult first = handler.handle(delivery).toCompletableFuture().join(); + + // when + HandleResult second = handler.handle(delivery).toCompletableFuture().join(); + + // then + assertThat(first).isInstanceOf(HandleResult.Retry.class); + assertThat(second).isInstanceOf(HandleResult.Success.class); + assertThat(effects).hasSize(2); + assertThat(inbox.reserved).hasSize(1); + } + + @Test + @DisplayName("handler가 예외를 던지면 예약이 롤백되고 실패한 stage로 전달된다") + void aThrowingHandlerRollsTheReservationBackAndFailsTheStage() { + // given + MessageHandler handler = + wrap( + ignored -> { + throw new IllegalStateException("database down"); + }); + + // when + CompletableFuture stage = + handler.handle(delivery(MessageId.newId())).toCompletableFuture(); + + // then + assertThat(stage).isCompletedExceptionally(); + assertThat(inbox.reserved).isEmpty(); + } + + private MessageHandler wrap(Function, HandleResult> body) { + MessageHandler business = + new MessageHandler<>() { + @Override + public MessageDestination destination() { + return new MessageDestination<>(ORDERS, new MessageType("order.created"), String.class); + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + effects.add(delivery.message().payload()); + return CompletableFuture.completedFuture(body.apply(delivery)); + } + }; + return new InboxMessageHandler<>( + business, new IdempotentConsumer(inbox, inbox), "order-projection", () -> NOW); + } + + private static MessageDelivery delivery(MessageId id) { + return new MessageDelivery<>( + new MessageEnvelope<>( + id, + new MessageType("order.created"), + new SchemaVersion(1), + NOW, + Optional.empty(), + new ProducerId("orders"), + Optional.empty(), + Optional.empty(), + ContentType.JSON, + Optional.empty(), + Optional.empty(), + Optional.empty(), + TraceContext.none(), + MessageHeaders.empty(), + "o-1"), + new DeliveryMetadata( + ORDERS, 1, false, Optional.empty(), Optional.empty(), Optional.empty(), NOW), + new DeliveryContext(NOW.plusSeconds(30), false, "order-projection")); + } + + private static FailureDescriptor failure() { + return FailureDescriptor.of( + FailureCategory.PROCESSING_TRANSIENT, "DOWNSTREAM_BUSY", "try again later"); + } + + /** An inbox whose reservations disappear when the transaction around them throws. */ + private static final class RollbackInbox + implements InboxRepository, IdempotentConsumer.TransactionRunner { + + private final Set reserved = new LinkedHashSet<>(); + + @Override + public T inTransaction(Supplier work) { + Set before = new LinkedHashSet<>(reserved); + try { + return work.get(); + } catch (RuntimeException rolledBack) { + reserved.clear(); + reserved.addAll(before); + throw rolledBack; + } + } + + @Override + public boolean reserve(MessageId messageId, String consumerId, Instant now) { + return reserved.add(messageId.value() + "|" + consumerId); + } + + @Override + public boolean isProcessed(MessageId messageId, String consumerId) { + return reserved.contains(messageId.value() + "|" + consumerId); + } + + @Override + public int purgeProcessedBefore(Instant processedBefore, int limit) { + return 0; + } + } +} diff --git a/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java index b822834..d019b69 100644 --- a/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java @@ -7,7 +7,6 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy; import dev.caskeleton.messaging.api.MessageId; import dev.caskeleton.messaging.api.error.MessagingConfigurationException; import dev.caskeleton.messaging.reliability.InboxRepository; -import dev.caskeleton.messaging.reliability.InboxResult; import java.time.Duration; import java.time.Instant; import java.util.ArrayList; @@ -139,15 +138,6 @@ class InboxOperationsTest { assertThat(redelivery.result()).isEmpty(); } - @Test - void anAlreadyAppliedMessageIsSafeToSettleButAClaimedOneIsNot() { - assertThat(InboxResult.APPLIED.isSafeToSettle()).isTrue(); - assertThat(InboxResult.ALREADY_APPLIED.isSafeToSettle()).isTrue(); - assertThat(InboxResult.CLAIMED_ELSEWHERE.isSafeToSettle()) - .as("the other transaction may still roll back, and this is the last copy") - .isFalse(); - } - /** Runs the work inline; the real runner opens a database transaction. */ private static final class DirectRunner implements IdempotentConsumer.TransactionRunner { diff --git a/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java index fdc8591..62d45f8 100644 --- a/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java @@ -24,7 +24,7 @@ import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.ConsumerPolicy; import dev.caskeleton.messaging.policy.DeadLetterPolicy; import dev.caskeleton.messaging.policy.DestinationProfile; @@ -630,7 +630,7 @@ final class RealPublisher { RetryPolicy.none(), DeadLetterPolicy.disabled(), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, false, false, false); diff --git a/messaging/messaging-runtime-core/build.gradle b/messaging/messaging-runtime-core/build.gradle index 5fb6829..06c462f 100644 --- a/messaging/messaging-runtime-core/build.gradle +++ b/messaging/messaging-runtime-core/build.gradle @@ -11,7 +11,6 @@ plugins { // with an application-supplied fake would pass a context test while running none of them. dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') api project(':messaging:messaging-policy') api project(':messaging:messaging-transport-spi') api project(':messaging:messaging-security') diff --git a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessor.java b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessor.java index 7c9dfd7..4f1169d 100644 --- a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessor.java +++ b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessor.java @@ -1,110 +1,255 @@ package dev.caskeleton.messaging.runtime; import dev.caskeleton.messaging.api.MessageEnvelope; +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.delivery.DeliveryContext; +import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.publish.PublishCompletion; import dev.caskeleton.messaging.api.settlement.SettlementResult; +import dev.caskeleton.messaging.policy.DestinationProfile; import dev.caskeleton.messaging.schema.EncodedMessage; +import dev.caskeleton.messaging.schema.MessageCodec; +import dev.caskeleton.messaging.schema.MessageCodecRegistry; import dev.caskeleton.messaging.transport.TransportDelivery; import java.time.Duration; +import java.time.InstantSource; +import java.util.HashMap; +import java.util.List; +import java.util.Map; import java.util.Objects; import java.util.Optional; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionStage; import java.util.concurrent.atomic.AtomicBoolean; -import java.util.function.Function; +import java.util.function.Supplier; /** - * Turns one handler outcome into exactly one broker settlement. + * Turns one still-encoded delivery into a typed {@link MessageHandler} call and exactly one broker + * settlement. * - *

    Nothing connected {@code HandleResult} to a settlement. Each broker adapter decided for itself - * what a retry or a dead-letter meant, so "the platform decides when and in what order the - * settlement happens" — the comment on {@code TransportSettlement} — described a decision nobody - * made in one place. + *

    This is the consume path the platform had three half-contracts for and no assembly (S7-#2): a + * typed {@code MessageHandler} nothing called, a processor that took an untyped function, and an + * inbox handler with its own signature. There is now one handler contract, and this class is the + * one place that decodes for it, calls it, and settles. * - *

    Two rules are enforced here rather than left to each adapter: + *

    Rules enforced here rather than left to each adapter: * *

      *
    • One terminal call. A delivery is acknowledged, requeued or discarded once. - * A second call is a programming error, not something to let through: acknowledging after a - * requeue tells the broker the message is done while a copy is already in flight. - *
    • Dead-letter before acknowledgement. The source is acknowledged only after - * the dead-letter publish is confirmed. Acknowledging first and publishing afterwards loses - * the message whenever the second step fails, which is exactly when it matters. + * Acknowledging after a requeue tells the broker the message is done while a copy is already + * in flight. + *
    • Acknowledge only after success. {@code Success} acknowledges; a handler + * that throws, fails its stage or returns nothing is requeued, not discarded. + *
    • Dead letter before acknowledgement. The source is acknowledged only after + * the dead-letter publish is confirmed; an unconfirmed one requeues, because a duplicate is + * the recoverable half of that choice. + *
    • Discard only where the profile allows it. {@code Reject} discards on an + * {@code AT_MOST_ONCE} destination and is dead-lettered everywhere else (design §19.2). + *
    • Undecodable is not retryable. A message whose type has no handler, or + * whose payload the codec refuses, is dead-lettered rather than retried forever. *
    + * + *

    How many times a requeued delivery comes back, and what happens when the attempts run out, is + * the transport's retry policy — {@code requeue} is a request, and the Kafka consumer counts it. */ public final class DefaultDeliveryProcessor { - private final Function, HandleResult> handler; - + private final DestinationProfile profile; + private final Map> handlers; + private final MessageCodecRegistry codecs; private final DeadLetterPublisher deadLetters; + private final String consumerId; + private final InstantSource clock; - private final Duration retryDelay; - + /** + * Creates a processor for one destination. + * + * @param profile the destination being consumed + * @param handlers the handlers for that destination, at most one per message type + * @param codecs decodes payloads by content type + * @param deadLetters publishes a delivery to the destination's dead-letter destination + * @param consumerId the stable, low-cardinality consumer identity handlers see + * @param clock the clock handler deadlines are measured on + */ public DefaultDeliveryProcessor( - Function, HandleResult> handler, + DestinationProfile profile, + List> handlers, + MessageCodecRegistry codecs, DeadLetterPublisher deadLetters, - Duration retryDelay) { - this.handler = Objects.requireNonNull(handler, "handler must not be null"); + String consumerId, + InstantSource clock) { + this.profile = Objects.requireNonNull(profile, "profile must not be null"); + this.codecs = Objects.requireNonNull(codecs, "codecs must not be null"); this.deadLetters = Objects.requireNonNull(deadLetters, "deadLetters must not be null"); - this.retryDelay = Objects.requireNonNull(retryDelay, "retryDelay must not be null"); - if (retryDelay.isNegative()) { - throw new IllegalArgumentException("a retry delay must not be negative"); + this.clock = Objects.requireNonNull(clock, "clock must not be null"); + if (consumerId == null || consumerId.isBlank()) { + throw new IllegalArgumentException("consumerId must not be blank"); } + this.consumerId = consumerId; + Map> byType = new HashMap<>(); + for (MessageHandler handler : Objects.requireNonNull(handlers, "handlers")) { + if (!handler.destination().name().equals(profile.name())) { + throw new IllegalArgumentException( + "handler for " + + handler.destination().name().value() + + " cannot consume " + + profile.name().value()); + } + if (byType.put(handler.destination().messageType(), handler) != null) { + throw new IllegalArgumentException( + "destination " + + profile.name().value() + + " has two handlers for " + + handler.destination().messageType().value() + + "; which one runs would depend on bean order"); + } + } + if (byType.isEmpty()) { + throw new IllegalArgumentException("a processor needs at least one handler"); + } + this.handlers = Map.copyOf(byType); } /** * Handles one delivery and settles it once. * - * @return a stage completing with the settlement the broker performed + * @param delivery the still-encoded delivery + * @return a stage completing once the settlement call has returned */ - public CompletionStage process(TransportDelivery delivery) { + public CompletionStage process(TransportDelivery delivery) { Objects.requireNonNull(delivery, "delivery must not be null"); OneShotSettlement settlement = new OneShotSettlement(delivery); - HandleResult result; - try { - result = handler.apply(delivery.envelope()); - } catch (RuntimeException handlerFailed) { - // A handler that threw is a retry, not a discard. Treating an exception as "this message is - // undeliverable" is how a transient bug in one consumer silently drops a day of traffic — - // and it is exactly what the Rabbit consumer did by folding handler exceptions into its - // deserialization-failure path. - return settlement.requeue(retryDelay); + MessageEnvelope envelope = delivery.envelope(); + MessageHandler handler = handlers.get(envelope.messageType()); + if (handler == null) { + return deadLetter( + delivery, + settlement, + FailureDescriptor.of( + FailureCategory.DESERIALIZATION, + "NO_HANDLER_FOR_MESSAGE_TYPE", + "no handler is registered for this message type on this destination")); } - if (result == null) { - return settlement.requeue(retryDelay); + return dispatch(handler, delivery, settlement); + } + + private CompletionStage dispatch( + MessageHandler handler, TransportDelivery delivery, OneShotSettlement settlement) { + MessageEnvelope envelope = delivery.envelope(); + T payload; + try { + MessageCodec codec = + codecs + .find(envelope.contentType()) + .orElseThrow( + () -> + new IllegalStateException( + "no codec for content type " + envelope.contentType().value())); + payload = + codec.decode( + envelope.messageType(), + envelope.schemaVersion(), + envelope.payload(), + handler.destination().payloadType()); + } catch (RuntimeException undecodable) { + return deadLetter( + delivery, + settlement, + new FailureDescriptor( + FailureCategory.DESERIALIZATION, + "PAYLOAD_UNDECODABLE", + false, + "the payload could not be decoded into the handler's type", + Optional.of(undecodable.getClass().getSimpleName()))); } + Duration timeout = profile.consumer().handlerTimeout(); + MessageDelivery typed = + new MessageDelivery<>( + envelope.withPayload(payload), + delivery.metadata(), + new DeliveryContext(clock.instant().plus(timeout), false, consumerId)); + + CompletionStage outcome; + try { + outcome = handler.handle(typed); + } catch (RuntimeException handlerFailed) { + // A handler that threw is a retry, not a discard. Treating an exception as "this message is + // undeliverable" is how a transient bug in one consumer silently drops a day of traffic. + return settlement.requeue(); + } + if (outcome == null) { + return settlement.requeue(); + } + return outcome + .handle( + (result, failure) -> + failure == null ? Optional.ofNullable(result) : Optional.empty()) + .thenCompose( + result -> + result.isEmpty() + ? settlement.requeue() + : settle(result.get(), delivery, settlement)); + } + + private CompletionStage settle( + HandleResult result, TransportDelivery delivery, OneShotSettlement settlement) { return switch (result) { case HandleResult.Success ignored -> settlement.acknowledge(); - case HandleResult.Retry ignored -> settlement.requeue(retryDelay); - case HandleResult.Reject ignored -> settlement.discard(); + case HandleResult.Retry ignored -> settlement.requeue(); + case HandleResult.Reject reject -> + profile.deliveryGuarantee() == DeliveryGuarantee.AT_MOST_ONCE + ? settlement.discard() + // Discarding on an at-least-once destination would drop a message the profile + // promised to deliver, so the rejection is kept where someone can look at it. + : deadLetter(delivery, settlement, reject.failure()); case HandleResult.DeadLetter deadLetter -> - deadLetters - .publish(delivery.envelope(), deadLetter.failure()) - .thenCompose( - completion -> - completion == PublishCompletion.CONFIRMED - // Confirmed: the message exists somewhere else, so removing it here is - // safe. - ? settlement.acknowledge() - // Not confirmed — rejected or ambiguous. Requeueing risks a duplicate; - // acknowledging loses the message outright, and a duplicate is the - // recoverable half of that choice. - : settlement.requeue(retryDelay)); + deadLetter(delivery, settlement, deadLetter.failure()); }; } + private CompletionStage deadLetter( + TransportDelivery delivery, OneShotSettlement settlement, FailureDescriptor failure) { + if (!profile.deadLetter().enabled()) { + // Nowhere to put it. Requeueing keeps the only copy; the transport's retry budget decides + // whether the partition then parks it or keeps it. + return settlement.requeue(); + } + MessageDelivery encoded = + new MessageDelivery<>( + delivery.envelope(), + delivery.metadata(), + new DeliveryContext(clock.instant(), false, consumerId)); + CompletionStage published; + try { + published = deadLetters.publish(encoded, failure); + } catch (RuntimeException publishFailed) { + return settlement.requeue(); + } + return published + .handle((completion, error) -> error == null && completion == PublishCompletion.CONFIRMED) + .thenCompose(confirmed -> confirmed ? settlement.acknowledge() : settlement.requeue()); + } + /** Publishes a delivery that cannot be handled to its dead-letter destination. */ @FunctionalInterface public interface DeadLetterPublisher { - /** Publishes and reports what the broker confirmed. */ + /** + * Publishes and reports what the broker confirmed. + * + * @param delivery the delivery, still encoded + * @param failure why it is being dead-lettered + * @return a stage completing with the publish completion + */ CompletionStage publish( - MessageEnvelope envelope, FailureDescriptor failure); + MessageDelivery delivery, FailureDescriptor failure); } /** A settlement that can be used once. */ @@ -118,20 +263,21 @@ public final class DefaultDeliveryProcessor { this.delivery = delivery; } - private CompletionStage acknowledge() { + private CompletionStage acknowledge() { return once("acknowledge", () -> delivery.settlement().acknowledge()); } - private CompletionStage requeue(Duration delay) { - return once("requeue", () -> delivery.settlement().requeue(delay)); + private CompletionStage requeue() { + // The delay is the transport's retry policy; zero asks for "as soon as that policy allows". + return once("requeue", () -> delivery.settlement().requeue(Duration.ZERO)); } - private CompletionStage discard() { + private CompletionStage discard() { return once("discard", () -> delivery.settlement().discard()); } - private CompletionStage once( - String what, java.util.function.Supplier> call) { + private CompletionStage once( + String what, Supplier> call) { if (!settled.compareAndSet(false, true)) { return CompletableFuture.failedFuture( new IllegalStateException( @@ -139,17 +285,7 @@ public final class DefaultDeliveryProcessor { + what + " would tell the broker something the first call already contradicted")); } - return call.get(); + return call.get().thenApply(ignored -> null); } } - - /** A descriptor for a handler that returned nothing, kept out of the hot path. */ - static FailureDescriptor missingResult() { - return new FailureDescriptor( - FailureCategory.PROCESSING_TRANSIENT, - "HANDLER_RETURNED_NOTHING", - true, - "the handler returned no result, which is treated as a retry rather than a discard", - Optional.empty()); - } } diff --git a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java index 46c7546..0f6d9e2 100644 --- a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java +++ b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java @@ -5,11 +5,8 @@ import dev.caskeleton.messaging.api.destination.MessageDestination; import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; import dev.caskeleton.messaging.policy.DestinationProfile; import dev.caskeleton.messaging.policy.MessagingAdmissionController; import dev.caskeleton.messaging.schema.EncodedMessage; @@ -288,7 +285,12 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM */ private void requireSupportedOptions(DestinationProfile profile, PublishOptions options) { if (options.deduplication().isPresent() - && !transport.capabilities(profile.name()).capabilities().deduplicatedPublish()) { + && !transport + .capabilities(profile.name()) + .capabilities() + .supports( + dev.caskeleton.messaging.api.destination.MessagingCapabilities.Capability + .DEDUPLICATED_PUBLISH)) { throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( "PUBLISH_DEDUPLICATION_UNSUPPORTED", "destination '" @@ -332,27 +334,13 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM private CompletionStage refused(FailureDescriptor failure, long startedAt) { return CompletableFuture.completedFuture( - new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsedSince(startedAt), - Optional.of(failure))); + PublishResult.rejected(elapsedSince(startedAt), failure)); } private PublishResult ambiguousResult(String code, String reason, long startedAt) { - return new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, + return PublishResult.ambiguous( elapsedSince(startedAt), - Optional.of( - new FailureDescriptor( - FailureCategory.AMBIGUOUS, code, true, reason, Optional.empty()))); + new FailureDescriptor(FailureCategory.AMBIGUOUS, code, true, reason, Optional.empty())); } /** diff --git a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessorTest.java b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessorTest.java index 9b2d007..8cff2f2 100644 --- a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessorTest.java +++ b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessorTest.java @@ -1,6 +1,7 @@ package dev.caskeleton.messaging.runtime; import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.catchThrowable; import dev.caskeleton.messaging.api.ContentType; import dev.caskeleton.messaging.api.MessageEnvelope; @@ -9,123 +10,318 @@ import dev.caskeleton.messaging.api.MessageType; import dev.caskeleton.messaging.api.ProducerId; import dev.caskeleton.messaging.api.SchemaVersion; import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; import dev.caskeleton.messaging.api.delivery.DeliveryMetadata; import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.header.MessageHeaders; import dev.caskeleton.messaging.api.publish.PublishCompletion; import dev.caskeleton.messaging.api.settlement.SettlementResult; +import dev.caskeleton.messaging.policy.DeadLetterPolicy; +import dev.caskeleton.messaging.policy.DestinationProfile; import dev.caskeleton.messaging.schema.EncodedMessage; +import dev.caskeleton.messaging.schema.MessageCodec; import dev.caskeleton.messaging.transport.TransportDelivery; import dev.caskeleton.messaging.transport.TransportSettlement; import java.nio.charset.StandardCharsets; import java.time.Duration; import java.time.Instant; +import java.time.InstantSource; import java.util.ArrayList; import java.util.List; import java.util.Optional; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionStage; +import java.util.function.Function; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; /** - * One handler outcome, one settlement. + * One typed handler, one outcome, one settlement. * - *

    Nothing turned a {@code HandleResult} into a settlement: each broker adapter decided for - * itself, and the Rabbit consumer folded handler exceptions into its deserialization-failure path - * and discarded them. + *

    The processor took an untyped function and nothing assembled it; the typed {@code + * MessageHandler} had no caller (S7-#2). These tests hold the settlement rules on the one contract + * that is left. */ class DefaultDeliveryProcessorTest { + private static final DestinationName ORDERS = new DestinationName("order-events"); + private static final MessageType CREATED = new MessageType("order.created"); + private static final Instant NOW = Instant.parse("2026-08-14T00:00:00Z"); + private final RecordingSettlement settlement = new RecordingSettlement(); @Test - @DisplayName("success acknowledges once") - void successAcknowledgesOnce() { - process(envelope -> new HandleResult.Success(), confirmedDeadLetters()); + @DisplayName("성공하면 handler가 decode된 payload를 받고 한 번 acknowledge된다") + void aSuccessfulHandlerReceivesTheDecodedPayloadAndIsAcknowledgedOnce() { + // given + List received = new ArrayList<>(); + MessageHandler handler = + handler( + delivery -> { + received.add(delivery.message().payload()); + return HandleResult.success(); + }); + // when + process(handler, confirmedDeadLetters(), RuntimeFixtures.profile(ORDERS)); + + // then + assertThat(received).containsExactly("{\"orderId\":\"o-1\"}"); assertThat(settlement.calls()).containsExactly("acknowledge"); } @Test - @DisplayName("retry requeues with the configured delay") - void retryRequeues() { - process(envelope -> new HandleResult.Retry(failure()), confirmedDeadLetters()); + @DisplayName("Retry 결과는 requeue된다") + void aRetryResultIsRequeued() { + // when + process( + handler(delivery -> new HandleResult.Retry(failure())), + confirmedDeadLetters(), + RuntimeFixtures.profile(ORDERS)); + // then assertThat(settlement.calls()).containsExactly("requeue"); } @Test - @DisplayName("a handler that throws is retried, not discarded") - void aThrowingHandlerIsRetried() { + @DisplayName("예외를 던진 handler는 버려지지 않고 requeue된다") + void aThrowingHandlerIsRequeuedNotDiscarded() { + // when process( - envelope -> { - throw new IllegalStateException("a bug in the projection"); - }, - confirmedDeadLetters()); + handler( + delivery -> { + throw new IllegalStateException("a bug in the projection"); + }), + confirmedDeadLetters(), + RuntimeFixtures.profile(ORDERS)); - assertThat(settlement.calls()) - .as("treating an exception as undeliverable drops a day of traffic over a transient bug") - .containsExactly("requeue"); + // then + assertThat(settlement.calls()).containsExactly("requeue"); } @Test - @DisplayName("a dead letter is published before the source is acknowledged") - void aDeadLetterIsPublishedBeforeAcknowledgement() { + @DisplayName("실패로 끝난 stage를 돌려준 handler는 requeue된다") + void aHandlerWhoseStageFailsIsRequeued() { + // given + MessageHandler handler = + new StringHandler( + delivery -> { + throw new UnsupportedOperationException("unused"); + }) { + @Override + public CompletionStage handle(MessageDelivery delivery) { + return CompletableFuture.failedFuture(new IllegalStateException("database down")); + } + }; + + // when + process(handler, confirmedDeadLetters(), RuntimeFixtures.profile(ORDERS)); + + // then + assertThat(settlement.calls()).containsExactly("requeue"); + } + + @Test + @DisplayName("dead letter는 발행이 확인된 뒤에 source를 acknowledge한다") + void aDeadLetterIsPublishedBeforeTheSourceIsAcknowledged() { + // given List order = new ArrayList<>(); DefaultDeliveryProcessor.DeadLetterPublisher publisher = - (envelope, failure) -> { + (delivery, failure) -> { order.add("dead-letter published"); return CompletableFuture.completedFuture(PublishCompletion.CONFIRMED); }; settlement.recordInto(order); - process(envelope -> new HandleResult.DeadLetter(failure()), publisher); + // when + process( + handler(delivery -> new HandleResult.DeadLetter(failure())), + publisher, + RuntimeFixtures.profile(ORDERS)); + // then assertThat(order).containsExactly("dead-letter published", "acknowledge"); } @Test - @DisplayName("an unconfirmed dead letter requeues rather than acknowledging") - void anUnconfirmedDeadLetterRequeues() { + @DisplayName("확인되지 않은 dead letter는 acknowledge하지 않고 requeue한다") + void anUnconfirmedDeadLetterIsRequeued() { + // when process( - envelope -> new HandleResult.DeadLetter(failure()), - (envelope, failure) -> CompletableFuture.completedFuture(PublishCompletion.AMBIGUOUS)); + handler(delivery -> new HandleResult.DeadLetter(failure())), + (delivery, failure) -> CompletableFuture.completedFuture(PublishCompletion.AMBIGUOUS), + RuntimeFixtures.profile(ORDERS)); - assertThat(settlement.calls()) - .as("acknowledging loses the message outright; a duplicate is the recoverable half") - .containsExactly("requeue"); + // then + assertThat(settlement.calls()).containsExactly("requeue"); } @Test - @DisplayName("a rejected message is discarded") - void aRejectedMessageIsDiscarded() { - process(envelope -> new HandleResult.Reject(failure()), confirmedDeadLetters()); + @DisplayName("at-least-once destination의 Reject는 버리지 않고 dead letter로 보낸다") + void aRejectOnAnAtLeastOnceDestinationIsDeadLetteredNotDiscarded() { + // given + List deadLettered = new ArrayList<>(); + // when + process( + handler(delivery -> new HandleResult.Reject(failure())), + (delivery, failure) -> { + deadLettered.add(failure); + return CompletableFuture.completedFuture(PublishCompletion.CONFIRMED); + }, + RuntimeFixtures.profile(ORDERS)); + + // then + assertThat(deadLettered).containsExactly(failure()); + assertThat(settlement.calls()).containsExactly("acknowledge"); + } + + @Test + @DisplayName("at-most-once destination의 Reject는 discard된다") + void aRejectOnAnAtMostOnceDestinationIsDiscarded() { + // when + process( + handler(delivery -> new HandleResult.Reject(failure())), + confirmedDeadLetters(), + atMostOnce(RuntimeFixtures.profile(ORDERS))); + + // then assertThat(settlement.calls()).containsExactly("discard"); } @Test - @DisplayName("a handler returning nothing is retried") - void aHandlerReturningNothingIsRetried() { - process(envelope -> null, confirmedDeadLetters()); + @DisplayName("handler가 없는 message type은 재시도하지 않고 dead letter로 보낸다") + void aMessageTypeWithNoHandlerIsDeadLetteredInsteadOfRetried() { + // given + List codes = new ArrayList<>(); + MessageHandler otherType = + new StringHandler( + new MessageDestination<>(ORDERS, new MessageType("order.cancelled"), String.class), + delivery -> HandleResult.success()); + // when + process( + otherType, + (delivery, failure) -> { + codes.add(failure.code()); + return CompletableFuture.completedFuture(PublishCompletion.CONFIRMED); + }, + RuntimeFixtures.profile(ORDERS)); + + // then + assertThat(codes).containsExactly("NO_HANDLER_FOR_MESSAGE_TYPE"); + assertThat(settlement.calls()).containsExactly("acknowledge"); + } + + @Test + @DisplayName("dead letter destination이 없으면 dead letter 대신 requeue해 유일한 사본을 지킨다") + void withoutADeadLetterDestinationADeadLetterIsRequeuedToKeepTheOnlyCopy() { + // when + process( + handler(delivery -> new HandleResult.DeadLetter(failure())), + confirmedDeadLetters(), + withoutDeadLetter(RuntimeFixtures.profile(ORDERS))); + + // then assertThat(settlement.calls()).containsExactly("requeue"); } + @Test + @DisplayName("한 message type에 handler가 둘이면 processor를 만들 수 없다") + void twoHandlersForOneMessageTypeAreRefused() { + // given + List> twice = + List.of(handler(delivery -> HandleResult.success()), handler(delivery -> null)); + + // when + Throwable thrown = + catchThrowable( + () -> + new DefaultDeliveryProcessor( + RuntimeFixtures.profile(ORDERS), + twice, + RegisteredMessageCodecs.of(new Utf8Codec()), + confirmedDeadLetters(), + "order-projection", + InstantSource.fixed(NOW))); + + // then + assertThat(thrown) + .isInstanceOf(IllegalArgumentException.class) + .hasMessageContaining("two handlers for order.created"); + } + private void process( - java.util.function.Function, HandleResult> handler, - DefaultDeliveryProcessor.DeadLetterPublisher deadLetters) { - new DefaultDeliveryProcessor(handler, deadLetters, Duration.ofSeconds(5)) + MessageHandler handler, + DefaultDeliveryProcessor.DeadLetterPublisher deadLetters, + DestinationProfile profile) { + new DefaultDeliveryProcessor( + profile, + List.of(handler), + RegisteredMessageCodecs.of(new Utf8Codec()), + deadLetters, + "order-projection", + InstantSource.fixed(NOW)) .process(new TransportDelivery(envelope(), metadata(), settlement)) .toCompletableFuture() .join(); } + private static MessageHandler handler( + Function, HandleResult> body) { + return new StringHandler(body); + } + private static DefaultDeliveryProcessor.DeadLetterPublisher confirmedDeadLetters() { - return (envelope, failure) -> CompletableFuture.completedFuture(PublishCompletion.CONFIRMED); + return (delivery, failure) -> CompletableFuture.completedFuture(PublishCompletion.CONFIRMED); + } + + private static DestinationProfile atMostOnce(DestinationProfile base) { + return new DestinationProfile( + base.name(), + base.broker(), + base.kind(), + base.physical(), + base.schema(), + DeliveryGuarantee.AT_MOST_ONCE, + base.orderingScope(), + base.externalSideEffectGuarantee(), + base.producer(), + base.consumer(), + base.retry(), + base.deadLetter(), + base.payload(), + base.tier(), + base.production(), + base.keyResolverConfigured(), + base.topologyAutoCreate()); + } + + private static DestinationProfile withoutDeadLetter(DestinationProfile base) { + return new DestinationProfile( + base.name(), + base.broker(), + base.kind(), + base.physical(), + base.schema(), + base.deliveryGuarantee(), + base.orderingScope(), + base.externalSideEffectGuarantee(), + base.producer(), + base.consumer(), + base.retry(), + DeadLetterPolicy.disabled(), + base.payload(), + base.tier(), + base.production(), + base.keyResolverConfigured(), + base.topologyAutoCreate()); } private static FailureDescriptor failure() { @@ -139,21 +335,15 @@ class DefaultDeliveryProcessorTest { private static DeliveryMetadata metadata() { return new DeliveryMetadata( - new dev.caskeleton.messaging.api.destination.DestinationName("order-events"), - 1, - false, - Optional.empty(), - Optional.empty(), - Optional.of("order-projection"), - Instant.parse("2026-08-14T00:00:00Z")); + ORDERS, 1, false, Optional.empty(), Optional.empty(), Optional.of("order-projection"), NOW); } private static MessageEnvelope envelope() { return new MessageEnvelope<>( MessageId.newId(), - new MessageType("order.created"), + CREATED, new SchemaVersion(1), - Instant.parse("2026-08-14T00:00:00Z"), + NOW, Optional.empty(), new ProducerId("orders"), Optional.empty(), @@ -165,7 +355,58 @@ class DefaultDeliveryProcessorTest { TraceContext.none(), MessageHeaders.empty(), new EncodedMessage( - "{}".getBytes(StandardCharsets.UTF_8), ContentType.JSON, Optional.empty())); + "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), + ContentType.JSON, + Optional.empty())); + } + + /** A handler for {@code order.created} whose body is a function. */ + private static class StringHandler implements MessageHandler { + + private final MessageDestination destination; + private final Function, HandleResult> body; + + StringHandler(Function, HandleResult> body) { + this(new MessageDestination<>(ORDERS, CREATED, String.class), body); + } + + StringHandler( + MessageDestination destination, + Function, HandleResult> body) { + this.destination = destination; + this.body = body; + } + + @Override + public MessageDestination destination() { + return destination; + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + return CompletableFuture.completedFuture(body.apply(delivery)); + } + } + + /** Decodes JSON bytes to the raw string, which is all these tests need. */ + private static final class Utf8Codec implements MessageCodec { + + @Override + public ContentType contentType() { + return ContentType.JSON; + } + + @Override + public EncodedMessage encode(MessageType type, SchemaVersion version, Object payload) { + return new EncodedMessage( + payload.toString().getBytes(StandardCharsets.UTF_8), ContentType.JSON, Optional.empty()); + } + + @Override + public T decode( + MessageType type, SchemaVersion version, byte[] encoded, Class payloadType) { + return payloadType.cast(new String(encoded, StandardCharsets.UTF_8)); + } } /** Records which terminal call the processor made, and in what order. */ diff --git a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/RuntimeFixtures.java b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/RuntimeFixtures.java index 6509b88..d341d54 100644 --- a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/RuntimeFixtures.java +++ b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/RuntimeFixtures.java @@ -3,7 +3,7 @@ package dev.caskeleton.messaging.runtime; import dev.caskeleton.messaging.api.ContentType; import dev.caskeleton.messaging.api.MessageType; import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.ConsumerPolicy; import dev.caskeleton.messaging.policy.DeadLetterPolicy; import dev.caskeleton.messaging.policy.DestinationProfile; @@ -36,7 +36,7 @@ final class RuntimeFixtures { dev.caskeleton.messaging.policy.RetryPolicy.none(), DeadLetterPolicy.to(new DestinationName("order-events-dlq")), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, false, false, false); diff --git a/messaging/messaging-schema-api/build.gradle b/messaging/messaging-schema-api/build.gradle deleted file mode 100644 index fac7b20..0000000 --- a/messaging/messaging-schema-api/build.gradle +++ /dev/null @@ -1,7 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') -} diff --git a/messaging/messaging-schema-api/gradle.lockfile b/messaging/messaging-schema-api/gradle.lockfile deleted file mode 100644 index e2c9585..0000000 --- a/messaging/messaging-schema-api/gradle.lockfile +++ /dev/null @@ -1,84 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty=compileClasspath,runtimeClasspath diff --git a/messaging/messaging-schema-avro/build.gradle b/messaging/messaging-schema-avro/build.gradle index 4044538..b1e8f84 100644 --- a/messaging/messaging-schema-avro/build.gradle +++ b/messaging/messaging-schema-avro/build.gradle @@ -4,7 +4,6 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') // api: AvroMessageCodec's constructors take a registry of org.apache.avro.Schema and // AvroCompatibilityGate.check takes and compares them. A consumer cannot build that diff --git a/messaging/messaging-schema-json/build.gradle b/messaging/messaging-schema-json/build.gradle index aebe0fe..e87b8c5 100644 --- a/messaging/messaging-schema-json/build.gradle +++ b/messaging/messaging-schema-json/build.gradle @@ -4,7 +4,6 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') implementation 'tools.jackson.core:jackson-databind' } diff --git a/messaging/messaging-schema-protobuf/build.gradle b/messaging/messaging-schema-protobuf/build.gradle index 3c431b4..58f78aa 100644 --- a/messaging/messaging-schema-protobuf/build.gradle +++ b/messaging/messaging-schema-protobuf/build.gradle @@ -4,7 +4,6 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') // api: ProtobufMessageContract is a public record over com.google.protobuf.Message and // Parser, and registering a contract is the first thing a consumer of this codec does. diff --git a/messaging/messaging-spring-boot-starter-kafka/build.gradle b/messaging/messaging-spring-boot-starter-kafka/build.gradle index 9333a84..a985d71 100644 --- a/messaging/messaging-spring-boot-starter-kafka/build.gradle +++ b/messaging/messaging-spring-boot-starter-kafka/build.gradle @@ -8,6 +8,8 @@ dependencies { // Kafka client classes. api project(':messaging:messaging-spring-boot-starter') api project(':messaging:messaging-kafka') + // KafkaConsumerConfiguration's public consumer-factory seam names kafka-clients types. + api 'org.apache.kafka:kafka-clients' implementation 'org.springframework.boot:spring-boot-autoconfigure' } diff --git a/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile b/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile index ffc4211..3e1b642 100644 --- a/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile +++ b/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile @@ -39,7 +39,6 @@ io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotatio io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.micrometer:micrometer-core:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.micrometer:micrometer-observation:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.projectreactor:reactor-core:3.8.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath @@ -87,7 +86,6 @@ org.ow2.asm:asm-tree:9.10.1=spotbugs org.ow2.asm:asm-util:9.10.1=spotbugs org.ow2.asm:asm:9.10.1=spotbugs org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reactivestreams:reactive-streams:1.0.4=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.reflections:reflections:0.10.2=checkstyle org.slf4j:slf4j-api:2.0.18=runtimeClasspath,spotbugs,spotbugsSlf4j,testRuntimeClasspath org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerCatalog.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerCatalog.java new file mode 100644 index 0000000..726df06 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerCatalog.java @@ -0,0 +1,37 @@ +package dev.caskeleton.messaging.autoconfigure; + +import java.util.LinkedHashMap; +import java.util.Map; +import java.util.Objects; + +/** + * The Kafka client settings of every {@code app.messaging.brokers} entry of type {@code kafka}. + * + * @param brokers the Kafka settings, keyed by logical broker name, in declaration order + */ +public record KafkaBrokerCatalog(Map brokers) { + + public KafkaBrokerCatalog { + brokers = + java.util.Collections.unmodifiableMap( + new LinkedHashMap<>(Objects.requireNonNull(brokers, "brokers must not be null"))); + } + + /** + * The one Kafka broker the producer and the consumers connect to. + * + * @return its logical name and settings + * @throws IllegalStateException unless exactly one Kafka entry is configured + */ + public Map.Entry single() { + if (brokers.size() != 1) { + throw new IllegalStateException( + "app.messaging.broker=kafka needs exactly one app.messaging.brokers. entry with" + + " type kafka and its bootstrap-servers; found " + + brokers.size() + + ". A producer with no broker address defaults to localhost:9092 and fails at the" + + " first publish rather than at startup"); + } + return brokers.entrySet().iterator().next(); + } +} diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerSettings.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerSettings.java new file mode 100644 index 0000000..5f38035 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerSettings.java @@ -0,0 +1,86 @@ +package dev.caskeleton.messaging.autoconfigure; + +import java.time.Duration; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Objects; +import org.springframework.boot.context.properties.bind.Bindable; +import org.springframework.boot.context.properties.bind.Binder; +import org.springframework.boot.context.properties.bind.DefaultValue; +import org.springframework.core.env.Environment; + +/** + * The Kafka client part of one {@code app.messaging.brokers.} entry of type {@code kafka}. + * + *

    Bound here, by the Kafka starter, from the same entry the broker-neutral {@link + * BrokerSettings} reads its family and posture from (S7-#9b). These fields used to sit in {@code + * BrokerSettings}, so the neutral starter bound Kafka client settings for deployments with no Kafka + * transport. + * + * @param bootstrapServers Kafka broker addresses + * @param enableIdempotence Kafka producer duplicate suppression on internal retry + * @param acks the Kafka acknowledgement level + * @param maxInFlightRequestsPerConnection the Kafka in-flight request ceiling + * @param deliveryTimeout the Kafka producer delivery deadline + * @param enableAutoCommit whether the Kafka consumer commits offsets on a timer; the platform + * consumer commits itself and refuses {@code true} + * @param consumerGroup the consumer group a destination without its own {@code consumer.group} + * joins + */ +public record KafkaBrokerSettings( + @DefaultValue List bootstrapServers, + @DefaultValue("true") boolean enableIdempotence, + @DefaultValue("all") String acks, + @DefaultValue("5") int maxInFlightRequestsPerConnection, + @DefaultValue("30s") Duration deliveryTimeout, + @DefaultValue("false") boolean enableAutoCommit, + @DefaultValue("") String consumerGroup) { + + public KafkaBrokerSettings { + bootstrapServers = List.copyOf(Objects.requireNonNull(bootstrapServers, "bootstrapServers")); + Objects.requireNonNull(acks, "acks must not be null"); + Objects.requireNonNull(deliveryTimeout, "deliveryTimeout must not be null"); + Objects.requireNonNull(consumerGroup, "consumerGroup must not be null"); + } + + /** + * Binds the Kafka settings of every broker entry whose family is {@code kafka}. + * + * @param environment the resolved environment + * @param compiled the compiled configuration, which names the Kafka entries + * @return the Kafka settings, keyed by logical broker name, in declaration order + * @throws dev.caskeleton.messaging.api.error.MessagingConfigurationException when a Kafka entry + * declares no address + */ + public static Map bindAll( + Environment environment, CompiledMessagingConfiguration compiled) { + Binder binder = Binder.get(environment); + Map kafka = new LinkedHashMap<>(); + compiled + .brokers() + .forEach( + (name, broker) -> { + if (broker.type() == null + || !BrokerSettings.KAFKA.equals(broker.type().toLowerCase(Locale.ROOT))) { + return; + } + String prefix = MessagingSettings.PREFIX + ".brokers." + name; + KafkaBrokerSettings settings = + binder + .bind(prefix, Bindable.of(KafkaBrokerSettings.class)) + .orElseGet(() -> binder.bindOrCreate(prefix, KafkaBrokerSettings.class)); + if (settings.bootstrapServers().isEmpty()) { + throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( + "MESSAGING_CONFIGURATION_INVALID", + prefix + + ".bootstrap-servers: a Kafka broker needs at least one address; a client" + + " with none defaults to localhost:9092 and fails at the first publish" + + " rather than at startup"); + } + kafka.put(name, settings); + }); + return kafka; + } +} diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerConfiguration.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerConfiguration.java new file mode 100644 index 0000000..8b38523 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerConfiguration.java @@ -0,0 +1,165 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.kafka.KafkaQuarantinePublisher; +import dev.caskeleton.messaging.kafka.KafkaSecurityConfigurer; +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import dev.caskeleton.messaging.transport.MessagingTransport; +import dev.caskeleton.messaging.transport.TransportConsumerRegistration; +import dev.caskeleton.messaging.transport.TransportConsumerSpec; +import java.util.HashMap; +import java.util.Map; +import java.util.function.Function; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.producer.Producer; +import org.apache.kafka.common.serialization.ByteArrayDeserializer; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * Opens a Kafka consumer for every destination a {@code MessageHandler} bean names (owner decision + * 10, S7-#2). + * + *

    On when present, like the rest of the capability: an application with no handler bean gets no + * consumer, no thread and no connection; {@code app.messaging.enabled=false} removes the platform + * altogether. The broker-neutral pipeline — decode, inbox, handler, dead letter, settlement — is + * {@link MessageConsumers}; this class adds only what is Kafka's, and it reaches the broker through + * {@link MessagingTransport#register}, which refused every subscription with {@code + * KAFKA_CONSUMER_NOT_CONFIGURED} until it had a consumer factory: + * + *

      + *
    • the client, from the one Kafka broker entry and the broker's consumer credential; + *
    • {@code group.id} from the destination's {@code consumer.group}; + *
    • {@code enable.auto.commit=false} always — the platform commits the contiguous watermark of + * settled deliveries, and a broker entry that asks for auto-commit is refused rather than + * overridden; + *
    • {@code auto.offset.reset=earliest}, so a new group reads what was published before it first + * joined instead of starting at the end and skipping it; + *
    • {@code isolation.level=read_committed}, so an aborted transactional write is never handed + * to a handler; + *
    • the dead-letter topic as the parking place for a record that cannot be rebuilt or whose + * attempt budget ({@code retry.max-attempts}) is spent. Without a dead-letter destination the + * record is retried and its partition stops behind it; a production destination without one + * is refused. + *
    + */ +@Configuration(proxyBeanMethods = false) +public class KafkaConsumerConfiguration { + + /** + * Builds a live subscription for a destination; the transport's consumer factory. + * + * @param compiled the compiled configuration + * @param catalog the Kafka client settings + * @param destinations the validated destination registry, for dead-letter topics + * @param producer the platform producer the dead-letter copy is written with, resolved when a + * subscription opens so that the startup validations run before a client is built + * @param declaredSecurity the security profiles the application declared as beans + * @param securityConfigurer the Kafka security configurer, present only with a credential source + * @param settings the bound settings supplying the drain deadline + * @return the factory + */ + @Bean + public KafkaConsumerFactory messagingKafkaConsumerFactory( + CompiledMessagingConfiguration compiled, + KafkaBrokerCatalog catalog, + ValidatedDestinationRegistry destinations, + ObjectProvider> producer, + ObjectProvider declaredSecurity, + ObjectProvider securityConfigurer, + MessagingSettings settings) { + return spec -> { + KafkaBrokerSettings broker = catalog.single().getValue(); + if (broker.enableAutoCommit()) { + throw new IllegalStateException( + "app.messaging.brokers." + + catalog.single().getKey() + + ".enable-auto-commit=true: the platform consumer commits an offset only after" + + " its delivery was settled, and a timer commit would move past unprocessed" + + " messages"); + } + return KafkaPolledRegistration.open( + spec, + consumerConfig(spec.profile(), compiled, catalog, declaredSecurity, securityConfigurer), + quarantine(spec.profile(), destinations, producer.getObject()), + settings.getShutdown().getDrainDeadline()); + }; + } + + /** + * Subscribes the consumed destinations when the context starts; empty without a handler bean. + * + * @param consumers the consume pipelines, present when a handler bean exists + * @param transport the selected transport + * @return the lifecycle + */ + @Bean + public KafkaConsumerLifecycle messagingKafkaConsumers( + ObjectProvider consumers, ObjectProvider transport) { + return new KafkaConsumerLifecycle( + () -> consumers.stream().flatMap(found -> found.consumers().stream()).toList(), + transport::getObject); + } + + /** The consumer's client configuration; package-visible so a test can read what it holds. */ + static Map consumerConfig( + DestinationProfile profile, + CompiledMessagingConfiguration compiled, + KafkaBrokerCatalog catalog, + ObjectProvider declaredSecurity, + ObjectProvider securityConfigurer) { + KafkaBrokerSettings broker = catalog.single().getValue(); + Map config = new HashMap<>(); + config.put( + ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, String.join(",", broker.bootstrapServers())); + config.put( + ConsumerConfig.GROUP_ID_CONFIG, + profile + .consumer() + .group() + .filter(group -> !group.isBlank()) + .orElseThrow( + () -> + new IllegalStateException( + "destination '" + + profile.name().value() + + "' is consumed but declares no consumer.group"))); + config.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false); + config.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); + config.put(ConsumerConfig.ISOLATION_LEVEL_CONFIG, "read_committed"); + config.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, profile.consumer().prefetch()); + config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class); + config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class); + config.putAll( + KafkaMessagingAutoConfiguration.clientSecurity( + compiled, declaredSecurity, securityConfigurer, BrokerSecurityProfile::consumer)); + return config; + } + + private static KafkaQuarantinePublisher quarantine( + DestinationProfile profile, + ValidatedDestinationRegistry destinations, + Producer producer) { + if (!profile.deadLetter().enabled()) { + return KafkaQuarantinePublisher.unavailable(); + } + DestinationName deadLetter = profile.deadLetter().destination().orElseThrow(); + String topic = + destinations + .require(deadLetter) + .physical() + .topic() + .orElseThrow( + () -> + new IllegalStateException( + "dead-letter destination '" + deadLetter.value() + "' has no Kafka topic")); + return new KafkaDeadLetterQuarantine(producer, topic, profile.producer().timeout()); + } + + /** The transport's consumer factory: a spec in, a live subscription out. */ + @FunctionalInterface + public interface KafkaConsumerFactory + extends Function {} +} diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerLifecycle.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerLifecycle.java new file mode 100644 index 0000000..e12e352 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerLifecycle.java @@ -0,0 +1,69 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.transport.MessagingTransport; +import dev.caskeleton.messaging.transport.TransportConsumerRegistration; +import dev.caskeleton.messaging.transport.TransportConsumerSpec; +import java.util.ArrayList; +import java.util.List; +import java.util.Objects; +import java.util.function.Supplier; +import org.springframework.context.SmartLifecycle; + +/** + * Subscribes every consumed destination through the transport when the context starts, and closes + * the subscriptions when it stops. + * + *

    An application with no handler bean has no consumed destination, and this opens nothing. Both + * collaborators are looked up at start rather than injected, so a context that fails a startup + * validation fails on that validation, not on a client this bean asked for too early. + */ +final class KafkaConsumerLifecycle implements SmartLifecycle { + + private final Supplier> consumers; + private final Supplier transport; + private final List registrations = new ArrayList<>(); + + KafkaConsumerLifecycle( + Supplier> consumers, Supplier transport) { + this.consumers = Objects.requireNonNull(consumers, "consumers must not be null"); + this.transport = Objects.requireNonNull(transport, "transport must not be null"); + } + + @Override + public synchronized void start() { + if (!registrations.isEmpty()) { + return; + } + List consumed = consumers.get(); + if (consumed.isEmpty()) { + return; + } + MessagingTransport selected = transport.get(); + for (MessageConsumers.Consumer consumer : consumed) { + registrations.add( + selected.register(new TransportConsumerSpec(consumer.profile(), consumer.sink()))); + } + } + + @Override + public synchronized void stop() { + registrations.forEach(TransportConsumerRegistration::close); + registrations.clear(); + } + + @Override + public synchronized boolean isRunning() { + return !registrations.isEmpty(); + } + + /** + * Stops before the publish drain, because a handler still running may dead-letter through the + * publisher that drain closes. + * + * @return the lifecycle phase + */ + @Override + public int getPhase() { + return Integer.MAX_VALUE - 512; + } +} diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java new file mode 100644 index 0000000..8185ef6 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java @@ -0,0 +1,65 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.api.header.ReservedHeaders; +import dev.caskeleton.messaging.kafka.KafkaQuarantinePublisher; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.Objects; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.producer.Producer; +import org.apache.kafka.clients.producer.ProducerRecord; +import org.apache.kafka.common.header.internals.RecordHeaders; + +/** + * Parks a record the consumer cannot deliver on the destination's dead-letter topic. + * + *

    Used by the Kafka consumer for the two cases the typed pipeline never sees: a record that + * cannot even be rebuilt into an envelope, and a delivery whose attempt budget is spent. The record + * is copied as it arrived — key, value and headers — with the failure category and code added, so + * the message a redrive reads is the message that failed. + * + *

    It answers {@code true} only once the broker has acknowledged the copy. The consumer commits + * past the source offset on that answer, so an unconfirmed write must say {@code false} and leave + * the source to be read again. + */ +final class KafkaDeadLetterQuarantine implements KafkaQuarantinePublisher { + + private final Producer producer; + private final String topic; + private final Duration timeout; + + KafkaDeadLetterQuarantine(Producer producer, String topic, Duration timeout) { + this.producer = Objects.requireNonNull(producer, "producer must not be null"); + this.topic = Objects.requireNonNull(topic, "topic must not be null"); + this.timeout = Objects.requireNonNull(timeout, "timeout must not be null"); + } + + @Override + public boolean quarantine(ConsumerRecord record, Throwable reason) { + RecordHeaders headers = new RecordHeaders(record.headers().toArray()); + headers.remove(ReservedHeaders.FAILURE_CATEGORY); + headers.remove(ReservedHeaders.FAILURE_CODE); + headers.add(ReservedHeaders.FAILURE_CATEGORY, bytes("POISON_MESSAGE")); + headers.add( + ReservedHeaders.FAILURE_CODE, + bytes(reason == null ? "UNKNOWN" : reason.getClass().getSimpleName())); + try { + producer + .send(new ProducerRecord<>(topic, null, record.key(), record.value(), headers)) + .get(timeout.toMillis(), TimeUnit.MILLISECONDS); + return true; + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + return false; + } catch (ExecutionException | TimeoutException | RuntimeException notConfirmed) { + return false; + } + } + + private static byte[] bytes(String value) { + return value.getBytes(StandardCharsets.UTF_8); + } +} diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java index 49a1ccb..05eadb6 100644 --- a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java @@ -30,6 +30,7 @@ import org.springframework.context.annotation.Configuration; * assembled. */ @Configuration(proxyBeanMethods = false) +@org.springframework.context.annotation.Import(KafkaConsumerConfiguration.class) public class KafkaMessagingAutoConfiguration { /** @@ -62,15 +63,31 @@ public class KafkaMessagingAutoConfiguration { kafkaProfileStartupValidation( ObjectProvider profiles, CompiledMessagingConfiguration compiled, + KafkaBrokerCatalog catalog, KafkaProfileValidator validator) { return new StartupProfileValidation<>( () -> java.util.stream.Stream.concat( - configuredKafkaProfiles(compiled).stream(), profiles.orderedStream()) + configuredKafkaProfiles(compiled, catalog).stream(), profiles.orderedStream()) .toList(), validator::validate); } + /** + * The Kafka client settings of every Kafka broker entry, bound by this starter (S7-#9b). + * + * @param environment the resolved environment + * @param compiled the compiled configuration naming the Kafka entries + * @return the catalog + */ + @Bean + @ConditionalOnMissingBean + public KafkaBrokerCatalog kafkaBrokerCatalog( + org.springframework.core.env.Environment environment, + CompiledMessagingConfiguration compiled) { + return new KafkaBrokerCatalog(KafkaBrokerSettings.bindAll(environment, compiled)); + } + /** * Returns the Kafka transaction profile validator. * @@ -138,18 +155,20 @@ public class KafkaMessagingAutoConfiguration { @ConditionalOnMissingBean public org.apache.kafka.clients.producer.Producer messagingKafkaProducer( CompiledMessagingConfiguration compiled, + KafkaBrokerCatalog catalog, ObjectProvider declaredSecurity, ObjectProvider securityConfigurer) { return new org.apache.kafka.clients.producer.KafkaProducer<>( - producerConfig(compiled, declaredSecurity, securityConfigurer)); + producerConfig(compiled, catalog, declaredSecurity, securityConfigurer)); } /** The producer's client configuration; package-visible so a test can read what it holds. */ static java.util.Map producerConfig( CompiledMessagingConfiguration compiled, + KafkaBrokerCatalog catalog, ObjectProvider declaredSecurity, ObjectProvider securityConfigurer) { - BrokerSettings broker = kafkaBroker(compiled); + KafkaBrokerSettings broker = catalog.single().getValue(); java.util.Map config = new java.util.HashMap<>(); config.put( org.apache.kafka.clients.producer.ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, @@ -185,38 +204,29 @@ public class KafkaMessagingAutoConfiguration { config.put( org.apache.kafka.clients.producer.ProducerConfig.MAX_BLOCK_MS_CONFIG, dev.caskeleton.messaging.api.publish.PublishOptions.defaults().timeout().toMillis()); - config.putAll(producerSecurity(compiled, declaredSecurity, securityConfigurer)); + config.putAll( + clientSecurity( + compiled, declaredSecurity, securityConfigurer, BrokerSecurityProfile::producer)); return config; } - /** The one Kafka broker entry the producer connects to. */ - private static BrokerSettings kafkaBroker(CompiledMessagingConfiguration compiled) { - java.util.List kafka = - compiled.brokers().values().stream() - .filter(broker -> BrokerSettings.KAFKA.equalsIgnoreCase(broker.type())) - .toList(); - if (kafka.size() != 1) { - throw new IllegalStateException( - "app.messaging.broker=kafka needs exactly one app.messaging.brokers. entry with" - + " type kafka and its bootstrap-servers; found " - + kafka.size() - + ". A producer with no broker address defaults to localhost:9092 and fails at the" - + " first publish rather than at startup"); - } - return kafka.getFirst(); - } - /** - * The producer's security properties, from the one security profile of the Kafka broker. + * A client's security properties, from the one security profile of the Kafka broker. * *

    {@code KafkaSecurityConfigurer} was a bean nothing called, so a broker that passed the TLS * and credential checks was reached over plaintext with no authentication. Every case that cannot - * be applied fails startup rather than falling back to that. + * be applied fails startup rather than falling back to that. The producer and the consumers use + * the same rule, each with its own credential. + * + * @param role picks the producer or the consumer credential of the profile */ - private static java.util.Map producerSecurity( + static java.util.Map clientSecurity( CompiledMessagingConfiguration compiled, ObjectProvider declaredSecurity, - ObjectProvider securityConfigurer) { + ObjectProvider securityConfigurer, + java.util.function.Function< + BrokerSecurityProfile, dev.caskeleton.messaging.security.BrokerCredentialProfile> + role) { java.util.Map kafkaBrokers = new java.util.TreeMap<>(); compiled.brokers().entrySet().stream() .filter(entry -> BrokerSettings.KAFKA.equalsIgnoreCase(entry.getValue().type())) @@ -239,14 +249,14 @@ public class KafkaMessagingAutoConfiguration { + name + "' enables TLS or authentication but has no app.messaging.security." + name - + " entry; the producer cannot apply either without one"); + + " entry; a Kafka client cannot apply either without one"); } }); return java.util.Map.of(); } if (profiles.size() > 1) { throw new IllegalStateException( - "the Kafka producer has one security posture but " + "a Kafka client has one security posture but " + profiles.size() + " security profiles match " + names); @@ -257,50 +267,53 @@ public class KafkaMessagingAutoConfiguration { throw new IllegalStateException( "app.messaging.security." + profile.broker() - + " is configured but no CredentialProvider bean exists; the producer cannot" + + " is configured but no CredentialProvider bean exists; a Kafka client cannot" + " resolve its credential and must not connect without it"); } return configurer.configure( - profile, profile.producer(), TLS_PROTOCOLS, java.time.Instant.now()); + profile, role.apply(profile), TLS_PROTOCOLS, java.time.Instant.now()); } /** - * The Kafka transport the publisher writes through. + * The Kafka transport the publisher writes through and the consumers subscribe through. + * + *

    It was built publish-only, so {@code register} refused every subscription with {@code + * KAFKA_CONSUMER_NOT_CONFIGURED} and the consumer runtime was never constructed (S7-#2). * * @param producer the configured producer + * @param consumers opens a live subscription for a destination * @return the transport */ @Bean @ConditionalOnMissingBean(dev.caskeleton.messaging.transport.MessagingTransport.class) public dev.caskeleton.messaging.kafka.KafkaMessagingTransport messagingKafkaTransport( - org.apache.kafka.clients.producer.Producer producer) { + org.apache.kafka.clients.producer.Producer producer, + KafkaConsumerConfiguration.KafkaConsumerFactory consumers) { // Generation 1 for the life of the producer. A SASL credential rotation does not replace the // producer: KafkaCredentialCallbackHandler reads the rotated secret at the next authentication, // so there is no second transport generation to install. - return new dev.caskeleton.messaging.kafka.KafkaMessagingTransport("kafka", 1L, producer); + return new dev.caskeleton.messaging.kafka.KafkaMessagingTransport( + "kafka", 1L, producer, consumers); } private static java.util.List - configuredKafkaProfiles(CompiledMessagingConfiguration compiled) { - return compiled.brokers().entrySet().stream() - .filter( - entry -> - entry.getValue().type() != null - && entry.getValue().type().equalsIgnoreCase(BrokerSettings.KAFKA)) + configuredKafkaProfiles(CompiledMessagingConfiguration compiled, KafkaBrokerCatalog catalog) { + return catalog.brokers().entrySet().stream() .map( entry -> { - BrokerSettings broker = entry.getValue(); + BrokerSettings broker = compiled.brokers().get(entry.getKey()); + KafkaBrokerSettings client = entry.getValue(); return new dev.caskeleton.messaging.kafka.KafkaBrokerProfile( entry.getKey(), broker.stable(), broker.production(), - broker.bootstrapServers(), - broker.enableIdempotence(), - broker.acks(), - broker.maxInFlightRequestsPerConnection(), - broker.deliveryTimeout(), - broker.enableAutoCommit(), - broker.consumerGroup(), + client.bootstrapServers(), + client.enableIdempotence(), + client.acks(), + client.maxInFlightRequestsPerConnection(), + client.deliveryTimeout(), + client.enableAutoCommit(), + client.consumerGroup(), broker.tlsEnabled(), broker.authenticationEnabled()); }) diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaPolledRegistration.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaPolledRegistration.java new file mode 100644 index 0000000..c309a13 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaPolledRegistration.java @@ -0,0 +1,169 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.kafka.KafkaConsumerRegistrar; +import dev.caskeleton.messaging.kafka.KafkaQuarantinePublisher; +import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; +import dev.caskeleton.messaging.transport.TransportConsumerRegistration; +import dev.caskeleton.messaging.transport.TransportConsumerSpec; +import java.time.Duration; +import java.time.Instant; +import java.util.Map; +import java.util.concurrent.CompletionStage; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeUnit; +import org.apache.kafka.clients.consumer.KafkaConsumer; + +/** + * One live Kafka subscription: a {@link KafkaConsumerRegistrar}, the poll thread that drives it and + * the bounded pool its handlers run on. + * + *

    The poll thread is the only thread that touches the {@code KafkaConsumer}: the registrar + * polls, commits and closes there, and handler work runs on a pool sized by the destination's + * {@code consumer.concurrency}. Offsets are committed through the contiguous watermark of settled + * deliveries, never on a timer. + * + *

    {@link #close()} stops dispatching, lets running handlers finish up to the drain deadline, + * commits what they settled and closes the client. Work still running at the deadline is abandoned + * unsettled, so the broker redelivers it rather than the platform pretending it completed. + */ +final class KafkaPolledRegistration implements TransportConsumerRegistration { + + private static final System.Logger LOG = + System.getLogger(KafkaPolledRegistration.class.getName()); + + private static final Duration POLL_TIMEOUT = Duration.ofMillis(200); + + private final String name; + private final KafkaConsumerRegistrar registrar; + private final GracefulShutdownCoordinator drain; + private final ExecutorService handlers; + private final Duration drainDeadline; + private final Thread poller; + private volatile boolean stopping; + + private KafkaPolledRegistration( + TransportConsumerSpec spec, + Map clientConfig, + KafkaQuarantinePublisher quarantine, + Duration drainDeadline) { + this.name = spec.profile().name().value(); + this.drainDeadline = drainDeadline; + String topic = + spec.profile() + .physical() + .topic() + .orElseThrow( + () -> + new IllegalStateException( + "destination '" + name + "' is consumed from Kafka but has no topic")); + this.drain = new GracefulShutdownCoordinator(drainDeadline); + this.handlers = + Executors.newFixedThreadPool( + spec.profile().consumer().concurrency(), + Thread.ofPlatform().name("messaging-kafka-handler-" + name + "-", 0).factory()); + this.registrar = + new KafkaConsumerRegistrar( + spec, + new KafkaConsumer(clientConfig), + handlers, + drain, + POLL_TIMEOUT, + quarantine); + registrar.subscribe(topic); + this.poller = Thread.ofPlatform().name("messaging-kafka-poll-" + name).unstarted(this::loop); + } + + /** + * Opens the client, subscribes and starts polling. + * + * @param spec the destination and the pipeline its deliveries go through + * @param clientConfig the consumer client configuration + * @param quarantine where a record the consumer cannot deliver is parked + * @param drainDeadline how long {@link #close()} lets running handlers finish + * @return the live registration + */ + static KafkaPolledRegistration open( + TransportConsumerSpec spec, + Map clientConfig, + KafkaQuarantinePublisher quarantine, + Duration drainDeadline) { + KafkaPolledRegistration registration = + new KafkaPolledRegistration(spec, clientConfig, quarantine, drainDeadline); + registration.poller.start(); + return registration; + } + + @Override + public CompletionStage pause(String scope) { + return registrar.pause(scope); + } + + @Override + public CompletionStage resume(String scope) { + return registrar.resume(scope); + } + + @Override + public boolean isActive() { + return registrar.isActive() && !stopping; + } + + @Override + public void close() { + Instant startedAt = Instant.now(); + drain.beginDrain(startedAt); + Instant deadline = startedAt.plus(drainDeadline); + while (!drain.isDrained(Instant.now()) && Instant.now().isBefore(deadline)) { + if (!sleep(Duration.ofMillis(50))) { + break; + } + } + stopping = true; + try { + poller.join(POLL_TIMEOUT.toMillis() * 10); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + } + handlers.shutdownNow(); + try { + handlers.awaitTermination(1, TimeUnit.SECONDS); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + } + } + + private void loop() { + try { + while (!stopping) { + try { + registrar.pollOnce(Instant.now()); + } catch (RuntimeException failure) { + // A failed poll or commit is not a reason to stop consuming for good: nothing past the + // last successful commit is lost, and the next cycle retries it. + LOG.log( + System.Logger.Level.WARNING, + "Kafka consumer for " + name + " failed a poll cycle: " + failure); + sleep(POLL_TIMEOUT); + } + if (!drain.isAcceptingWork()) { + // pollOnce no longer polls once draining begins; it only applies settlements and + // commits. Without a pause the loop would spin. + sleep(Duration.ofMillis(20)); + } + } + } finally { + registrar.close(); + } + } + + private static boolean sleep(Duration duration) { + try { + Thread.sleep(duration.toMillis()); + return true; + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + return false; + } + } +} diff --git a/messaging/messaging-spring-boot-starter/build.gradle b/messaging/messaging-spring-boot-starter/build.gradle index 88f4717..7f3f781 100644 --- a/messaging/messaging-spring-boot-starter/build.gradle +++ b/messaging/messaging-spring-boot-starter/build.gradle @@ -18,7 +18,6 @@ auxiliarySourceSets { dependencies { // Types an adopter writes against: envelopes, destinations, publish results, profiles, ports. api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') api project(':messaging:messaging-policy') api project(':messaging:messaging-transport-spi') api project(':messaging:messaging-observability') @@ -34,18 +33,11 @@ dependencies { // The relay and consumer logic, and the PostgreSQL stores behind their ports. implementation project(':messaging:messaging-reliability') implementation project(':messaging:messaging-reliability-jdbc-postgresql') - implementation project(':messaging:messaging-admin-api') - implementation project(':messaging:messaging-admin-runtime') + implementation project(':messaging:messaging-admin') implementation 'org.springframework.boot:spring-boot-autoconfigure' implementation 'org.springframework.boot:spring-boot-actuator' - // The Reactor facade lives here, not in core-api: the core contract stays CompletionStage so - // that a service which does not use Reactor never inherits it. api, because - // ReactiveMessagePublisher returns Mono — a facade whose return type an adopter cannot name - // is a facade they cannot call. - api 'io.projectreactor:reactor-core' - testImplementation 'io.projectreactor:reactor-test' testImplementation 'org.springframework.boot:spring-boot-test' testImplementation 'org.springframework:spring-test' // Provider auto-configurations live in capability artifacts. Tests that exercise provider @@ -63,6 +55,13 @@ dependencies { integrationTestImplementation project(':messaging:messaging-spring-boot-starter-kafka') integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' integrationTestImplementation 'org.testcontainers:testcontainers-kafka' + // The consume round trip's inbox lane: a real PostgreSQL behind the JDBC inbox, joined to the + // handler's own write by a real transaction manager. + integrationTestImplementation project(':messaging:messaging-reliability') + integrationTestImplementation project(':messaging:messaging-reliability-jdbc-postgresql') + integrationTestImplementation 'org.springframework:spring-jdbc' + integrationTestImplementation 'org.testcontainers:testcontainers-postgresql' + integrationTestImplementation 'org.postgresql:postgresql' } strictTestLanes { diff --git a/messaging/messaging-spring-boot-starter/gradle.lockfile b/messaging/messaging-spring-boot-starter/gradle.lockfile index d606cfc..d178e83 100644 --- a/messaging/messaging-spring-boot-starter/gradle.lockfile +++ b/messaging/messaging-spring-boot-starter/gradle.lockfile @@ -44,8 +44,6 @@ io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTe io.micrometer:micrometer-commons:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.micrometer:micrometer-core:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.projectreactor:reactor-core:3.8.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.projectreactor:reactor-test:3.8.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath @@ -69,6 +67,7 @@ org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle org.apache.xbean:xbean-reflect:3.7=checkstyle org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,testCompileClasspath org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath +org.checkerframework:checker-qual:3.55.1=integrationTestRuntimeClasspath org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle @@ -96,7 +95,7 @@ org.ow2.asm:asm-tree:9.10.1=spotbugs org.ow2.asm:asm-util:9.10.1=spotbugs org.ow2.asm:asm:9.10.1=spotbugs org.pcollections:pcollections:4.0.1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -org.reactivestreams:reactive-streams:1.0.4=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath +org.postgresql:postgresql:42.7.13=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.reflections:reflections:0.10.2=checkstyle org.rnorth.duct-tape:duct-tape:1.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.slf4j:slf4j-api:2.0.18=integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs,spotbugsSlf4j,testRuntimeClasspath @@ -112,11 +111,14 @@ org.springframework:spring-beans:7.0.9=compileClasspath,integrationTestCompileCl org.springframework:spring-context:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework:spring-core:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework:spring-expression:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-jdbc:7.0.9=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath +org.springframework:spring-jdbc:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath org.springframework:spring-test:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-tx:7.0.9=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath +org.springframework:spring-tx:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath +org.testcontainers:testcontainers-database-commons:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath +org.testcontainers:testcontainers-jdbc:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.testcontainers:testcontainers-junit-jupiter:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.testcontainers:testcontainers-kafka:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath +org.testcontainers:testcontainers-postgresql:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.testcontainers:testcontainers:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.xerial.snappy:snappy-java:1.1.10.7=integrationTestRuntimeClasspath,testRuntimeClasspath org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs diff --git a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingKafkaConsumeQualificationTest.java b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingKafkaConsumeQualificationTest.java new file mode 100644 index 0000000..7083bc6 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingKafkaConsumeQualificationTest.java @@ -0,0 +1,460 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.ContentType; +import dev.caskeleton.messaging.api.MessageEnvelope; +import dev.caskeleton.messaging.api.MessageId; +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.ProducerId; +import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; +import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; +import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.delivery.OrderingScope; +import dev.caskeleton.messaging.api.destination.DestinationKind; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.error.FailureCategory; +import dev.caskeleton.messaging.api.error.FailureDescriptor; +import dev.caskeleton.messaging.api.header.MessageHeaders; +import dev.caskeleton.messaging.api.header.ReservedHeaders; +import dev.caskeleton.messaging.api.publish.MessagePublisher; +import dev.caskeleton.messaging.api.publish.PublishCompletion; +import dev.caskeleton.messaging.api.publish.PublishOptions; +import dev.caskeleton.messaging.inbox.IdempotentConsumer; +import dev.caskeleton.messaging.inbox.JdbcInboxRepository; +import dev.caskeleton.messaging.policy.ApiTier; +import dev.caskeleton.messaging.policy.ConsumerPolicy; +import dev.caskeleton.messaging.policy.DeadLetterPolicy; +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.policy.OrderingImpact; +import dev.caskeleton.messaging.policy.PayloadPolicy; +import dev.caskeleton.messaging.policy.PhysicalDestination; +import dev.caskeleton.messaging.policy.ProducerPolicy; +import dev.caskeleton.messaging.policy.RetryMode; +import dev.caskeleton.messaging.policy.RetryPolicy; +import dev.caskeleton.messaging.policy.SchemaPolicy; +import dev.caskeleton.messaging.schema.MessageContractKey; +import dev.caskeleton.messaging.schema.SchemaCompatibility; +import java.io.IOException; +import java.io.InputStream; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.time.Instant; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Properties; +import java.util.Set; +import java.util.concurrent.BlockingQueue; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionStage; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.Function; +import javax.sql.DataSource; +import org.apache.kafka.clients.admin.Admin; +import org.apache.kafka.clients.admin.AdminClientConfig; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.clients.consumer.OffsetAndMetadata; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.common.serialization.ByteArrayDeserializer; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.condition.EnabledIf; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.jdbc.datasource.DataSourceTransactionManager; +import org.springframework.jdbc.datasource.DriverManagerDataSource; +import org.springframework.transaction.support.TransactionTemplate; +import org.testcontainers.DockerClientFactory; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.kafka.KafkaContainer; +import org.testcontainers.postgresql.PostgreSQLContainer; + +/** + * A message published through the assembled platform is consumed by a {@code MessageHandler} bean + * on a real broker (owner decision 10, S7-#2). + * + *

    The consume side had three contracts and no assembly: the typed handler had no caller, the + * Kafka consumer runtime was never constructed, and the transport the starter built refused every + * subscription with {@code KAFKA_CONSUMER_NOT_CONFIGURED}. These lanes run the whole path — + * publish, poll, decode, inbox, handler, settlement, offset commit — against Kafka, and against + * PostgreSQL for the inbox. + */ +@Testcontainers +@EnabledIf(value = "dockerAvailable", disabledReason = "Docker not available for the live broker") +class MessagingKafkaConsumeQualificationTest { + + private static final MessageType CREATED = new MessageType("order.created"); + + @Container static final KafkaContainer KAFKA = new KafkaContainer("apache/kafka:4.1.0"); + + // @Container: the Testcontainers JUnit extension owns this lifecycle. + @SuppressWarnings("resource") + @Container + static final PostgreSQLContainer POSTGRES = + new PostgreSQLContainer("postgres:16-alpine") + .withDatabaseName("messaging") + .withUsername("messaging") + .withPassword("messaging"); + + static boolean dockerAvailable() { + return DockerClientFactory.instance().isDockerAvailable(); + } + + /** The payload the handlers are typed on. */ + record OrderCreated(String orderId) {} + + @Test + @DisplayName("플랫폼으로 발행한 메시지를 handler bean이 typed payload로 받고 offset이 commit된다") + void aPublishedMessageIsConsumedByTheHandlerBeanAndItsOffsetIsCommitted() { + // given + DestinationName orders = new DestinationName("consume-round-trip"); + BlockingQueue received = new LinkedBlockingQueue<>(); + MessageHandler handler = + handler( + orders, + delivery -> { + received.add(delivery.message().payload()); + return HandleResult.success(); + }); + + runner(orders, handler, RetryPolicy.none(), ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED) + .run( + context -> { + assertThat(context).hasNotFailed(); + + // when + publish(context.getBean(MessagePublisher.class), orders, envelope(MessageId.newId())); + + // then + assertThat(received.poll(60, TimeUnit.SECONDS)) + .as("the handler bean is called with the decoded payload") + .isEqualTo(new OrderCreated("o-1")); + assertThat(committedOffset(group(orders), orders.value(), 1)) + .as("the offset is committed once the delivery was settled") + .isEqualTo(1L); + }); + } + + @Test + @DisplayName("같은 메시지가 다시 배달돼도 inbox가 handler의 효과를 한 번만 남긴다") + void aRedeliveredMessageIsAppliedOnceThroughThePostgresInbox() throws Exception { + // given + DestinationName orders = new DestinationName("consume-inbox"); + DataSource dataSource = dataSource(); + JdbcTemplate jdbc = new JdbcTemplate(dataSource); + jdbc.execute("DROP TABLE IF EXISTS messaging_inbox"); + jdbc.execute("DROP TABLE IF EXISTS order_projection"); + jdbc.execute(inboxMigration()); + jdbc.execute("CREATE TABLE order_projection (order_id TEXT NOT NULL)"); + TransactionTemplate transactions = + new TransactionTemplate(new DataSourceTransactionManager(dataSource)); + IdempotentConsumer inbox = + new IdempotentConsumer( + new JdbcInboxRepository(dataSource), + new IdempotentConsumer.TransactionRunner() { + @Override + public T inTransaction(java.util.function.Supplier work) { + return transactions.execute(status -> work.get()); + } + }); + AtomicInteger calls = new AtomicInteger(); + MessageHandler handler = + handler( + orders, + delivery -> { + calls.incrementAndGet(); + // The business write joins the transaction the inbox reservation runs in. + jdbc.update( + "INSERT INTO order_projection (order_id) VALUES (?)", + delivery.message().payload().orderId()); + return HandleResult.success(); + }); + MessageEnvelope message = envelope(MessageId.newId()); + + runner(orders, handler, RetryPolicy.none(), ExternalSideEffectGuarantee.INBOX_TRANSACTIONAL) + .withBean(IdempotentConsumer.class, () -> inbox) + .run( + context -> { + assertThat(context).hasNotFailed(); + MessagePublisher publisher = context.getBean(MessagePublisher.class); + + // when + publish(publisher, orders, message); + publish(publisher, orders, message); + + // then + assertThat(committedOffset(group(orders), orders.value(), 2)) + .as("both copies were settled") + .isEqualTo(2L); + assertThat(jdbc.queryForObject("SELECT count(*) FROM order_projection", Long.class)) + .as("the second copy found the reservation and did not apply the effect again") + .isEqualTo(1L); + assertThat(calls.get()).isEqualTo(1); + }); + } + + @Test + @DisplayName("계속 Retry로 답하는 메시지는 시도 횟수를 다 쓰면 DLQ 토픽에 남고 offset은 넘어간다") + void aMessageItsHandlerKeepsRetryingIsParkedOnTheDeadLetterTopicOnceItsAttemptsAreSpent() { + // given + DestinationName orders = new DestinationName("consume-retry"); + AtomicInteger attempts = new AtomicInteger(); + MessageHandler handler = + handler( + orders, + delivery -> { + attempts.incrementAndGet(); + return new HandleResult.Retry( + FailureDescriptor.of( + FailureCategory.PROCESSING_TRANSIENT, "DOWNSTREAM_BUSY", "try later")); + }); + RetryPolicy twoAttempts = + new RetryPolicy( + RetryMode.PAUSE_PARTITION, + 2, + Duration.ZERO, + Duration.ZERO, + 1.0, + false, + OrderingImpact.PRESERVE, + Set.of(), + Set.of()); + MessageId id = MessageId.newId(); + + runner(orders, handler, twoAttempts, ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED) + .run( + context -> { + assertThat(context).hasNotFailed(); + + // when + publish(context.getBean(MessagePublisher.class), orders, envelope(id)); + + // then + assertThat(readMessageIds(dlq(orders).value())) + .as("the message is parked where someone can redrive it, identity intact") + .contains(id.value().toString()); + assertThat(committedOffset(group(orders), orders.value(), 1)).isEqualTo(1L); + assertThat(attempts.get()).isEqualTo(2); + }); + } + + private static ApplicationContextRunner runner( + DestinationName orders, + MessageHandler handler, + RetryPolicy retry, + ExternalSideEffectGuarantee sideEffect) { + // Created up front. A consumer that subscribes before its topic exists waits for the next + // metadata refresh — five minutes by default — before it is assigned anything. + createTopics(orders.value(), dlq(orders).value()); + return new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=" + KAFKA.getBootstrapServers(), + "app.messaging.shutdown.drain-deadline=5s") + .withBean( + MessageContracts.class, + () -> + new MessageContracts( + Map.of( + new MessageContractKey(CREATED, new SchemaVersion(1)), OrderCreated.class))) + .withBean( + orders.value() + "Profile", + DestinationProfile.class, + () -> profile(orders, dlq(orders), retry, sideEffect)) + .withBean( + orders.value() + "DlqProfile", + DestinationProfile.class, + () -> + profile( + dlq(orders), + null, + RetryPolicy.none(), + ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED)) + .withBean(orders.value() + "Handler", MessageHandler.class, () -> handler); + } + + private static MessageHandler handler( + DestinationName orders, Function, HandleResult> body) { + return new MessageHandler<>() { + @Override + public MessageDestination destination() { + return new MessageDestination<>(orders, CREATED, OrderCreated.class); + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + return CompletableFuture.completedFuture(body.apply(delivery)); + } + }; + } + + private static void publish( + MessagePublisher publisher, DestinationName orders, MessageEnvelope message) { + PublishCompletion completion = + publisher + .publish( + new MessageDestination<>(orders, CREATED, OrderCreated.class), + message, + PublishOptions.defaults()) + .toCompletableFuture() + .join() + .completion(); + assertThat(completion).isEqualTo(PublishCompletion.CONFIRMED); + } + + private static DestinationName dlq(DestinationName source) { + return new DestinationName(source.value() + "-dlq"); + } + + private static String group(DestinationName destination) { + return destination.value() + "-projection"; + } + + private static DestinationProfile profile( + DestinationName name, + DestinationName deadLetter, + RetryPolicy retry, + ExternalSideEffectGuarantee sideEffect) { + ConsumerPolicy defaults = ConsumerPolicy.defaults(group(name)); + return new DestinationProfile( + name, + "kafka", + DestinationKind.EVENT_STREAM, + PhysicalDestination.kafkaTopic(name.value()), + new SchemaPolicy( + ContentType.JSON, SchemaCompatibility.BACKWARD_TRANSITIVE, Set.of(CREATED)), + deadLetter == null ? DeliveryGuarantee.AT_MOST_ONCE : DeliveryGuarantee.AT_LEAST_ONCE, + OrderingScope.NONE, + sideEffect, + ProducerPolicy.defaults(), + new ConsumerPolicy( + defaults.group(), + defaults.concurrency(), + defaults.maxInFlightPerOrderingUnit(), + defaults.prefetch(), + Duration.ofSeconds(10), + false), + retry, + deadLetter == null ? DeadLetterPolicy.disabled() : DeadLetterPolicy.to(deadLetter), + PayloadPolicy.defaults(), + ApiTier.M1, + false, + false, + true); + } + + private static MessageEnvelope envelope(MessageId id) { + return new MessageEnvelope<>( + id, + CREATED, + new SchemaVersion(1), + Instant.parse("2026-09-29T00:00:00Z"), + Optional.empty(), + new ProducerId("orders"), + Optional.empty(), + Optional.empty(), + ContentType.JSON, + Optional.empty(), + Optional.empty(), + Optional.empty(), + TraceContext.none(), + MessageHeaders.empty(), + new OrderCreated("o-1")); + } + + private static void createTopics(String... topics) { + try (Admin admin = + Admin.create( + Map.of(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA.getBootstrapServers()))) { + admin + .createTopics( + java.util.Arrays.stream(topics) + .map(topic -> new org.apache.kafka.clients.admin.NewTopic(topic, 1, (short) 1)) + .toList()) + .all() + .get(30, TimeUnit.SECONDS); + } catch (Exception failed) { + throw new IllegalStateException("could not create " + List.of(topics), failed); + } + } + + /** Waits, bounded, until the group has committed at least {@code expected} on partition 0. */ + private static long committedOffset(String group, String topic, long expected) throws Exception { + try (Admin admin = + Admin.create( + Map.of(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA.getBootstrapServers()))) { + Instant deadline = Instant.now().plusSeconds(60); + long seen = -1; + while (Instant.now().isBefore(deadline)) { + Map offsets = + admin.listConsumerGroupOffsets(group).partitionsToOffsetAndMetadata().get(); + OffsetAndMetadata committed = offsets.get(new TopicPartition(topic, 0)); + seen = committed == null ? -1 : committed.offset(); + if (seen >= expected) { + return seen; + } + TimeUnit.MILLISECONDS.sleep(200); + } + return seen; + } + } + + /** Reads the message ids that reached a topic, with a bound rather than an open-ended loop. */ + private static List readMessageIds(String topic) { + Properties config = new Properties(); + config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA.getBootstrapServers()); + config.put(ConsumerConfig.GROUP_ID_CONFIG, topic + "-reader"); + config.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); + config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class); + config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class); + try (KafkaConsumer consumer = new KafkaConsumer<>(config)) { + consumer.subscribe(List.of(topic)); + Instant deadline = Instant.now().plusSeconds(60); + while (Instant.now().isBefore(deadline)) { + List ids = + java.util.stream.StreamSupport.stream( + consumer.poll(Duration.ofSeconds(1)).records(topic).spliterator(), false) + .map(MessagingKafkaConsumeQualificationTest::messageId) + .toList(); + if (!ids.isEmpty()) { + return ids; + } + } + return List.of(); + } + } + + private static String messageId(ConsumerRecord record) { + return new String( + record.headers().lastHeader(ReservedHeaders.MESSAGE_ID).value(), StandardCharsets.UTF_8); + } + + private static DataSource dataSource() { + return new DriverManagerDataSource( + POSTGRES.getJdbcUrl(), POSTGRES.getUsername(), POSTGRES.getPassword()); + } + + private static String inboxMigration() throws IOException { + try (InputStream sql = + MessagingKafkaConsumeQualificationTest.class.getResourceAsStream( + "/db/migration/messaging/V1101__messaging_inbox.sql")) { + assertThat(sql).as("the inbox migration ships with the JDBC inbox").isNotNull(); + return new String(sql.readAllBytes(), StandardCharsets.UTF_8); + } + } +} diff --git a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java index 65c4340..42c1d15 100644 --- a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java +++ b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java @@ -193,7 +193,7 @@ class MessagingLiveRoundTripQualificationTest { ? dev.caskeleton.messaging.policy.DeadLetterPolicy.disabled() : dev.caskeleton.messaging.policy.DeadLetterPolicy.to(deadLetter), dev.caskeleton.messaging.policy.PayloadPolicy.defaults(), - dev.caskeleton.messaging.policy.CapabilityTier.M1, + dev.caskeleton.messaging.policy.ApiTier.M1, false, false, false); diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java index fb3a111..1b491be 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java @@ -1,15 +1,17 @@ package dev.caskeleton.messaging.autoconfigure; -import java.time.Duration; -import java.util.List; import org.springframework.boot.context.properties.bind.DefaultValue; /** - * One entry of {@code app.messaging.brokers}. + * The broker-neutral part of one {@code app.messaging.brokers} entry. * - *

    Kafka is the one broker family this platform has a transport for, so the record carries only - * Kafka fields. RabbitMQ fields used to live here too and bound with no transport behind them: a - * deployment could configure a RabbitMQ broker that nothing would ever connect to. + *

    This starter is broker-neutral, and its settings are too. The client settings of a family — + * Kafka's bootstrap servers, acks, idempotence — are bound by that family's starter from the same + * entry ({@code KafkaBrokerSettings} in {@code messaging-spring-boot-starter-kafka}). They used to + * be fields here, so the neutral starter carried the union of every broker's client knobs and a + * deployment that shipped no Kafka transport still bound Kafka settings. {@link + * MessagingConfigurationKeyValidator} accepts a family's keys only when that family's settings + * record is on the classpath ({@link MessagingProviderSelection#PROVIDER_BROKER_SETTINGS}). * *

    {@code tlsEnabled} and {@code production} live here rather than beside the credentials, so a * broker is described once. They were previously implied by whichever object happened to be @@ -21,27 +23,13 @@ import org.springframework.boot.context.properties.bind.DefaultValue; * @param production whether this profile is used in production * @param tlsEnabled whether transport encryption is on * @param authenticationEnabled whether broker authentication is on - * @param bootstrapServers Kafka broker addresses - * @param enableIdempotence Kafka producer duplicate suppression on internal retry - * @param acks the Kafka acknowledgement level - * @param maxInFlightRequestsPerConnection the Kafka in-flight request ceiling - * @param deliveryTimeout the Kafka producer delivery deadline - * @param enableAutoCommit whether the Kafka consumer commits offsets on a timer - * @param consumerGroup the Kafka consumer group */ public record BrokerSettings( String type, @DefaultValue("false") boolean stable, @DefaultValue("false") boolean production, @DefaultValue("false") boolean tlsEnabled, - @DefaultValue("false") boolean authenticationEnabled, - @DefaultValue List bootstrapServers, - @DefaultValue("true") boolean enableIdempotence, - @DefaultValue("all") String acks, - @DefaultValue("5") int maxInFlightRequestsPerConnection, - @DefaultValue("30s") Duration deliveryTimeout, - @DefaultValue("false") boolean enableAutoCommit, - @DefaultValue("") String consumerGroup) { + @DefaultValue("false") boolean authenticationEnabled) { /** The broker family a Kafka entry declares. */ public static final String KAFKA = "kafka"; diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBatchMessagePublisher.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBatchMessagePublisher.java deleted file mode 100644 index c45124d..0000000 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBatchMessagePublisher.java +++ /dev/null @@ -1,262 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.error.MessageTooLargeException; -import dev.caskeleton.messaging.api.publish.BatchMessagePublisher; -import dev.caskeleton.messaging.api.publish.BatchPublishItemResult; -import dev.caskeleton.messaging.api.publish.BatchPublishOptions; -import dev.caskeleton.messaging.api.publish.BatchPublishResult; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishRequest; -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Objects; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.function.Supplier; - -/** - * Fans a batch out over the single-message publisher and keeps every entry's outcome. - * - *

    Entries are submitted concurrently and awaited together, because a batch's value is the - * overlap: submitting serially would make a 500-entry batch take 500 round trips, which is slower - * than the loop the caller was trying to avoid. - * - *

    A failed entry never fails the batch. Each entry's exception is converted into that entry's - * result, so an oversized message at index 7 does not hide the outcomes of the other 499 — the - * caller needs to know exactly which entries to resubmit, and a thrown exception tells it nothing. - * - *

    {@code stopOnFirstRejection} short-circuits the submission, not the awaiting. Entries - * already in flight when the first rejection arrives cannot be recalled, so they are still awaited - * and reported; pretending otherwise would leave the caller unsure whether they landed. - */ -public final class DefaultBatchMessagePublisher implements BatchMessagePublisher, AutoCloseable { - - private final MessagePublisher publisher; - private final Supplier clock; - private final java.util.concurrent.ScheduledExecutorService deadlines; - - /** - * Creates a batch publisher over the single-message publisher. - * - * @param publisher the platform publisher - * @param clock supplies the current instant - */ - public DefaultBatchMessagePublisher(MessagePublisher publisher, Supplier clock) { - this( - publisher, - clock, - java.util.concurrent.Executors.newSingleThreadScheduledExecutor( - runnable -> { - Thread thread = new Thread(runnable, "messaging-batch-deadline"); - thread.setDaemon(true); - return thread; - })); - } - - DefaultBatchMessagePublisher( - MessagePublisher publisher, - Supplier clock, - java.util.concurrent.ScheduledExecutorService deadlines) { - this.publisher = Objects.requireNonNull(publisher, "publisher must not be null"); - this.clock = Objects.requireNonNull(clock, "clock must not be null"); - this.deadlines = Objects.requireNonNull(deadlines, "deadlines must not be null"); - } - - @Override - public CompletionStage publish( - List> requests, BatchPublishOptions options) { - Objects.requireNonNull(requests, "requests must not be null"); - Objects.requireNonNull(options, "options must not be null"); - - if (requests.size() > options.maxBatchSize()) { - throw new MessageTooLargeException( - "BATCH_COUNT_EXCEEDED", - "batch of %d entries exceeds the %d entry limit" - .formatted(requests.size(), options.maxBatchSize())); - } - - Instant startedAt = clock.get(); - List> pending = new ArrayList<>(requests.size()); - // Index and message id per slot, so an entry still outstanding at the deadline can still be - // reported against the input position it came from. - List placeholders = new ArrayList<>(requests.size()); - - for (int index = 0; index < requests.size(); index++) { - placeholders.add(outstanding(index, requests.get(index).message().messageId())); - if (options.stopOnFirstRejection() && anyRejected(pending)) { - // Every remaining index still gets a result. Breaking out shortened the result list, so a - // caller could not tell "not submitted" from "result lost" — and those need opposite - // actions. - pending.add( - CompletableFuture.completedFuture( - notSubmitted(index, requests.get(index).message().messageId(), startedAt))); - continue; - } - CompletableFuture submitted = submit(requests.get(index), index); - pending.add(submitted); - if (options.stopOnFirstRejection()) { - // Sequential under stop-on-first-rejection: waiting for this entry's outcome is the only - // way the flag can mean anything. The previous loop only saw a rejection that had already - // resolved, which for an asynchronous broker is never — so every remaining entry was - // submitted anyway and the option did nothing. - awaitQuietly(submitted, remaining(startedAt, options.timeout())); - } - } - - CompletableFuture completed = new CompletableFuture<>(); - CompletableFuture all = - CompletableFuture.allOf(pending.toArray(CompletableFuture[]::new)); - // The returned stage is deliberately dropped: `completed` is what the caller holds, and this - // callback's only job is to fill it in. Nothing downstream observes the intermediate stage. - CompletableFuture ignoredCompletion = - all.whenComplete( - (ignored, failure) -> - completed.complete( - new BatchPublishResult( - resolveAll(pending, placeholders), - Duration.between(startedAt, clock.get())))); - assert ignoredCompletion != null; - - // The whole-batch deadline the options document. It was never read, so a publisher that never - // completed left the caller's stage pending forever — a timeout that exists only in the - // javadoc. - // Same for the scheduled future: cancelling it early would only save a no-op, and the task - // checks `isDone` before completing anything. - java.util.concurrent.ScheduledFuture ignoredDeadline = - deadlines.schedule( - () -> { - if (!completed.isDone()) { - completed.complete( - new BatchPublishResult( - resolveAll(pending, placeholders), - Duration.between(startedAt, clock.get()))); - } - }, - options.timeout().toMillis(), - java.util.concurrent.TimeUnit.MILLISECONDS); - assert ignoredDeadline != null; - return completed; - } - - @Override - public void close() { - deadlines.shutdownNow(); - } - - private boolean anyRejected(List> pending) { - return pending.stream() - .anyMatch( - future -> - future.isDone() - && !future.isCompletedExceptionally() - && future.join().result().completion() == PublishCompletion.REJECTED); - } - - private void awaitQuietly(CompletableFuture future, Duration budget) { - if (budget.isNegative() || budget.isZero()) { - return; - } - try { - future.get(budget.toMillis(), java.util.concurrent.TimeUnit.MILLISECONDS); - } catch (InterruptedException interrupted) { - Thread.currentThread().interrupt(); - } catch (java.util.concurrent.ExecutionException - | java.util.concurrent.TimeoutException other) { - // Both are outcomes the result already carries; nothing to add here. - } - } - - private Duration remaining(Instant startedAt, Duration timeout) { - return timeout.minus(Duration.between(startedAt, clock.get())); - } - - /** - * An entry the batch never submitted because an earlier one was rejected. - * - *

    Distinct from a failure: nothing was sent, so the caller may submit it unchanged. - */ - private BatchPublishItemResult notSubmitted( - int index, dev.caskeleton.messaging.api.MessageId messageId, Instant startedAt) { - return new BatchPublishItemResult( - index, - messageId, - new dev.caskeleton.messaging.api.publish.PublishResult( - PublishCompletion.REJECTED, - dev.caskeleton.messaging.api.publish.PublishEvidence.notTransmitted(), - dev.caskeleton.messaging.api.publish.RoutingOutcome.UNKNOWN, - java.util.Optional.empty(), - 1, - Duration.between(startedAt, clock.get()), - java.util.Optional.of( - dev.caskeleton.messaging.api.error.FailureDescriptor.of( - dev.caskeleton.messaging.api.error.FailureCategory.CONFIGURATION, - "NOT_SUBMITTED_AFTER_REJECTION", - "an earlier entry was rejected and this batch stops on the first rejection")))); - } - - private List resolveAll( - List> pending, - List placeholders) { - List resolved = new ArrayList<>(pending.size()); - for (int index = 0; index < pending.size(); index++) { - CompletableFuture future = pending.get(index); - resolved.add( - future.isDone() && !future.isCompletedExceptionally() - ? future.join() - : placeholders.get(index)); - } - return List.copyOf(resolved); - } - - /** An entry still outstanding when the deadline arrived is ambiguous, never a success. */ - private BatchPublishItemResult outstanding( - int index, dev.caskeleton.messaging.api.MessageId messageId) { - return new BatchPublishItemResult( - index, - messageId, - new dev.caskeleton.messaging.api.publish.PublishResult( - PublishCompletion.AMBIGUOUS, - dev.caskeleton.messaging.api.publish.PublishEvidence.ambiguous(), - dev.caskeleton.messaging.api.publish.RoutingOutcome.UNKNOWN, - java.util.Optional.empty(), - 1, - Duration.ZERO, - java.util.Optional.of( - dev.caskeleton.messaging.api.error.FailureDescriptor.of( - dev.caskeleton.messaging.api.error.FailureCategory.AMBIGUOUS, - "BATCH_DEADLINE_EXCEEDED", - "the batch deadline elapsed before this entry produced evidence")))); - } - - private CompletableFuture submit( - PublishRequest request, int index) { - @SuppressWarnings("unchecked") - PublishRequest typed = (PublishRequest) request; - MessageDestination destination = typed.destination(); - MessageEnvelope message = typed.message(); - - try { - return publisher - .publish(destination, message, typed.options()) - .toCompletableFuture() - .handle( - (result, failure) -> - new BatchPublishItemResult( - index, - message.messageId(), - failure == null ? result : PublishResults.from(failure))) - .toCompletableFuture(); - } catch (RuntimeException synchronousFailure) { - // A publisher that validates eagerly throws instead of returning a failed stage. Converting - // it here keeps the "one result per index" contract that the caller resubmits from. - return CompletableFuture.completedFuture( - new BatchPublishItemResult( - index, message.messageId(), PublishResults.from(synchronousFailure))); - } - } -} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBlockingMessagePublisher.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBlockingMessagePublisher.java deleted file mode 100644 index 9931b91..0000000 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBlockingMessagePublisher.java +++ /dev/null @@ -1,65 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.error.MessagePublishTimeoutException; -import dev.caskeleton.messaging.api.publish.BlockingMessagePublisher; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import java.util.Objects; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.TimeoutException; - -/** - * Converts the asynchronous publish lifecycle into a blocking call. - * - *

    Three details make this safe rather than a thread-pool trap. The wait is bounded by the call's - * own deadline. A timeout is reported as {@link MessagePublishTimeoutException}, which the error - * model classifies as ambiguous — the broker may still hold the message, and the caller must not - * treat it as a failed send. And an interrupt restores the thread's interrupt flag instead of - * swallowing it, so a shutdown that interrupts a blocked publisher still unwinds. - */ -public final class DefaultBlockingMessagePublisher implements BlockingMessagePublisher { - - private final MessagePublisher delegate; - - /** - * Creates a blocking facade. - * - * @param delegate the asynchronous publisher - */ - public DefaultBlockingMessagePublisher(MessagePublisher delegate) { - this.delegate = Objects.requireNonNull(delegate, "delegate must not be null"); - } - - @Override - public PublishResult publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options) { - Objects.requireNonNull(options, "options must not be null"); - CompletableFuture future = - delegate.publish(destination, message, options).toCompletableFuture(); - try { - return future.get(options.timeout().toMillis(), TimeUnit.MILLISECONDS); - } catch (TimeoutException exception) { - future.cancel(true); - throw new MessagePublishTimeoutException( - "PUBLISH_TIMEOUT", - "no publish outcome within " + options.timeout() + "; the broker may hold the message", - exception); - } catch (InterruptedException exception) { - Thread.currentThread().interrupt(); - future.cancel(true); - throw new MessagePublishTimeoutException( - "PUBLISH_INTERRUPTED", "the publishing thread was interrupted", exception); - } catch (ExecutionException exception) { - Throwable cause = exception.getCause(); - if (cause instanceof RuntimeException runtime) { - throw runtime; - } - throw new IllegalStateException("publish failed", cause); - } - } -} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java deleted file mode 100644 index 5c3a25b..0000000 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java +++ /dev/null @@ -1,43 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import java.util.Objects; -import reactor.core.publisher.Mono; - -/** - * Converts the asynchronous publish lifecycle into a cold, cancellable {@code Mono}. - * - *

    {@code Mono.fromCompletionStage} with a supplier is used deliberately: the eager overload - * would start the publish when the {@code Mono} is assembled rather than when it is subscribed, - * which makes a request never sent by an unsubscribed pipeline indistinguishable from one that was. - */ -public final class DefaultReactiveMessagePublisher implements ReactiveMessagePublisher { - - private final MessagePublisher delegate; - - /** - * Creates a Reactor facade. - * - * @param delegate the asynchronous publisher - */ - public DefaultReactiveMessagePublisher(MessagePublisher delegate) { - this.delegate = Objects.requireNonNull(delegate, "delegate must not be null"); - } - - @Override - public Mono publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(message, "message must not be null"); - Objects.requireNonNull(options, "options must not be null"); - // publish() is not purely asynchronous: admission may wait for a permit and the Kafka producer - // may wait for metadata before the stage exists. Subscribing on a Netty event loop would pin - // that loop for the wait, so the call runs on the bounded elastic pool instead. - return Mono.fromCompletionStage(() -> delegate.publish(destination, message, options)) - .subscribeOn(reactor.core.scheduler.Schedulers.boundedElastic()); - } -} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DestinationSettings.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DestinationSettings.java index 35304c7..8184d6a 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DestinationSettings.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DestinationSettings.java @@ -5,7 +5,7 @@ import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; import dev.caskeleton.messaging.api.delivery.OrderingScope; import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.OrderingImpact; import dev.caskeleton.messaging.policy.RetryMode; import dev.caskeleton.messaging.schema.SchemaCompatibility; @@ -48,7 +48,7 @@ import org.springframework.boot.context.properties.bind.DefaultValue; public record DestinationSettings( String broker, DestinationKind kind, - @DefaultValue("M1") CapabilityTier tier, + @DefaultValue("M1") ApiTier tier, @DefaultValue Physical physical, @DefaultValue Schema schema, @DefaultValue Guarantees guarantees, diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessageConsumers.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessageConsumers.java new file mode 100644 index 0000000..332baa5 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessageConsumers.java @@ -0,0 +1,46 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.transport.TransportDelivery; +import java.util.List; +import java.util.Objects; +import java.util.concurrent.CompletionStage; +import java.util.function.Function; + +/** + * The destinations this application consumes, each with the broker-neutral pipeline its deliveries + * go through. + * + *

    Built by {@link MessagingConsumeConfiguration} from the {@code MessageHandler} beans; a broker + * starter opens one consumer per entry and hands every delivery to its {@link Consumer#sink()}. The + * split keeps decoding, inbox, dead-lettering and settlement rules out of each broker starter. + * + * @param consumers one entry per consumed destination + */ +public record MessageConsumers(List consumers) { + + public MessageConsumers { + consumers = List.copyOf(Objects.requireNonNull(consumers, "consumers must not be null")); + } + + /** + * One consumed destination. + * + * @param profile the destination profile + * @param group the consumer group, which is also the inbox consumer identity + * @param sink the pipeline each still-encoded delivery goes through + */ + public record Consumer( + DestinationProfile profile, + String group, + Function> sink) { + + public Consumer { + Objects.requireNonNull(profile, "profile must not be null"); + Objects.requireNonNull(sink, "sink must not be null"); + if (group == null || group.isBlank()) { + throw new IllegalArgumentException("group must not be blank"); + } + } + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java index 876b390..2a24d32 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java @@ -93,20 +93,13 @@ public final class MessagingConfigurationCompiler { } private static void validateBrokerSettings(String name, BrokerSettings broker) { - switch (familyOf(name, broker)) { - case BrokerSettings.KAFKA -> { - if (broker.bootstrapServers().isEmpty()) { - throw configurationError( - key("brokers", name, "bootstrap-servers"), - "a Kafka broker needs at least one address; a client with none defaults to " - + "localhost:9092 and fails at the first publish rather than at startup"); - } - } - default -> - throw configurationError( - key("brokers", name, "type"), - "'%s' is not a broker family this platform has a transport for; use '%s'" - .formatted(broker.type(), BrokerSettings.KAFKA)); + // Only the family is checked here. Its client settings — addresses, acks — belong to the + // family's starter, which binds and validates them when it builds the client. + if (!MessagingProviderSelection.registeredBrokerIds().contains(familyOf(name, broker))) { + throw configurationError( + key("brokers", name, "type"), + "'%s' is not a broker family this platform has a transport for; use one of %s" + .formatted(broker.type(), MessagingProviderSelection.registeredBrokerIds())); } } diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java index b5c534a..56c5b67 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java @@ -5,6 +5,7 @@ import java.lang.reflect.RecordComponent; import java.util.ArrayList; import java.util.LinkedHashSet; import java.util.List; +import java.util.Locale; import java.util.Map; import java.util.Objects; import java.util.Set; @@ -12,6 +13,7 @@ import org.springframework.beans.factory.InitializingBean; import org.springframework.core.env.ConfigurableEnvironment; import org.springframework.core.env.EnumerablePropertySource; import org.springframework.core.env.PropertySource; +import org.springframework.util.ClassUtils; /** * Refuses a key inside the platform's own configuration sections that nothing binds. @@ -68,12 +70,14 @@ public final class MessagingConfigurationKeyValidator implements InitializingBea List rejected = new ArrayList<>(); sections.forEach( (section, shape) -> { - Set legal = leafPaths(shape); + Set common = leafPaths(shape); for (String key : keysUnder(MessagingSettings.PREFIX + '.' + section + '.')) { Entry entry = split(key, MessagingSettings.PREFIX + '.' + section + '.'); if (entry.leaf().isEmpty()) { continue; } + Set legal = + "brokers".equals(section) ? brokerLeafPaths(entry.name(), common) : common; if (!legal.contains(entry.leaf())) { rejected.add( "%s binds nothing; %s entries accept %s" @@ -99,6 +103,33 @@ public final class MessagingConfigurationKeyValidator implements InitializingBea + (rejected.size() > 10 ? " (and " + (rejected.size() - 10) + " more)" : "")); } + /** + * The keys a broker entry accepts: the neutral ones, plus its family's own when the family's + * starter is on the classpath. + * + *

    A family whose settings record is absent accepts only the neutral keys. Its client keys bind + * nothing in that deployment — there is no transport to hand them to — and {@link + * MessagingProviderSelection} refuses a selected family whose starter is missing anyway. + */ + private Set brokerLeafPaths(String brokerName, Set common) { + String type = + environment.getProperty(MessagingSettings.PREFIX + ".brokers." + brokerName + ".type", ""); + String shapeName = + MessagingProviderSelection.PROVIDER_BROKER_SETTINGS.get( + type.trim().toLowerCase(Locale.ROOT)); + if (shapeName == null + || !ClassUtils.isPresent( + shapeName, MessagingConfigurationKeyValidator.class.getClassLoader())) { + return common; + } + Set legal = new LinkedHashSet<>(common); + legal.addAll( + leafPaths( + ClassUtils.resolveClassName( + shapeName, MessagingConfigurationKeyValidator.class.getClassLoader()))); + return legal; + } + /** One configuration key split into the map entry it configures and the leaf inside it. */ private record Entry(String name, String leaf) {} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfiguration.java new file mode 100644 index 0000000..0dc8879 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfiguration.java @@ -0,0 +1,151 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.error.MessagingConfigurationException; +import dev.caskeleton.messaging.inbox.IdempotentConsumer; +import dev.caskeleton.messaging.inbox.InboxMessageHandler; +import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.runtime.DefaultDeliveryProcessor; +import dev.caskeleton.messaging.schema.MessageCodecRegistry; +import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import dev.caskeleton.messaging.security.DestinationAccessPolicy; +import java.time.Instant; +import java.time.InstantSource; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.CompletableFuture; +import java.util.stream.Stream; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * The broker-neutral consume pipeline, assembled from the application's {@link MessageHandler} + * beans (owner decision 10, S7-#2). + * + *

    On when present: no handler bean, no consumer. {@code app.messaging.enabled=false} turns the + * whole platform, consumers included, off. Every contradiction a handler can declare fails startup + * here, before any broker connection opens: + * + *

      + *
    • a handler for a destination nobody declared; + *
    • a consumed destination without {@code consumer.group} — the group is also the inbox + * identity, and inventing one would split or merge consumers silently; + *
    • a destination the broker's security profile does not list as consumable; + *
    • {@code INBOX_TRANSACTIONAL} without an {@link IdempotentConsumer} bean to run it. + *
    + */ +@Configuration(proxyBeanMethods = false) +public class MessagingConsumeConfiguration { + + /** + * One consume pipeline per destination at least one handler names. + * + * @param handlers the application's handlers + * @param destinations the validated destination registry + * @param codecs the codec registry + * @param deadLetters publishes to a destination's dead-letter destination + * @param inbox runs a handler inside the inbox reservation's transaction, when supplied + * @param compiled the security profiles compiled from configuration + * @param declaredSecurity the security profiles the application declared as beans + * @return the consumers + */ + @Bean + @ConditionalOnBean(MessageHandler.class) + @ConditionalOnMissingBean + public MessageConsumers messageConsumers( + ObjectProvider> handlers, + ValidatedDestinationRegistry destinations, + MessageCodecRegistry codecs, + DeadLetterOrchestrator deadLetters, + ObjectProvider inbox, + CompiledMessagingConfiguration compiled, + ObjectProvider declaredSecurity) { + Map>> byDestination = new LinkedHashMap<>(); + handlers + .orderedStream() + .forEach( + handler -> + byDestination + .computeIfAbsent(handler.destination().name(), name -> new ArrayList<>()) + .add(handler)); + + Map access = new LinkedHashMap<>(); + Stream.concat(compiled.security().stream(), declaredSecurity.orderedStream()) + .forEach(profile -> access.putIfAbsent(profile.broker(), profile.access())); + + List consumers = new ArrayList<>(); + byDestination.forEach( + (name, forDestination) -> { + DestinationProfile profile = destinations.require(name); + String group = + profile + .consumer() + .group() + .filter(value -> !value.isBlank()) + .orElseThrow( + () -> + refused( + name, + "is consumed by a handler but declares no consumer.group; the group" + + " is also the inbox identity, so the platform does not invent" + + " one")); + DestinationAccessPolicy acl = access.get(profile.broker()); + if (acl != null && !acl.mayConsume(name)) { + throw refused( + name, + "is not in app.messaging.security." + profile.broker() + ".access.consumable"); + } + List> pipeline = + profile.externalSideEffectGuarantee() + == ExternalSideEffectGuarantee.INBOX_TRANSACTIONAL + ? forDestination.stream() + .>map( + handler -> inboxed(handler, inbox.getIfAvailable(), group, name)) + .toList() + : forDestination; + DefaultDeliveryProcessor processor = + new DefaultDeliveryProcessor( + profile, + pipeline, + codecs, + (delivery, failure) -> + // The processor acknowledges the source itself once the publish confirms, + // so the orchestrator's settlement callback has nothing left to do. + deadLetters + .deadLetter( + profile, + delivery, + failure, + () -> CompletableFuture.completedFuture(null)) + .thenApply(result -> result.publishResult().completion()), + group, + InstantSource.system()); + consumers.add(new MessageConsumers.Consumer(profile, group, processor::process)); + }); + return new MessageConsumers(consumers); + } + + private static MessageHandler inboxed( + MessageHandler handler, IdempotentConsumer inbox, String group, DestinationName name) { + if (inbox == null) { + throw refused( + name, + "declares INBOX_TRANSACTIONAL but no IdempotentConsumer bean exists; without one the" + + " handler's side effect is not protected against redelivery"); + } + return new InboxMessageHandler<>(handler, inbox, group, Instant::now); + } + + private static MessagingConfigurationException refused(DestinationName name, String reason) { + return new MessagingConfigurationException( + "MESSAGING_CONSUMER_INVALID", "destination '" + name.value() + "' " + reason); + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java index 0f1c771..c7fa030 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java @@ -1,583 +1,26 @@ package dev.caskeleton.messaging.autoconfigure; -import dev.caskeleton.messaging.api.publish.BatchMessagePublisher; -import dev.caskeleton.messaging.api.publish.BlockingMessagePublisher; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.observation.CardinalityGuard; -import dev.caskeleton.messaging.observation.MessagingRedactor; -import dev.caskeleton.messaging.policy.BackoffCalculator; -import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; -import dev.caskeleton.messaging.policy.DefaultRetryDecisionEngine; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.DestinationProfileValidator; -import dev.caskeleton.messaging.policy.RetryDecisionEngine; -import dev.caskeleton.messaging.security.BrokerTlsPolicy; -import dev.caskeleton.messaging.security.CredentialProvider; -import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; -import dev.caskeleton.messaging.security.MessageSecurityValidator; -import dev.caskeleton.messaging.transport.DefaultMessagingRuntimeRegistry; -import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; -import dev.caskeleton.messaging.transport.MessagingRuntimeRegistry; -import java.time.Instant; -import java.util.List; -import org.springframework.beans.factory.ObjectProvider; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; /** - * Wires the platform's policy, transport, and observability primitives. + * Wires the platform's configuration, publish pipeline and observability. * - *

    Destination profiles are validated at context refresh through a bean that fails to construct - * when the registry is contradictory. Startup failure is the point: a profile that promises ordered - * delivery while configuring a reordering retry does not fail on the happy path, it fails months - * later on the first retry, looking like a data bug rather than a configuration one. + *

    It was one 550-line class with 28 beans (S7-#9a). It is now the entry point that imports the + * three parts, plus the consume pipeline (S7-#2), so an adopter or a test that names this class + * still gets the whole core. * - *

    The M2 and M3 surfaces are deliberately not auto-configured. Manual settlement, native - * capabilities, and the Admin Plane are opt-in beans with their own credentials, so a service - * cannot acquire them by adding the starter to its classpath. + * @see MessagingValidationConfiguration + * @see MessagingPublishConfiguration + * @see MessagingObservabilityConfiguration + * @see MessagingConsumeConfiguration */ @Configuration(proxyBeanMethods = false) @EnableConfigurationProperties(MessagingSettings.class) -public class MessagingCoreAutoConfiguration { - - /** - * Fails the context when a production broker profile has no credential source. - * - *

    One actionable error instead of a missing-bean message naming an internal registry type. - */ - @Bean - public MessagingCredentialRequirementValidator messagingCredentialRequirementValidator( - org.springframework.beans.factory.ObjectProvider< - dev.caskeleton.messaging.security.BrokerSecurityProfile> - profiles, - CompiledMessagingConfiguration compiled, - org.springframework.beans.factory.ObjectProvider< - dev.caskeleton.messaging.security.CredentialProvider> - credentials, - org.springframework.core.env.Environment environment) { - return new MessagingCredentialRequirementValidator( - profiles, compiled.security(), credentials, environment); - } - - /** - * Fails the context when configuration uses a prefix nothing binds. - * - *

    Registered here rather than left to review because the failure it catches is silent: a key - * under {@code backend.messaging} or a bare {@code messaging} looks configured and does nothing. - */ - @Bean - public MessagingPrefixMigrationValidator messagingPrefixMigrationValidator( - org.springframework.core.env.ConfigurableEnvironment environment) { - return new MessagingPrefixMigrationValidator(environment); - } - - /** - * Compiles {@code app.messaging.destinations}, {@code .brokers} and {@code .security} into the - * platform's own profile objects. - * - *

    Separate from the registry bean so that a configuration error fails while compiling, with - * the offending key in the message, rather than later as a validation failure over an object - * whose origin the operator then has to work out. - * - * @param settings the bound configuration - * @return the compiled profiles - */ - @Bean - @ConditionalOnMissingBean - public CompiledMessagingConfiguration compiledMessagingConfiguration(MessagingSettings settings) { - return new MessagingConfigurationCompiler(settings).compile(); - } - - /** - * Refuses a key under the platform's own sections that nothing binds. - * - *

    Registered here rather than left to review because a misspelt key inside a bound section is - * silent in a way a misspelt prefix is not: the section binds, the entry appears, and the one - * setting the operator came to change is the one that did not take. - * - * @param environment the resolved environment - * @return the key validator - */ - @Bean - @ConditionalOnMissingBean - public MessagingConfigurationKeyValidator messagingConfigurationKeyValidator( - org.springframework.core.env.ConfigurableEnvironment environment) { - return new MessagingConfigurationKeyValidator(environment); - } - - /** - * Validates every registered destination profile at startup. - * - *

    Returns a marker rather than a service because its only job is to fail the context when the - * registry is inconsistent. - * - *

    Configured profiles and bean-declared ones are merged before validation, not after, because - * the rules that matter most across a registry — a dead letter destination that exists, a retry - * graph with no cycle — are questions about the whole set. Validating the two halves separately - * would pass a deployment whose configured destination dead-letters into a bean-declared one that - * points back. - * - * @param profiles the profiles the application declared as beans - * @param compiled the profiles compiled from configuration - * @return the validated registry - */ - @Bean - @ConditionalOnMissingBean - public ValidatedDestinationRegistry validatedDestinationRegistry( - ObjectProvider profiles, CompiledMessagingConfiguration compiled) { - List registered = - java.util.stream.Stream.concat(compiled.destinations().stream(), profiles.orderedStream()) - .toList(); - new DestinationProfileValidator().validateAll(registered); - return new ValidatedDestinationRegistry(registered); - } - - /** - * Returns the destination profile validator. - * - * @return the validator - */ - @Bean - @ConditionalOnMissingBean - public DestinationProfileValidator destinationProfileValidator() { - return new DestinationProfileValidator(); - } - - /** - * Returns the broker security validator. - * - * @return the validator - */ - @Bean - @ConditionalOnMissingBean - public MessageSecurityValidator messageSecurityValidator() { - return new MessageSecurityValidator(); - } - - /** - * Returns the retry decision engine. - * - * @return the engine - */ - @Bean - @ConditionalOnMissingBean - public RetryDecisionEngine retryDecisionEngine() { - return new DefaultRetryDecisionEngine(new BackoffCalculator()); - } - - /** - * Returns the dead letter orchestrator. - * - * @param publisher the platform publisher - * @return the orchestrator - */ - @Bean - @ConditionalOnMissingBean - public DeadLetterOrchestrator deadLetterOrchestrator(MessagePublisher publisher) { - return new DeadLetterOrchestrator(publisher); - } - - /** - * Returns the runtime registry used for credential and topology rotation. - * - * @param properties the bound configuration - * @return the registry - */ - @Bean - @ConditionalOnMissingBean - public MessagingRuntimeRegistry messagingRuntimeRegistry(MessagingSettings properties) { - return new DefaultMessagingRuntimeRegistry(properties.getShutdown().getDrainDeadline()); - } - - /** - * Returns the graceful shutdown coordinator. - * - * @param properties the bound configuration - * @return the coordinator - */ - @Bean - @ConditionalOnMissingBean - public GracefulShutdownCoordinator gracefulShutdownCoordinator(MessagingSettings properties) { - return new GracefulShutdownCoordinator(properties.getShutdown().getDrainDeadline()); - } - - /** - * Runs the security validator over every broker security profile, declared or configured. - * - * @param profiles the security profiles the application declared as beans - * @param compiled the security profiles compiled from configuration - * @param validator the security validator - * @return the startup validation - */ - @Bean - public StartupProfileValidation - brokerSecurityStartupValidation( - ObjectProvider profiles, - CompiledMessagingConfiguration compiled, - MessageSecurityValidator validator) { - return new StartupProfileValidation<>( - () -> - java.util.stream.Stream.concat(compiled.security().stream(), profiles.orderedStream()) - .toList(), - validator::validate); - } - - /** - * Drives the shutdown phases the admission gate and the drain coordinator only implemented. - * - * @param admission the publish admission gate - * @param drain the in-flight work coordinator - * @param properties the bound configuration supplying the drain deadline - * @return the lifecycle - */ - @Bean - @ConditionalOnMissingBean - public MessagingShutdownLifecycle messagingShutdownLifecycle( - dev.caskeleton.messaging.policy.MessagingAdmissionController admission, - GracefulShutdownCoordinator drain, - MessagingSettings properties) { - return new MessagingShutdownLifecycle( - admission, drain, properties.getShutdown().getDrainDeadline()); - } - - /** - * Returns the diagnostic redactor. - * - * @return the redactor - */ - @Bean - @ConditionalOnMissingBean - public MessagingRedactor messagingRedactor() { - return new MessagingRedactor(); - } - - /** - * Returns the metric cardinality guard. - * - * @return the guard - */ - @Bean - @ConditionalOnMissingBean - public CardinalityGuard cardinalityGuard() { - return new CardinalityGuard(); - } - - /** - * Binds the platform observation seam to Micrometer when a registry is available. - * - *

    Narrow tests and non-observed adopters may omit Micrometer entirely. A normal Boot runtime - * with Actuator contributes a registry, in which case publish outcomes must not fall back to the - * no-op observation constructor. - * - * @param registry the Micrometer registry - * @param guard the bounded-cardinality guard - * @param redactor the diagnostics redactor - * @return the messaging metrics binding - */ - @Bean - @ConditionalOnBean(io.micrometer.core.instrument.MeterRegistry.class) - @ConditionalOnMissingBean(dev.caskeleton.messaging.observation.MessagingObservation.class) - public dev.caskeleton.messaging.observation.MessagingObservation messagingObservation( - io.micrometer.core.instrument.MeterRegistry registry, - CardinalityGuard guard, - MessagingRedactor redactor) { - return new dev.caskeleton.messaging.observation.MessagingMetrics(registry, guard, redactor); - } - - /** - * Returns the blocking publish facade. - * - * @param publisher the asynchronous publisher - * @return the blocking facade - */ - @Bean - @ConditionalOnMissingBean - public BlockingMessagePublisher blockingMessagePublisher(MessagePublisher publisher) { - return new DefaultBlockingMessagePublisher(publisher); - } - - /** - * Returns the Reactor publish facade. - * - * @param publisher the asynchronous publisher - * @return the Reactor facade - */ - @Bean - @ConditionalOnMissingBean - public ReactiveMessagePublisher reactiveMessagePublisher(MessagePublisher publisher) { - return new DefaultReactiveMessagePublisher(publisher); - } - - /** - * Returns the M2 batch publish facade. - * - * @param publisher the asynchronous publisher - * @return the batch publisher - */ - @Bean - @ConditionalOnMissingBean - public BatchMessagePublisher batchMessagePublisher(MessagePublisher publisher) { - return new DefaultBatchMessagePublisher(publisher, Instant::now); - } - - /** - * Returns the credential runtime registry, when the application supplied a credential provider. - * - *

    Conditional on the provider because there is no safe default: a registry with no source - * would resolve nothing, and the adapters would connect with whatever their client's defaults - * are. - * - * @param provider the application's credential source - * @return the credential runtime registry - */ - @Bean - @ConditionalOnBean(CredentialProvider.class) - @ConditionalOnMissingBean - public CredentialRuntimeRegistry credentialRuntimeRegistry(CredentialProvider provider) { - return new CredentialRuntimeRegistry(provider); - } - - /** - * Returns the transport security policy. - * - * @return the TLS policy - */ - @Bean - @ConditionalOnMissingBean - public BrokerTlsPolicy brokerTlsPolicy() { - return new BrokerTlsPolicy(); - } - - /** - * Returns the read-only actuator endpoint. - * - * @param registry the validated destination registry - * @param admission the gate every publish passes through - * @return the endpoint - */ - @Bean - @ConditionalOnMissingBean - public MessagingEndpoint messagingEndpoint( - ValidatedDestinationRegistry registry, - dev.caskeleton.messaging.policy.MessagingAdmissionController admission) { - return new MessagingEndpoint(registry, admission); - } - - /** - * The codec registry, which had no implementation anywhere (MSG-INT-003). - * - *

    The message contracts come from the deployment — a {@code Map} of message type and schema - * version to payload class — and default to empty, which means no message type can be - * encoded and every publish is refused at preparation. That is fail-closed on purpose: a codec - * that accepted an unregistered type would serialise whatever object it was handed and put bytes - * on a topic that consumers compiled against a different shape will read. - * - * @param contracts the declared message contracts, if the deployment registered any - * @return the registry, defaulting to JSON - */ - @Bean - @ConditionalOnMissingBean(dev.caskeleton.messaging.schema.MessageCodecRegistry.class) - public dev.caskeleton.messaging.runtime.RegisteredMessageCodecs messagingCodecs( - ObjectProvider contracts) { - return dev.caskeleton.messaging.runtime.RegisteredMessageCodecs.of( - dev.caskeleton.messaging.schema.json.JacksonMessageCodec.of( - contracts.getIfAvailable(MessageContracts::none).byKey())); - } - - /** - * The destination profiles the publisher resolves against, from the validated registry. - * - * @param destinations the validated registry - * @return the profile registry - */ - @Bean - @ConditionalOnMissingBean - public dev.caskeleton.messaging.runtime.DestinationProfileRegistry messagingDestinationProfiles( - ValidatedDestinationRegistry destinations) { - return dev.caskeleton.messaging.runtime.DestinationProfileRegistry.of(destinations.all()); - } - - /** - * What this deployment may publish to. - * - *

    The declared destinations, and nothing else. Deny-all assembles a platform that refuses - * every publish a correctly configured deployment makes; allow-all makes the check decoration. - * Declaring a destination profile is already an act of configuration, so a message to one nobody - * declared is a typo or a module reaching past its contract — which is what this check is for. - * - *

    A broker with a security entry narrows that further to its {@code access.publishable} list. - * The list was compiled and then read by nothing, so a destination the security entry left out - * was published to anyway. - * - * @param destinations the validated registry - * @param compiled the security profiles compiled from configuration - * @param declared the security profiles the application declared as beans - * @return the access policy - */ - @Bean - @ConditionalOnMissingBean - public dev.caskeleton.messaging.security.DestinationAccessPolicy messagingDestinationAccess( - ValidatedDestinationRegistry destinations, - CompiledMessagingConfiguration compiled, - ObjectProvider declared) { - java.util.Map byBroker = - new java.util.HashMap<>(); - java.util.stream.Stream.concat(compiled.security().stream(), declared.orderedStream()) - .forEach( - profile -> { - if (byBroker.putIfAbsent(profile.broker(), profile.access()) != null) { - throw new IllegalStateException( - "broker '" - + profile.broker() - + "' has more than one security profile; its access list is ambiguous"); - } - }); - java.util.Map< - dev.caskeleton.messaging.api.destination.DestinationName, - dev.caskeleton.messaging.policy.DestinationProfile> - permitted = new java.util.HashMap<>(); - destinations - .all() - .forEach( - (name, profile) -> { - dev.caskeleton.messaging.security.DestinationAccessPolicy acl = - byBroker.get(profile.broker()); - if (acl == null || acl.mayPublish(name)) { - permitted.put(name, profile); - } - }); - return dev.caskeleton.messaging.runtime.DeclaredDestinationAccess.of(permitted); - } - - /** - * Payload and in-flight admission. - * - * @param properties the bound settings - * @return the admission controller - */ - @Bean - @ConditionalOnMissingBean - public dev.caskeleton.messaging.policy.MessagingAdmissionController messagingAdmission( - MessagingSettings properties) { - return new dev.caskeleton.messaging.policy.MessagingAdmissionController( - new dev.caskeleton.messaging.policy.PayloadLimitGuard( - new dev.caskeleton.messaging.policy.PayloadPolicy( - dev.caskeleton.messaging.schema.json.JacksonMessageCodec.DEFAULT_MAX_BYTES, - dev.caskeleton.messaging.schema.json.JacksonMessageCodec.DEFAULT_MAX_BYTES / 2)), - new dev.caskeleton.messaging.policy.InFlightLimiter( - properties.getBackpressure().getGlobalLimit(), java.time.Duration.ofSeconds(5)), - properties.getBackpressure().getPerDestinationLimit()); - } - - /** - * The publisher this configuration has been consuming since it was written (MSG-INT-003). - * - *

    {@code deadLetterOrchestrator}, {@code blockingMessagePublisher}, {@code - * reactiveMessagePublisher} and {@code batchMessagePublisher} all take a {@link - * MessagePublisher}, and no configuration produced one. A selected transport therefore could not - * assemble: the platform failed on a missing bean rather than publishing anywhere. {@code - * DefaultMessagePublisher} existed in {@code messaging-runtime-core} the whole time; the starter - * did not depend on that leaf. - * - * @param destinations the destination profiles - * @param access what may be published to - * @param codecs the codec registry - * @param admission payload and in-flight admission - * @param runtimes the runtime registry - * @param transport the selected transport - * @return the publisher - */ - @Bean - @ConditionalOnMissingBean(MessagePublisher.class) - public dev.caskeleton.messaging.runtime.DefaultMessagePublisher messagingPublisher( - dev.caskeleton.messaging.runtime.DestinationProfileRegistry destinations, - dev.caskeleton.messaging.security.DestinationAccessPolicy access, - dev.caskeleton.messaging.schema.MessageCodecRegistry codecs, - dev.caskeleton.messaging.policy.MessagingAdmissionController admission, - MessagingRuntimeRegistry runtimes, - dev.caskeleton.messaging.transport.MessagingTransport transport, - ObjectProvider observation) { - dev.caskeleton.messaging.observation.MessagingObservation configured = - observation.getIfAvailable(); - if (configured == null) { - return new dev.caskeleton.messaging.runtime.DefaultMessagePublisher( - destinations, access, codecs, admission, runtimes, transport); - } - return new dev.caskeleton.messaging.runtime.DefaultMessagePublisher( - destinations, access, codecs, admission, runtimes, transport, System::nanoTime, configured); - } - - /** - * Installs the selected transport into the runtime registry (MSG-INT-003). - * - *

    The registry was constructed empty and nothing ever installed anything, so every publish - * failed at {@code PUBLISH_RUNTIME_UNAVAILABLE} — after resolving the destination, checking - * access and encoding the payload, which is as far as a message could get. The same "registry - * built empty" shape the notification provider registry had. - * - * @param runtimes the registry - * @param transport the selected transport - * @return the startup installation - */ - @Bean - public org.springframework.beans.factory.InitializingBean messagingRuntimeInstallation( - MessagingRuntimeRegistry runtimes, - ObjectProvider transport, - CompiledMessagingConfiguration compiled) { - return () -> { - dev.caskeleton.messaging.transport.MessagingTransport selected = transport.getIfAvailable(); - if (selected == null) { - // Not a silent skip of a check: MessagingProviderSelection is what guarantees a transport - // when a broker is selected, and it refuses startup by name when one is not. This - // configuration is also loadable on its own — an adopter composing the policy primitives - // without a transport — and demanding one here would refuse that. - return; - } - - String runtimeName = logicalRuntimeName(selected, compiled); - runtimes.install( - new dev.caskeleton.messaging.runtime.TransportMessagingRuntime( - runtimeName, selected.generation(), selected)); - }; - } - - private static String logicalRuntimeName( - dev.caskeleton.messaging.transport.MessagingTransport selected, - CompiledMessagingConfiguration compiled) { - String configuredFamily = - switch (selected.brokerName()) { - case "kafka" -> BrokerSettings.KAFKA; - default -> selected.brokerName(); - }; - java.util.List configuredNames = - compiled.brokers().entrySet().stream() - .filter( - entry -> - entry.getValue().type() != null - && entry.getValue().type().equalsIgnoreCase(configuredFamily)) - .map(java.util.Map.Entry::getKey) - .sorted() - .toList(); - - if (configuredNames.isEmpty()) { - // Backward compatibility for applications that declare all profiles as beans and therefore - // have no app.messaging.brokers entries. Their destination profiles historically named the - // transport family directly. - return selected.brokerName(); - } - if (configuredNames.size() != 1) { - throw new IllegalStateException( - "the selected " - + selected.brokerName() - + " transport has one client/runtime but app.messaging.brokers declares " - + configuredNames.size() - + " logical brokers " - + configuredNames - + "; configure exactly one logical broker for the selected transport"); - } - return configuredNames.getFirst(); - } -} +@org.springframework.context.annotation.Import({ + MessagingValidationConfiguration.class, + MessagingPublishConfiguration.class, + MessagingObservabilityConfiguration.class, + MessagingConsumeConfiguration.class +}) +public class MessagingCoreAutoConfiguration {} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingObservabilityConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingObservabilityConfiguration.java new file mode 100644 index 0000000..bd8ca53 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingObservabilityConfiguration.java @@ -0,0 +1,77 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.observation.CardinalityGuard; +import dev.caskeleton.messaging.observation.MessagingRedactor; +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * Metrics, diagnostic redaction and the read-only actuator endpoint. + * + *

    The Micrometer binding is conditional on a registry, so a narrow test or a non-observed + * adopter assembles the platform without Micrometer. + */ +@Configuration(proxyBeanMethods = false) +public class MessagingObservabilityConfiguration { + + /** + * Returns the diagnostic redactor. + * + * @return the redactor + */ + @Bean + @ConditionalOnMissingBean + public MessagingRedactor messagingRedactor() { + return new MessagingRedactor(); + } + + /** + * Returns the metric cardinality guard. + * + * @return the guard + */ + @Bean + @ConditionalOnMissingBean + public CardinalityGuard cardinalityGuard() { + return new CardinalityGuard(); + } + + /** + * Binds the platform observation seam to Micrometer when a registry is available. + * + *

    Narrow tests and non-observed adopters may omit Micrometer entirely. A normal Boot runtime + * with Actuator contributes a registry, in which case publish outcomes must not fall back to the + * no-op observation constructor. + * + * @param registry the Micrometer registry + * @param guard the bounded-cardinality guard + * @param redactor the diagnostics redactor + * @return the messaging metrics binding + */ + @Bean + @ConditionalOnBean(io.micrometer.core.instrument.MeterRegistry.class) + @ConditionalOnMissingBean(dev.caskeleton.messaging.observation.MessagingObservation.class) + public dev.caskeleton.messaging.observation.MessagingObservation messagingObservation( + io.micrometer.core.instrument.MeterRegistry registry, + CardinalityGuard guard, + MessagingRedactor redactor) { + return new dev.caskeleton.messaging.observation.MessagingMetrics(registry, guard, redactor); + } + + /** + * Returns the read-only actuator endpoint. + * + * @param registry the validated destination registry + * @param admission the gate every publish passes through + * @return the endpoint + */ + @Bean + @ConditionalOnMissingBean + public MessagingEndpoint messagingEndpoint( + ValidatedDestinationRegistry registry, + dev.caskeleton.messaging.policy.MessagingAdmissionController admission) { + return new MessagingEndpoint(registry, admission); + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java index 91cf33b..b7a248f 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java @@ -45,6 +45,16 @@ public class MessagingProviderSelection { static final Map PROVIDER_CONFIGURATIONS = Map.of("kafka", "dev.caskeleton.messaging.autoconfigure.KafkaMessagingAutoConfiguration"); + /** + * Broker id to the record its starter binds the family's own keys of an {@code + * app.messaging.brokers.} entry into. + * + *

    Named rather than referenced for the same reason as {@link #PROVIDER_CONFIGURATIONS}: this + * starter is broker-neutral and cannot see the class. The key validator loads it when present. + */ + static final Map PROVIDER_BROKER_SETTINGS = + Map.of("kafka", "dev.caskeleton.messaging.autoconfigure.KafkaBrokerSettings"); + /** Broker id to the starter a deployment adds to its build to obtain that transport. */ static final Map PROVIDER_STARTERS = Map.of("kafka", "messaging-spring-boot-starter-kafka"); diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPublishConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPublishConfiguration.java new file mode 100644 index 0000000..79995f1 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPublishConfiguration.java @@ -0,0 +1,304 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.api.publish.MessagePublisher; +import dev.caskeleton.messaging.policy.BackoffCalculator; +import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; +import dev.caskeleton.messaging.policy.DefaultRetryDecisionEngine; +import dev.caskeleton.messaging.policy.RetryDecisionEngine; +import dev.caskeleton.messaging.transport.DefaultMessagingRuntimeRegistry; +import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; +import dev.caskeleton.messaging.transport.MessagingRuntimeRegistry; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * The publish pipeline: codecs, destination access, admission, the publisher, the runtime it + * publishes through, and the shutdown order that drains it. + * + *

    The M2 and M3 surfaces are deliberately not auto-configured. Manual settlement, native + * capabilities, and the Admin Plane are opt-in beans with their own credentials, so a service + * cannot acquire them by adding the starter to its classpath. + */ +@Configuration(proxyBeanMethods = false) +public class MessagingPublishConfiguration { + + /** + * Returns the retry decision engine. + * + * @return the engine + */ + @Bean + @ConditionalOnMissingBean + public RetryDecisionEngine retryDecisionEngine() { + return new DefaultRetryDecisionEngine(new BackoffCalculator()); + } + + /** + * Returns the dead letter orchestrator. + * + * @param publisher the platform publisher + * @return the orchestrator + */ + @Bean + @ConditionalOnMissingBean + public DeadLetterOrchestrator deadLetterOrchestrator(MessagePublisher publisher) { + return new DeadLetterOrchestrator(publisher); + } + + /** + * Returns the runtime registry used for credential and topology rotation. + * + * @param properties the bound configuration + * @return the registry + */ + @Bean + @ConditionalOnMissingBean + public MessagingRuntimeRegistry messagingRuntimeRegistry(MessagingSettings properties) { + return new DefaultMessagingRuntimeRegistry(properties.getShutdown().getDrainDeadline()); + } + + /** + * Returns the graceful shutdown coordinator. + * + * @param properties the bound configuration + * @return the coordinator + */ + @Bean + @ConditionalOnMissingBean + public GracefulShutdownCoordinator gracefulShutdownCoordinator(MessagingSettings properties) { + return new GracefulShutdownCoordinator(properties.getShutdown().getDrainDeadline()); + } + + /** + * Drives the shutdown phases the admission gate and the drain coordinator only implemented. + * + * @param admission the publish admission gate + * @param drain the in-flight work coordinator + * @param properties the bound configuration supplying the drain deadline + * @return the lifecycle + */ + @Bean + @ConditionalOnMissingBean + public MessagingShutdownLifecycle messagingShutdownLifecycle( + dev.caskeleton.messaging.policy.MessagingAdmissionController admission, + GracefulShutdownCoordinator drain, + MessagingSettings properties) { + return new MessagingShutdownLifecycle( + admission, drain, properties.getShutdown().getDrainDeadline()); + } + + /** + * The codec registry, which had no implementation anywhere (MSG-INT-003). + * + *

    The message contracts come from the deployment — a {@code Map} of message type and schema + * version to payload class — and default to empty, which means no message type can be + * encoded and every publish is refused at preparation. That is fail-closed on purpose: a codec + * that accepted an unregistered type would serialise whatever object it was handed and put bytes + * on a topic that consumers compiled against a different shape will read. + * + * @param contracts the declared message contracts, if the deployment registered any + * @return the registry, defaulting to JSON + */ + @Bean + @ConditionalOnMissingBean(dev.caskeleton.messaging.schema.MessageCodecRegistry.class) + public dev.caskeleton.messaging.runtime.RegisteredMessageCodecs messagingCodecs( + ObjectProvider contracts) { + return dev.caskeleton.messaging.runtime.RegisteredMessageCodecs.of( + dev.caskeleton.messaging.schema.json.JacksonMessageCodec.of( + contracts.getIfAvailable(MessageContracts::none).byKey())); + } + + /** + * The destination profiles the publisher resolves against, from the validated registry. + * + * @param destinations the validated registry + * @return the profile registry + */ + @Bean + @ConditionalOnMissingBean + public dev.caskeleton.messaging.runtime.DestinationProfileRegistry messagingDestinationProfiles( + ValidatedDestinationRegistry destinations) { + return dev.caskeleton.messaging.runtime.DestinationProfileRegistry.of(destinations.all()); + } + + /** + * What this deployment may publish to. + * + *

    The declared destinations, and nothing else. Deny-all assembles a platform that refuses + * every publish a correctly configured deployment makes; allow-all makes the check decoration. + * Declaring a destination profile is already an act of configuration, so a message to one nobody + * declared is a typo or a module reaching past its contract — which is what this check is for. + * + *

    A broker with a security entry narrows that further to its {@code access.publishable} list. + * The list was compiled and then read by nothing, so a destination the security entry left out + * was published to anyway. + * + * @param destinations the validated registry + * @param compiled the security profiles compiled from configuration + * @param declared the security profiles the application declared as beans + * @return the access policy + */ + @Bean + @ConditionalOnMissingBean + public dev.caskeleton.messaging.security.DestinationAccessPolicy messagingDestinationAccess( + ValidatedDestinationRegistry destinations, + CompiledMessagingConfiguration compiled, + ObjectProvider declared) { + java.util.Map byBroker = + new java.util.HashMap<>(); + java.util.stream.Stream.concat(compiled.security().stream(), declared.orderedStream()) + .forEach( + profile -> { + if (byBroker.putIfAbsent(profile.broker(), profile.access()) != null) { + throw new IllegalStateException( + "broker '" + + profile.broker() + + "' has more than one security profile; its access list is ambiguous"); + } + }); + java.util.Map< + dev.caskeleton.messaging.api.destination.DestinationName, + dev.caskeleton.messaging.policy.DestinationProfile> + permitted = new java.util.HashMap<>(); + destinations + .all() + .forEach( + (name, profile) -> { + dev.caskeleton.messaging.security.DestinationAccessPolicy acl = + byBroker.get(profile.broker()); + if (acl == null || acl.mayPublish(name)) { + permitted.put(name, profile); + } + }); + return dev.caskeleton.messaging.runtime.DeclaredDestinationAccess.of(permitted); + } + + /** + * Payload and in-flight admission. + * + * @param properties the bound settings + * @return the admission controller + */ + @Bean + @ConditionalOnMissingBean + public dev.caskeleton.messaging.policy.MessagingAdmissionController messagingAdmission( + MessagingSettings properties) { + return new dev.caskeleton.messaging.policy.MessagingAdmissionController( + new dev.caskeleton.messaging.policy.PayloadLimitGuard( + new dev.caskeleton.messaging.policy.PayloadPolicy( + dev.caskeleton.messaging.schema.json.JacksonMessageCodec.DEFAULT_MAX_BYTES, + dev.caskeleton.messaging.schema.json.JacksonMessageCodec.DEFAULT_MAX_BYTES / 2)), + new dev.caskeleton.messaging.policy.InFlightLimiter( + properties.getBackpressure().getGlobalLimit(), java.time.Duration.ofSeconds(5)), + properties.getBackpressure().getPerDestinationLimit()); + } + + /** + * The publisher this configuration has been consuming since it was written (MSG-INT-003). + * + *

    {@code deadLetterOrchestrator} takes a {@link MessagePublisher}, and no configuration + * produced one. A selected transport therefore could not assemble: the platform failed on a + * missing bean rather than publishing anywhere. {@code DefaultMessagePublisher} existed in {@code + * messaging-runtime-core} the whole time; the starter did not depend on that leaf. + * + * @param destinations the destination profiles + * @param access what may be published to + * @param codecs the codec registry + * @param admission payload and in-flight admission + * @param runtimes the runtime registry + * @param transport the selected transport + * @return the publisher + */ + @Bean + @ConditionalOnMissingBean(MessagePublisher.class) + public dev.caskeleton.messaging.runtime.DefaultMessagePublisher messagingPublisher( + dev.caskeleton.messaging.runtime.DestinationProfileRegistry destinations, + dev.caskeleton.messaging.security.DestinationAccessPolicy access, + dev.caskeleton.messaging.schema.MessageCodecRegistry codecs, + dev.caskeleton.messaging.policy.MessagingAdmissionController admission, + MessagingRuntimeRegistry runtimes, + dev.caskeleton.messaging.transport.MessagingTransport transport, + ObjectProvider observation) { + dev.caskeleton.messaging.observation.MessagingObservation configured = + observation.getIfAvailable(); + if (configured == null) { + return new dev.caskeleton.messaging.runtime.DefaultMessagePublisher( + destinations, access, codecs, admission, runtimes, transport); + } + return new dev.caskeleton.messaging.runtime.DefaultMessagePublisher( + destinations, access, codecs, admission, runtimes, transport, System::nanoTime, configured); + } + + /** + * Installs the selected transport into the runtime registry (MSG-INT-003). + * + *

    The registry was constructed empty and nothing ever installed anything, so every publish + * failed at {@code PUBLISH_RUNTIME_UNAVAILABLE} — after resolving the destination, checking + * access and encoding the payload, which is as far as a message could get. The same "registry + * built empty" shape the notification provider registry had. + * + * @param runtimes the registry + * @param transport the selected transport + * @return the startup installation + */ + @Bean + public org.springframework.beans.factory.InitializingBean messagingRuntimeInstallation( + MessagingRuntimeRegistry runtimes, + ObjectProvider transport, + CompiledMessagingConfiguration compiled) { + return () -> { + dev.caskeleton.messaging.transport.MessagingTransport selected = transport.getIfAvailable(); + if (selected == null) { + // Not a silent skip of a check: MessagingProviderSelection is what guarantees a transport + // when a broker is selected, and it refuses startup by name when one is not. This + // configuration is also loadable on its own — an adopter composing the policy primitives + // without a transport — and demanding one here would refuse that. + return; + } + + String runtimeName = logicalRuntimeName(selected, compiled); + runtimes.install( + new dev.caskeleton.messaging.runtime.TransportMessagingRuntime( + runtimeName, selected.generation(), selected)); + }; + } + + private static String logicalRuntimeName( + dev.caskeleton.messaging.transport.MessagingTransport selected, + CompiledMessagingConfiguration compiled) { + String configuredFamily = + switch (selected.brokerName()) { + case "kafka" -> BrokerSettings.KAFKA; + default -> selected.brokerName(); + }; + java.util.List configuredNames = + compiled.brokers().entrySet().stream() + .filter( + entry -> + entry.getValue().type() != null + && entry.getValue().type().equalsIgnoreCase(configuredFamily)) + .map(java.util.Map.Entry::getKey) + .sorted() + .toList(); + + if (configuredNames.isEmpty()) { + // Backward compatibility for applications that declare all profiles as beans and therefore + // have no app.messaging.brokers entries. Their destination profiles historically named the + // transport family directly. + return selected.brokerName(); + } + if (configuredNames.size() != 1) { + throw new IllegalStateException( + "the selected " + + selected.brokerName() + + " transport has one client/runtime but app.messaging.brokers declares " + + configuredNames.size() + + " logical brokers " + + configuredNames + + "; configure exactly one logical broker for the selected transport"); + } + return configuredNames.getFirst(); + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java index 0fa0d2d..f6e096a 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java @@ -1,10 +1,8 @@ package dev.caskeleton.messaging.autoconfigure; import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.inbox.IdempotentConsumer; import dev.caskeleton.messaging.inbox.InboxCleanupJob; import dev.caskeleton.messaging.inbox.InboxRetentionPolicy; -import dev.caskeleton.messaging.inbox.TransactionalInboxHandler; import dev.caskeleton.messaging.outbox.OutboxCleanupJob; import dev.caskeleton.messaging.outbox.OutboxEnvelopeFactory; import dev.caskeleton.messaging.outbox.OutboxProperties; @@ -14,7 +12,6 @@ import dev.caskeleton.messaging.outbox.OutboxRetryScheduler; import dev.caskeleton.messaging.reliability.InboxRepository; import dev.caskeleton.messaging.reliability.OutboxRepository; import java.time.Duration; -import java.time.Instant; import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; import org.springframework.context.annotation.Bean; @@ -196,17 +193,4 @@ public class MessagingReliabilityAutoConfiguration { public InboxCleanupJob inboxCleanupJob(InboxRepository inbox, InboxRetentionPolicy policy) { return new InboxCleanupJob(inbox, policy, 20); } - - /** - * Returns the transactional inbox handler. - * - * @param consumer the reservation-and-effect runner the application supplied - * @return the handler - */ - @Bean - @ConditionalOnBean(IdempotentConsumer.class) - @ConditionalOnMissingBean - public TransactionalInboxHandler transactionalInboxHandler(IdempotentConsumer consumer) { - return new TransactionalInboxHandler<>(consumer, Instant::now); - } } diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingValidationConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingValidationConfiguration.java new file mode 100644 index 0000000..0d6a60a --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingValidationConfiguration.java @@ -0,0 +1,194 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.policy.DestinationProfileValidator; +import dev.caskeleton.messaging.security.BrokerTlsPolicy; +import dev.caskeleton.messaging.security.CredentialProvider; +import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; +import dev.caskeleton.messaging.security.MessageSecurityValidator; +import java.util.List; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * Compiles and validates the messaging configuration, and fails the context when it is + * contradictory. + * + *

    Destination profiles are validated at context refresh through a bean that fails to construct + * when the registry is contradictory. Startup failure is the point: a profile that promises ordered + * delivery while configuring a reordering retry does not fail on the happy path, it fails months + * later on the first retry, looking like a data bug rather than a configuration one. + * + *

    One of the three parts {@link MessagingCoreAutoConfiguration} used to be in a single class + * (S7-#9a): configuration and validation here, the publish pipeline in {@link + * MessagingPublishConfiguration}, observability in {@link MessagingObservabilityConfiguration}. + */ +@Configuration(proxyBeanMethods = false) +public class MessagingValidationConfiguration { + + /** + * Fails the context when a production broker profile has no credential source. + * + *

    One actionable error instead of a missing-bean message naming an internal registry type. + */ + @Bean + public MessagingCredentialRequirementValidator messagingCredentialRequirementValidator( + org.springframework.beans.factory.ObjectProvider< + dev.caskeleton.messaging.security.BrokerSecurityProfile> + profiles, + CompiledMessagingConfiguration compiled, + org.springframework.beans.factory.ObjectProvider< + dev.caskeleton.messaging.security.CredentialProvider> + credentials, + org.springframework.core.env.Environment environment) { + return new MessagingCredentialRequirementValidator( + profiles, compiled.security(), credentials, environment); + } + + /** + * Fails the context when configuration uses a prefix nothing binds. + * + *

    Registered here rather than left to review because the failure it catches is silent: a key + * under {@code backend.messaging} or a bare {@code messaging} looks configured and does nothing. + */ + @Bean + public MessagingPrefixMigrationValidator messagingPrefixMigrationValidator( + org.springframework.core.env.ConfigurableEnvironment environment) { + return new MessagingPrefixMigrationValidator(environment); + } + + /** + * Compiles {@code app.messaging.destinations}, {@code .brokers} and {@code .security} into the + * platform's own profile objects. + * + *

    Separate from the registry bean so that a configuration error fails while compiling, with + * the offending key in the message, rather than later as a validation failure over an object + * whose origin the operator then has to work out. + * + * @param settings the bound configuration + * @return the compiled profiles + */ + @Bean + @ConditionalOnMissingBean + public CompiledMessagingConfiguration compiledMessagingConfiguration(MessagingSettings settings) { + return new MessagingConfigurationCompiler(settings).compile(); + } + + /** + * Refuses a key under the platform's own sections that nothing binds. + * + *

    Registered here rather than left to review because a misspelt key inside a bound section is + * silent in a way a misspelt prefix is not: the section binds, the entry appears, and the one + * setting the operator came to change is the one that did not take. + * + * @param environment the resolved environment + * @return the key validator + */ + @Bean + @ConditionalOnMissingBean + public MessagingConfigurationKeyValidator messagingConfigurationKeyValidator( + org.springframework.core.env.ConfigurableEnvironment environment) { + return new MessagingConfigurationKeyValidator(environment); + } + + /** + * Validates every registered destination profile at startup. + * + *

    Returns a marker rather than a service because its only job is to fail the context when the + * registry is inconsistent. + * + *

    Configured profiles and bean-declared ones are merged before validation, not after, because + * the rules that matter most across a registry — a dead letter destination that exists, a retry + * graph with no cycle — are questions about the whole set. Validating the two halves separately + * would pass a deployment whose configured destination dead-letters into a bean-declared one that + * points back. + * + * @param profiles the profiles the application declared as beans + * @param compiled the profiles compiled from configuration + * @return the validated registry + */ + @Bean + @ConditionalOnMissingBean + public ValidatedDestinationRegistry validatedDestinationRegistry( + ObjectProvider profiles, CompiledMessagingConfiguration compiled) { + List registered = + java.util.stream.Stream.concat(compiled.destinations().stream(), profiles.orderedStream()) + .toList(); + new DestinationProfileValidator().validateAll(registered); + return new ValidatedDestinationRegistry(registered); + } + + /** + * Returns the destination profile validator. + * + * @return the validator + */ + @Bean + @ConditionalOnMissingBean + public DestinationProfileValidator destinationProfileValidator() { + return new DestinationProfileValidator(); + } + + /** + * Returns the broker security validator. + * + * @return the validator + */ + @Bean + @ConditionalOnMissingBean + public MessageSecurityValidator messageSecurityValidator() { + return new MessageSecurityValidator(); + } + + /** + * Runs the security validator over every broker security profile, declared or configured. + * + * @param profiles the security profiles the application declared as beans + * @param compiled the security profiles compiled from configuration + * @param validator the security validator + * @return the startup validation + */ + @Bean + public StartupProfileValidation + brokerSecurityStartupValidation( + ObjectProvider profiles, + CompiledMessagingConfiguration compiled, + MessageSecurityValidator validator) { + return new StartupProfileValidation<>( + () -> + java.util.stream.Stream.concat(compiled.security().stream(), profiles.orderedStream()) + .toList(), + validator::validate); + } + + /** + * Returns the credential runtime registry, when the application supplied a credential provider. + * + *

    Conditional on the provider because there is no safe default: a registry with no source + * would resolve nothing, and the adapters would connect with whatever their client's defaults + * are. + * + * @param provider the application's credential source + * @return the credential runtime registry + */ + @Bean + @ConditionalOnBean(CredentialProvider.class) + @ConditionalOnMissingBean + public CredentialRuntimeRegistry credentialRuntimeRegistry(CredentialProvider provider) { + return new CredentialRuntimeRegistry(provider); + } + + /** + * Returns the transport security policy. + * + * @return the TLS policy + */ + @Bean + @ConditionalOnMissingBean + public BrokerTlsPolicy brokerTlsPolicy() { + return new BrokerTlsPolicy(); + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/PublishResults.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/PublishResults.java deleted file mode 100644 index 4dd0178..0000000 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/PublishResults.java +++ /dev/null @@ -1,71 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessagingException; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.time.Duration; -import java.util.Optional; -import java.util.concurrent.CompletionException; - -/** - * Converts an exception that escaped a publish into a result the caller can act on. - * - *

    A {@link MessagingException} already carries a sanitized descriptor, including whether it is - * retryable, so its own classification is used. Anything else is reported {@code AMBIGUOUS}: an - * unrecognised exception proves nothing about whether the message reached the broker, and the - * conservative reading is the one that does not turn a lost confirmation into a duplicate. - */ -final class PublishResults { - - private PublishResults() {} - - /** - * Converts a failure into a publish result. - * - * @param failure the exception that escaped the publish - * @return the equivalent result - */ - static PublishResult from(Throwable failure) { - Throwable cause = failure instanceof CompletionException ? failure.getCause() : failure; - - if (cause instanceof MessagingException messagingFailure) { - FailureDescriptor descriptor = messagingFailure.failure(); - boolean transmitted = descriptor.category() == FailureCategory.AMBIGUOUS; - return new PublishResult( - transmitted ? PublishCompletion.AMBIGUOUS : PublishCompletion.REJECTED, - transmitted - ? new PublishEvidence( - true, - TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, - false, - ConfirmationLevel.NONE) - : PublishEvidence.notTransmitted(), - transmitted ? RoutingOutcome.UNKNOWN : RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of(descriptor)); - } - - return new PublishResult( - PublishCompletion.AMBIGUOUS, - new PublishEvidence( - true, TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "UNCLASSIFIED_PUBLISH_FAILURE", - "the publish failed with an unrecognised error: " - + (cause == null ? "unknown" : cause.getClass().getSimpleName())))); - } -} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/ReactiveMessagePublisher.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/ReactiveMessagePublisher.java deleted file mode 100644 index f59bdf1..0000000 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/ReactiveMessagePublisher.java +++ /dev/null @@ -1,32 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import reactor.core.publisher.Mono; - -/** - * The Reactor facade over the asynchronous publisher. - * - *

    Lives in the starter rather than the core so that {@code messaging-core-api} never depends on - * Reactor. A service that does not use Reactor should not inherit it through a messaging contract. - * - *

    The returned {@code Mono} is cold and cancellable: nothing is published until it is - * subscribed, and cancelling the subscription cancels the underlying stage rather than leaving an - * orphaned publish in flight. - */ -public interface ReactiveMessagePublisher { - - /** - * Publishes one message. - * - * @param the payload type - * @param destination the logical destination - * @param message the envelope to publish - * @param options per-call options - * @return a cold {@code Mono} emitting the outcome and its evidence - */ - Mono publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options); -} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BatchPublisherTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BatchPublisherTest.java deleted file mode 100644 index ee8855a..0000000 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BatchPublisherTest.java +++ /dev/null @@ -1,328 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.error.MessageTooLargeException; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.BatchPublishOptions; -import dev.caskeleton.messaging.api.publish.BatchPublishResult; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishRequest; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import org.junit.jupiter.api.Test; - -class BatchPublisherTest { - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - - private static PublishResult rejected() { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of( - dev.caskeleton.messaging.api.error.FailureDescriptor.of( - dev.caskeleton.messaging.api.error.FailureCategory.PERMANENT_BUSINESS, - "BROKER_REJECTED", - "the broker refused the message"))); - } - - private static PublishResult confirmed() { - return new PublishResult( - PublishCompletion.CONFIRMED, - new PublishEvidence( - true, - TransmissionEvidence.TRANSMITTED, - true, - ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK), - RoutingOutcome.ROUTED, - Optional.empty(), - 1, - Duration.ZERO, - Optional.empty()); - } - - /** Publishes according to a per-index script, so partial failures are reproducible. */ - private static MessagePublisher scripted(List script, List calls) { - return new MessagePublisher() { - private int index; - - @Override - public CompletionStage publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options) { - int current = index++; - calls.add(current); - return switch (script.get(current)) { - case CONFIRM -> CompletableFuture.completedFuture(confirmed()); - case FAIL_ASYNC -> - CompletableFuture.failedFuture( - new MessageTooLargeException("PAYLOAD_LIMIT_EXCEEDED", "too large")); - case THROW_SYNC -> - throw new MessageTooLargeException("PAYLOAD_LIMIT_EXCEEDED", "too large"); - }; - } - }; - } - - private enum Outcome { - CONFIRM, - FAIL_ASYNC, - THROW_SYNC - } - - private static PublishRequest request() { - MessageDestination destination = - new MessageDestination<>( - new DestinationName("orders.v1"), new MessageType("order.created"), String.class); - MessageEnvelope envelope = - new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - "{}"); - return PublishRequest.of(destination, envelope); - } - - private static List> requests(int count) { - List> requests = new ArrayList<>(); - for (int index = 0; index < count; index++) { - requests.add(request()); - } - return requests; - } - - private static BatchPublishResult publish(List script, BatchPublishOptions options) { - List calls = new ArrayList<>(); - return new DefaultBatchMessagePublisher(scripted(script, calls), () -> NOW) - .publish(requests(script.size()), options) - .toCompletableFuture() - .join(); - } - - @Test - void closingTheBatchPublisherStopsItsDeadlineScheduler() { - var deadlines = new java.util.concurrent.ScheduledThreadPoolExecutor(1); - var publisher = - new DefaultBatchMessagePublisher( - scripted(List.of(Outcome.CONFIRM), new ArrayList<>()), () -> NOW, deadlines); - - publisher.close(); - - assertThat(deadlines.isShutdown()).isTrue(); - } - - @Test - void everyEntryGetsItsOwnResultKeyedByIndex() { - BatchPublishResult result = - publish( - List.of(Outcome.CONFIRM, Outcome.CONFIRM, Outcome.CONFIRM), - BatchPublishOptions.defaults()); - - assertThat(result.items()).hasSize(3); - assertThat(result.items()).extracting(item -> item.index()).containsExactly(0, 1, 2); - assertThat(result.allConfirmed()).isTrue(); - } - - @Test - void oneFailedEntryDoesNotHideTheOthers() { - BatchPublishResult result = - publish( - List.of(Outcome.CONFIRM, Outcome.FAIL_ASYNC, Outcome.CONFIRM), - BatchPublishOptions.defaults()); - - assertThat(result.withCompletion(PublishCompletion.CONFIRMED)).hasSize(2); - assertThat(result.withCompletion(PublishCompletion.REJECTED)) - .as("the caller needs to know exactly which entries to resubmit") - .hasSize(1); - } - - @Test - void aSynchronousFailureStillProducesAnEntryResult() { - BatchPublishResult result = - publish(List.of(Outcome.THROW_SYNC, Outcome.CONFIRM), BatchPublishOptions.defaults()); - - assertThat(result.items()).hasSize(2); - assertThat(result.items().get(0).result().completion()).isEqualTo(PublishCompletion.REJECTED); - } - - @Test - void aRejectedEntryKeepsItsSanitizedFailureCode() { - BatchPublishResult result = - publish(List.of(Outcome.FAIL_ASYNC), BatchPublishOptions.defaults()); - - assertThat(result.items().get(0).result().failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("PAYLOAD_LIMIT_EXCEEDED")); - } - - @Test - void aBatchLargerThanItsLimitIsRefusedBeforeAnythingIsPublished() { - List calls = new ArrayList<>(); - DefaultBatchMessagePublisher publisher = - new DefaultBatchMessagePublisher(scripted(List.of(Outcome.CONFIRM), calls), () -> NOW); - - assertThatThrownBy( - () -> - publisher.publish( - requests(5), new BatchPublishOptions(Duration.ofSeconds(5), 2, false))) - .isInstanceOf(MessageTooLargeException.class); - assertThat(calls).isEmpty(); - } - - @Test - void stopOnFirstRejectionSkipsTheRemainingSubmissions() { - List calls = new ArrayList<>(); - BatchPublishResult result = - new DefaultBatchMessagePublisher( - scripted( - List.of(Outcome.CONFIRM, Outcome.THROW_SYNC, Outcome.CONFIRM, Outcome.CONFIRM), - calls), - () -> NOW) - .publish(requests(4), new BatchPublishOptions(Duration.ofSeconds(5), 100, true)) - .toCompletableFuture() - .join(); - - assertThat(calls).containsExactly(0, 1); - assertThat(result.items()) - .as("truncating the list left the caller unable to tell 'not submitted' from 'lost'") - .hasSize(4); - assertThat(result.items().get(3).result().failure().orElseThrow().code()) - .isEqualTo("NOT_SUBMITTED_AFTER_REJECTION"); - } - - @Test - @org.junit.jupiter.api.DisplayName( - "the batch deadline completes outstanding entries as ambiguous") - void theBatchDeadlineCompletesOutstandingEntries() { - BatchPublishResult result = - new DefaultBatchMessagePublisher( - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - return new CompletableFuture<>(); - } - }, - () -> NOW) - .publish(requests(2), new BatchPublishOptions(Duration.ofMillis(50), 100, false)) - .toCompletableFuture() - .join(); - - assertThat(result.items()) - .as("options.timeout() was never read, so a publisher that never completes hung forever") - .hasSize(2); - assertThat(result.items()) - .allSatisfy( - item -> - assertThat(item.result().completion()) - .isEqualTo(dev.caskeleton.messaging.api.publish.PublishCompletion.AMBIGUOUS)); - } - - @Test - @org.junit.jupiter.api.DisplayName("a delayed rejection still stops the entries after it") - void aDelayedRejectionStopsLaterEntries() { - List calls = new ArrayList<>(); - java.util.concurrent.CompletableFuture - delayed = new java.util.concurrent.CompletableFuture<>(); - - BatchPublishResult result = - new DefaultBatchMessagePublisher( - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - int index = calls.size(); - calls.add(index); - if (index == 0) { - // Resolves only after the publisher returns, which is what an asynchronous - // broker rejection looks like. The old loop only saw an already-completed - // failure, so every later entry was submitted anyway. - // Completed on another thread so the rejection lands after this method - // returns — the shape an asynchronous broker rejection actually has. - new Thread(() -> delayed.complete(rejected())).start(); - return delayed; - } - return CompletableFuture.completedFuture(confirmed()); - } - }, - () -> NOW) - .publish(requests(3), new BatchPublishOptions(Duration.ofSeconds(2), 100, true)) - .toCompletableFuture() - .join(); - - assertThat(calls).containsExactly(0); - assertThat(result.items()).hasSize(3); - assertThat(result.items().get(2).result().failure().orElseThrow().code()) - .isEqualTo("NOT_SUBMITTED_AFTER_REJECTION"); - } - - @Test - void withoutStopOnFirstRejectionEveryEntryIsAttempted() { - List calls = new ArrayList<>(); - new DefaultBatchMessagePublisher( - scripted(List.of(Outcome.THROW_SYNC, Outcome.CONFIRM, Outcome.CONFIRM), calls), - () -> NOW) - .publish(requests(3), BatchPublishOptions.defaults()) - .toCompletableFuture() - .join(); - - assertThat(calls).containsExactly(0, 1, 2); - } - - @Test - void aBatchIsNotATransactionSoConfirmedEntriesStayConfirmed() { - BatchPublishResult result = - publish(List.of(Outcome.CONFIRM, Outcome.FAIL_ASYNC), BatchPublishOptions.defaults()); - - assertThat(result.items().get(0).result().completion()) - .as("nothing can un-publish an entry the broker already confirmed") - .isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.allConfirmed()).isFalse(); - } - - @Test - void theDefaultOptionsAttemptEveryEntry() { - assertThat(BatchPublishOptions.defaults().stopOnFirstRejection()).isFalse(); - } -} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BlockingFacadeTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BlockingFacadeTest.java deleted file mode 100644 index 0b60b6e..0000000 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BlockingFacadeTest.java +++ /dev/null @@ -1,148 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessagePublishTimeoutException; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.BatchPublishItemResult; -import dev.caskeleton.messaging.api.publish.BatchPublishResult; -import dev.caskeleton.messaging.api.publish.BlockingMessagePublisher; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import java.time.Duration; -import java.time.Instant; -import java.util.List; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import org.junit.jupiter.api.Test; - -class BlockingFacadeTest { - - private static final MessageDestination ORDERS = - new MessageDestination<>( - new DestinationName("order-events"), new MessageType("order.created"), String.class); - - @Test - void aConfirmedPublishIsReturnedDirectly() { - BlockingMessagePublisher publisher = - new DefaultBlockingMessagePublisher( - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - return CompletableFuture.completedFuture(confirmed()); - } - }); - - PublishResult result = publisher.publish(ORDERS, envelope(), PublishOptions.defaults()); - - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - } - - @Test - void aStalledPublishTimesOutAsAmbiguousRatherThanBlockingForever() { - MessagePublisher stalled = - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - return new CompletableFuture<>(); - } - }; - PublishOptions shortDeadline = - new PublishOptions( - Duration.ofMillis(50), - dev.caskeleton.messaging.api.destination.ConfirmationRequirement - .REPLICATION_OR_PERSISTENCE_ACK, - Optional.empty()); - - assertThatThrownBy( - () -> - new DefaultBlockingMessagePublisher(stalled) - .publish(ORDERS, envelope(), shortDeadline)) - .isInstanceOf(MessagePublishTimeoutException.class) - .satisfies( - thrown -> - assertThat(((MessagePublishTimeoutException) thrown).category()) - .isEqualTo(FailureCategory.AMBIGUOUS)); - } - - @Test - void aBatchKeepsPerItemOutcomes() { - BatchPublishResult batch = - new BatchPublishResult( - List.of( - new BatchPublishItemResult(0, MessageId.newId(), confirmed()), - new BatchPublishItemResult(1, MessageId.newId(), ambiguous())), - Duration.ofMillis(20)); - - assertThat(batch.allConfirmed()).isFalse(); - assertThat(batch.withCompletion(PublishCompletion.CONFIRMED)).hasSize(1); - assertThat(batch.withCompletion(PublishCompletion.AMBIGUOUS)).hasSize(1); - } - - private static MessageEnvelope envelope() { - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - Instant.parse("2026-08-10T09:15:00Z"), - Optional.of(Instant.parse("2026-08-10T09:15:00Z")), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - "payload"); - } - - private static PublishResult confirmed() { - return new PublishResult( - PublishCompletion.CONFIRMED, - PublishEvidence.confirmed(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ofMillis(3), - Optional.empty()); - } - - private static PublishResult ambiguous() { - return new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - Duration.ofSeconds(5), - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, "CONFIRM_TIMEOUT", "confirm timed out"))); - } -} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java index 643a088..0340c8a 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java @@ -90,6 +90,7 @@ class KafkaProducerConfigTest { private static Map producerConfig(AssertableApplicationContext context) { return KafkaMessagingAutoConfiguration.producerConfig( context.getBean(CompiledMessagingConfiguration.class), + context.getBean(KafkaBrokerCatalog.class), context.getBeanProvider(BrokerSecurityProfile.class), context.getBeanProvider(KafkaSecurityConfigurer.class)); } diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java index 98c4bde..cfca082 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java @@ -228,6 +228,7 @@ class KafkaProducerSecurityTest { private static Map producerConfig(AssertableApplicationContext context) { return KafkaMessagingAutoConfiguration.producerConfig( context.getBean(CompiledMessagingConfiguration.class), + context.getBean(KafkaBrokerCatalog.class), context.getBeanProvider(BrokerSecurityProfile.class), context.getBeanProvider(KafkaSecurityConfigurer.class)); } diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java index 0d4708c..d3c02ea 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java @@ -11,11 +11,10 @@ import dev.caskeleton.messaging.api.delivery.OrderingScope; import dev.caskeleton.messaging.api.destination.DestinationKind; import dev.caskeleton.messaging.api.destination.DestinationName; import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.publish.BlockingMessagePublisher; import dev.caskeleton.messaging.api.publish.MessagePublisher; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.ConsumerPolicy; import dev.caskeleton.messaging.policy.DeadLetterPolicy; import dev.caskeleton.messaging.policy.DestinationProfile; @@ -49,8 +48,6 @@ class MessagingAutoConfigurationTest { context -> assertThat(context) .hasSingleBean(ValidatedDestinationRegistry.class) - .hasSingleBean(BlockingMessagePublisher.class) - .hasSingleBean(ReactiveMessagePublisher.class) .hasSingleBean(MessagingAdmissionController.class) .hasSingleBean(GracefulShutdownCoordinator.class)); } @@ -300,7 +297,7 @@ class MessagingAutoConfigurationTest { retry, DeadLetterPolicy.to(new DestinationName("order-events-dlq")), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, false, false, false); diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java index c3239aa..f365f6e 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java @@ -68,8 +68,11 @@ class MessagingConfigurationBindingTest { CompiledMessagingConfiguration compiled = context.getBean(CompiledMessagingConfiguration.class); assertThat(compiled.brokers()).containsOnlyKeys("kafka-primary"); - BrokerSettings kafka = compiled.brokers().get("kafka-primary"); - assertThat(kafka.type()).isEqualTo(BrokerSettings.KAFKA); + assertThat(compiled.brokers().get("kafka-primary").type()) + .isEqualTo(BrokerSettings.KAFKA); + KafkaBrokerSettings kafka = + KafkaBrokerSettings.bindAll(context.getEnvironment(), compiled) + .get("kafka-primary"); assertThat(kafka.bootstrapServers()) .containsExactly("broker-1:9093", "broker-2:9093"); assertThat(kafka.enableAutoCommit()).isFalse(); @@ -253,6 +256,26 @@ class MessagingConfigurationBindingTest { .hasMessageContaining("TRUST_ME")); } + @Test + @org.junit.jupiter.api.DisplayName("주소가 없는 Kafka broker는 Kafka starter가 그 키를 이름으로 대며 기동을 거부한다") + void aKafkaBrokerWithoutAnAddressIsRefusedByTheKafkaStarterNamingTheKey() { + // given + ApplicationContextRunner kafka = + runner + .withConfiguration(AutoConfigurations.of(KafkaMessagingAutoConfiguration.class)) + .withPropertyValues("app.messaging.brokers.kafka-primary.type=kafka"); + + // when + kafka.run( + context -> + // then + assertThat(context) + .hasFailed() + .getFailure() + .rootCause() + .hasMessageContaining("app.messaging.brokers.kafka-primary.bootstrap-servers")); + } + @Test void aProductionKafkaBrokerWithoutTransportSecurityFailsStartup() { runner diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfigurationTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfigurationTest.java new file mode 100644 index 0000000..b8471f8 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfigurationTest.java @@ -0,0 +1,117 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionStage; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; + +/** The consume pipeline turns on with a handler bean and refuses what it cannot honour. */ +class MessagingConsumeConfigurationTest { + + private final ApplicationContextRunner runner = + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingCoreAutoConfiguration.class)) + .withUserConfiguration(MessagingAutoConfigurationTest.PublisherConfiguration.class) + .withPropertyValues( + "app.messaging.brokers.kafka-primary.type=kafka", + "app.messaging.brokers.kafka-primary.bootstrap-servers[0]=broker-1:9093", + "app.messaging.destinations.order-events.broker=kafka-primary", + "app.messaging.destinations.order-events.kind=EVENT_STREAM", + "app.messaging.destinations.order-events.physical.topic=order.events.v1", + "app.messaging.destinations.order-events.schema.message-types[0]=order.created", + "app.messaging.destinations.order-events.guarantees.delivery=AT_MOST_ONCE"); + + @Test + @DisplayName("handler bean이 없으면 consumer를 만들지 않는다") + void withoutAHandlerBeanNoConsumerIsAssembled() { + // when + runner.run( + context -> + // then + assertThat(context).hasNotFailed().doesNotHaveBean(MessageConsumers.class)); + } + + @Test + @DisplayName("handler bean이 있으면 그 destination의 consumer가 group과 함께 조립된다") + void aHandlerBeanAssemblesAConsumerForItsDestinationWithItsGroup() { + // when + runner + .withPropertyValues("app.messaging.destinations.order-events.consumer.group=projection") + .withBean("ordersHandler", MessageHandler.class, MessagingConsumeConfigurationTest::handler) + .run( + context -> { + // then + assertThat(context).hasNotFailed(); + assertThat(context.getBean(MessageConsumers.class).consumers()) + .singleElement() + .satisfies( + consumer -> { + assertThat(consumer.profile().name().value()).isEqualTo("order-events"); + assertThat(consumer.group()).isEqualTo("projection"); + }); + }); + } + + @Test + @DisplayName("consumer.group이 없는 destination을 소비하는 handler는 기동을 거부한다") + void aHandlerForADestinationWithoutAConsumerGroupFailsStartup() { + // when + runner + .withBean("ordersHandler", MessageHandler.class, MessagingConsumeConfigurationTest::handler) + .run( + context -> + // then + assertThat(context) + .hasFailed() + .getFailure() + .rootCause() + .hasMessageContaining("order-events") + .hasMessageContaining("consumer.group")); + } + + @Test + @DisplayName("INBOX_TRANSACTIONAL destination에 IdempotentConsumer가 없으면 기동을 거부한다") + void anInboxDestinationWithoutAnIdempotentConsumerFailsStartup() { + // when + runner + .withPropertyValues( + "app.messaging.destinations.order-events.consumer.group=projection", + "app.messaging.destinations.order-events.guarantees.external-side-effect=" + + "INBOX_TRANSACTIONAL") + .withBean("ordersHandler", MessageHandler.class, MessagingConsumeConfigurationTest::handler) + .run( + context -> + // then + assertThat(context) + .hasFailed() + .getFailure() + .rootCause() + .hasMessageContaining("INBOX_TRANSACTIONAL") + .hasMessageContaining("IdempotentConsumer")); + } + + private static MessageHandler handler() { + return new MessageHandler<>() { + @Override + public MessageDestination destination() { + return new MessageDestination<>( + new DestinationName("order-events"), new MessageType("order.created"), String.class); + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + return CompletableFuture.completedFuture(HandleResult.success()); + } + }; + } +} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingEndpointTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingEndpointTest.java index d69b9f2..75c1bfa 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingEndpointTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingEndpointTest.java @@ -9,7 +9,7 @@ import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; import dev.caskeleton.messaging.api.delivery.OrderingScope; import dev.caskeleton.messaging.api.destination.DestinationKind; import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.ConsumerPolicy; import dev.caskeleton.messaging.policy.DeadLetterPolicy; import dev.caskeleton.messaging.policy.DestinationProfile; @@ -47,7 +47,7 @@ class MessagingEndpointTest { RetryPolicy.none(), DeadLetterPolicy.to(new DestinationName("orders.v1.dlq")), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, true, true, false); diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java deleted file mode 100644 index 91e82fe..0000000 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java +++ /dev/null @@ -1,162 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import static org.assertj.core.api.Assertions.assertThat; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import java.time.Duration; -import java.time.Instant; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicInteger; -import org.junit.jupiter.api.Test; -import reactor.core.publisher.Mono; -import reactor.test.StepVerifier; - -class ReactiveFacadeTest { - - private static final MessageDestination ORDERS = - new MessageDestination<>( - new DestinationName("order-events"), new MessageType("order.created"), String.class); - - private final AtomicInteger publishes = new AtomicInteger(); - - @Test - void nothingIsPublishedUntilTheMonoIsSubscribed() { - ReactiveMessagePublisher publisher = new DefaultReactiveMessagePublisher(counting()); - - Mono mono = publisher.publish(ORDERS, envelope(), PublishOptions.defaults()); - - assertThat(publishes.get()).as("assembly must not publish").isZero(); - - mono.block(); - - assertThat(publishes.get()).isEqualTo(1); - } - - @Test - void eachSubscriptionPublishesOnce() { - ReactiveMessagePublisher publisher = new DefaultReactiveMessagePublisher(counting()); - - Mono mono = publisher.publish(ORDERS, envelope(), PublishOptions.defaults()); - mono.block(); - mono.block(); - - assertThat(publishes.get()).isEqualTo(2); - } - - @Test - void theOutcomeIsEmittedAndTheMonoCompletes() { - ReactiveMessagePublisher publisher = new DefaultReactiveMessagePublisher(counting()); - - StepVerifier.create(publisher.publish(ORDERS, envelope(), PublishOptions.defaults())) - .assertNext( - result -> assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED)) - .verifyComplete(); - } - - @Test - void aFailedPublishPropagatesAsAnErrorSignal() { - MessagePublisher failing = - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - return CompletableFuture.failedFuture(new IllegalStateException("broker down")); - } - }; - - StepVerifier.create( - new DefaultReactiveMessagePublisher(failing) - .publish(ORDERS, envelope(), PublishOptions.defaults())) - .expectErrorMessage("broker down") - .verify(); - } - - @Test - void theBlockingPublishRunsOffTheSubscribingThread() { - // given - java.util.concurrent.atomic.AtomicReference publishedOn = - new java.util.concurrent.atomic.AtomicReference<>(); - MessagePublisher recording = - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - publishedOn.set(Thread.currentThread()); - return CompletableFuture.completedFuture(confirmed()); - } - }; - - // when - new DefaultReactiveMessagePublisher(recording) - .publish(ORDERS, envelope(), PublishOptions.defaults()) - .block(); - - // then - assertThat(publishedOn.get()) - .as("admission and the producer's metadata wait block; an event loop must not pay for it") - .isNotSameAs(Thread.currentThread()); - } - - private MessagePublisher counting() { - return new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options) { - publishes.incrementAndGet(); - return CompletableFuture.completedFuture(confirmed()); - } - }; - } - - private static PublishResult confirmed() { - return new PublishResult( - PublishCompletion.CONFIRMED, - PublishEvidence.confirmed(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ofMillis(2), - Optional.empty()); - } - - private static MessageEnvelope envelope() { - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - Instant.parse("2026-08-10T09:15:00Z"), - Optional.of(Instant.parse("2026-08-10T09:15:00Z")), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - "payload"); - } -} diff --git a/messaging/messaging-testkit/build.gradle b/messaging/messaging-testkit/build.gradle index a7492ce..0b4b657 100644 --- a/messaging/messaging-testkit/build.gradle +++ b/messaging/messaging-testkit/build.gradle @@ -10,7 +10,6 @@ jmh { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') api project(':messaging:messaging-policy') api project(':messaging:messaging-transport-spi') diff --git a/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java b/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java index 42d4271..482d9b4 100644 --- a/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java +++ b/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java @@ -5,22 +5,22 @@ import java.util.Map; import java.util.Objects; /** - * The certified broker versions and support tier for every adapter. + * The certified broker versions and support grade for every adapter. * *

    Held as code with a test over it rather than as a document, because a support matrix that * lives only in prose drifts the first time an adapter is added. The test asserts the rules the * design fixes: a Stable adapter must run the shared contract and be certified against a live * broker, and an Experimental adapter must be disabled by default. * - *

    The certification claim is derived from lane evidence rather than declared beside the tier, so - * an adapter cannot be promoted by editing a boolean. + *

    The certification claim is derived from lane evidence rather than declared beside the grade, + * so an adapter cannot be promoted by editing a boolean. */ public final class CompatibilityMatrix { private CompatibilityMatrix() {} - /** The support tier an adapter has earned. */ - public enum Tier { + /** The support grade an adapter has earned. */ + public enum SupportGrade { /** Passes the shared adapter contract unchanged. */ STABLE, @@ -36,14 +36,14 @@ public final class CompatibilityMatrix { * * @param adapter the module name * @param brokerVersions the broker versions the adapter is certified against - * @param tier the support tier + * @param grade the support grade * @param runsSharedContract whether the module runs {@link MessagingAdapterContract} * @param enabledByDefault whether the adapter is active without an explicit opt-in */ public record Entry( String adapter, List brokerVersions, - Tier tier, + SupportGrade grade, boolean runsSharedContract, boolean enabledByDefault) { @@ -51,7 +51,7 @@ public final class CompatibilityMatrix { * Reports whether a lane has run this adapter against a real broker. * *

    Read from the evidence rather than declared. As a field it was a boolean an author set - * next to the tier, and an Experimental adapter carried {@code true} while no fault scenario + * next to the grade, and an Experimental adapter carried {@code true} while no fault scenario * had ever been executed against it — the same shape as the coverage matrix that used to assert * its own constants. * @@ -63,7 +63,7 @@ public final class CompatibilityMatrix { } public Entry { - Objects.requireNonNull(tier, "tier must not be null"); + Objects.requireNonNull(grade, "grade must not be null"); Objects.requireNonNull(brokerVersions, "brokerVersions must not be null"); if (adapter == null || adapter.isBlank()) { throw new IllegalArgumentException("adapter must not be blank"); @@ -82,7 +82,7 @@ public final class CompatibilityMatrix { // the locked client is 4.1.1, so the matrix and the support document agreed with each // other and with nothing that had executed. A version returns here when a lane runs // against its image. - new Entry("messaging-kafka", List.of("4.1"), Tier.STABLE, true, true)); + new Entry("messaging-kafka", List.of("4.1"), SupportGrade.STABLE, true, true)); /** * Returns every certified adapter. diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java index 7cccaf2..2bff244 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java @@ -23,7 +23,7 @@ class CompatibilityMatrixTest { @Test void everyStableAdapterRunsTheSharedContract() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .isNotEmpty() .allSatisfy(entry -> assertThat(entry.runsSharedContract()).isTrue()); } @@ -31,14 +31,14 @@ class CompatibilityMatrixTest { @Test void noExperimentalAdapterIsEnabledByDefault() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.EXPERIMENTAL) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.EXPERIMENTAL) .allSatisfy(entry -> assertThat(entry.enabledByDefault()).isFalse()); } @Test void theStableSetIsExactlyWhatALaneHasCertified() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .extracting(CompatibilityMatrix.Entry::adapter) .containsExactly("messaging-kafka"); } @@ -59,7 +59,7 @@ class CompatibilityMatrixTest { @Test void everyStableAdapterHasOnlyExplicitlyApprovedEvidenceGaps() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .allSatisfy( entry -> assertThat(CertifiedEvidence.approvedGaps(entry.adapter())) @@ -98,14 +98,14 @@ class CompatibilityMatrixTest { assertThatThrownBy( () -> new CompatibilityMatrix.Entry( - "messaging-x", List.of(), CompatibilityMatrix.Tier.STABLE, true, true)) + "messaging-x", List.of(), CompatibilityMatrix.SupportGrade.STABLE, true, true)) .isInstanceOf(IllegalArgumentException.class); } @Test void everyStableAdapterIsCertifiedAgainstALiveBroker() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .isNotEmpty() .allSatisfy( entry -> @@ -119,7 +119,7 @@ class CompatibilityMatrixTest { @Test void noExperimentalAdapterClaimsLiveBrokerCertification() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.EXPERIMENTAL) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.EXPERIMENTAL) .allSatisfy(entry -> assertThat(entry.hasLiveBrokerCertification()).isFalse()); } @@ -127,7 +127,11 @@ class CompatibilityMatrixTest { void anAdapterWithoutEvidenceCannotBeCalledStable() { CompatibilityMatrix.Entry undeclared = new CompatibilityMatrix.Entry( - "messaging-artemis", List.of("2.42"), CompatibilityMatrix.Tier.STABLE, true, false); + "messaging-artemis", + List.of("2.42"), + CompatibilityMatrix.SupportGrade.STABLE, + true, + false); assertThat(undeclared.hasLiveBrokerCertification()) .as("promoting an adapter used to be a boolean an author could set beside the tier") diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java index ff0b29b..be044b0 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java @@ -35,7 +35,7 @@ class CrossBrokerContractSuite { private static List stableAdapters() { return CompatibilityMatrix.entries().values().stream() - .filter(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filter(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .map(CompatibilityMatrix.Entry::adapter) .toList(); } @@ -88,7 +88,7 @@ class CrossBrokerContractSuite { void noExperimentalAdapterClaimsLiveBrokerFaultCoverage() { List experimental = CompatibilityMatrix.entries().values().stream() - .filter(entry -> entry.tier() == CompatibilityMatrix.Tier.EXPERIMENTAL) + .filter(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.EXPERIMENTAL) .map(CompatibilityMatrix.Entry::adapter) .toList(); diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java index e97c8ac..cc5755b 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java @@ -62,7 +62,7 @@ class MessagingDocumentationContractTest { String matrix = read("support-matrix.md"); CompatibilityMatrix.entries().values().stream() - .filter(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filter(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .forEach(entry -> assertThat(matrix).contains(brokerLabel(entry.adapter()))); } @@ -71,7 +71,7 @@ class MessagingDocumentationContractTest { String matrix = read("support-matrix.md"); CompatibilityMatrix.entries().values().stream() - .filter(entry -> entry.tier() == CompatibilityMatrix.Tier.EXPERIMENTAL) + .filter(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.EXPERIMENTAL) .forEach( entry -> assertThat(matrix) diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/contract/MessagingPublicSurfaceContractTest.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/contract/MessagingPublicSurfaceContractTest.java index 8cd6a3c..00756f7 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/contract/MessagingPublicSurfaceContractTest.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/contract/MessagingPublicSurfaceContractTest.java @@ -96,7 +96,7 @@ class MessagingPublicSurfaceContractTest { assertThat(leaves) .as("a scan that found no leaf would report every violation as absent") - .hasSizeGreaterThan(20); + .hasSizeGreaterThan(15); assertThat( exposedLibraries( leaves.stream().filter(isNamed("messaging-kafka")).findFirst().orElseThrow())) @@ -123,8 +123,8 @@ class MessagingPublicSurfaceContractTest { assertThat(reachable) .as( "an empty closure would satisfy the assertion below without proving anything; the " - + "metrics registry and the reactive facade are exported on purpose") - .contains("io.micrometer:micrometer-core", "io.projectreactor:reactor-core"); + + "metrics registry is exported on purpose (the reactive facade was deleted, S7-#9c)") + .contains("io.micrometer:micrometer-core"); assertThat(reachable) .as( "both adapters were exported, so every adopter compiled against two broker SDKs at " diff --git a/messaging/messaging-transport-spi/build.gradle b/messaging/messaging-transport-spi/build.gradle index 81e3e70..55c8be5 100644 --- a/messaging/messaging-transport-spi/build.gradle +++ b/messaging/messaging-transport-spi/build.gradle @@ -4,6 +4,5 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') api project(':messaging:messaging-policy') } diff --git a/messaging/messaging-transport-spi/src/main/java/dev/caskeleton/messaging/transport/MessagingLifecycle.java b/messaging/messaging-transport-spi/src/main/java/dev/caskeleton/messaging/transport/MessagingLifecycle.java deleted file mode 100644 index e45aa2b..0000000 --- a/messaging/messaging-transport-spi/src/main/java/dev/caskeleton/messaging/transport/MessagingLifecycle.java +++ /dev/null @@ -1,67 +0,0 @@ -package dev.caskeleton.messaging.transport; - -import java.time.Duration; - -/** - * The ordered lifecycle every messaging runtime implements. - * - *

    The order in {@link ShutdownPhase} is the contract, not an implementation detail. Closing - * connections before settlements have been transmitted loses the settlements, and pausing consumers - * after draining lets fresh deliveries arrive into a runtime that is already shutting down. Each - * adapter implements the phases; none of them chooses the order. - * - *

    Implementations are driven by the Spring lifecycle rather than a JVM shutdown hook alone. A - * shutdown hook runs after the context has already begun disposing beans, so a handler mid-drain - * can find its datasource closed underneath it. - */ -public interface MessagingLifecycle { - - /** The fixed phases of an orderly shutdown, in the order they must run. */ - enum ShutdownPhase { - /** Refuse new publishes. Nothing in flight is affected. */ - STOP_PUBLISH_ADMISSION, - /** Refuse to start new delivery handlers. */ - STOP_NEW_HANDLERS, - /** Ask the broker to stop delivering. */ - PAUSE_CONSUMERS, - /** Let running handlers finish. */ - DRAIN_HANDLERS, - /** Transmit the settlements those handlers produced. */ - FLUSH_SETTLEMENTS, - /** Wait for outstanding producer confirms so publishes are not left ambiguous. */ - AWAIT_PRODUCER_CONFIRMS, - /** Return outbox leases so another relay can claim the rows immediately. */ - RELEASE_OUTBOX_LEASES, - /** Close connections and channels. */ - CLOSE_CONNECTIONS - } - - /** The default drain budget from the design. */ - Duration DEFAULT_DRAIN_DEADLINE = Duration.ofSeconds(30); - - /** - * Starts the runtime. - * - * @throws IllegalStateException when the runtime was already started - */ - void start(); - - /** - * Runs the shutdown phases in order, bounded by the drain deadline. - * - *

    Work still running at the deadline is abandoned unsettled so the broker redelivers - * it. Unconfirmed publishes are recorded as ambiguous rather than as successes. - * - * @param drainDeadline how long in-flight work may take - * @return the phase that was still running when the deadline passed, or {@link - * ShutdownPhase#CLOSE_CONNECTIONS} when the shutdown completed - */ - ShutdownPhase shutdown(Duration drainDeadline); - - /** - * Reports whether the runtime is running and accepting work. - * - * @return true between {@link #start()} and the first shutdown phase - */ - boolean isRunning(); -} diff --git a/messaging/messaging-transport-spi/src/test/java/dev/caskeleton/messaging/transport/MessagingLifecycleTest.java b/messaging/messaging-transport-spi/src/test/java/dev/caskeleton/messaging/transport/MessagingLifecycleTest.java deleted file mode 100644 index beabf82..0000000 --- a/messaging/messaging-transport-spi/src/test/java/dev/caskeleton/messaging/transport/MessagingLifecycleTest.java +++ /dev/null @@ -1,59 +0,0 @@ -package dev.caskeleton.messaging.transport; - -import static org.assertj.core.api.Assertions.assertThat; - -import dev.caskeleton.messaging.transport.MessagingLifecycle.ShutdownPhase; -import java.time.Duration; -import java.util.List; -import org.junit.jupiter.api.Test; - -class MessagingLifecycleTest { - - @Test - void publishAdmissionClosesBeforeConsumersArePaused() { - List order = List.of(ShutdownPhase.values()); - - assertThat(order.indexOf(ShutdownPhase.STOP_PUBLISH_ADMISSION)) - .isLessThan(order.indexOf(ShutdownPhase.PAUSE_CONSUMERS)); - } - - @Test - void handlersDrainBeforeTheirSettlementsAreFlushed() { - List order = List.of(ShutdownPhase.values()); - - assertThat(order.indexOf(ShutdownPhase.DRAIN_HANDLERS)) - .as("flushing before the handlers finish would lose the settlements they produce") - .isLessThan(order.indexOf(ShutdownPhase.FLUSH_SETTLEMENTS)); - } - - @Test - void connectionsCloseLastOfAll() { - ShutdownPhase[] order = ShutdownPhase.values(); - - assertThat(order[order.length - 1]) - .as("every earlier phase needs the connection that this one closes") - .isEqualTo(ShutdownPhase.CLOSE_CONNECTIONS); - } - - @Test - void producerConfirmsAreAwaitedBeforeTheConnectionGoesAway() { - List order = List.of(ShutdownPhase.values()); - - assertThat(order.indexOf(ShutdownPhase.AWAIT_PRODUCER_CONFIRMS)) - .as("a confirm that arrives after close cannot be observed, leaving the publish ambiguous") - .isLessThan(order.indexOf(ShutdownPhase.CLOSE_CONNECTIONS)); - } - - @Test - void outboxLeasesAreReleasedBeforeClosing() { - List order = List.of(ShutdownPhase.values()); - - assertThat(order.indexOf(ShutdownPhase.RELEASE_OUTBOX_LEASES)) - .isLessThan(order.indexOf(ShutdownPhase.CLOSE_CONNECTIONS)); - } - - @Test - void theDefaultDrainDeadlineMatchesTheDesign() { - assertThat(MessagingLifecycle.DEFAULT_DRAIN_DEADLINE).isEqualTo(Duration.ofSeconds(30)); - } -} From 5093193cb5f2dbd8a5c8f7d3bd82332052dbe7c9 Mon Sep 17 00:00:00 2001 From: donghyeon-ka Date: Tue, 29 Sep 2026 14:54:06 +0900 Subject: [PATCH 6/7] =?UTF-8?q?review-followup=20Wave=203=20=EB=9D=BC?= =?UTF-8?q?=EC=9A=B4=EB=93=9C=203(P28=C2=B7P29):=20fileserver=20use=20case?= =?UTF-8?q?=C2=B7file:*=20=EA=B6=8C=ED=95=9C,=20=EB=B3=B4=EC=95=88=20?= =?UTF-8?q?=ED=8C=8C=EC=9D=BC=20primitive=20=EA=B3=B5=EC=9A=A9=ED=99=94,?= =?UTF-8?q?=20jvm-test-suite?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 결정 32: fileserver transport 연산 21개를 `port.in` use case로, 쓰기·읽기에 `file:*` 권한(소유자·tenant 정책은 유지), inbound 서비스 이름 `*UseCase`, port 구현 이름 `` 규칙(S2-#13b, 24개) - 디렉터리 descriptor 파일 primitive를 `adapter:outbound:support`로(S4-lead-2 일부 — fileserver는 번역만), persistence-fileserver를 scan·클래스별 조건 대신 명시적 `@Bean` 설정으로(S2-#8a 일부), `FileVerifier`를 adapter로 - 호출자 0인 `poster.image-attached` 이벤트 삭제, SES SDK client를 레지스트리 종료 때 close, fileserver migration V1–V5 고정 - P29: auxiliary 레인 → Gradle `jvm-test-suite`(BUILD-O3-5, build-logic 16,768 → 5,851줄), redis-sdk가 runtime을 api로(P22-new-3) - 통합: JPA API surface 기준선 재승인(이름 규칙 변경), migration lane에 V19 반영, messaging 통합 테스트가 jar 안 migration 목록 조회(ORCH-new-5) - P28·P29 에이전트가 세션 재시작으로 중단 — 커밋된 것만 합치고 결과 파일은 오케스트레이터가 확정, S6-#10a는 P30으로 검증: ./gradlew ci verifyDependencyLocks --warning-mode=fail(714 task) · mainCheck architectureCheck verifyPublicPathSnapshot :app-bootstrap:verifyEnvKeys configContractCheck · :app-bootstrap:integrationTest :sample-portfolio:integrationTest :sample-portfolio:systemTest integrationCheck · -p build-logic test · -p optional-platforms ci · releaseCheck qualificationBootJar 통과 Co-Authored-By: Claude Opus 5.5 (1M context) --- messaging/messaging-kafka/build.gradle | 16 ++++++-- .../build.gradle | 15 ++++++-- .../messaging/outbox/OutboxPostgresIT.java | 38 ++++++++++++++++--- .../build.gradle | 15 ++++++-- 4 files changed, 66 insertions(+), 18 deletions(-) diff --git a/messaging/messaging-kafka/build.gradle b/messaging/messaging-kafka/build.gradle index 8f28164..00c9892 100644 --- a/messaging/messaging-kafka/build.gradle +++ b/messaging/messaging-kafka/build.gradle @@ -1,6 +1,5 @@ plugins { id 'java-test-fixtures' - id 'ca.auxiliary-source-set' id 'ca.platform-module' id 'me.champeau.jmh' } @@ -10,11 +9,20 @@ jmh { includeTests = true } -auxiliarySourceSets { - sourceSet('integrationTest') { - compilesAgainst 'main', 'testFixtures' +testing { + suites { + integrationTest(JvmTestSuite) { + dependencies { + implementation project() + implementation testFixtures(project()) + } + } } } +configurations { + integrationTestImplementation.extendsFrom testImplementation + integrationTestRuntimeOnly.extendsFrom testRuntimeOnly +} dependencies { // No spring-kafka. It was declared here and imported by nothing: this adapter drives the plain diff --git a/messaging/messaging-reliability-jdbc-postgresql/build.gradle b/messaging/messaging-reliability-jdbc-postgresql/build.gradle index cfa5217..f549b2d 100644 --- a/messaging/messaging-reliability-jdbc-postgresql/build.gradle +++ b/messaging/messaging-reliability-jdbc-postgresql/build.gradle @@ -1,13 +1,20 @@ plugins { - id 'ca.auxiliary-source-set' id 'ca.platform-module' } -auxiliarySourceSets { - sourceSet('integrationTest') { - compilesAgainst 'main' +testing { + suites { + integrationTest(JvmTestSuite) { + dependencies { + implementation project() + } + } } } +configurations { + integrationTestImplementation.extendsFrom testImplementation + integrationTestRuntimeOnly.extendsFrom testRuntimeOnly +} // The relational implementations of the reliability ports: the outbox repository, the inbox // repository and the destructive-operation journal. Spring JDBC for the transaction join, and SQL diff --git a/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java index f3f1d2a..16becff 100644 --- a/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java +++ b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java @@ -290,14 +290,31 @@ class OutboxPostgresIT { if (directory == null) { throw new IOException("the messaging migrations are missing from the classpath"); } - java.io.File[] files = - new java.io.File(java.net.URI.create(directory.toString())) - .listFiles((dir, name) -> name.endsWith(".sql")); - if (files == null || files.length == 0) { + // The migrations may sit in a directory or, when the suite runs against the main jar, inside + // it; a jar URL is not a file URI, so list through a filesystem that understands either. + java.net.URI uri; + try { + uri = directory.toURI(); + } catch (java.net.URISyntaxException malformed) { + throw new IOException(malformed); + } + java.util.List names; + if ("jar".equals(uri.getScheme())) { + try (java.nio.file.FileSystem jar = + java.nio.file.FileSystems.newFileSystem(uri, java.util.Map.of()); + java.util.stream.Stream listed = + java.nio.file.Files.list(jar.getPath("db/migration/messaging"))) { + names = sqlNames(listed); + } + } else { + try (java.util.stream.Stream listed = + java.nio.file.Files.list(java.nio.file.Path.of(uri))) { + names = sqlNames(listed); + } + } + if (names.isEmpty()) { throw new IOException("no migration applied; the schema under test would be empty"); } - java.util.List names = - java.util.Arrays.stream(files).map(java.io.File::getName).sorted().toList(); java.util.List statements = new java.util.ArrayList<>(); for (String name : names) { statements.add(migration(name)); @@ -305,6 +322,15 @@ class OutboxPostgresIT { return statements; } + private static java.util.List sqlNames( + java.util.stream.Stream listed) { + return listed + .map(path -> path.getFileName().toString()) + .filter(name -> name.endsWith(".sql")) + .sorted() + .toList(); + } + private int countWhere(String predicate) { try (Connection connection = dataSource.getConnection(); Statement statement = connection.createStatement(); diff --git a/messaging/messaging-spring-boot-starter/build.gradle b/messaging/messaging-spring-boot-starter/build.gradle index 7f3f781..efed4b9 100644 --- a/messaging/messaging-spring-boot-starter/build.gradle +++ b/messaging/messaging-spring-boot-starter/build.gradle @@ -1,5 +1,4 @@ plugins { - id 'ca.auxiliary-source-set' id 'ca.platform-module' id 'ca.spring-config' } @@ -9,11 +8,19 @@ plugins { // `api` is the types that genuinely appear in this module's public signatures; everything else is // `implementation`. Broker transports live in capability starters (messaging-spring-boot-starter-kafka), // so this starter's compile classpath carries no broker SDK. -auxiliarySourceSets { - sourceSet('integrationTest') { - compilesAgainst 'main' +testing { + suites { + integrationTest(JvmTestSuite) { + dependencies { + implementation project() + } + } } } +configurations { + integrationTestImplementation.extendsFrom testImplementation + integrationTestRuntimeOnly.extendsFrom testRuntimeOnly +} dependencies { // Types an adopter writes against: envelopes, destinations, publish results, profiles, ports. From cacc69c0b087a954b4473605ce8391953abac9b8 Mon Sep 17 00:00:00 2001 From: donghyeon-ka Date: Tue, 29 Sep 2026 17:01:54 +0900 Subject: [PATCH 7/7] =?UTF-8?q?review-followup=20Wave=203:=20=EC=84=A4?= =?UTF-8?q?=EC=A0=95=20prefix=20app.*=20=ED=86=B5=EC=9D=BC(P20),=20Wave=20?= =?UTF-8?q?3=20=EC=A2=85=EB=A3=8C?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 결정 16: `ca-skeleton.*`·`backend.graphql` 등 → `app.*`(APP_* env 키 이름 유지), selector 키 상수 하나(`CapabilitySelectors`), 바인딩만 되던 `streaming-idle` 삭제 - `StrictSettingsBinder`가 env 오타까지 거부 — 모든 capability(graphql·websocket·mongo는 edge 제약으로 파일만 strict), httpclient 리플렉션 `HttpClientEnvironmentKeys` 삭제 - persistence-jpa가 `APP_DATASOURCE_PASSWORD` secret 요구를 기여(bootstrap이 JPA 스위치를 읽지 않음), Redis ACL fixture에서 pub/sub 권한 제거 - 결함 수정: SMTP 설정 env 이름 충돌(P20-new-1), `bindExcept`가 system env까지 filter(P20-new-2), graphql 빈 deployment-mode 메시지(P20-new-6) - 통합: P30과 충돌 11건(P30의 명시적 bean 구조 유지), P30 코드의 옛 키를 app.*로, persistence-jpa 포맷 - 현황표: 소유자 결정 36(persistence-jpa 설정은 플랫폼 키만 app.persistence-jpa.*로, 공유 기능 트리 예외), 재개 문서에 일시 정지 상태 Wave 3 종료 검증: check · ci verifyDependencyLocks --warning-mode=fail(714 task) · mainCheck · architectureCheck verifyPublicPathSnapshot :app-bootstrap:verifyEnvKeys configContractCheck · :app-bootstrap:integrationTest :sample-portfolio:integrationTest :sample-portfolio:systemTest · integrationCheck · -p build-logic test · -p optional-platforms ci · :app-bootstrap:qualificationBootJar releaseCheck 통과 Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/messaging/configuration-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/messaging/configuration-reference.md b/docs/messaging/configuration-reference.md index 0f87d73..3fc19e9 100644 --- a/docs/messaging/configuration-reference.md +++ b/docs/messaging/configuration-reference.md @@ -28,7 +28,7 @@ producing-service identity다. outbox는 canonical 하나다(2026-09-28). use case가 `IntegrationEventEncoderPort`로 만든 `ValidatedIntegrationEvent`를 `OutboxAppendPort`로 append하고, relay는 claim한 row의 exact envelope -bytes를 `IntegrationEventPublishPort`로 보낸다. 예전의 `ca-skeleton.outbox.canonical-transport-enabled` +bytes를 `IntegrationEventPublishPort`로 보낸다. 예전의 `app.outbox.canonical-transport-enabled` gate와 legacy `MessageBroker → KafkaSender` 경로는 삭제됐다. 발행할 contract의 logical destination마다 아래 destination profile을 선언한다(템플릿의 durable operation event는 `durable-operations`).