diff --git a/docs/messaging/configuration-reference.md b/docs/messaging/configuration-reference.md index ea220b4..3fc19e9 100644 --- a/docs/messaging/configuration-reference.md +++ b/docs/messaging/configuration-reference.md @@ -21,20 +21,16 @@ Application의 canonical integration event를 platform publish pipeline으로 producing-service identity다. 값이 없으면 `IntegrationEventPublishPort` bridge 자체를 만들지 않는다. `spring.application.name`이나 -현재 process 이름으로 추론하지 않는다. 기존 legacy `OutboxEvent`/realtime 경로는 별도 cutover가 -끝날 때까지 `app.messaging.broker` 경로를 유지한다. +현재 process 이름으로 추론하지 않는다. 모든 발행(outbox relay 포함)이 이 bridge를 지나므로 messaging +모듈이 켜진 상태에서 값이 없으면 기동이 거부된다. -## Outbox canonical transport-only cutover +## Outbox -`APP_OUTBOX_CANONICAL_TRANSPORT_ENABLED` / `ca-skeleton.outbox.canonical-transport-enabled`은 -기존 `outbox_event` writer/claim/status authority를 유지한 채 canonical row의 **transport만** platform -publish path로 보내는 compatibility gate다. 기본값은 `false`이며 `POLLING_V2`를 활성화하지 않는다. - -`true`일 때는 `OutboxAppendPort`가 `ValidatedIntegrationEvent`의 exact envelope bytes와 canonical -metadata를 기존 outbox row에 저장하고, claim된 canonical row는 `IntegrationEventPublishPort`로 간다. -legacy row는 계속 `MessageBroker`를 사용한다. 따라서 mixed-row compatibility 기간에는 relay가 켜져 -있다면 `app.messaging.broker`도 계속 필요하며, canonical path를 위해 `IntegrationEventPublishPort`도 -추가로 필요하다. legacy backlog가 0이라는 별도 증거 없이 broker 요구를 제거하지 않는다. +outbox는 canonical 하나다(2026-09-28). use case가 `IntegrationEventEncoderPort`로 만든 +`ValidatedIntegrationEvent`를 `OutboxAppendPort`로 append하고, relay는 claim한 row의 exact envelope +bytes를 `IntegrationEventPublishPort`로 보낸다. 예전의 `app.outbox.canonical-transport-enabled` +gate와 legacy `MessageBroker → KafkaSender` 경로는 삭제됐다. 발행할 contract의 logical destination마다 +아래 destination profile을 선언한다(템플릿의 durable operation event는 `durable-operations`). ## Destination profile @@ -135,9 +131,10 @@ app: ## Broker profile -브로커는 `app.messaging.brokers` 아래에 한 번만 기술한다. `type`이 어느 계열의 설정이 적용되는지 -결정하며, 다른 계열의 키(Kafka 항목의 `prefetch` 같은)는 무시되지 않고 부팅 실패로 거절된다 — -무시하면 그 줄을 쓴 사람은 무언가가 적용됐다고 믿게 된다. +브로커는 `app.messaging.brokers` 아래에 한 번만 기술한다. `type`은 `kafka`만 받는다 — transport가 +있는 계열이 Kafka 하나뿐이다. RabbitMQ 키(`addresses`, `prefetch`, `quorum-queues` 등)와 +`type: rabbitmq`는 2026-09-25에 설정 모델에서 삭제됐다. 연결할 transport 없이 바인딩만 되던 키라, +지금은 무시되지 않고 키 이름을 담아 부팅 실패로 거절된다. ### Kafka @@ -160,32 +157,17 @@ app: authentication-enabled: false # production이면 필수 ``` -### RabbitMQ - -```yaml -app: - messaging: - brokers: - rabbit-primary: - type: rabbitmq - stable: true - production: false - addresses: [rabbit-1:5671] - publisher-confirms: true # stable에서 필수 - publisher-returns: true # stable에서 필수 - mandatory: true # stable에서 필수 - confirm-timeout: 5s - auto-ack: false # 항상 금지 - prefetch: 16 - quorum-queues: true # durable work queue 필수 - tls-enabled: false - authentication-enabled: false -``` - -`production: true`인 브로커는 `tls-enabled`와 `authentication-enabled`가 모두 참이어야 하고, -그렇지 않으면 `KafkaProfileValidator` / `RabbitProfileValidator`가 부팅을 거절한다. 위 예시가 +`production: true`인 Kafka 브로커는 `tls-enabled`와 `authentication-enabled`가 모두 참이어야 하고, +그렇지 않으면 `KafkaProfileValidator`가 부팅을 거절한다. `app.messaging.broker`로 고를 수 있는 값은 +`kafka`뿐이다. 위 예시가 `production: false`인 것은 이 페이지가 그대로 실행되는 fixture이기 때문이며, 실 배포는 셋 다 참이다. +Kafka producer는 이 항목 하나로 만든다. `bootstrap-servers`, `acks`, `enable-idempotence`, +`max-in-flight-requests-per-connection`, `delivery-timeout`이 그대로 client 설정이 된다. +`spring.kafka.bootstrap-servers`는 읽지 않는다. type이 `kafka`인 항목이 정확히 하나가 아니면 기동에 +실패한다. `send()`가 metadata를 기다리며 호출 스레드를 막는 시간(`max.block.ms`)은 기본 publish +timeout(5s)이다. + ## 보안 ```yaml @@ -197,6 +179,8 @@ app: consumer: { type: SASL_SCRAM, credential-id: kafka-consumer } # admin은 application runtime에 설정하지 않는다 hostname-verification: true + # 사설 CA로 서명된 TLS 브로커면(tls-enabled: true일 때만): + # trust-store: { location: /etc/kafka/ca.p12, type: PKCS12, password-credential-id: kafka-truststore } access: publishable: [order-events] consumable: [] @@ -211,21 +195,37 @@ app: 연결 시점에 해석하므로, 설정 덤프나 힙 덤프에서 나오는 것은 이름뿐이다. producer와 consumer는 서로 다른 `credential-id`를 써야 하며, 같으면 부팅에 실패한다. -## Experimental / Optional +`SASL_SCRAM`·`USERNAME_PASSWORD`의 사용자 이름은 `CredentialProvider.username(credentialId)`가 +준다. credential id를 사용자 이름으로 보내지 않는다. 비밀번호와 사용자 이름은 JAAS 문자열에 들어가지 +않고, 연결이 인증할 때마다 provider에서 읽힌다. 그래서 provider가 만료 시각을 주는 credential은 +재기동 없이 rotation된다(`security.md`). -기본값은 전부 `false`다. +`MUTUAL_TLS`는 클라이언트 인증서를 key store로 제출한다. `credential-id`는 key store 비밀번호를 +가리킨다. -```yaml -app: - messaging: - experimental: - kafka-share: false - pulsar: false - nats: false - bridge: - spring-cloud-stream: false +```text +producer: + type: MUTUAL_TLS + credential-id: kafka-producer-tls # key store 비밀번호 + key-store: { location: /etc/kafka/producer.p12, type: PKCS12 } ``` +`trust-store`를 생략하면 JVM 기본 trust store를 쓴다. 사설 CA 브로커는 `trust-store.location` +(PKCS12·JKS·PEM)과, 잠긴 store면 `password-credential-id`를 준다. + +Kafka producer는 이 항목을 실제 클라이언트 설정(`security.protocol`, `sasl.mechanism`, +`sasl.jaas.config`, `sasl.client.callback.handler.class`, `ssl.truststore.*`, `ssl.keystore.*`, +`ssl.enabled.protocols=TLSv1.3,TLSv1.2`, hostname 검증)으로 옮긴다. 평문으로 조용히 넘어가지 +않도록 다음 경우 부팅에 실패한다: 브로커가 `tls-enabled`나 `authentication-enabled`를 켰는데 +`app.messaging.security.`가 없을 때, security 항목은 있는데 `CredentialProvider` bean이 없을 +때, 같은 브로커에 맞는 항목이 둘 이상일 때, provider가 SASL 사용자 이름을 주지 않을 때, +`MUTUAL_TLS`인데 `key-store`가 없거나 `tls-enabled`가 꺼져 있을 때, `trust-store`를 줬는데 +`tls-enabled`가 꺼져 있을 때, store 파일을 열 수 없을 때. + +`access.publishable`은 강제된다. security 항목이 있는 브로커의 destination은 이 목록에 있어야 +publish된다. 목록 밖 publish는 `REJECTED`(`PUBLISH_FORBIDDEN`)이고, 빈 목록이면 아무것도 publish되지 +않는다. + ## Backpressure ```yaml diff --git a/docs/messaging/cutover.md b/docs/messaging/cutover.md index a65e341..f697177 100644 --- a/docs/messaging/cutover.md +++ b/docs/messaging/cutover.md @@ -27,8 +27,9 @@ `PublishResult` → `OutboxPublishOutcome`. registry에 leaf를 추가하는 변경이라 별도 커밋. - golden contract 테스트: event/message ID, type, schema revision, partition/order/correlation/ causation/tenant/trace, payload digest, wire version이 bytes 단위로 보존되는지. -- 단일 publication authority: 기존 `OutboxPublicationAuthority` fence를 재사용해 writer/relay가 - 동시에 ACTIVE가 되지 않도록. **dual write/publish는 금지** — 한 business fact가 두 durable store와 +- 단일 publication authority: writer/relay가 동시에 ACTIVE가 되지 않도록 하는 fence. 이 용도로 쓰려던 + outbox V2의 `OutboxPublicationAuthority`는 한 번도 조립되지 않은 채 2026-09-25에 삭제됐으므로 + (`docs/capability-status.md` "제거된 코드") cutover가 자기 fence를 새로 가져야 한다. **dual write/publish는 금지** — 한 business fact가 두 durable store와 두 relay로 나가는 상태가 cutover에서 가장 위험하다. - 첫 cutover 범위는 **transport만** 교체(저장소는 기존 유지). storage migration은 shadow read → authority switch → old backlog drain 순서로 별도 release. diff --git a/docs/messaging/experimental-policy.md b/docs/messaging/experimental-policy.md index e2052c4..de8f5b1 100644 --- a/docs/messaging/experimental-policy.md +++ b/docs/messaging/experimental-policy.md @@ -17,24 +17,22 @@ stopsAcceptingNewWorkDuringShutdown **Experimental**은 아직 그 증명이 끝나지 않은 어댑터다. +현재 Experimental 어댑터는 없다. Kafka Share Group·Pulsar·NATS JetStream(Experimental), RabbitMQ, +Spring Cloud Stream bridge(Optional)는 SDK 없는 seam이거나 선택할 수 없는 상태여서 2026-09-25에 +삭제했다(`docs/capability-status.md` 제거 표). 이 문서는 새 어댑터를 들일 때 지킬 규칙이다. + ## 규칙 ### 1. 기본 비활성 -```yaml -messaging.experimental.kafka-share: false -messaging.experimental.pulsar: false -messaging.experimental.nats: false -``` - +새 Experimental 어댑터는 자기 활성화 플래그를 가지고 기본값은 `false`다. 활성화하지 않으면 validator가 `MessagingCapabilityUnavailableException`을 던진다. Contract Suite가 아직 증명 중인 어댑터가 누군가의 기본 설정 때문에 load-bearing이 되어서는 안 된다. ### 2. Stable 모듈이 Experimental 모듈에 의존하지 않는다 -Gradle 의존 그래프로 강제된다. `messaging-spring-boot-starter`의 `allowed_dependencies`에 -`messaging-kafka-share-experimental`, `messaging-pulsar-experimental`, -`messaging-nats-experimental`, `messaging-spring-cloud-stream-bridge`가 **없다**. +Gradle 의존 그래프로 강제된다. Experimental leaf는 `messaging-spring-boot-starter`의 +`allowed_dependencies`에 넣지 않는다. `verifyCleanArchitectureDependencies`가 위반을 빌드 실패로 만든다. @@ -45,45 +43,8 @@ Experimental 어댑터는 브로커의 차이를 `MessagingCapabilities`로 표 ### 4. 없는 기능을 광고하지 않는다 -| 어댑터 | 광고하지 않는 것 | 이유 | -|---|---|---| -| Kafka Share Group | orderedStream, keyedOrdering, replay, brokerTransaction | 경쟁 소비자 + 개별 ack는 partition 순서를 유지할 수 없다 | -| Pulsar | brokerTransaction | Pulsar에 있지만 플랫폼 Contract Suite로 증명되지 않았다 | -| Pulsar (Shared) | keyedOrdering | round-robin 분배 | -| NATS JetStream | nativeDeadLetter | delivery limit 초과 시 terminate할 뿐 라우팅하지 않는다 | -| NATS JetStream | keyedOrdering | subject 기반 모델에 per-key 순서가 없다 | - -`false`인 capability를 요구하는 profile은 startup에서 실패한다. -조용히 약화되지 않는다. - -### 5. 명시적 거부 - -| 조합 | 결과 | -|---|---| -| Kafka Share Group + ordering != NONE | 거부 | -| Kafka Share Group + pause/resume | `MessagingCapabilityUnavailableException` | -| Pulsar Shared + ordering=KEY | 거부 (Key_Shared 필요) | -| Pulsar + ordering=DESTINATION | 거부 | -| NATS Core + AT_LEAST_ONCE | 거부 (JetStream 필요) | -| NATS ordered consumer + 경쟁 워커 > 1 | 거부 | -| NATS + ordering=KEY | 거부 | - -## Spring Cloud Stream bridge - -Experimental이 아니라 **Optional**이다. 위험이 다르다. - -Stream은 자체 binder 설정을 소유하므로, binding이 destination profile이 모르는 -serializer·error handling·acknowledgement mode를 조용히 획득할 수 있다. - -따라서 브리지는 **플랫폼 보장에 의존하지 않는 destination에만** 허용한다. - -```text -ordering scope 선언 → 거부 -retry policy 선언 → 거부 -dead letter 선언 → 거부 -``` - -이 셋 중 하나라도 필요하면 native adapter를 쓴다. 거기서만 실제로 강제되기 때문이다. +Contract Suite로 증명하지 않은 capability는 `MessagingCapabilities`에서 `false`로 둔다. +`false`인 capability를 요구하는 profile은 startup에서 실패한다. 조용히 약화되지 않는다. ## 승격 조건 diff --git a/docs/messaging/operations.md b/docs/messaging/operations.md index 5cea33b..5bccc75 100644 --- a/docs/messaging/operations.md +++ b/docs/messaging/operations.md @@ -3,8 +3,7 @@ ## 배포 전 체크 ```bash -./gradlew verifyCleanArchitectureDependencies --console=plain -./gradlew verifyRuntimeModuleMembership --console=plain +./gradlew architectureCheck --console=plain ./gradlew checkstyleMain --console=plain ``` diff --git a/docs/messaging/outbox-inbox.md b/docs/messaging/outbox-inbox.md index 4dac3ee..bf68398 100644 --- a/docs/messaging/outbox-inbox.md +++ b/docs/messaging/outbox-inbox.md @@ -21,8 +21,8 @@ Flyway version `1000..1999`를 예약해서 애플리케이션의 일반적인 classpath:db/migration/messaging ``` -현재 순서는 `V1001..V1004`(outbox/admin)와 `V1101`(inbox)이다. starter 테스트가 두 JDBC -모듈을 실제 combined classpath로 올려 version 중복과 예약 대역 이탈을 검사한다. 개별 DDL의 +현재 순서는 `V1001..V1005`(outbox/admin)와 `V1101`(inbox)이고, 모두 +`messaging-reliability-jdbc-postgresql`에 있다. starter 테스트가 그 모듈을 실제 combined classpath로 올려 version 중복과 예약 대역 이탈을 검사한다. 개별 DDL의 의미와 전환 순서는 [migration-guide.md](migration-guide.md)를 따른다. ## Outbox @@ -39,13 +39,18 @@ COMMIT; ### relay ```text -leaseBatch(100, 30s) -- lease로 다중 relay 인스턴스 안전 +claimBatch(owner, 100, 30s, now, maxAttempts) -- owner + fencing token으로 다중 relay 인스턴스 안전 → publish (messageId 그대로) → CONFIRMED → markPublished → AMBIGUOUS → markAmbiguous (같은 messageId로 재시도 가능) → REJECTED → markFailed ``` +row 하나의 실패는 그 row에서 끝난다. envelope로 만들 수 없는 row(예약 header `msg.*`를 담은 row +등)는 `FAILED`가 되고, publisher가 예외를 던지면 결과를 모르므로 `AMBIGUOUS`가 된다. 같은 batch의 +나머지 row는 그대로 publish된다. pass 전체가 실패하면(DB 연결 등) WARN 로그를 남기고 +`messaging.outbox.relay.failed.passes` 카운터를 올린다. + ### 핵심 규칙: ambiguous는 같은 messageId로 재시도 새 id를 발급하면 "전달됐을 수도 있는 메시지"가 "확실히 두 번째인 메시지"가 되어 @@ -61,8 +66,16 @@ status IN ('PENDING','AMBIGUOUS','IN_FLIGHT') AND (lease_expires_at IS NULL OR lease_expires_at <= now) AND next_attempt_at <= now AND attempts < maxAttempts + AND (ordering_key IS NULL + OR 같은 destination·ordering_key의 더 오래된 미완료 row가 없음) ``` +`ordering_key`가 있는 row는 같은 key의 더 오래된 row가 `PENDING`·`AMBIGUOUS`·`IN_FLIGHT`인 동안 +claim되지 않는다. 이 조건이 없으면 두 relay가 같은 key의 두 row를 나눠 가져가거나, AMBIGUOUS로 +재시도되는 row보다 뒤의 row가 먼저 나간다(`OutboxPostgresIT#aNewerRowWaitsWhileAnOlderRowWithTheSameOrderingKeyIsUnfinished`). +`V1005`의 partial index가 이 확인을 받친다. 더 오래된 row가 `FAILED`·`EXHAUSTED`가 되면 뒤의 row는 +진행한다. + `IN_FLIGHT`가 목록에 있는 것이 핵심이다. relay가 publish 도중 죽으면 row는 `IN_FLIGHT`로 남는데, 이를 제외하면 그 메시지는 **영원히** 발행되지 않는다 — outbox가 막으려던 바로 그 실패다. 대신 lease가 만료됐을 때만 회수하므로, 살아 있는 relay가 들고 있는 row는 회수되지 않는다. @@ -95,6 +108,24 @@ transactions.inTransaction(() -> { 별도 transaction으로 예약하면 Inbox가 닫으려던 바로 그 창이 다시 열린다. +### 소비와 연결 (소유자 결정 10) + +consume 계약은 `MessageHandler` 하나다. handler를 bean으로 등록하면 `messaging-spring-boot-starter-kafka`가 +그 handler가 가리키는 destination마다 Kafka consumer를 연다(`app.messaging.enabled=false`면 없다). + +- destination profile이 `guarantees.external-side-effect: INBOX_TRANSACTIONAL`이면 platform이 handler를 + `InboxMessageHandler`로 감싼다. 예약과 handler의 쓰기가 같은 transaction에서 commit되고, 이미 예약된 + message는 handler를 부르지 않고 `Success`로 settle한다. 이 경우 애플리케이션이 `IdempotentConsumer` + bean(inbox repository + transaction runner)을 둬야 하며, 없으면 기동이 실패한다. +- inbox의 consumer 식별자와 Kafka `group.id`는 destination의 `consumer.group`이다. 없으면 기동이 실패한다. +- offset은 handler가 `Success`로 답하고 (inbox destination이면) 그 transaction이 commit된 뒤에만, 또는 + DLQ 토픽 기록이 확인된 뒤에만 넘어간다. auto-commit은 쓰지 않는다. +- `Retry`·예외·timeout은 `retry.max-attempts`까지 partition을 멈추고 다시 읽으며, 다 쓰면 DLQ 토픽에 + 원본 record를 남기고 넘어간다. DLQ가 없는 destination(at-most-once)은 넘어가지 않고 멈춘다. +- `Reject`는 at-most-once destination에서만 버리고, 그 밖에서는 dead letter로 보낸다. + +실제 왕복·중복 제거·DLQ는 `MessagingKafkaConsumeQualificationTest`(Kafka + PostgreSQL Testcontainers)가 확인한다. + ### 복합 키 `PRIMARY KEY (message_id, consumer_id)`. @@ -107,13 +138,14 @@ message_id만으로 중복 제거하면 같은 event를 소비하는 두 번째 broker의 최대 redelivery window보다 **길어야** 한다. row를 먼저 지우면 늦게 도착한 redelivery가 두 번 처리된다. -## Debezium CDC 대안 +outbox·inbox cleanup job은 한 번에 최대 1,000행씩 지우고, 지운 수가 1,000보다 적으면 멈춘다. +repository port에는 개수 제한 없는 삭제 메서드가 없다. -polling relay 대신 WAL을 읽는다. polling interval과 lease 경합이 사라지지만 -인프라와 그 자체의 실패 모드가 추가된다. +## CDC 대안 -wire contract는 동일하다. `DebeziumOutboxEventRouter`가 polling relay와 같은 reserved header를 -방출하므로 consumer는 어느 쪽이 발행했는지 구분할 수 없고, 전환은 배포 결정일 뿐 계약 변경이 아니다. +relay는 polling 하나뿐이다. Debezium Event Router 매핑(`DebeziumOutboxEventRouter` 등)은 조립하는 +곳이 없어 2026-09-25에 삭제했다(`docs/capability-status.md` 제거 표). `messaging_outbox`의 +`routing_key` 컬럼은 남아 있으므로 CDC를 다시 들일 때 그 컬럼을 message key로 쓴다. ## Claim Check diff --git a/docs/messaging/retry-dlq-redrive.md b/docs/messaging/retry-dlq-redrive.md index 39355e6..9d64b63 100644 --- a/docs/messaging/retry-dlq-redrive.md +++ b/docs/messaging/retry-dlq-redrive.md @@ -49,6 +49,15 @@ full jitter는 `[0, delay]` 균등 분포다. jitter가 없으면 같은 초에 pause-and-seek는 메시지가 로그의 자기 자리를 떠나지 않는다. partition을 멈추고, 기다리고, 같은 offset으로 seek해 재전달한다. 뒤의 메시지도 함께 기다리며 이것이 의도된 동작이다. +`KafkaConsumerRegistrar`가 handler 실패(예외, settle 없이 끝남, timeout)를 다루는 방식: + +- 재전달은 destination `retry`의 backoff 뒤에 한다. 최소 1초. 0초 재시도는 없다. +- 같은 offset의 실패 횟수가 `retry.maxAttempts`에 닿으면 `KafkaQuarantinePublisher`에 쓰고, + 쓰기가 확인되면 offset을 넘긴다. quarantine이 없으면 backoff로 계속 재시도한다(메시지를 버리지 + 않는다). 그래서 `production` destination은 quarantine 없이 만들 수 없다(`KAFKA_QUARANTINE_REQUIRED`). +- handler timeout이 지나도 handler가 끝날 때까지 partition slot을 잡고 있는다. `CompletableFuture`를 + cancel해도 handler는 멈추지 않으므로, slot을 먼저 풀면 같은 offset이 두 번 동시에 돈다. + ## RabbitMQ: delayed retry queue core broker에 per-message delay가 없으므로 **TTL + DLX**로 구현한다. diff --git a/docs/messaging/security.md b/docs/messaging/security.md index c6dc392..a847205 100644 --- a/docs/messaging/security.md +++ b/docs/messaging/security.md @@ -36,17 +36,30 @@ heap dump나 설정 출력에서 사용 가능한 credential이 나오지 않는 `CredentialIds`는 `bearer `, `sk-`, `-----begin`, `eyJ` 같은 접두사를 거부한다. 참조가 들어갈 자리에 secret 자체를 붙여넣는 가장 흔한 사고를 막는다. +SASL(PLAIN·SCRAM) 사용자 이름도 `CredentialProvider.username(credentialId)`가 준다. credential id는 +저장소 안의 참조이지 broker가 아는 사용자 이름이 아니다. provider가 이름을 주지 않으면 기동에 +실패한다. + +key store·trust store는 경로만 설정에 둔다. 비밀번호는 credential id로 provider가 해석한다 +(`configuration-reference.md` 보안 절). + ## Rotation `CredentialRotationPlan.isDue()`는 만료 **전에** 참이 된다. broker가 연결을 거부하기 시작한 시점에는 이미 publish가 실패하고 consumer가 멈춰 있다. -rotation은 세대 교체다. `DefaultMessagingRuntimeRegistry.install()`이 새 세대를 원자적으로 -게시하고, 이전 세대는 마지막 lease가 닫힐 때까지 열려 있다가 닫힌다. -진행 중인 publish는 시작한 연결에서 confirm을 받는다. +Kafka SASL credential은 producer를 다시 만들지 않고 바뀐다. JAAS 설정에는 login module 이름만 +있고, `KafkaCredentialCallbackHandler`가 연결이 인증할 때마다 `CredentialRuntimeRegistry`에서 +사용자 이름과 비밀번호를 읽는다. registry는 provider가 준 만료 시각의 rotation lead(기본 30분) +안으로 들어오면 새로 가져오므로, 새 연결과 재인증(broker `connections.max.reauth.ms`)이 새 값을 +쓴다. 만료 시각을 주지 않는 provider의 credential은 캐시된 값을 계속 쓴다. -drain deadline이 이 대기를 제한한다. 없으면 lease 하나가 새면 폐기된 credential이 -무기한 열려 있고, rotation이 보안상 무의미해진다. +key store·trust store 비밀번호는 예외다. Kafka가 SSL context를 만들 때 한 번 읽으므로 바꾸려면 +producer를 다시 만들어야 한다(재기동). + +transport 세대 교체(`DefaultMessagingRuntimeRegistry.install()`)는 transport 자체를 바꿀 때 쓴다. +새 세대를 원자적으로 게시하고, 이전 세대는 마지막 lease가 닫힐 때까지 열려 있다가 닫힌다. +drain deadline이 이 대기를 제한한다. ## Header @@ -66,10 +79,13 @@ logical identity가 호출자 제어가 된다. ## ACL -`DestinationAccessValidator`가 broker ACL **이전에** 검사한다. +publisher가 broker ACL **이전에** 검사한다(`PUBLISH_FORBIDDEN`). broker ACL 거부는 애플리케이션 컨텍스트가 없는 연결 수준 오류로 도착하므로 "어느 모듈이 어디에 publish하려 했는가"가 조사 대상이 된다. +publish 가능한 destination은 선언된 destination이다. broker에 `app.messaging.security.` +항목이 있으면 그중 `access.publishable`에 적힌 것만 남는다(빈 목록이면 publish 불가). + ## 관측성 누출 `MessagingRedactor`는 denylist다. diff --git a/docs/messaging/support-matrix.md b/docs/messaging/support-matrix.md index 563bd88..bd3164c 100644 --- a/docs/messaging/support-matrix.md +++ b/docs/messaging/support-matrix.md @@ -4,10 +4,12 @@ 여기 없는 조합은 지원되지 않는다. > **등급은 증거를 따른다.** `CompatibilityMatrix.Entry.hasLiveBrokerCertification()`은 선언된 -> boolean이 아니라 `CertifiedEvidence`가 가진 레인 증거에서 파생된다. RabbitMQ가 Stable에서 내려온 -> 이유가 이것이다 — 어댑터는 공유 contract 7개를 통과하고 `RabbitBrokerIT`가 실 컨테이너에서 정상 -> 경로를 돌리지만, 이 저장소의 Stable 기준인 **장애 시나리오 증거**가 하나도 없다. 레인이 생겨 -> 증거를 내면 등급은 코드 수정 없이 따라 올라간다. +> boolean이 아니라 `CertifiedEvidence`가 가진 레인 증거에서 파생된다. 레인이 생겨 증거를 내면 등급은 +> 코드 수정 없이 따라 올라간다. + +> **삭제된 어댑터.** RabbitMQ(선택할 수 없던 validator·설정 모음), Pulsar·NATS JetStream·Kafka Share +> Group(Experimental, SDK 없는 seam), Spring Cloud Stream bridge, Debezium outbox 매핑은 2026-09-25에 +> 삭제했다. 복구 경로는 `docs/capability-status.md`의 제거 표에 있다. > **인증 근거.** 이 표의 버전은 이 저장소의 컨테이너 레인이 실제로 실행한 이미지다. 이전 판은 > Kafka 4.2/4.3을 선언했지만 fixture는 `apache/kafka:4.1.0`, lockfile client는 4.1.1이었다 — 표와 @@ -15,8 +17,10 @@ > `BrokerFailureMatrix.shipped()` 하드코딩이 아니라 레인이 낸 증거(`BrokerCertificationEvidence`)에서 > 나온다. 증거가 없는 조합은 `NOT_COVERED`다 (MSG-014). -> **모듈 이름과 런타임 편입.** `messaging-outbox-jdbc-postgresql` / `messaging-inbox-jdbc-postgresql`은 -> 이전에 `-jpa`로 불렸다. 구현은 Spring JDBC이고 SQL은 PostgreSQL 전용(`?::jsonb`, +> **모듈 이름과 런타임 편입.** 관계형 신뢰성 구현은 `messaging-reliability-jdbc-postgresql`, +> 그 port만 보는 relay·consumer 로직은 `messaging-reliability`다(2026-09-28에 +> `messaging-outbox-jdbc-postgresql` / `messaging-inbox-jdbc-postgresql`을 이렇게 나눴다). 그 전에는 +> `-jpa`로 불렸다. 구현은 Spring JDBC이고 SQL은 PostgreSQL 전용(`jsonb`, > `FOR UPDATE SKIP LOCKED`, `ON CONFLICT`, `TIMESTAMPTZ`)이므로, 그 이름은 쓰지 않는 기술을 > 광고하고 vendor 중립 port(`messaging-reliability-api`)의 위치를 가렸다 (MSG-023). > @@ -29,10 +33,7 @@ | 브로커 | 등급 | 인증 기준 | Stable 기능 | 제한 | |---|---|---|---|---| -| Kafka | Stable | 4.1.x | producer idempotence, consumer group, batch, pause/resume, replay, transaction capability | Share Group은 Experimental | -| RabbitMQ | Experimental | 4.3.x | exchange/routing, publisher confirm, mandatory return, manual ACK, quorum queue, retry queue, DLQ | **현재 starter에서 선택 불가** — production `RabbitChannelPublisher`/transport bridge가 없어 `app.messaging.broker=rabbit`은 startup에서 거부된다. 장애 시나리오 증거도 아직 없음 | -| Pulsar | Experimental | 4.0 LTS + 4.2 | typed publish/consume, Shared, Key_Shared, schema | transaction 미승격, 기본 비활성 | -| NATS JetStream | Experimental | 2.14.x | stream, durable consumer, explicit ACK, dedupe, replay | native DLQ 없음(플랫폼이 대행), 기본 비활성 | +| Kafka | Stable | 4.1.x | producer idempotence, consumer group, batch, pause/resume, replay, transaction capability | Share Group 미지원 | | Artemis/JMS | Extension | 범위 밖 | adapter SPI만 | 별도 ADR + Contract Suite 통과 필요 | ## Capability 매트릭스 @@ -40,27 +41,20 @@ `MessagingCapabilities`가 런타임에 선언하는 값이다. `false`인 기능을 요구하는 destination profile은 **startup에서 실패**하며, 조용히 약화되지 않는다. -| Capability | Kafka | Kafka Share | RabbitMQ | Pulsar | NATS JS | -|---|---|---|---|---|---| -| brokerAcknowledgement | O | O | O | O | O | -| replicationOrPersistenceEvidence | O | O | O | O | O | -| perMessageSettlement | O | O | O | O | O | -| batchSettlement | O | X | X | O | O | -| orderedStream | O | **X** | X | X | O | -| keyedOrdering | O | **X** | X | Key_Shared만 | X | -| replay | O | X | X | O | O | -| delayedDelivery | X | X | retry queue로 대행 | O | X | -| brokerTransaction | O | X | X | 미승격 | X | -| deduplicatedPublish | O | X | X | X | O | -| nativeDeadLetter | X | X | O | O | **X** | -| topologyManagement | O | X | O | O | O | - -Kafka Share Group이 ordering 전부 `X`인 것은 설계 결정이다. share group은 개별 record를 -경쟁 소비자에게 나눠주고 개별 ack하므로 partition 순서를 유지할 수 없다. ordered destination을 -share group에 설정하면 `KafkaShareProfileValidator`가 거부한다. - -NATS JetStream의 `nativeDeadLetter=X`도 마찬가지다. JetStream은 delivery limit 초과 시 메시지를 -**terminate**할 뿐 어디로도 라우팅하지 않으므로, 플랫폼이 DLQ publish를 직접 수행한다. +| Capability | Kafka | +|---|---| +| brokerAcknowledgement | O | +| replicationOrPersistenceEvidence | O | +| perMessageSettlement | O | +| batchSettlement | O | +| orderedStream | O | +| keyedOrdering | O | +| replay | O | +| delayedDelivery | X | +| brokerTransaction | O | +| deduplicatedPublish | O | +| nativeDeadLetter | X | +| topologyManagement | O | ## 기능 등급 @@ -73,8 +67,6 @@ NATS JetStream의 `nativeDeadLetter=X`도 마찬가지다. JetStream은 delivery | Batch / Manual settlement / Pause·Resume / Delayed / Replay 요청 | M2 | | Broker transaction / partition / routing / subscription | M3 | | Replay 실행 / Redrive / offset reset / purge / delete | M4 Admin | -| Kafka Share Group, Pulsar, NATS | Experimental | -| Spring Cloud Stream bridge | Optional | ## 무엇이 "Stable"을 증명하는가 @@ -82,7 +74,7 @@ Stable 등급은 두 가지를 **모두** 통과해야 한다. `CompatibilityMat ### 1. 공유 Contract Suite (`MessagingAdapterContract`, 7개) -Kafka와 RabbitMQ가 동일한 7개 테스트를 변경 없이 통과한다. 결정적 하네스를 쓰므로 +Kafka가 7개 테스트를 변경 없이 통과한다. 결정적 하네스를 쓰므로 확인 유실·settlement 유실 같은 장애를 요청 시점에 재현할 수 있다. ### 2. 실 브로커 인증 (Testcontainers) @@ -92,7 +84,6 @@ Kafka와 RabbitMQ가 동일한 7개 테스트를 변경 없이 통과한다. 결 | `KafkaBrokerIT` | `acks=all`이 실제 replication 증거를 만든다 / 잘못된 토픽은 `REJECTED` / 발행-소비 왕복에서 identity 보존 및 contiguous commit | | `KafkaAmbiguityChaosIT` | 브로커를 `docker pause`로 멈춘 상태의 publish가 **`AMBIGUOUS`** 로 보고된다 (broker acceptance 없음, confirmation level `NONE`, 비-retryable) | | `KafkaBrokerCertificationIT` | 인증 레인. Toxiproxy를 broker 앞에 두고 connection cut / confirm 유실 / 지연 / settlement 유실을 각각 주입하고, 통과한 시나리오마다 `BrokerCertificationEvidence` 한 줄을 manifest에 쓴다 | -| `RabbitBrokerIT` | exchange가 confirm했는데 어떤 큐에도 바인딩되지 않은 publish가 **`REJECTED` + `UNROUTABLE`** 로 보고된다 | | `OutboxPostgresIT` | 롤백된 트랜잭션은 발행 가능한 행을 남기지 않는다 / `SKIP LOCKED` lease가 두 relay를 분리한다 / ambiguous 행이 같은 `messageId`로 재클레임된다 | | `InboxPostgresIT` | 재전달이 side effect를 두 번 적용하지 않는다 / 롤백은 예약도 되돌린다 | @@ -137,7 +128,7 @@ delivery timeout으로 나타난다. 이를 `REJECTED`로 보고하는 것은 1. **Outbox `IN_FLIGHT` 고아 행** — lease 쿼리가 `PENDING`/`AMBIGUOUS`만 클레임 대상으로 봐서, publish 도중 죽은 relay가 남긴 행이 lease 만료 후에도 영영 회수되지 않았다. -2. **Rabbit confirm 경합** — transport가 publish *후에* confirm을 등록해서, 연결 스레드에서 +2. **Rabbit confirm 경합**(2026-09-25에 삭제된 Rabbit transport) — transport가 publish *후에* confirm을 등록해서, 연결 스레드에서 confirm이 먼저 도착하면 유실되고 호출자가 무한 대기했다. 둘 다 인메모리 double이 실제보다 관대해서 통과하고 있었다. @@ -154,21 +145,10 @@ delivery timeout으로 나타난다. 이를 `REJECTED`로 보고하는 것은 - DLQ publish 확인 전 source ACK -## R1 compiler status - -`adapter:outbound:messaging`의 contract/catalog/envelope compiler 군은 현재 **의도적으로 미배선**이다. -`DISABLED` + 빈 catalog/binding이 zero-resource 상태이며, 이 R1 compiler를 legacy -`MessagingConfig`에 연결해 Kafka ACK, wire compatibility, `ACTIVE_READY`, R2 qualification을 -주장하지 않는다. 코드는 다음 세대 계약을 보존하지만 현재 채택 경로가 아니며, 실제 활성화는 별도 R2 -composition/cutover에서 명시적으로 이뤄져야 한다. - - -### Optional mapping / bridge utilities +### Optional mapping utilities - `messaging-cloudevents`는 starter runtime에 자동 편입되지 않는 **opt-in mapping utility**다. 내부 envelope와 CloudEvent 객체 사이의 제한된 profile mapping을 제공하지만 binary/structured CloudEvents wire mode 전체를 구현하거나 임의 CloudEvents producer와의 1.0 interoperability를 인증하지 않는다. -- `messaging-spring-cloud-stream-bridge`는 현재 `ChannelSend` seam과 publish-evidence 판단을 제공하는 - framework-neutral bridge contract다. Spring Cloud Stream `StreamBridge` dependency/auto-configuration은 - 제공하지 않으므로 shipped Spring Cloud Stream integration으로 분류하지 않는다. +- `messaging-claim-check`도 starter에 편입되지 않는 opt-in utility다. diff --git a/messaging/CLAUDE.md b/messaging/CLAUDE.md index d051e29..14c58e8 100644 --- a/messaging/CLAUDE.md +++ b/messaging/CLAUDE.md @@ -14,25 +14,28 @@ leaf의 정확한 id는 registry에서 읽는다. family는 **디렉터리 이 | Family | 역할 | 허용 framework | | --- | --- | --- | -| `*-api` (`core-api`, `schema-api`, `policy`, `reliability-api`, `admin-api`, `transport-spi`) | 값 타입, 포트, 정책 계약 | Java stdlib만. Spring/broker SDK/logging framework 금지 | +| `*-api` (`core-api` — `api`·`schema` 패키지, `policy`, `reliability-api`, `transport-spi`) | 값 타입, 포트, 정책 계약 | Java stdlib만. Spring/broker SDK/logging framework 금지 | | `schema-*` (json/avro/protobuf), `cloudevents` | 인코딩 구현 | 해당 codec 라이브러리만. Spring 금지 | -| `runtime-core`, `security`, `observability`, `admin-runtime` | framework-free 실행 로직 | Micrometer는 `observability`에서만. 나머지는 stdlib | -| `kafka`, `rabbit`, `pulsar-experimental`, `nats-experimental`, `kafka-share-experimental` | broker adapter | 해당 broker client. Spring은 `rabbit`의 `spring-rabbit`처럼 client가 곧 Spring인 경우만 | -| `outbox-jdbc-postgresql`, `inbox-jdbc-postgresql` | 관계형 신뢰성 adapter | Spring JDBC / Spring TX. **JPA 아님** — 이름이 곧 계약이다 | -| `spring-boot-starter`, `spring-cloud-stream-bridge` | 조립 경계 | Spring Boot autoconfigure. 여기서만 | +| `runtime-core`, `security`, `observability`, `admin`(계약 `admin` + 실행 `admin.runtime`), `reliability` | framework-free 실행 로직 | Micrometer는 `observability`에서만. 나머지는 stdlib | +| `kafka` | broker adapter | 해당 broker client. Spring은 client가 곧 Spring인 경우만 | +| `reliability-jdbc-postgresql` | 관계형 신뢰성 adapter (outbox·inbox repository, admin journal) | Spring JDBC / Spring TX. **JPA 아님** — 이름이 곧 계약이다 | +| `spring-boot-starter`, `spring-boot-starter-kafka` | 조립 경계 | Spring Boot autoconfigure. 여기서만 | | `testkit` | 테스트 지원 | Testcontainers, JUnit. production leaf가 의존하지 않는다 | -`outbox`/`inbox` leaf가 `jdbc-postgresql`로 명명된 이유: 구현은 Spring JDBC이고 SQL은 -`?::jsonb` · `FOR UPDATE SKIP LOCKED` · `ON CONFLICT` · `TIMESTAMPTZ`로 PostgreSQL 전용이다. -`jpa`라는 이름은 사용하지 않는 기술을 광고하고, vendor 중립 port가 어디 있는지도 숨겼다. vendor -중립 계약은 `messaging-reliability-api`에 있다. +신뢰성 기능은 세 leaf다. `messaging-reliability-api`가 vendor 중립 port(`OutboxRepository`, +`InboxRepository`)를, `messaging-reliability`가 그 port만 보는 relay·consumer 로직(stdlib)을, +`messaging-reliability-jdbc-postgresql`이 port의 관계형 구현과 migration(`V1001..V1005`, `V1101`)을 +갖는다. 예전에는 `outbox-jdbc-postgresql` 하나가 relay 로직과 PG repository와 admin journal을 함께 +가졌고 inbox도 같은 모양이었다(S7-#5b). 구현 leaf가 `jdbc-postgresql`로 명명된 이유: 구현은 Spring +JDBC이고 SQL은 `jsonb` · `FOR UPDATE SKIP LOCKED` · `ON CONFLICT` · `TIMESTAMPTZ`로 PostgreSQL +전용이다. `jpa`라는 이름은 사용하지 않는 기술을 광고하고, vendor 중립 port가 어디 있는지도 숨겼다. ## Public surface 정책 - **vendor 타입을 public signature에 노출하면 그 의존성은 `api`로 선언한다.** 숨긴다고 타입이 숨겨지지 않는다 — consumer가 이 모듈이 문서화한 API에 대해 컴파일하지 못하게 만들 뿐이다. - (`kafka-clients`, `spring-rabbit`, `cloudevents-api`, `micrometer-core`, `avro`, - `protobuf-java`, `reactor-core`가 이 규칙으로 `api`다.) + (`kafka-clients`, `cloudevents-api`, `micrometer-core`, `avro`, + `protobuf-java`가 이 규칙으로 `api`다.) - 반대로 **노출하지 않는 것은 `implementation`으로 낮춘다.** starter가 노출하지 않는 하위 모듈을 `implementation`으로 내린 것이, JSON codec이 `ObjectMapper`를 private으로만 쓰고 Jackson을 `implementation`으로 둔 것이 그 예다. @@ -45,16 +48,19 @@ leaf의 정확한 id는 registry에서 읽는다. family는 **디렉터리 이 ## Application runtime에 포함되는 기준 -Messaging family는 기본 application이 `messaging-spring-boot-starter`를 dependency로 가지므로 그 -transitive project graph가 runtime에 포함된다. 이 사실의 SSOT는 Gradle `runtimeClasspath`이며 registry에 -동일한 membership 목록을 복제하지 않는다. `app.messaging.enabled=false`가 기본값이고 master switch가 -activation을 결정한다. +기본 조립에는 Messaging family가 없다. 서비스가 `app-bootstrap/build.gradle`에 +`runtimeOnly project(':adapter:outbound:messaging-kafka')` 한 줄을 넣으면 그 모듈이 bridge와 +`messaging-spring-boot-starter`(Kafka transport 포함)의 transitive project graph를 runtime에 싣는다. 이 +사실의 SSOT는 Gradle `runtimeClasspath`이며 registry에 동일한 membership 목록을 복제하지 않는다. +모듈이 있으면 켜지고(`app.messaging.enabled` 기본 `true`), `APP_MESSAGING_ENABLED=false`로만 끈다. -experimental transport, 일부 codec/bridge/testkit처럼 starter graph에 들어오지 않는 leaf는 build-only다. +일부 codec(cloudevents 등)·claim-check·testkit처럼 starter graph에 들어오지 않는 leaf는 build-only다. +RabbitMQ·Pulsar·NATS·Kafka Share·Spring Cloud Stream bridge leaf와 Debezium outbox 매핑은 +2026-09-25에 삭제했다(`docs/capability-status.md` 제거 표). 새 leaf를 application에 채택할 때는 starter 또는 composition dependency와 architecture edge policy, qualification/support 상태를 함께 변경한다. -## application이 이 family에 도달하는 경로 (MSG-015, canonical bridge 구현 / legacy cutover 잔존) +## application이 이 family에 도달하는 경로 (MSG-015, cutover 완료) 리뷰(`docs/reviews/2026-08-14-messaging-module-code-review.md` §6.1·§6.3)가 정한 `application-owned port → platform anti-corruption bridge → platform` 경로는 이제 존재한다. @@ -66,25 +72,24 @@ qualification/support 상태를 함께 변경한다. bridge는 exact canonical envelope bytes를 다시 serialize하지 않고, destination/access/admission/runtime lease/transport/outcome normalization/observation은 platform 중앙 publish path를 그대로 탄다. bridge -package는 `messaging-core-api`와 `messaging-schema-api`만 보며 concrete runtime/transport/Kafka 타입을 +package는 `messaging-core-api`만 보며 concrete runtime/transport/Kafka 타입을 직접 보지 않는다. -다만 **MSG-015의 전체 cutover가 끝난 것은 아니다.** legacy `OutboxEvent`와 realtime publisher는 -schema/order/tenant 등 canonical metadata가 부족해 아직 `MessageBroker → KafkaSender` R0 경로를 -사용한다. 그 metadata를 추측해 platform envelope를 만드는 것은 금지한다. 따라서 legacy outbox -storage/relay migration을 별도 change set으로 완료한 뒤에만 R0 seam을 제거한다. +**MSG-015 cutover는 끝났다(2026-09-28, 소유자 결정 7).** application의 outbox append는 canonical +`OutboxAppendPort` 하나이고, relay는 claim한 row의 validated event를 이 bridge로만 보낸다. legacy +`OutboxEvent`·`LegacyOutboxAppendPort`와 `MessageBroker → KafkaSender` R0 경로(TLS/SASL을 적용하지 +않던 두 번째 producer)는 삭제했다. 남은 Kafka producer는 platform의 것 하나이고, broker security +profile(`KafkaSecurityConfigurer`)을 적용한다. canonical bridge의 producer identity는 `app.messaging.producer-id`로 명시한다. 값이 없으면 bridge bean을 -만들지 않으며 `spring.application.name`/host/pod 이름에서 identity를 발명하지 않는다. +만들지 않으며 `spring.application.name`/host/pod 이름에서 identity를 발명하지 않는다. bridge가 없으면 +relay가 발행할 곳이 없으므로 messaging 모듈이 켜진 상태에서는 기동이 거부된다. -`ca-skeleton.outbox.canonical-transport-enabled=true`는 bridge 존재만으로 충분하지 않다. Stable -composition은 애플리케이션 계약 catalog/schema source를 임의로 만들지 않으므로 adopter가 -`IntegrationEventEncoderPort`를 명시적으로 제공해야 한다. 저장소의 후보 구현은 -`JsonSchemaIntegrationEventEncoder`이지만 이를 사용하려면 compiled contract descriptor, -`LocalJsonSchemaRegistry`, payload/envelope limits와 key material에 해당하는 application-owned 입력을 -같이 조립해야 한다. `OutboxCanonicalTransportRequirementValidator`는 publisher 또는 encoder 중 하나라도 -없으면 startup을 거부한다. 즉 canonical flag가 켜졌는데 정본 encoder가 없는 상태로 성공 기동하는 -경로는 없다. +`IntegrationEventEncoderPort`는 템플릿이 제공한다: `adapter:outbound:support`의 +`JsonIntegrationEventEncoder`가 envelope v1(`shared-contract`의 `contracts/messaging/envelope/v1.schema.json`) +안에 payload record를 결정적 JSON으로 싣는다. 계약 catalog는 애플리케이션이 bean으로 등록한 +`IntegrationEventContractContribution`뿐이고, 선언한 payload schema hash가 classpath의 schema 파일과 +다르면 기동이 실패한다. payload를 JSON Schema로 매번 검증하려는 서비스는 자기 encoder bean을 두면 된다. ## Stable 승격에 필요한 증거 diff --git a/messaging/messaging-admin-api/build.gradle b/messaging/messaging-admin-api/build.gradle deleted file mode 100644 index 55c8be5..0000000 --- a/messaging/messaging-admin-api/build.gradle +++ /dev/null @@ -1,8 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-policy') -} diff --git a/messaging/messaging-admin-api/gradle.lockfile b/messaging/messaging-admin-api/gradle.lockfile deleted file mode 100644 index e2c9585..0000000 --- a/messaging/messaging-admin-api/gradle.lockfile +++ /dev/null @@ -1,84 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty=compileClasspath,runtimeClasspath diff --git a/messaging/messaging-admin-runtime/build.gradle b/messaging/messaging-admin-runtime/build.gradle deleted file mode 100644 index feb5781..0000000 --- a/messaging/messaging-admin-runtime/build.gradle +++ /dev/null @@ -1,12 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-admin-api') - api project(':messaging:messaging-transport-spi') - api project(':messaging:messaging-security') - api project(':messaging:messaging-observability') -} diff --git a/messaging/messaging-admin/build.gradle b/messaging/messaging-admin/build.gradle new file mode 100644 index 0000000..a6b6ce5 --- /dev/null +++ b/messaging/messaging-admin/build.gradle @@ -0,0 +1,14 @@ +plugins { + id 'ca.platform-module' +} + +// The admin plane: its contract (package `admin`) and the stdlib services that execute it +// (package `admin.runtime`). They were two leaves, and every consumer of the runtime took the +// contract too, so the split added a registry entry without separating anything (S7-#12). +dependencies { + api project(':messaging:messaging-core-api') + + // ReplayService records a MessagingAuditEvent internally; no public signature names it, so + // Micrometer does not reach the journal adapters that implement the contract. + implementation project(':messaging:messaging-observability') +} diff --git a/messaging/messaging-admin-runtime/gradle.lockfile b/messaging/messaging-admin/gradle.lockfile similarity index 100% rename from messaging/messaging-admin-runtime/gradle.lockfile rename to messaging/messaging-admin/gradle.lockfile diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminApproval.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminApproval.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminApproval.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminApproval.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationJournal.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationJournal.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationJournal.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationJournal.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationLease.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationLease.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationLease.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationLease.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationRecord.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationRecord.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationRecord.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationRecord.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationState.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationState.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/AdminOperationState.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/AdminOperationState.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovalGrant.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovalGrant.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovalGrant.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovalGrant.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovalVerifier.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovalVerifier.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovalVerifier.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovalVerifier.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovedRedrivePlan.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovedRedrivePlan.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovedRedrivePlan.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovedRedrivePlan.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovedReplayPlan.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovedReplayPlan.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ApprovedReplayPlan.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ApprovedReplayPlan.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestinationTopology.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestinationTopology.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestinationTopology.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestinationTopology.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperation.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperation.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperation.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperation.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperationGuard.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperationGuard.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperationGuard.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/DestructiveOperationGuard.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/HmacApprovalVerifier.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/HmacApprovalVerifier.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/HmacApprovalVerifier.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/HmacApprovalVerifier.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/PlanDigest.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/PlanDigest.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/PlanDigest.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/PlanDigest.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedrivePlan.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedrivePlan.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedrivePlan.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedrivePlan.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedriveRequest.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedriveRequest.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedriveRequest.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedriveRequest.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedriveResult.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedriveResult.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/RedriveResult.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/RedriveResult.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayPlan.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayPlan.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayPlan.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayPlan.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayRequest.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayRequest.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayRequest.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayRequest.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayResult.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayResult.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/ReplayResult.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/ReplayResult.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyIssue.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyIssue.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyIssue.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyIssue.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyManagementMode.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyManagementMode.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyManagementMode.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyManagementMode.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyManifest.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyManifest.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyManifest.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyManifest.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyValidationReport.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyValidationReport.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/TopologyValidationReport.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/TopologyValidationReport.java diff --git a/messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/VerifiedApproval.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/VerifiedApproval.java similarity index 100% rename from messaging/messaging-admin-api/src/main/java/dev/caskeleton/messaging/admin/VerifiedApproval.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/VerifiedApproval.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/BrokerTopologyInspector.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/BrokerTopologyInspector.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/BrokerTopologyInspector.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/BrokerTopologyInspector.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/CompositeTopologyValidator.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/CompositeTopologyValidator.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/CompositeTopologyValidator.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/CompositeTopologyValidator.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/DefaultMessagingAdminService.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/DefaultMessagingAdminService.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/DefaultMessagingAdminService.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/DefaultMessagingAdminService.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/DestructiveMessagingAdmin.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/DestructiveMessagingAdmin.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/DestructiveMessagingAdmin.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/DestructiveMessagingAdmin.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/InMemoryAdminOperationJournal.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/InMemoryAdminOperationJournal.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/InMemoryAdminOperationJournal.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/InMemoryAdminOperationJournal.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/MessagingAdminService.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/MessagingAdminService.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/MessagingAdminService.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/MessagingAdminService.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveReport.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveReport.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveReport.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveReport.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveService.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveService.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveService.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/RedriveService.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayReport.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayReport.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayReport.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayReport.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayService.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayService.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayService.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/ReplayService.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntime.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntime.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntime.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntime.java diff --git a/messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidator.java b/messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidator.java similarity index 100% rename from messaging/messaging-admin-runtime/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidator.java rename to messaging/messaging-admin/src/main/java/dev/caskeleton/messaging/admin/runtime/TopologyValidator.java diff --git a/messaging/messaging-admin-api/src/test/java/dev/caskeleton/messaging/admin/DestructiveOperationGuardTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/DestructiveOperationGuardTest.java similarity index 100% rename from messaging/messaging-admin-api/src/test/java/dev/caskeleton/messaging/admin/DestructiveOperationGuardTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/DestructiveOperationGuardTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/AdminOperationJournalTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/AdminOperationJournalTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/AdminOperationJournalTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/AdminOperationJournalTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovalForgeryTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovalForgeryTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovalForgeryTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovalForgeryTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovedPlanExecutionTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovedPlanExecutionTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovedPlanExecutionTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ApprovedPlanExecutionTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/RedriveResumptionTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/RedriveResumptionTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/RedriveResumptionTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/RedriveResumptionTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ReplayServiceTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ReplayServiceTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/ReplayServiceTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/ReplayServiceTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntimeTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntimeTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntimeTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidationRuntimeTest.java diff --git a/messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidatorTest.java b/messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidatorTest.java similarity index 100% rename from messaging/messaging-admin-runtime/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidatorTest.java rename to messaging/messaging-admin/src/test/java/dev/caskeleton/messaging/admin/runtime/TopologyValidatorTest.java diff --git a/messaging/messaging-cloudevents/build.gradle b/messaging/messaging-cloudevents/build.gradle index b65cf72..80f04d9 100644 --- a/messaging/messaging-cloudevents/build.gradle +++ b/messaging/messaging-cloudevents/build.gradle @@ -4,7 +4,6 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') // api, because CloudEventMapper's public signatures return io.cloudevents.CloudEvent. // diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchDeliveryMetadata.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchDeliveryMetadata.java deleted file mode 100644 index 61b9ae7..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchDeliveryMetadata.java +++ /dev/null @@ -1,46 +0,0 @@ -package dev.caskeleton.messaging.api.delivery; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import java.time.Instant; -import java.util.Objects; -import java.util.Optional; - -/** - * Batch-wide facts about one delivered batch. - * - *

{@code orderingUnit} is what makes a batch safe to hand to an ordered destination. A batch - * drawn from two partitions cannot be settled or retried as a unit without reordering one of them, - * so an ordered destination requires the batch to name exactly one ordering unit and the runtime - * rejects a batch that spans more. - * - * @param destination the logical destination - * @param size the number of deliveries in the batch - * @param orderingUnit the single partition or queue the batch was drawn from, when it has one - * @param settlableAsBatch whether the broker can settle the whole batch in one operation - * @param receivedAt when the consumer assembled the batch - */ -public record BatchDeliveryMetadata( - DestinationName destination, - int size, - Optional orderingUnit, - boolean settlableAsBatch, - Instant receivedAt) { - - public BatchDeliveryMetadata { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(orderingUnit, "orderingUnit must not be null"); - Objects.requireNonNull(receivedAt, "receivedAt must not be null"); - if (size < 1) { - throw new IllegalArgumentException("a delivered batch has at least one delivery"); - } - } - - /** - * Reports whether this batch may be handed to a destination with strict ordering. - * - * @return true when the batch came from exactly one ordering unit - */ - public boolean isSafeForOrderedDestination() { - return orderingUnit.isPresent(); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageDelivery.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageDelivery.java deleted file mode 100644 index 54c035c..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageDelivery.java +++ /dev/null @@ -1,29 +0,0 @@ -package dev.caskeleton.messaging.api.delivery; - -import java.util.List; -import java.util.Objects; - -/** - * A batch of decoded messages handed to a batch handler. - * - * @param the payload type - * @param deliveries the individual deliveries, in broker order - * @param metadata batch-wide facts - */ -public record BatchMessageDelivery( - List> deliveries, BatchDeliveryMetadata metadata) { - - public BatchMessageDelivery { - Objects.requireNonNull(deliveries, "deliveries must not be null"); - Objects.requireNonNull(metadata, "metadata must not be null"); - deliveries = List.copyOf(deliveries); - if (deliveries.isEmpty()) { - throw new IllegalArgumentException("a delivered batch is never empty"); - } - if (deliveries.size() != metadata.size()) { - throw new IllegalArgumentException( - "metadata size %d does not match %d deliveries" - .formatted(metadata.size(), deliveries.size())); - } - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageHandler.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageHandler.java deleted file mode 100644 index 655e920..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/BatchMessageHandler.java +++ /dev/null @@ -1,24 +0,0 @@ -package dev.caskeleton.messaging.api.delivery; - -import java.util.concurrent.CompletionStage; - -/** - * The M2 batch consume entry point. - * - *

The handler returns one {@link HandleResult} for the whole batch. On a broker that settles - * batches atomically the runtime applies that result once; on a broker that settles per message it - * applies the same result to each delivery. Either way the handler is not asked to reason about - * which settlement mode it is running under. - * - * @param the payload type - */ -public interface BatchMessageHandler { - - /** - * Handles one delivered batch. - * - * @param batch the batch to handle - * @return a stage completing with the outcome for the whole batch - */ - CompletionStage handle(BatchMessageDelivery batch); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/MessageHandler.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/MessageHandler.java index 4fc54b1..0abdc89 100644 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/MessageHandler.java +++ b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/MessageHandler.java @@ -1,18 +1,33 @@ package dev.caskeleton.messaging.api.delivery; +import dev.caskeleton.messaging.api.destination.MessageDestination; import java.util.concurrent.CompletionStage; /** - * The M1 typed handler implemented by ordinary business code. + * The one consume contract, implemented by ordinary business code. * *

Handlers state an outcome and never touch broker acknowledgement APIs. Duplicate delivery is a - * normal condition, not an error: implementations are expected to be idempotent, or to sit behind - * the Inbox. + * normal condition, not an error: implementations are expected to be idempotent, or to consume a + * destination whose profile declares {@code INBOX_TRANSACTIONAL}, where the platform runs the + * handler inside the inbox reservation's transaction. + * + *

Registering an implementation as a bean is what turns consumption on: the broker starter opens + * a consumer for every destination at least one handler names. There is no second, batch or manual + * settlement contract; the platform settles, after the handler has answered. * * @param the payload type */ public interface MessageHandler { + /** + * The destination, message type and payload type this handler consumes. + * + *

A destination may carry several message types; each type has at most one handler. + * + * @return what this handler consumes + */ + MessageDestination destination(); + /** * Handles one delivery. * diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/PauseResumeController.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/PauseResumeController.java deleted file mode 100644 index 5c86238..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/PauseResumeController.java +++ /dev/null @@ -1,33 +0,0 @@ -package dev.caskeleton.messaging.api.delivery; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import java.util.concurrent.CompletionStage; - -/** - * The M2 consumer flow-control entry point. - * - *

Pausing stops new deliveries; it does not abandon the ones already in flight. A paused - * consumer stays a member of its group and keeps its assignment, which is the point: leaving the - * group to stop consuming would trigger a rebalance and hand the work to another instance that is - * just as overloaded. - */ -public interface PauseResumeController { - - /** - * Stops new deliveries for a destination scope. - * - * @param destination the logical destination - * @param scope the partition, queue, or {@code "*"} for every assigned unit - * @return a stage completing once no further deliveries will be dispatched - */ - CompletionStage pause(DestinationName destination, String scope); - - /** - * Resumes deliveries for a destination scope. - * - * @param destination the logical destination - * @param scope the partition, queue, or {@code "*"} for every assigned unit - * @return a stage completing once deliveries may flow again - */ - CompletionStage resume(DestinationName destination, String scope); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/ProcessingGuarantee.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/ProcessingGuarantee.java deleted file mode 100644 index f6f537d..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/delivery/ProcessingGuarantee.java +++ /dev/null @@ -1,11 +0,0 @@ -package dev.caskeleton.messaging.api.delivery; - -/** How duplicate processing is neutralised once a message has been delivered. */ -public enum ProcessingGuarantee { - - /** The handler suppresses duplicate effects using the message id or a business key. */ - APPLICATION_IDEMPOTENT, - - /** Atomicity holds only inside the transaction scope the broker itself defines. */ - BROKER_TRANSACTIONAL -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/CapabilityRegistry.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/CapabilityRegistry.java deleted file mode 100644 index b4628f5..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/CapabilityRegistry.java +++ /dev/null @@ -1,15 +0,0 @@ -package dev.caskeleton.messaging.api.destination; - -/** Resolves the capability snapshot for a logical destination. */ -public interface CapabilityRegistry { - - /** - * Returns the capabilities of a destination. - * - * @param destination the logical destination - * @return the capability snapshot - * @throws dev.caskeleton.messaging.api.error.MessagingConfigurationException when the destination - * is not registered - */ - DestinationCapabilities capabilities(DestinationName destination); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/MessagingCapabilities.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/MessagingCapabilities.java index fce026c..44cc6c5 100644 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/MessagingCapabilities.java +++ b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/destination/MessagingCapabilities.java @@ -1,5 +1,10 @@ package dev.caskeleton.messaging.api.destination; +import java.util.Collections; +import java.util.EnumSet; +import java.util.Objects; +import java.util.Set; + /** * What a broker adapter can actually prove or do. * @@ -8,32 +13,35 @@ package dev.caskeleton.messaging.api.destination; * capability exception — rather than quietly degrading, because a silently weakened guarantee is * indistinguishable from a working one until the incident. * - * @param brokerAcknowledgement the adapter can prove the broker accepted a publish - * @param replicationOrPersistenceEvidence the adapter can prove replication or persistence - * @param perMessageSettlement individual messages can be settled - * @param batchSettlement batches can be settled as a unit - * @param orderedStream the destination preserves order inside an ordering unit - * @param keyedOrdering order is preserved per key - * @param replay historical messages can be re-read - * @param delayedDelivery delivery can be scheduled for a future instant - * @param brokerTransaction the broker offers a transaction scope - * @param deduplicatedPublish the broker suppresses duplicate publishes of a stable id - * @param nativeDeadLetter the broker provides dead lettering itself - * @param topologyManagement topology can be inspected or created through the adapter + *

A set of named capabilities rather than a row of positional booleans. The record used to take + * twelve booleans in a fixed order, and an adapter declared itself with {@code new + * MessagingCapabilities(true, true, …, false, true)} — a line where swapping two arguments changes + * a guarantee and still compiles. Only the capabilities the platform reads are named: a flag + * nothing consults is a promise nobody checks. + * + * @param supported the capabilities the adapter can prove */ -public record MessagingCapabilities( - boolean brokerAcknowledgement, - boolean replicationOrPersistenceEvidence, - boolean perMessageSettlement, - boolean batchSettlement, - boolean orderedStream, - boolean keyedOrdering, - boolean replay, - boolean delayedDelivery, - boolean brokerTransaction, - boolean deduplicatedPublish, - boolean nativeDeadLetter, - boolean topologyManagement) { +public record MessagingCapabilities(Set supported) { + + /** One capability the platform consults before relying on it. */ + public enum Capability { + + /** The destination preserves order inside an ordering unit; required by pause-and-retry. */ + ORDERED_STREAM, + + /** Delivery can be scheduled for a future instant; required by broker-delayed retry. */ + DELAYED_DELIVERY, + + /** The broker suppresses duplicate publishes of a stable id; required by publish dedup. */ + DEDUPLICATED_PUBLISH + } + + public MessagingCapabilities { + Objects.requireNonNull(supported, "supported must not be null"); + supported = + Collections.unmodifiableSet( + supported.isEmpty() ? EnumSet.noneOf(Capability.class) : EnumSet.copyOf(supported)); + } /** * Returns a capability set with nothing enabled. @@ -41,7 +49,27 @@ public record MessagingCapabilities( * @return the empty capability set */ public static MessagingCapabilities none() { - return new MessagingCapabilities( - false, false, false, false, false, false, false, false, false, false, false, false); + return new MessagingCapabilities(EnumSet.noneOf(Capability.class)); + } + + /** + * Returns a capability set holding exactly the named capabilities. + * + * @param first the first capability + * @param rest the remaining capabilities + * @return the capability set + */ + public static MessagingCapabilities of(Capability first, Capability... rest) { + return new MessagingCapabilities(EnumSet.of(first, rest)); + } + + /** + * Reports whether the adapter can prove a capability. + * + * @param capability the capability to look up + * @return true when it is supported + */ + public boolean supports(Capability capability) { + return supported.contains(Objects.requireNonNull(capability, "capability must not be null")); } } diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageAuthenticationException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageAuthenticationException.java deleted file mode 100644 index c7bc1e8..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageAuthenticationException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** Broker authentication failed. */ -public class MessageAuthenticationException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.AUTHENTICATION; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageAuthenticationException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageAuthenticationException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageAuthenticationException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageBrokerUnavailableException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageBrokerUnavailableException.java deleted file mode 100644 index 65a9640..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageBrokerUnavailableException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** The broker could not be reached. */ -public class MessageBrokerUnavailableException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.TRANSIENT_INFRASTRUCTURE; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageBrokerUnavailableException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageBrokerUnavailableException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageBrokerUnavailableException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageConsumerException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageConsumerException.java deleted file mode 100644 index 3b3b095..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageConsumerException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** The consumer runtime failed while handling a delivery. */ -public class MessageConsumerException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PROCESSING_TRANSIENT; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageConsumerException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageConsumerException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageConsumerException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageDeadLetterException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageDeadLetterException.java deleted file mode 100644 index 210acd7..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageDeadLetterException.java +++ /dev/null @@ -1,54 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** - * Publishing to the dead letter destination failed. - * - *

The source message stays unsettled. Acknowledging a source whose DLQ publish failed would - * destroy the only remaining copy. - */ -public class MessageDeadLetterException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.TRANSIENT_INFRASTRUCTURE; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageDeadLetterException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageDeadLetterException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageDeadLetterException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHandlerTimeoutException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHandlerTimeoutException.java deleted file mode 100644 index 9632b7f..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHandlerTimeoutException.java +++ /dev/null @@ -1,54 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** - * The handler exceeded its timeout. - * - *

Whether the handler committed a side effect before the deadline is unknown, so the delivery is - * recorded as possibly duplicated when it is retried. - */ -public class MessageHandlerTimeoutException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PROCESSING_TRANSIENT; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageHandlerTimeoutException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageHandlerTimeoutException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageHandlerTimeoutException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHeaderRejectedException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHeaderRejectedException.java deleted file mode 100644 index 7730c35..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageHeaderRejectedException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** A header was reserved, secret, or exceeded a limit. */ -public class MessageHeaderRejectedException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PERMANENT_BUSINESS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageHeaderRejectedException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageHeaderRejectedException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageHeaderRejectedException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishAmbiguousException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishAmbiguousException.java deleted file mode 100644 index 26af714..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishAmbiguousException.java +++ /dev/null @@ -1,54 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** - * The publish outcome could not be determined. - * - *

The broker may hold the message. Republishing is permitted only under the same logical message - * id, so that broker deduplication or a downstream Inbox can collapse the duplicate. - */ -public class MessagePublishAmbiguousException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.AMBIGUOUS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessagePublishAmbiguousException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessagePublishAmbiguousException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessagePublishAmbiguousException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishRejectedException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishRejectedException.java deleted file mode 100644 index 6f913b1..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessagePublishRejectedException.java +++ /dev/null @@ -1,53 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** - * The broker definitively refused the publish. - * - *

The message is known not to be stored, so abandoning this attempt is safe. - */ -public class MessagePublishRejectedException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PERMANENT_BUSINESS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessagePublishRejectedException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessagePublishRejectedException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessagePublishRejectedException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRedriveException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRedriveException.java deleted file mode 100644 index 9d4b657..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRedriveException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** A redrive operation failed. */ -public class MessageRedriveException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.TRANSIENT_INFRASTRUCTURE; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageRedriveException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageRedriveException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageRedriveException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRetryExhaustedException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRetryExhaustedException.java deleted file mode 100644 index 2ffea59..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRetryExhaustedException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** The retry policy's attempt budget was exhausted. */ -public class MessageRetryExhaustedException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PERMANENT_BUSINESS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageRetryExhaustedException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageRetryExhaustedException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageRetryExhaustedException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRoutingException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRoutingException.java deleted file mode 100644 index c5ea68f..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageRoutingException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** The broker accepted the publish but could not route it. */ -public class MessageRoutingException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.PERMANENT_BUSINESS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageRoutingException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageRoutingException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageRoutingException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementException.java deleted file mode 100644 index 63cd72b..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementException.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** Settlement was refused by the broker. */ -public class MessageSettlementException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.TRANSIENT_INFRASTRUCTURE; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageSettlementException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, true, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageSettlementException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - true, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageSettlementException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementUnknownException.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementUnknownException.java deleted file mode 100644 index 27072f1..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/MessageSettlementUnknownException.java +++ /dev/null @@ -1,53 +0,0 @@ -package dev.caskeleton.messaging.api.error; - -import java.io.Serial; -import java.util.Optional; - -/** - * Settlement was sent but not confirmed. - * - *

Redelivery is possible. Callers must not treat this as a successful settlement. - */ -public class MessageSettlementUnknownException extends MessagingException { - - @Serial private static final long serialVersionUID = 1L; - - private static final FailureCategory CATEGORY = FailureCategory.AMBIGUOUS; - - /** - * Creates the exception with a stable code and sanitized message. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - */ - public MessageSettlementUnknownException(String code, String sanitizedMessage) { - super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty())); - } - - /** - * Creates the exception with an originating cause. - * - * @param code the stable failure code - * @param sanitizedMessage the operator-facing description - * @param cause the originating throwable - */ - public MessageSettlementUnknownException(String code, String sanitizedMessage, Throwable cause) { - super( - new FailureDescriptor( - CATEGORY, - code, - false, - sanitizedMessage, - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName())), - cause); - } - - /** - * Creates the exception from an explicit descriptor. - * - * @param failure the sanitized failure description - */ - public MessageSettlementUnknownException(FailureDescriptor failure) { - super(failure); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchMessagePublisher.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchMessagePublisher.java deleted file mode 100644 index 6a56824..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchMessagePublisher.java +++ /dev/null @@ -1,25 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import java.util.List; -import java.util.concurrent.CompletionStage; - -/** - * The M2 batch publish entry point. - * - *

A batch is a throughput optimisation, not a transaction. Entries succeed, fail, and go - * ambiguous independently, and {@link BatchPublishResult} keeps every entry's outcome against its - * submission index so the caller can act per entry. Nothing here rolls back: an adapter that - * reported a confirmed entry cannot un-publish it because a later entry was rejected. - */ -public interface BatchMessagePublisher { - - /** - * Publishes a batch of independent requests. - * - * @param requests the entries to publish, in submission order - * @param options the batch-wide options - * @return a stage completing with one outcome per submitted entry - */ - CompletionStage publish( - List> requests, BatchPublishOptions options); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishItemResult.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishItemResult.java deleted file mode 100644 index a275061..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishItemResult.java +++ /dev/null @@ -1,27 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import dev.caskeleton.messaging.api.MessageId; -import java.util.Objects; - -/** - * One item's outcome inside a batch publish. - * - *

Kept per index rather than collapsed into a batch-level success flag. A batch is not a - * transaction: some items confirm, some are rejected, and some are ambiguous, and the caller has to - * be able to act differently on each. A single boolean would force it to re-submit everything, - * which duplicates the items that already landed. - * - * @param index the item's position in the submitted list - * @param messageId the item's logical identity - * @param result the item's outcome and evidence - */ -public record BatchPublishItemResult(int index, MessageId messageId, PublishResult result) { - - public BatchPublishItemResult { - Objects.requireNonNull(messageId, "messageId must not be null"); - Objects.requireNonNull(result, "result must not be null"); - if (index < 0) { - throw new IllegalArgumentException("index must not be negative"); - } - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishOptions.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishOptions.java deleted file mode 100644 index 8361438..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishOptions.java +++ /dev/null @@ -1,42 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import java.time.Duration; -import java.util.Objects; - -/** - * Options covering a whole batch publish. - * - *

There is no retry setting. A batch is not a transaction, and retrying the batch would resubmit - * entries that already confirmed, so the platform never retries one on the caller's behalf; the - * caller resubmits the individual entries it knows are duplicate-safe under their original {@code - * messageId}. - * - * @param timeout the deadline for the whole batch - * @param maxBatchSize the largest number of entries accepted in one call - * @param stopOnFirstRejection whether to skip remaining entries once one is rejected - */ -public record BatchPublishOptions( - Duration timeout, int maxBatchSize, boolean stopOnFirstRejection) { - - private static final Duration DEFAULT_TIMEOUT = Duration.ofSeconds(30); - private static final int DEFAULT_MAX_BATCH_SIZE = 500; - - public BatchPublishOptions { - Objects.requireNonNull(timeout, "timeout must not be null"); - if (timeout.isNegative() || timeout.isZero()) { - throw new IllegalArgumentException("batch timeout must be positive"); - } - if (maxBatchSize < 1) { - throw new IllegalArgumentException("maxBatchSize must be at least 1"); - } - } - - /** - * Returns the defaults: thirty seconds, 500 entries, best-effort completion of every entry. - * - * @return the default options - */ - public static BatchPublishOptions defaults() { - return new BatchPublishOptions(DEFAULT_TIMEOUT, DEFAULT_MAX_BATCH_SIZE, false); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishResult.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishResult.java deleted file mode 100644 index 707bee0..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BatchPublishResult.java +++ /dev/null @@ -1,44 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import java.time.Duration; -import java.util.List; -import java.util.Objects; - -/** - * The outcome of one batch publish. - * - *

There is no automatic batch retry. Re-submitting a whole batch because part of it failed - * duplicates the part that succeeded; the caller re-submits only the items it decides are - * duplicate-safe, under their original message ids. - * - * @param items each item's outcome, in submission order - * @param elapsed how long the batch took - */ -public record BatchPublishResult(List items, Duration elapsed) { - - public BatchPublishResult { - Objects.requireNonNull(items, "items must not be null"); - Objects.requireNonNull(elapsed, "elapsed must not be null"); - items = List.copyOf(items); - } - - /** - * Returns the items with a given completion. - * - * @param completion the completion to select - * @return the matching items - */ - public List withCompletion(PublishCompletion completion) { - return items.stream().filter(item -> item.result().completion() == completion).toList(); - } - - /** - * Reports whether every item confirmed. - * - * @return true when nothing was rejected or ambiguous - */ - public boolean allConfirmed() { - return items.stream() - .allMatch(item -> item.result().completion() == PublishCompletion.CONFIRMED); - } -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BlockingMessagePublisher.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BlockingMessagePublisher.java deleted file mode 100644 index 5b81bf8..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/BlockingMessagePublisher.java +++ /dev/null @@ -1,30 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; - -/** - * The blocking facade over the asynchronous publisher. - * - *

Offered because most business code is written against a blocking data access layer, and - * forcing a {@code CompletionStage} through it produces worse code than a blocking call does. It is - * a facade rather than a second implementation: the core stays asynchronous, and this only converts - * the lifecycle. - * - *

It respects the publish deadline rather than waiting indefinitely, so a stalled broker - * surfaces as a timeout with ambiguous evidence instead of an exhausted thread pool. - */ -public interface BlockingMessagePublisher { - - /** - * Publishes one message and waits for its outcome. - * - * @param the payload type - * @param destination the logical destination - * @param message the envelope to publish - * @param options per-call options, including the deadline - * @return the outcome and its evidence - */ - PublishResult publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/DelayedMessagePublisher.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/DelayedMessagePublisher.java deleted file mode 100644 index 9b9749a..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/DelayedMessagePublisher.java +++ /dev/null @@ -1,28 +0,0 @@ -package dev.caskeleton.messaging.api.publish; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import java.time.Instant; -import java.util.concurrent.CompletionStage; - -/** - * The M2 scheduled-delivery entry point. - * - *

Only destinations whose runtime declares {@code delayedDelivery} accept these calls. A plain - * Kafka topic does not: emulating a delay by holding the message in the producer would make the - * publisher's own liveness the schedule, so the platform rejects the call instead of pretending. - */ -public interface DelayedMessagePublisher { - - /** - * Publishes a message the broker must not deliver before the given instant. - * - * @param the payload type - * @param destination the logical destination - * @param message the envelope to publish - * @param deliverAt the earliest delivery time - * @return a stage completing with the publish outcome and its evidence - */ - CompletionStage publish( - MessageDestination destination, MessageEnvelope message, Instant deliverAt); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/PublishResult.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/PublishResult.java index 7449c3a..3582dd7 100644 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/PublishResult.java +++ b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/publish/PublishResult.java @@ -100,6 +100,70 @@ public record PublishResult( } } + /** + * A publish the broker confirmed. + * + * @param level the confirmation the broker supplied + * @param routing whether the broker routed it; a broker with no routing step reports {@link + * RoutingOutcome#NOT_APPLICABLE} + * @param position the broker coordinate, when the broker reported one + * @param elapsed wall-clock time spent + * @return the confirmed result + */ + public static PublishResult confirmed( + ConfirmationLevel level, + RoutingOutcome routing, + Optional position, + Duration elapsed) { + return new PublishResult( + PublishCompletion.CONFIRMED, + PublishEvidence.confirmed(level), + routing, + position, + 1, + elapsed, + Optional.empty()); + } + + /** + * A publish refused before anything left the process. + * + *

Routing is {@link RoutingOutcome#NOT_APPLICABLE}: nothing reached a broker that could have + * routed it. + * + * @param elapsed wall-clock time spent + * @param failure why it was refused + * @return the rejected result + */ + public static PublishResult rejected(Duration elapsed, FailureDescriptor failure) { + return new PublishResult( + PublishCompletion.REJECTED, + PublishEvidence.notTransmitted(), + RoutingOutcome.NOT_APPLICABLE, + Optional.empty(), + 1, + elapsed, + Optional.of(failure)); + } + + /** + * A publish whose fate is unknown: the broker may or may not hold the message. + * + * @param elapsed wall-clock time spent + * @param failure what made the outcome unknowable + * @return the ambiguous result + */ + public static PublishResult ambiguous(Duration elapsed, FailureDescriptor failure) { + return new PublishResult( + PublishCompletion.AMBIGUOUS, + PublishEvidence.ambiguous(), + RoutingOutcome.UNKNOWN, + Optional.empty(), + 1, + elapsed, + Optional.of(failure)); + } + /** * Reports whether the broker may be holding this message. * diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/ManualMessageHandler.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/ManualMessageHandler.java deleted file mode 100644 index 69f1620..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/ManualMessageHandler.java +++ /dev/null @@ -1,25 +0,0 @@ -package dev.caskeleton.messaging.api.settlement; - -import dev.caskeleton.messaging.api.delivery.MessageDelivery; -import java.util.concurrent.CompletionStage; - -/** - * The M2 handler that settles its own deliveries. - * - *

Available only to modules granted the advanced capability. A handler that returns without - * calling a terminal settlement method is a defect the runtime detects and reports rather than - * silently leaving the message in flight. - * - * @param the payload type - */ -public interface ManualMessageHandler { - - /** - * Handles one delivery, settling it through the supplied controller. - * - * @param delivery the decoded message and its metadata - * @param settlement the settlement handle for this delivery - * @return a stage completing when handling has finished - */ - CompletionStage handle(MessageDelivery delivery, SettlementController settlement); -} diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/SettlementController.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/SettlementController.java deleted file mode 100644 index e17e6b6..0000000 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/settlement/SettlementController.java +++ /dev/null @@ -1,50 +0,0 @@ -package dev.caskeleton.messaging.api.settlement; - -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import java.time.Duration; -import java.util.concurrent.CompletionStage; - -/** - * The M2 manual settlement handle. - * - *

No method here accepts a broker channel, delivery tag, or offset. Manual settlement changes - * when the platform settles, never who settles, so the same DLQ-before-ack - * ordering and payload limits still apply. - * - *

Exactly one terminal call is permitted per delivery; a second call fails rather than racing - * the first. - */ -public interface SettlementController { - - /** - * Acknowledges the message. - * - * @return a stage completing with the settlement outcome - */ - CompletionStage ack(); - - /** - * Schedules the message for another attempt. - * - * @param delay how long to wait before redelivery - * @return a stage completing with the settlement outcome - */ - CompletionStage retry(Duration delay); - - /** - * Routes the message to the dead letter destination, settling the source only after that publish - * is confirmed. - * - * @param failure the sanitized failure description - * @return a stage completing with the settlement outcome - */ - CompletionStage deadLetter(FailureDescriptor failure); - - /** - * Discards the message without dead lettering. - * - * @param failure the sanitized failure description - * @return a stage completing with the settlement outcome - */ - CompletionStage reject(FailureDescriptor failure); -} diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/BoundedByteSink.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/BoundedByteSink.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/BoundedByteSink.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/BoundedByteSink.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessage.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessage.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessage.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessage.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessagePublisher.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessagePublisher.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessagePublisher.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/EncodedMessagePublisher.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodec.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodec.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodec.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodec.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodecRegistry.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodecRegistry.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodecRegistry.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageCodecRegistry.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageContractKey.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageContractKey.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/MessageContractKey.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/MessageContractKey.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/RawBytesMessageCodec.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/RawBytesMessageCodec.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/RawBytesMessageCodec.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/RawBytesMessageCodec.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibility.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibility.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibility.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibility.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidator.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidator.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidator.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidator.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaReference.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaReference.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaReference.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaReference.java diff --git a/messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaRegistry.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaRegistry.java similarity index 100% rename from messaging/messaging-schema-api/src/main/java/dev/caskeleton/messaging/schema/SchemaRegistry.java rename to messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/schema/SchemaRegistry.java diff --git a/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/destination/DestinationCapabilityTest.java b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/destination/DestinationCapabilityTest.java index eaf9e32..da9c270 100644 --- a/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/destination/DestinationCapabilityTest.java +++ b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/destination/DestinationCapabilityTest.java @@ -31,9 +31,24 @@ class DestinationCapabilityTest { void emptyCapabilitySetAdvertisesNothing() { MessagingCapabilities capabilities = MessagingCapabilities.none(); - assertThat(capabilities.brokerAcknowledgement()).isFalse(); - assertThat(capabilities.replay()).isFalse(); - assertThat(capabilities.brokerTransaction()).isFalse(); + assertThat(capabilities.supported()).isEmpty(); + } + + @Test + @org.junit.jupiter.api.DisplayName("이름 붙인 capability만 지원하고 넘긴 집합을 나중에 바꿔도 영향이 없다") + void aCapabilitySetSupportsExactlyWhatWasNamedAndIgnoresLaterChangesToTheInput() { + // given + java.util.Set input = + java.util.EnumSet.of(MessagingCapabilities.Capability.ORDERED_STREAM); + MessagingCapabilities capabilities = new MessagingCapabilities(input); + + // when + input.add(MessagingCapabilities.Capability.DEDUPLICATED_PUBLISH); + + // then + assertThat(capabilities.supports(MessagingCapabilities.Capability.ORDERED_STREAM)).isTrue(); + assertThat(capabilities.supports(MessagingCapabilities.Capability.DEDUPLICATED_PUBLISH)) + .isFalse(); } @Test diff --git a/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/publish/PublishResultTest.java b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/publish/PublishResultTest.java index 2e1add5..122ede7 100644 --- a/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/publish/PublishResultTest.java +++ b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/api/publish/PublishResultTest.java @@ -7,6 +7,7 @@ import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import java.time.Duration; import java.util.Optional; +import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; class PublishResultTest { @@ -239,4 +240,47 @@ class PublishResultTest { Optional.empty())) .isInstanceOf(IllegalArgumentException.class); } + + @Test + @DisplayName("거부 factory는 전송되지 않았고 라우팅 대상이 아닌 결과를 만든다") + void theRejectedFactoryReportsNothingWasTransmittedOrRouted() { + // when + PublishResult result = PublishResult.rejected(Duration.ofMillis(3), AMBIGUOUS_FAILURE); + + // then + assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); + assertThat(result.evidence()).isEqualTo(PublishEvidence.notTransmitted()); + assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.NOT_APPLICABLE); + assertThat(result.failure()).contains(AMBIGUOUS_FAILURE); + } + + @Test + @DisplayName("모호 factory는 브로커가 메시지를 가졌을 수 있다고 보고한다") + void theAmbiguousFactoryReportsTheBrokerMayHoldTheMessage() { + // when + PublishResult result = PublishResult.ambiguous(Duration.ofMillis(3), AMBIGUOUS_FAILURE); + + // then + assertThat(result.mayHaveBeenStored()).isTrue(); + assertThat(result.evidence()).isEqualTo(PublishEvidence.ambiguous()); + assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.UNKNOWN); + } + + @Test + @DisplayName("확인 factory는 브로커 수락과 위치를 담고 실패 설명은 없다") + void theConfirmedFactoryCarriesAcceptanceAndPositionWithoutAFailure() { + // when + PublishResult result = + PublishResult.confirmed( + ConfirmationLevel.BROKER_ACK, + RoutingOutcome.NOT_APPLICABLE, + Optional.of(POSITION), + Duration.ofMillis(3)); + + // then + assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); + assertThat(result.evidence().brokerAccepted()).isTrue(); + assertThat(result.position()).contains(POSITION); + assertThat(result.failure()).isEmpty(); + } } diff --git a/messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/BoundedByteSinkTest.java b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/BoundedByteSinkTest.java similarity index 100% rename from messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/BoundedByteSinkTest.java rename to messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/BoundedByteSinkTest.java diff --git a/messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/RawBytesMessageCodecTest.java b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/RawBytesMessageCodecTest.java similarity index 100% rename from messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/RawBytesMessageCodecTest.java rename to messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/RawBytesMessageCodecTest.java diff --git a/messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidatorTest.java b/messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidatorTest.java similarity index 100% rename from messaging/messaging-schema-api/src/test/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidatorTest.java rename to messaging/messaging-core-api/src/test/java/dev/caskeleton/messaging/schema/SchemaCompatibilityValidatorTest.java diff --git a/messaging/messaging-inbox-jdbc-postgresql/build.gradle b/messaging/messaging-inbox-jdbc-postgresql/build.gradle deleted file mode 100644 index bf339f0..0000000 --- a/messaging/messaging-inbox-jdbc-postgresql/build.gradle +++ /dev/null @@ -1,33 +0,0 @@ -plugins { - id 'ca.auxiliary-source-set' - id 'ca.platform-module' -} - -auxiliarySourceSets { - sourceSet('integrationTest') { - compilesAgainst 'main' - } -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-reliability-api') - - implementation 'org.springframework:spring-jdbc' - implementation 'org.springframework:spring-tx' - - // Live-database certification. The reliability patterns are claims about transaction - // boundaries and uniqueness constraints, and only a real database can settle them. - testImplementation project(':messaging:messaging-testkit') - integrationTestImplementation 'org.testcontainers:testcontainers-postgresql' - integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' - integrationTestImplementation 'org.postgresql:postgresql' -} - -strictTestLanes { - lane('integrationTest') { - integration() - sourceSet = 'integrationTest' - description = 'Runs inbox transaction and uniqueness contracts against real PostgreSQL.' - } -} diff --git a/messaging/messaging-inbox-jdbc-postgresql/gradle.lockfile b/messaging/messaging-inbox-jdbc-postgresql/gradle.lockfile deleted file mode 100644 index 865fc31..0000000 --- a/messaging/messaging-inbox-jdbc-postgresql/gradle.lockfile +++ /dev/null @@ -1,106 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.fasterxml.jackson.core:jackson-annotations:2.21=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.github.docker-java:docker-java-api:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.docker-java:docker-java-transport-zerodep:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.docker-java:docker-java-transport:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-codec:commons-codec:1.19.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.20.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -commons-io:commons-io:2.21.0=spotbugs -commons-logging:commons-logging:1.3.6=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -net.java.dev.jna:jna:5.18.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-compress:1.28.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.apache.commons:commons-lang3:3.20.0=checkstyle,integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,testCompileClasspath -org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.checkerframework:checker-qual:3.55.1=integrationTestRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.javassist:javassist:3.28.0-GA=checkstyle -org.jetbrains:annotations:17.0.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,integrationTestAnnotationProcessor,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=integrationTestRuntimeClasspath,testRuntimeClasspath -org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -org.postgresql:postgresql:42.7.13=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.reflections:reflections:0.10.2=checkstyle -org.rnorth.duct-tape:duct-tape:1.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.slf4j:slf4j-api:2.0.18=integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.springframework:spring-beans:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-core:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-jdbc:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-tx:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.testcontainers:testcontainers-database-commons:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers-jdbc:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers-junit-jupiter:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers-postgresql:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty= diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java b/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java deleted file mode 100644 index 7195d4e..0000000 --- a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/TransactionalInboxHandler.java +++ /dev/null @@ -1,104 +0,0 @@ -package dev.caskeleton.messaging.inbox; - -import dev.caskeleton.messaging.api.delivery.HandleResult; -import dev.caskeleton.messaging.api.delivery.MessageDelivery; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.reliability.IdempotentMessageHandler; -import dev.caskeleton.messaging.reliability.InboxResult; -import dev.caskeleton.messaging.reliability.TransactionalMessageAction; -import java.time.Instant; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.function.Supplier; - -/** - * The JDBC-backed {@link IdempotentMessageHandler}. - * - *

Reservation and effect commit together, in the runner's single transaction. Everything else - * about this class follows from that: it cannot settle the message (settlement is not - * transactional), it cannot publish (the publish would survive a rollback), and it cannot catch and - * swallow the action's exception (the rollback is how the reservation is undone). - * - *

An already-reserved message returns success. Under at-least-once delivery a duplicate is the - * normal case, not an error; reporting failure would send a message whose work is complete to the - * dead-letter queue. - * - * @param the payload type - */ -public final class TransactionalInboxHandler implements IdempotentMessageHandler { - - private final IdempotentConsumer consumer; - private final Supplier clock; - - /** - * Creates a handler. - * - * @param consumer the reservation-and-effect runner - * @param clock supplies the current instant - */ - public TransactionalInboxHandler(IdempotentConsumer consumer, Supplier clock) { - this.consumer = Objects.requireNonNull(consumer, "consumer must not be null"); - this.clock = Objects.requireNonNull(clock, "clock must not be null"); - } - - @Override - public CompletionStage handleOnce( - String consumerName, MessageDelivery delivery, TransactionalMessageAction action) { - Objects.requireNonNull(delivery, "delivery must not be null"); - Objects.requireNonNull(action, "action must not be null"); - if (consumerName == null || consumerName.isBlank()) { - throw new IllegalArgumentException("consumerName must not be blank"); - } - - try { - InboxOutcome outcome = - consumer.runOnce( - delivery.message().messageId(), - consumerName, - clock.get(), - () -> { - try { - action.apply(delivery); - } catch (Exception failure) { - // Wrapped, not swallowed: the transaction runner has to see a throw to roll the - // reservation back along with the effect. - throw new ActionFailedException(failure); - } - return InboxResult.APPLIED; - }); - - return CompletableFuture.completedFuture( - outcome.processed() ? HandleResult.success() : duplicateIsSuccess()); - } catch (ActionFailedException failure) { - return CompletableFuture.completedFuture(retryable(failure.getCause())); - } - } - - private static HandleResult duplicateIsSuccess() { - // The effect already ran in an earlier delivery. Settling is correct; redelivering is not. - return HandleResult.success(); - } - - private static HandleResult retryable(Throwable cause) { - return new HandleResult.Retry( - new FailureDescriptor( - FailureCategory.TRANSIENT_INFRASTRUCTURE, - "INBOX_ACTION_FAILED", - true, - "the transactional action failed and its reservation was rolled back", - Optional.ofNullable(cause).map(value -> value.getClass().getSimpleName()))); - } - - /** Carries the action's checked exception out through the transaction runner. */ - private static final class ActionFailedException extends RuntimeException { - - private static final long serialVersionUID = 1L; - - private ActionFailedException(Throwable cause) { - super(cause); - } - } -} diff --git a/messaging/messaging-kafka-share-experimental/build.gradle b/messaging/messaging-kafka-share-experimental/build.gradle deleted file mode 100644 index 33edc25..0000000 --- a/messaging/messaging-kafka-share-experimental/build.gradle +++ /dev/null @@ -1,11 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-transport-spi') - api project(':messaging:messaging-kafka') - -} diff --git a/messaging/messaging-kafka-share-experimental/gradle.lockfile b/messaging/messaging-kafka-share-experimental/gradle.lockfile deleted file mode 100644 index eaf97d5..0000000 --- a/messaging/messaging-kafka-share-experimental/gradle.lockfile +++ /dev/null @@ -1,93 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -at.yawk.lz4:lz4-java:1.10.1=runtimeClasspath,testRuntimeClasspath -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.luben:zstd-jni:1.5.6-10=runtimeClasspath,testRuntimeClasspath -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.kafka:kafka-clients:4.1.2=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=runtimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=runtimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=runtimeClasspath,spotbugs,spotbugsSlf4j,testRuntimeClasspath -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xerial.snappy:snappy-java:1.1.10.7=runtimeClasspath,testRuntimeClasspath -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty= diff --git a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareGroupRegistrar.java b/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareGroupRegistrar.java deleted file mode 100644 index 07c9ec1..0000000 --- a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareGroupRegistrar.java +++ /dev/null @@ -1,48 +0,0 @@ -package dev.caskeleton.messaging.kafka.share; - -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.transport.TransportConsumerRegistration; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import java.util.Objects; - -/** - * Registers a share group consumer, once the experimental guard has passed. - * - *

Pause and resume are refused rather than silently ignored. A share group has no partition - * assignment to pause, so accepting the call would let a retry policy that depends on pausing - * appear to work while doing nothing. - */ -public final class KafkaShareGroupRegistrar { - - private final KafkaShareProfileValidator validator; - - /** Creates a registrar with the default validator. */ - public KafkaShareGroupRegistrar() { - this(new KafkaShareProfileValidator()); - } - - /** - * Creates a registrar with an explicit validator. - * - * @param validator the share profile validator - */ - public KafkaShareGroupRegistrar(KafkaShareProfileValidator validator) { - this.validator = Objects.requireNonNull(validator, "validator must not be null"); - } - - /** - * Registers a share group consumer. - * - * @param profile the share group profile - * @param spec what to consume and where to deliver it - * @return the registration - */ - public TransportConsumerRegistration register( - KafkaShareProfile profile, TransportConsumerSpec spec) { - Objects.requireNonNull(spec, "spec must not be null"); - validator.validate(profile); - throw new MessagingCapabilityUnavailableException( - "KAFKA_SHARE_CONSUMER_NOT_IMPLEMENTED", - "Kafka Share registration is not implemented; refusing to advertise an active consumer"); - } -} diff --git a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfile.java b/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfile.java deleted file mode 100644 index 276207c..0000000 --- a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfile.java +++ /dev/null @@ -1,33 +0,0 @@ -package dev.caskeleton.messaging.kafka.share; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationName; -import java.util.Objects; - -/** - * Settings for a Kafka Share Group work queue. - * - * @param destination the logical destination - * @param shareGroup the share group name - * @param orderingScope the ordering the destination declares - * @param enabled whether the experimental adapter is switched on - * @param maxDeliveryCount how many times a record may be re-acquired before it is released - */ -public record KafkaShareProfile( - DestinationName destination, - String shareGroup, - OrderingScope orderingScope, - boolean enabled, - int maxDeliveryCount) { - - public KafkaShareProfile { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(orderingScope, "orderingScope must not be null"); - if (shareGroup == null || shareGroup.isBlank()) { - throw new IllegalArgumentException("shareGroup must not be blank"); - } - if (maxDeliveryCount < 1) { - throw new IllegalArgumentException("maxDeliveryCount must be at least 1"); - } - } -} diff --git a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidator.java b/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidator.java deleted file mode 100644 index 1bd0390..0000000 --- a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidator.java +++ /dev/null @@ -1,42 +0,0 @@ -package dev.caskeleton.messaging.kafka.share; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import java.util.Objects; - -/** - * Guards the Kafka Share Group experimental adapter. - * - *

A share group hands individual records to competing consumers and acknowledges them - * individually. That is a work queue, and it is fundamentally incompatible with partition ordering: - * two consumers in the same share group can process records from one partition concurrently and - * finish in either order. Configuring an ordered destination on a share group would therefore - * advertise a guarantee the broker is not providing, so it is refused rather than degraded. - * - *

The adapter is also off unless explicitly enabled, so an Experimental capability cannot drift - * into a Stable deployment by default. - */ -public final class KafkaShareProfileValidator { - - /** - * Validates a share group profile. - * - * @param profile the profile to validate - * @throws IllegalArgumentException when the destination requires ordering - * @throws MessagingCapabilityUnavailableException when the experimental adapter is disabled - */ - public void validate(KafkaShareProfile profile) { - Objects.requireNonNull(profile, "profile must not be null"); - - if (!profile.enabled()) { - throw new MessagingCapabilityUnavailableException( - "KAFKA_SHARE_DISABLED", - "the Kafka Share Group adapter is experimental and disabled unless " - + "backend.messaging.experimental.kafka-share=true"); - } - if (profile.orderingScope() != OrderingScope.NONE) { - throw new IllegalArgumentException( - "a Kafka share group cannot provide ordered delivery: " + profile.destination().value()); - } - } -} diff --git a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareWorkQueueCapability.java b/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareWorkQueueCapability.java deleted file mode 100644 index afa4907..0000000 --- a/messaging/messaging-kafka-share-experimental/src/main/java/dev/caskeleton/messaging/kafka/share/KafkaShareWorkQueueCapability.java +++ /dev/null @@ -1,27 +0,0 @@ -package dev.caskeleton.messaging.kafka.share; - -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; - -/** - * What a Kafka Share Group can and cannot do. - * - *

Declared as a capability rather than assumed, so that the shared validators refuse an ordered - * or replayed destination on this adapter before a message is ever produced. - */ -public final class KafkaShareWorkQueueCapability { - - private KafkaShareWorkQueueCapability() {} - - /** - * Returns the share group capability set. - * - *

Per-record settlement, yes. Ordering, replay, and transactions, no — a share group gives up - * exactly those to gain competing-consumer throughput. - * - * @return the capabilities - */ - public static MessagingCapabilities capabilities() { - return new MessagingCapabilities( - true, true, true, false, false, false, false, false, false, false, false, false); - } -} diff --git a/messaging/messaging-kafka-share-experimental/src/test/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidatorTest.java b/messaging/messaging-kafka-share-experimental/src/test/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidatorTest.java deleted file mode 100644 index e2ecf9a..0000000 --- a/messaging/messaging-kafka-share-experimental/src/test/java/dev/caskeleton/messaging/kafka/share/KafkaShareProfileValidatorTest.java +++ /dev/null @@ -1,113 +0,0 @@ -package dev.caskeleton.messaging.kafka.share; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import org.junit.jupiter.api.Test; - -class KafkaShareProfileValidatorTest { - - private final KafkaShareProfileValidator validator = new KafkaShareProfileValidator(); - - @Test - void rejectsOrderedStreamUse() { - KafkaShareProfile profile = KafkaShareProfileFixtures.profile(OrderingScope.KEY); - - assertThatThrownBy(() -> validator.validate(profile)) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("ordered"); - } - - @Test - void rejectsPartitionOrderingToo() { - KafkaShareProfile profile = KafkaShareProfileFixtures.profile(OrderingScope.PARTITION); - - assertThatThrownBy(() -> validator.validate(profile)) - .isInstanceOf(IllegalArgumentException.class); - } - - @Test - void staysDisabledUnlessExplicitlyEnabled() { - KafkaShareProfile profile = KafkaShareProfileFixtures.disabled(); - - assertThatThrownBy(() -> validator.validate(profile)) - .isInstanceOf(MessagingCapabilityUnavailableException.class) - .hasMessageContaining("experimental"); - } - - @Test - void acceptsAnUnorderedWorkQueue() { - assertThatCode(() -> validator.validate(KafkaShareProfileFixtures.profile(OrderingScope.NONE))) - .doesNotThrowAnyException(); - } - - @Test - void doesNotAdvertiseOrderingReplayOrTransactions() { - var capabilities = KafkaShareWorkQueueCapability.capabilities(); - - assertThat(capabilities.perMessageSettlement()).isTrue(); - assertThat(capabilities.orderedStream()).isFalse(); - assertThat(capabilities.keyedOrdering()).isFalse(); - assertThat(capabilities.replay()).isFalse(); - assertThat(capabilities.brokerTransaction()).isFalse(); - } - - @Test - void registerFailsClosedUntilAShareConsumerActuallyExists() { - assertThatThrownBy( - () -> - new KafkaShareGroupRegistrar() - .register( - KafkaShareProfileFixtures.profile(OrderingScope.NONE), - new dev.caskeleton.messaging.transport.TransportConsumerSpec( - KafkaShareProfileFixtures.destinationProfile(), - delivery -> - java.util.concurrent.CompletableFuture.completedFuture(null)))) - .isInstanceOf(MessagingCapabilityUnavailableException.class) - .hasMessageContaining("not implemented"); - } -} - -/** Builds Kafka share group fixtures. */ -final class KafkaShareProfileFixtures { - - private KafkaShareProfileFixtures() {} - - static KafkaShareProfile profile(OrderingScope orderingScope) { - return new KafkaShareProfile( - new DestinationName("email-work"), "email-share", orderingScope, true, 5); - } - - static KafkaShareProfile disabled() { - return new KafkaShareProfile( - new DestinationName("email-work"), "email-share", OrderingScope.NONE, false, 5); - } - - static dev.caskeleton.messaging.policy.DestinationProfile destinationProfile() { - return new dev.caskeleton.messaging.policy.DestinationProfile( - new DestinationName("email-work"), - "kafka-primary", - dev.caskeleton.messaging.api.destination.DestinationKind.WORK_QUEUE, - dev.caskeleton.messaging.policy.PhysicalDestination.kafkaTopic("email.work.v1"), - new dev.caskeleton.messaging.policy.SchemaPolicy( - dev.caskeleton.messaging.api.ContentType.JSON, - dev.caskeleton.messaging.schema.SchemaCompatibility.BACKWARD, - java.util.Set.of(new dev.caskeleton.messaging.api.MessageType("email.requested"))), - dev.caskeleton.messaging.api.delivery.DeliveryGuarantee.AT_LEAST_ONCE, - OrderingScope.NONE, - dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, - dev.caskeleton.messaging.policy.ProducerPolicy.defaults(), - dev.caskeleton.messaging.policy.ConsumerPolicy.defaults("email-share"), - dev.caskeleton.messaging.policy.RetryPolicy.none(), - dev.caskeleton.messaging.policy.DeadLetterPolicy.disabled(), - dev.caskeleton.messaging.policy.PayloadPolicy.defaults(), - dev.caskeleton.messaging.policy.CapabilityTier.M2, - false, - false, - false); - } -} diff --git a/messaging/messaging-kafka/build.gradle b/messaging/messaging-kafka/build.gradle index 789445b..00c9892 100644 --- a/messaging/messaging-kafka/build.gradle +++ b/messaging/messaging-kafka/build.gradle @@ -1,27 +1,39 @@ plugins { id 'java-test-fixtures' - id 'ca.auxiliary-source-set' - id 'ca.jmh-benchmarks' - id 'ca.messaging-certification' + id 'ca.platform-module' + id 'me.champeau.jmh' } -auxiliarySourceSets { - sourceSet('integrationTest') { - compilesAgainst 'main', 'testFixtures' +jmh { + jmhVersion = libs.versions.jmh.get() + includeTests = true +} + +testing { + suites { + integrationTest(JvmTestSuite) { + dependencies { + implementation project() + implementation testFixtures(project()) + } + } } } +configurations { + integrationTestImplementation.extendsFrom testImplementation + integrationTestRuntimeOnly.extendsFrom testRuntimeOnly +} dependencies { // No spring-kafka. It was declared here and imported by nothing: this adapter drives the plain // `org.apache.kafka` producer API directly, and a declared library that no source names is a // resolution, a lock entry and a transitive surface bought for nothing. api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') api project(':messaging:messaging-policy') api project(':messaging:messaging-transport-spi') api project(':messaging:messaging-observability') api project(':messaging:messaging-security') - api project(':messaging:messaging-admin-api') + api project(':messaging:messaging-admin') // api: the public constructors and mappers take and return Producer, ProducerRecord and // RecordMetadata. Hiding the dependency does not hide the types — it only stops a consumer @@ -33,10 +45,6 @@ dependencies { testImplementation project(':messaging:messaging-testkit') testFixturesImplementation project(':messaging:messaging-testkit') - // Test-only. The topology validator is broker-neutral, so it lives in the admin runtime; the - // live-broker suite is here because only this module has a Kafka container to describe. - testImplementation project(':messaging:messaging-admin-runtime') - // Live-broker certification. The deterministic suite proves the platform's semantics; these // prove the client actually behaves that way against the version the matrix claims. integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' @@ -82,3 +90,30 @@ strictTestLanes { .toString() } } + +// The certification lane writes the scenarios it proved; the committed manifest in messaging-testkit +// may claim no more and no fewer. gitCommit and observedAt are provenance, not claim identity. +tasks.register('verifyMessagingCertificationEvidence') { + group = 'verification' + description = 'Fails when the committed broker certification manifest claims a scenario the certification lane did not produce.' + dependsOn 'messagingCertificationTest' + def produced = layout.buildDirectory.file('messaging-certification/broker-certification-evidence.jsonl') + def committed = rootProject.file('messaging/messaging-testkit/src/main/resources/messaging/broker-certification-evidence.jsonl') + inputs.file(produced) + inputs.file(committed) + doLast { + def claims = { File file -> + file.readLines('UTF-8').findAll { !it.trim().isEmpty() }.collect { + it.replaceAll(/,"gitCommit":"[^"]*"/, '').replaceAll(/,"observedAt":"[^"]*"/, '') + } as Set + } + def ran = claims(produced.get().asFile) + def shipped = claims(committed) + if (ran != shipped) { + throw new GradleException("the committed certification manifest does not match this run.\n" + + " claimed but not produced: ${shipped - ran}\n produced but not claimed: ${ran - shipped}\n" + + "Copy ${produced.get().asFile} over ${committed} — the manifest is a record of a run.") + } + } +} + diff --git a/messaging/messaging-kafka/gradle.lockfile b/messaging/messaging-kafka/gradle.lockfile index b0dd01f..ecffd2e 100644 --- a/messaging/messaging-kafka/gradle.lockfile +++ b/messaging/messaging-kafka/gradle.lockfile @@ -46,15 +46,15 @@ io.micrometer:micrometer-core:1.16.7=compileClasspath,integrationTestCompileClas io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath net.java.dev.jna:jna:5.18.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -net.sf.jopt-simple:jopt-simple:5.0.4=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath +net.sf.jopt-simple:jopt-simple:5.0.4=jmh,jmhCompileClasspath,jmhRuntimeClasspath net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs org.antlr:antlr4-runtime:4.13.2=checkstyle org.apache.bcel:bcel:6.12.0=spotbugs org.apache.commons:commons-compress:1.28.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.apache.commons:commons-lang3:3.20.0=checkstyle,integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs -org.apache.commons:commons-math3:3.6.1=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath +org.apache.commons:commons-math3:3.6.1=jmh,jmhCompileClasspath,jmhRuntimeClasspath org.apache.commons:commons-text:1.15.0=spotbugs org.apache.commons:commons-text:1.3=checkstyle org.apache.httpcomponents:httpclient:4.5.13=checkstyle @@ -67,8 +67,8 @@ org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,jmhCompileClasspath,testCompileClasspath,testFixturesCompileClasspath -org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,testCompileClasspath,testFixturesCompileClasspath +org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle @@ -78,24 +78,27 @@ org.hdrhistogram:HdrHistogram:2.2.2=integrationTestRuntimeClasspath,jmhRuntimeCl org.javassist:javassist:3.28.0-GA=checkstyle org.jetbrains:annotations:17.0.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,integrationTestAnnotationProcessor,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testFixturesAnnotationProcessor,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.junit.jupiter:junit-jupiter-api:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.junit.jupiter:junit-jupiter-engine:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.junit.jupiter:junit-jupiter-params:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.junit.jupiter:junit-jupiter:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.junit.platform:junit-platform-commons:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.junit.platform:junit-platform-engine:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.junit.platform:junit-platform-launcher:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testRuntimeClasspath -org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.junit:junit-bom:6.1.0=spotbugs org.latencyutils:LatencyUtils:2.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,runtimeClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.mockito:mockito-core:5.20.0=mockitoAgent -org.openjdk.jmh:jmh-core:1.37=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath -org.openjdk.jmh:jmh-generator-annprocess:1.37=jmhAnnotationProcessor -org.opentest4j:opentest4j:1.3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath +org.openjdk.jmh:jmh-core:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-asm:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-bytecode:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-reflection:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.opentest4j:opentest4j:1.3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhRuntimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath org.ow2.asm:asm-analysis:9.10.1=spotbugs org.ow2.asm:asm-commons:9.10.1=spotbugs org.ow2.asm:asm-tree:9.10.1=spotbugs org.ow2.asm:asm-util:9.10.1=spotbugs +org.ow2.asm:asm:9.0=jmh,jmhCompileClasspath,jmhRuntimeClasspath org.ow2.asm:asm:9.10.1=spotbugs org.pcollections:pcollections:4.0.1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor org.reflections:reflections:0.10.2=checkstyle diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaBatchConsumerRegistrar.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaBatchConsumerRegistrar.java deleted file mode 100644 index 6ba2dc3..0000000 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaBatchConsumerRegistrar.java +++ /dev/null @@ -1,146 +0,0 @@ -package dev.caskeleton.messaging.kafka; - -import dev.caskeleton.messaging.api.delivery.BatchDeliveryMetadata; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.time.Instant; -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; -import org.apache.kafka.clients.consumer.ConsumerRecord; -import org.apache.kafka.common.TopicPartition; - -/** - * Groups polled records into batches a batch handler can safely settle. - * - *

Batches never span partitions. A poll returns records from every assigned partition at once, - * and handing that mixture to a handler makes the batch unsettleable as a unit: committing the - * highest offset per partition after a partial failure would acknowledge records the handler never - * finished. Splitting by partition first makes each batch exactly one ordering unit, which is also - * what lets an ordered destination use batching at all. - * - *

Batch order within a partition is preserved as Kafka returned it. Re-ordering inside a batch - * would break ordering just as thoroughly as mixing partitions would, and less visibly. - */ -public final class KafkaBatchConsumerRegistrar { - - private final DestinationProfile profile; - private final int maxBatchSize; - - /** - * Creates a batch grouper for a destination. - * - * @param profile the validated destination profile - * @param maxBatchSize the largest batch handed to a handler - */ - public KafkaBatchConsumerRegistrar(DestinationProfile profile, int maxBatchSize) { - this.profile = Objects.requireNonNull(profile, "profile must not be null"); - if (maxBatchSize < 1) { - throw new IllegalArgumentException("maxBatchSize must be at least 1"); - } - this.maxBatchSize = maxBatchSize; - } - - /** - * Splits polled records into per-partition batches. - * - * @param records the records returned by one poll - * @param now the current instant - * @return the batches, each drawn from exactly one partition - */ - public List group(List> records, Instant now) { - Objects.requireNonNull(records, "records must not be null"); - Objects.requireNonNull(now, "now must not be null"); - - Map>> byPartition = new LinkedHashMap<>(); - for (ConsumerRecord record : records) { - byPartition - .computeIfAbsent( - new TopicPartition(record.topic(), record.partition()), key -> new ArrayList<>()) - .add(record); - } - - List batches = new ArrayList<>(); - for (Map.Entry>> entry : - byPartition.entrySet()) { - List> partitionRecords = entry.getValue(); - for (int start = 0; start < partitionRecords.size(); start += maxBatchSize) { - List> slice = - List.copyOf( - partitionRecords.subList( - start, Math.min(start + maxBatchSize, partitionRecords.size()))); - batches.add( - new PartitionBatch(entry.getKey(), slice, metadataFor(entry.getKey(), slice, now))); - } - } - return List.copyOf(batches); - } - - /** - * Refuses batch consumption on a destination whose ordering it cannot honour. - * - *

Destination-wide ordering and batching are incompatible on Kafka: ordering across the whole - * destination requires a single partition, and a single-partition destination gains nothing from - * batching while losing the ability to recover a partial failure. - * - * @throws MessagingConfigurationException when the destination cannot be consumed in batches - */ - public void requireBatchable() { - if (profile.orderingScope() == OrderingScope.DESTINATION) { - throw new MessagingConfigurationException( - "BATCH_ON_DESTINATION_ORDERED", - "destination %s declares destination-wide ordering, which batching cannot preserve " - .formatted(profile.name().value()) - + "across a partial batch failure"); - } - } - - private BatchDeliveryMetadata metadataFor( - TopicPartition partition, List> slice, Instant now) { - return new BatchDeliveryMetadata( - profile.name(), - slice.size(), - Optional.of(partition.toString()), - // Kafka commits an offset per partition, so a single-partition batch is settleable as one. - true, - now); - } - - /** - * One batch drawn from a single partition. - * - * @param partition the partition the batch came from - * @param records the records, in broker order - * @param metadata the batch-wide metadata handed to the handler - */ - public record PartitionBatch( - TopicPartition partition, - List> records, - BatchDeliveryMetadata metadata) { - - public PartitionBatch { - Objects.requireNonNull(partition, "partition must not be null"); - Objects.requireNonNull(metadata, "metadata must not be null"); - records = List.copyOf(records); - if (records.isEmpty()) { - throw new IllegalArgumentException("a batch is never empty"); - } - } - - /** - * Returns the offset to commit once the whole batch succeeds. - * - *

The last record's offset plus one, because Kafka commits the next offset to read. - * Committing the last record's own offset would replay it on every restart. - * - * @return the offset to commit - */ - public long commitOffset() { - return records.get(records.size() - 1).offset() + 1; - } - } -} diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java index c285b5c..4b85f2f 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java @@ -66,6 +66,35 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati private final KafkaQuarantinePublisher quarantine; + private final java.time.InstantSource clock; + + /** The shortest wait before a failed delivery is read again, whatever the policy computes. */ + private static final Duration MIN_RETRY_DELAY = Duration.ofSeconds(1); + + private static final dev.caskeleton.messaging.policy.BackoffCalculator BACKOFF = + new dev.caskeleton.messaging.policy.BackoffCalculator(); + + /** A delivery that failed, by where it sits in the log. */ + private record FailedOffset(TopicPartition partition, long offset) {} + + /** + * How many times each offset has failed without being settled. + * + *

Pause-and-seek re-reads the same record, so the envelope's own attempt header never moves; + * the count has to live here. Without it a poison message was retried with no delay and no limit, + * and its partition never advanced again. + */ + private final Map failures = new ConcurrentHashMap<>(); + + /** + * Partitions paused because a record could not be dispatched — the ceiling was reached, the + * runtime was draining, or the pool refused. Poll thread only. + * + *

They were paused and never resumed: an ordered partition handed two records by one poll ran + * the first and then stopped reading for good. + */ + private final Set saturated = new java.util.HashSet<>(); + private final Map committed = new ConcurrentHashMap<>(); /** @@ -127,8 +156,38 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati GracefulShutdownCoordinator shutdown, Duration pollTimeout, KafkaQuarantinePublisher quarantine) { + this( + spec, + consumer, + handlerPool, + shutdown, + pollTimeout, + quarantine, + java.time.InstantSource.system()); + } + + /** A consumer runtime with the clock retry delays are measured on, for tests. */ + KafkaConsumerRegistrar( + TransportConsumerSpec spec, + Consumer consumer, + Executor handlerPool, + GracefulShutdownCoordinator shutdown, + Duration pollTimeout, + KafkaQuarantinePublisher quarantine, + java.time.InstantSource clock) { + this.clock = Objects.requireNonNull(clock, "clock must not be null"); this.quarantine = Objects.requireNonNull(quarantine, "quarantine must not be null"); this.spec = Objects.requireNonNull(spec, "spec must not be null"); + if (spec.profile().production() && quarantine == KafkaQuarantinePublisher.unavailable()) { + // Without somewhere to park a message that keeps failing, its partition retries it forever + // and nothing behind it is ever read. + throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( + "KAFKA_QUARANTINE_REQUIRED", + "production destination " + + spec.profile().name().value() + + " needs a KafkaQuarantinePublisher; without one a message that exhausts its" + + " attempts stops its partition for good"); + } this.consumer = Objects.requireNonNull(consumer, "consumer must not be null"); this.handlerPool = Objects.requireNonNull(handlerPool, "handlerPool must not be null"); this.shutdown = Objects.requireNonNull(shutdown, "shutdown must not be null"); @@ -169,6 +228,8 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati coordinator.forget(partition); retries.forget(partition); committed.remove(partition); + saturated.remove(partition); + failures.keySet().removeIf(failed -> failed.partition().equals(partition)); }); } @@ -199,6 +260,7 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati syncEpochs(); applySettlements(); applyDueResumes(now); + resumeSaturated(); commitContiguous(); if (!shutdown.isAcceptingWork()) { @@ -251,6 +313,7 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati // Only this partition pauses. Its earliest unsubmitted offset is where it resumes. consumer.pause(Set.of(partition)); consumer.seek(partition, seekBackTo); + saturated.add(partition); } } return dispatched; @@ -270,7 +333,8 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati try { handlerPool.execute( () -> { - QueuedSettlement settlement = new QueuedSettlement(partition, record.offset(), epoch); + QueuedSettlement settlement = + new QueuedSettlement(partition, record, record.offset(), epoch); dev.caskeleton.messaging.api.MessageEnvelope< dev.caskeleton.messaging.schema.EncodedMessage> envelope; @@ -283,7 +347,7 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati if (quarantine.quarantine(record, undecodable)) { settlement.acknowledgeAfterQuarantine(); } else { - settlement.requeueAfterFailure(); + settlement.requeueAfterFailure(undecodable); } coordinator.release(partition); shutdown.endWork(); @@ -292,7 +356,13 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati java.util.concurrent.CompletableFuture handler = null; boolean releaseImmediately = true; try { - int attempt = retryMetadataMapper.attemptOf(envelope); + // Pause-and-seek re-reads the same record, so its retry header never moves; the + // count this runtime keeps for the offset is what tells the handler which attempt it + // is. A retry-topic hop carries its count in the header instead. + int attempt = + Math.max( + retryMetadataMapper.attemptOf(envelope), + failures.getOrDefault(new FailedOffset(partition, record.offset()), 0) + 1); TransportDelivery delivery = new TransportDelivery( envelope, @@ -308,37 +378,35 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati handler = spec.sink().apply(delivery).toCompletableFuture(); handler.get(handlerTimeout.toMillis(), java.util.concurrent.TimeUnit.MILLISECONDS); } catch (java.util.concurrent.TimeoutException timedOut) { - // A timed-out handler may still be applying side effects. Cancel it before releasing - // the partition permit; otherwise the same offset can be dispatched again while the - // first handler is still running. + // A timed-out handler may still be applying side effects, and cancelling the + // CompletableFuture does not stop it — it only completes the stage this worker holds, + // which released the permit and let the same offset run again beside the first. So + // the slot stays taken until the handler really finishes, and only then is a delivery + // it left unsettled requeued. java.util.concurrent.CompletableFuture timedOutHandler = handler; - settlement.requeueAfterFailure(); - if (timedOutHandler != null - && !timedOutHandler.cancel(true) - && !timedOutHandler.isDone()) { - releaseImmediately = false; - java.util.concurrent.CompletableFuture releaseAfterCompletion = - timedOutHandler.whenComplete( - (ignored, failure) -> { - coordinator.release(partition); - shutdown.endWork(); - }); - // Keep the continuation strongly referenced through this worker frame. The - // completion action owns the delayed permit/work-count release; its own result is - // intentionally not joined here. - Objects.requireNonNull(releaseAfterCompletion); - } + releaseImmediately = false; + java.util.concurrent.CompletableFuture releaseAfterCompletion = + timedOutHandler.whenComplete( + (ignored, failure) -> { + settlement.requeueAfterFailure(failure == null ? timedOut : failure); + coordinator.release(partition); + shutdown.endWork(); + }); + // Keep the continuation strongly referenced through this worker frame. The + // completion action owns the delayed permit/work-count release; its own result is + // intentionally not joined here. + Objects.requireNonNull(releaseAfterCompletion); } catch (InterruptedException interrupted) { Thread.currentThread().interrupt(); if (handler != null) { handler.cancel(true); } - settlement.requeueAfterFailure(); + settlement.requeueAfterFailure(interrupted); } catch (java.util.concurrent.ExecutionException | RuntimeException failure) { // A handler failure is retried, not parked. PARKED completed the offset without any // quarantine or dead-letter write, so a transient database outage silently discarded // every message it touched. - settlement.requeueAfterFailure(); + settlement.requeueAfterFailure(failure); } finally { if (releaseImmediately) { coordinator.release(partition); @@ -365,7 +433,10 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati continue; } switch (command.kind()) { - case COMPLETE, PARKED -> offsets.completed(command.partition(), command.offset()); + case COMPLETE, PARKED -> { + offsets.completed(command.partition(), command.offset()); + failures.remove(new FailedOffset(command.partition(), command.offset())); + } case PAUSE_AND_SEEK -> { coordinator.pause(command.partition()); consumer.pause(Set.of(command.partition())); @@ -391,6 +462,22 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati current.forEach( partition -> epochs.computeIfAbsent(partition, key -> epochSequence.incrementAndGet())); epochs.keySet().removeIf(partition -> !current.contains(partition)); + saturated.retainAll(current); + } + + /** Resumes a partition paused for lack of a slot once it has one and nothing else holds it. */ + private void resumeSaturated() { + int ceiling = spec.profile().consumer().maxInFlightPerOrderingUnit(); + saturated.removeIf( + partition -> { + if (coordinator.isPaused(partition) + || retries.isPaused(partition) + || coordinator.inFlight(partition) >= ceiling) { + return false; + } + consumer.resume(Set.of(partition)); + return true; + }); } private void applyDueResumes(Instant now) { @@ -553,13 +640,16 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati implements dev.caskeleton.messaging.transport.TransportSettlement { private final TopicPartition partition; + private final ConsumerRecord record; private final long offset; private final long epoch; private final java.util.concurrent.atomic.AtomicReference terminal = new java.util.concurrent.atomic.AtomicReference<>(); - private QueuedSettlement(TopicPartition partition, long offset, long epoch) { + private QueuedSettlement( + TopicPartition partition, ConsumerRecord record, long offset, long epoch) { this.partition = partition; + this.record = record; this.offset = offset; this.epoch = epoch; } @@ -576,10 +666,23 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati }); } + /** + * A requeue the handler asked for, counted against the same attempt budget as a failure. + * + *

It used to pause and seek with no count, so a handler that kept answering {@code Retry} + * re-read its message forever and its partition never advanced — the retry policy's {@code + * maxAttempts} applied to exceptions only. + */ @Override public CompletionStage requeue( Duration delay) { - return terminalOnce("requeue", () -> enqueueRequeue(delay)); + return terminalOnce( + "requeue", + () -> + retryOrQuarantine( + new IllegalStateException( + "the handler asked for another attempt and the retry budget is spent"), + delay)); } @Override @@ -603,16 +706,54 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati } } - /** Used by the worker when the handler failed or timed out and never settled. */ - private void requeueAfterFailure() { - if (terminal.compareAndSet(null, "requeue")) { - enqueueRequeue(Duration.ZERO); + /** + * Used by the worker when the handler failed or timed out and never settled. + * + *

Retried after the destination's backoff, never sooner than {@code MIN_RETRY_DELAY}; once + * the attempt budget is spent the record is written to quarantine and committed past. When + * there is no quarantine it keeps being retried — the offset may not advance past a message + * that exists nowhere else — which is why a production destination must have one. + */ + private void requeueAfterFailure(Throwable reason) { + if (!terminal.compareAndSet(null, "requeue")) { + // The handler settled before it failed or before its late completion; that answer stands. + return; } + retryOrQuarantine(reason, Duration.ZERO); + } + + /** + * Counts one more attempt, then either parks the record or pauses its partition for a retry. + * + *

The caller has already claimed the terminal slot. Once the attempt budget is spent the + * record is written to quarantine and committed past; when there is no quarantine it keeps + * being retried — the offset may not advance past a message that exists nowhere else. + */ + private dev.caskeleton.messaging.api.settlement.SettlementResult retryOrQuarantine( + Throwable reason, Duration requested) { + FailedOffset key = new FailedOffset(partition, offset); + int attempt = failures.merge(key, 1, Integer::sum); + dev.caskeleton.messaging.policy.RetryPolicy retry = spec.profile().retry(); + if (attempt >= retry.maxAttempts() && quarantine.quarantine(record, reason)) { + failures.remove(key); + settlements.enqueue( + new KafkaSettlementCommand( + partition, offset, KafkaSettlementCommand.Kind.PARKED, epoch)); + return dev.caskeleton.messaging.api.settlement.SettlementResult.settled(); + } + Duration backoff = BACKOFF.delayFor(retry, attempt); + if (backoff.compareTo(MIN_RETRY_DELAY) < 0) { + backoff = MIN_RETRY_DELAY; + } + if (requested != null && backoff.compareTo(requested) < 0) { + backoff = requested; + } + return enqueueRequeue(backoff); } private dev.caskeleton.messaging.api.settlement.SettlementResult enqueueRequeue( Duration delay) { - retries.pauseUntil(partition, offset, delay, Instant.now()); + retries.pauseUntil(partition, offset, delay, clock.instant()); settlements.enqueue( new KafkaSettlementCommand( partition, offset, KafkaSettlementCommand.Kind.PAUSE_AND_SEEK, epoch)); diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaCredentialCallbackHandler.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaCredentialCallbackHandler.java new file mode 100644 index 0000000..b503424 --- /dev/null +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaCredentialCallbackHandler.java @@ -0,0 +1,84 @@ +package dev.caskeleton.messaging.kafka; + +import dev.caskeleton.messaging.api.error.MessagingConfigurationException; +import dev.caskeleton.messaging.security.CredentialRuntime; +import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; +import java.time.Instant; +import java.util.Arrays; +import java.util.List; +import java.util.Map; +import javax.security.auth.callback.Callback; +import javax.security.auth.callback.NameCallback; +import javax.security.auth.callback.PasswordCallback; +import javax.security.auth.callback.UnsupportedCallbackException; +import javax.security.auth.login.AppConfigurationEntry; +import org.apache.kafka.common.security.auth.AuthenticateCallbackHandler; + +/** + * Answers the SASL client's user name and password from {@link CredentialRuntimeRegistry}, at each + * authentication. + * + *

The JAAS entry used to carry both values, fixed when the producer was built: a rotated secret + * never reached a connection until the process restarted, and the user name was the credential id. + * Kafka asks this handler every time a connection authenticates — a new connection, or a + * re-authentication when the broker sets {@code connections.max.reauth.ms} — so each one reads the + * registry's current generation. + * + *

Kafka instantiates this class itself, so the registry and the credential id arrive through the + * client configuration under {@link #REGISTRY} and {@link #CREDENTIAL_ID}. + */ +public final class KafkaCredentialCallbackHandler implements AuthenticateCallbackHandler { + + /** Client configuration key carrying the {@link CredentialRuntimeRegistry}. */ + public static final String REGISTRY = "caskeleton.messaging.credential.registry"; + + /** Client configuration key carrying the credential id this client authenticates with. */ + public static final String CREDENTIAL_ID = "caskeleton.messaging.credential.id"; + + private CredentialRuntimeRegistry registry; + + private String credentialId; + + /** Created by Kafka through reflection. */ + public KafkaCredentialCallbackHandler() {} + + @Override + public void configure( + Map configs, String saslMechanism, List jaasConfigEntries) { + if (!(configs.get(REGISTRY) instanceof CredentialRuntimeRegistry configured) + || !(configs.get(CREDENTIAL_ID) instanceof String id)) { + throw new MessagingConfigurationException( + "SASL_CALLBACK_NOT_CONFIGURED", + "the Kafka client names this callback handler but does not carry " + + REGISTRY + + " and " + + CREDENTIAL_ID); + } + this.registry = configured; + this.credentialId = id; + } + + @Override + public void handle(Callback[] callbacks) throws UnsupportedCallbackException { + CredentialRuntime credential = registry.resolve(credentialId, Instant.now()); + for (Callback callback : callbacks) { + if (callback instanceof NameCallback name) { + name.setName( + credential + .username() + .orElseThrow(() -> KafkaSecurityConfigurer.usernameMissing(credentialId))); + } else if (callback instanceof PasswordCallback password) { + char[] material = credential.material(); + // PasswordCallback keeps its own copy, so this one is wiped as soon as it is handed over. + password.setPassword(material); + Arrays.fill(material, '\0'); + } else { + // SCRAM asks for extensions and carries on without them when the handler declines. + throw new UnsupportedCallbackException(callback); + } + } + } + + @Override + public void close() {} +} diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaMessagingTransport.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaMessagingTransport.java index 6f1a369..e80cff4 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaMessagingTransport.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaMessagingTransport.java @@ -6,10 +6,7 @@ import dev.caskeleton.messaging.api.destination.MessagingCapabilities; import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; import dev.caskeleton.messaging.transport.MessagingTransport; import dev.caskeleton.messaging.transport.TransportConsumerRegistration; import dev.caskeleton.messaging.transport.TransportConsumerSpec; @@ -17,7 +14,6 @@ import dev.caskeleton.messaging.transport.TransportPublishRequest; import dev.caskeleton.messaging.transport.TransportPublishResult; import java.time.Duration; import java.util.Objects; -import java.util.Optional; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionStage; import java.util.concurrent.atomic.AtomicBoolean; @@ -41,7 +37,7 @@ import org.apache.kafka.clients.producer.ProducerRecord; public final class KafkaMessagingTransport implements MessagingTransport { /** - * What this adapter can prove, with deduplicated publish set to false. + * What this adapter can prove, with deduplicated publish absent. * *

It was {@code true}, justified by producer idempotence. Producer idempotence deduplicates * sequence retries within one producer session: the producer id is reassigned on @@ -55,13 +51,10 @@ public final class KafkaMessagingTransport implements MessagingTransport { * build. False makes that request a startup failure instead, which is the whole reason the flag * exists. * - *

The order of these flags is the record's: brokerAcknowledgement, replication evidence, - * per-message settlement, batch settlement, ordered stream, keyed ordering, replay, delayed - * delivery, broker transaction, deduplicated publish, native dead letter, topology management. + *

Delayed delivery is absent too: Kafka has no broker-side schedule. */ private static final MessagingCapabilities CAPABILITIES = - new MessagingCapabilities( - true, true, true, true, true, true, true, false, true, false, false, true); + MessagingCapabilities.of(MessagingCapabilities.Capability.ORDERED_STREAM); private final String brokerName; private final long generation; @@ -199,14 +192,9 @@ public final class KafkaMessagingTransport implements MessagingTransport { private static TransportPublishResult rejectedLocally(String code, String message) { return new TransportPublishResult( - new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, + PublishResult.rejected( Duration.ZERO, - Optional.of(FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message)))); + FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message))); } private static CompletionStage completed(TransportPublishResult result) { diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java index 0d33a58..3436871 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java @@ -2,10 +2,7 @@ package dev.caskeleton.messaging.kafka; import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; import java.time.Duration; import java.util.Optional; import org.apache.kafka.common.errors.AuthenticationException; @@ -40,23 +37,9 @@ public final class KafkaPublishFailureClassifier { public PublishResult classify(Throwable error, Duration elapsed) { Throwable cause = unwrap(error); if (isDefinitiveRejection(cause)) { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - elapsed, - Optional.of(descriptor(cause, rejectionCategory(cause), false))); + return PublishResult.rejected(elapsed, descriptor(cause, rejectionCategory(cause), false)); } - return new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsed, - Optional.of(descriptor(cause, FailureCategory.AMBIGUOUS, false))); + return PublishResult.ambiguous(elapsed, descriptor(cause, FailureCategory.AMBIGUOUS, false)); } /** diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishMapper.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishMapper.java index 39f98ad..7c74f13 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishMapper.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishMapper.java @@ -4,8 +4,6 @@ import dev.caskeleton.messaging.api.MessageEnvelope; import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; import dev.caskeleton.messaging.api.error.MessagingConfigurationException; import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishResult; import dev.caskeleton.messaging.api.publish.RoutingOutcome; import dev.caskeleton.messaging.policy.DestinationProfile; @@ -99,13 +97,10 @@ public final class KafkaPublishMapper { ? ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK : ConfirmationLevel.BROKER_ACK; - return new PublishResult( - PublishCompletion.CONFIRMED, - PublishEvidence.confirmed(level), + return PublishResult.confirmed( + level, RoutingOutcome.NOT_APPLICABLE, Optional.of(new KafkaPosition(metadata.topic(), metadata.partition(), metadata.offset())), - 1, - elapsed, - Optional.empty()); + elapsed); } } diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java index bdb4365..20489c9 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java @@ -24,8 +24,21 @@ public interface KafkaQuarantinePublisher { */ boolean quarantine(ConsumerRecord record, Throwable reason); - /** The default: nothing is configured, so nothing may be committed past. */ + /** + * The default: nothing is configured, so nothing may be committed past. + * + *

One instance, so a consumer can tell it apart from a configured quarantine and refuse it on + * a production destination. + */ static KafkaQuarantinePublisher unavailable() { - return (record, reason) -> false; + return Unavailable.INSTANCE; + } + + /** Holds the single "not configured" quarantine. */ + final class Unavailable { + + private static final KafkaQuarantinePublisher INSTANCE = (record, reason) -> false; + + private Unavailable() {} } } diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurer.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurer.java index e9ca621..391ece3 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurer.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurer.java @@ -1,5 +1,6 @@ package dev.caskeleton.messaging.kafka; +import dev.caskeleton.messaging.api.error.MessagingConfigurationException; import dev.caskeleton.messaging.security.BrokerCredentialProfile; import dev.caskeleton.messaging.security.BrokerSecurityProfile; import dev.caskeleton.messaging.security.BrokerTlsPolicy; @@ -14,17 +15,18 @@ import java.util.Objects; /** * Builds the Kafka client security properties from a validated security profile. * - *

Nothing here reads an environment variable or a property file. The credential arrives already - * resolved from {@link CredentialRuntimeRegistry}, which is what makes rotation possible: a client - * configured from a value read once at startup holds that value until the process restarts, so the - * rotation the credential store performs never reaches the broker connection. + *

Nothing here reads an environment variable or a property file. Credentials resolve through + * {@link CredentialRuntimeRegistry}. * - *

The JAAS config is assembled into a {@code String} because that is the shape Kafka's client - * configuration accepts. The configurer keeps no field reference after returning the map, but the - * returned immutable String contains the credential and cannot be zeroized; its lifetime is the - * Kafka/client configuration lifetime. The source {@link CredentialRuntime} material is still - * zeroizable, but converting it to JAAS necessarily weakens that property and must not be described - * as if the String never existed. + *

SASL user names and passwords are not written into the configuration at all. The JAAS entry + * names the login module and nothing else, and {@link KafkaCredentialCallbackHandler} answers the + * user name and password from the registry each time a connection authenticates — so a credential + * the store rotates (one whose provider reports an expiry) reaches the next connection without a + * restart, and no immutable {@code String} holds the password for the client's lifetime. + * + *

Key and trust store passwords are the exception: Kafka reads them once, when the client builds + * its SSL context, and accepts them only as configuration values. They are resolved here and live + * as long as the client does; rotating one means rebuilding the client. */ public final class KafkaSecurityConfigurer { @@ -37,6 +39,9 @@ public final class KafkaSecurityConfigurer { /** Kafka's key for the JAAS configuration. */ public static final String SASL_JAAS_CONFIG = "sasl.jaas.config"; + /** Kafka's key for the SASL client callback handler class. */ + public static final String SASL_CLIENT_CALLBACK_HANDLER = "sasl.client.callback.handler.class"; + /** Kafka's key for the endpoint identification algorithm. */ public static final String ENDPOINT_IDENTIFICATION = "ssl.endpoint.identification.algorithm"; @@ -78,6 +83,9 @@ public final class KafkaSecurityConfigurer { // Validate before resolving. A profile that will be refused should not cause a credential to be // fetched and briefly held in memory for nothing. tlsPolicy.validate(profile, enabledProtocols); + if (!profile.tlsEnabled()) { + requireNoTlsMaterial(profile, credential); + } Map properties = new LinkedHashMap<>(); properties.put(SECURITY_PROTOCOL, securityProtocol(profile, credential)); @@ -87,31 +95,51 @@ public final class KafkaSecurityConfigurer { // Kafka disables hostname verification by setting this to the empty string. The policy has // already refused that combination, so the value here is always the verifying one. properties.put(ENDPOINT_IDENTIFICATION, "https"); + profile + .trustStore() + .ifPresent( + store -> { + properties.put("ssl.truststore.location", store.location()); + properties.put("ssl.truststore.type", store.type()); + store + .passwordCredentialId() + .ifPresent(id -> properties.put("ssl.truststore.password", secret(id, now))); + }); } switch (credential) { - case BrokerCredentialProfile.SaslScram scram -> { - CredentialRuntime resolved = credentials.resolve(scram.credentialId(), now); - properties.put(SASL_MECHANISM, "SCRAM-SHA-512"); - properties.put(SASL_JAAS_CONFIG, scramJaas(scram.credentialId(), resolved)); - } + case BrokerCredentialProfile.SaslScram scram -> + properties.putAll( + sasl( + "SCRAM-SHA-512", + "org.apache.kafka.common.security.scram.ScramLoginModule", + scram.credentialId(), + now)); case BrokerCredentialProfile.OAuth2 oauth -> { // Refused rather than half-configured. Setting the mechanism name without a callback // handler produces a client that authenticates with nothing and fails at connect time, // which is later and harder to attribute than failing here. - throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( + throw new MessagingConfigurationException( "SASL_OAUTH_NOT_CONFIGURED", "OAuth bearer needs a login callback handler and a token endpoint; this platform" + " configures the mechanism name only, so the profile is refused rather than" + " producing a client that cannot authenticate: " + oauth.credentialId()); } - case BrokerCredentialProfile.MutualTls ignored -> properties.put(SASL_MECHANISM, "NONE"); - case BrokerCredentialProfile.UsernamePassword plain -> { - CredentialRuntime resolved = credentials.resolve(plain.credentialId(), now); - properties.put(SASL_MECHANISM, "PLAIN"); - properties.put(SASL_JAAS_CONFIG, plainJaas(plain.credentialId(), resolved)); + case BrokerCredentialProfile.MutualTls tls -> { + String password = secret(tls.credentialId(), now); + properties.put("ssl.keystore.location", tls.keyStoreLocation()); + properties.put("ssl.keystore.type", tls.keyStoreType()); + properties.put("ssl.keystore.password", password); + properties.put("ssl.key.password", password); } + case BrokerCredentialProfile.UsernamePassword plain -> + properties.putAll( + sasl( + "PLAIN", + "org.apache.kafka.common.security.plain.PlainLoginModule", + plain.credentialId(), + now)); case BrokerCredentialProfile.Nkey ignored -> throw new IllegalArgumentException( "NKey credentials are a NATS concept, not a Kafka one"); @@ -119,6 +147,62 @@ public final class KafkaSecurityConfigurer { return Map.copyOf(properties); } + /** + * The SASL properties: a JAAS entry naming only the login module, and the handler that answers. + * + *

The credential is resolved once here so that a provider with no user name, or no secret, + * fails startup rather than the first connection. + */ + private Map sasl( + String mechanism, String loginModule, String credentialId, Instant now) { + if (credentials.resolve(credentialId, now).username().isEmpty()) { + throw usernameMissing(credentialId); + } + return Map.of( + SASL_MECHANISM, + mechanism, + SASL_JAAS_CONFIG, + loginModule + " required;", + SASL_CLIENT_CALLBACK_HANDLER, + KafkaCredentialCallbackHandler.class, + KafkaCredentialCallbackHandler.REGISTRY, + credentials, + KafkaCredentialCallbackHandler.CREDENTIAL_ID, + credentialId); + } + + private String secret(String credentialId, Instant now) { + CredentialRuntime resolved = credentials.resolve(credentialId, now); + return new String(resolved.material()); + } + + /** TLS material on a plaintext connection is a configuration that cannot mean what it says. */ + private static void requireNoTlsMaterial( + BrokerSecurityProfile profile, BrokerCredentialProfile credential) { + if (profile.trustStore().isPresent()) { + throw new MessagingConfigurationException( + "TRUST_STORE_REQUIRES_TLS", + "broker %s configures a trust store but not TLS; nothing would ever read it" + .formatted(profile.broker())); + } + if (credential instanceof BrokerCredentialProfile.MutualTls) { + throw new MessagingConfigurationException( + "MUTUAL_TLS_REQUIRES_TLS", + "broker %s authenticates with a client certificate but does not enable TLS, and a" + .formatted(profile.broker()) + + " certificate is only presented inside a TLS handshake"); + } + } + + static MessagingConfigurationException usernameMissing(String credentialId) { + return new MessagingConfigurationException( + "SASL_USERNAME_MISSING", + "credential " + + credentialId + + " authenticates with a user name and password, and CredentialProvider.username" + + " returned none; the credential id is a reference, not the broker's user name"); + } + private static String securityProtocol( BrokerSecurityProfile profile, BrokerCredentialProfile credential) { boolean sasl = !(credential instanceof BrokerCredentialProfile.MutualTls); @@ -127,50 +211,4 @@ public final class KafkaSecurityConfigurer { } return sasl ? "SASL_PLAINTEXT" : "PLAINTEXT"; } - - private static String scramJaas(String credentialId, CredentialRuntime resolved) { - return jaas("org.apache.kafka.common.security.scram.ScramLoginModule", credentialId, resolved); - } - - private static String plainJaas(String credentialId, CredentialRuntime resolved) { - return jaas("org.apache.kafka.common.security.plain.PlainLoginModule", credentialId, resolved); - } - - /** - * Builds a JAAS entry with both values escaped. - * - *

The password was interpolated into a quoted string unescaped. A perfectly ordinary generated - * secret containing a quote or a backslash then produced a JAAS config that either fails to parse - * — an authentication error that names nothing useful — or, with a quote and a semicolon, - * terminates the option early and injects another one. A secret is attacker-influenced in exactly - * one sense that matters here: nobody reads it before it is used. - */ - private static String jaas(String loginModule, String credentialId, CredentialRuntime resolved) { - return loginModule - + " required username=\"" - + escapeJaas(credentialId) - + "\" password=\"" - + escapeJaas(new String(resolved.material())) - + "\";"; - } - - /** - * Escapes a JAAS quoted-string value. - * - *

Backslash first, then quote — the other order double-escapes the backslash it just added. - * Control characters are refused rather than escaped: JAAS has no escape for them, and a secret - * containing one cannot be expressed in this format at all. - */ - private static String escapeJaas(String value) { - for (int index = 0; index < value.length(); index++) { - char character = value.charAt(index); - if (character < 0x20 || character == 0x7F) { - throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( - "JAAS_VALUE_UNREPRESENTABLE", - "a credential containing a control character cannot be expressed in a JAAS entry;" - + " configure the client with a callback handler instead"); - } - } - return value.replace("\\", "\\\\").replace("\"", "\\\""); - } } diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaTopologyInspector.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaTopologyInspector.java deleted file mode 100644 index 195cf29..0000000 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaTopologyInspector.java +++ /dev/null @@ -1,45 +0,0 @@ -package dev.caskeleton.messaging.kafka; - -import java.util.List; -import java.util.Map; - -/** - * Read-only inspection of Kafka topology. - * - *

Describe and validate only. Creation, deletion, and configuration changes belong to the Admin - * Plane with its own credential: an application runtime that could alter topology is one bad - * deployment away from deleting a topic it merely meant to read. - */ -public interface KafkaTopologyInspector { - - /** - * Returns the partition count for a topic. - * - * @param topic the topic - * @return the number of partitions - */ - int partitionCount(String topic); - - /** - * Returns the replication factor for a topic. - * - * @param topic the topic - * @return the replication factor - */ - short replicationFactor(String topic); - - /** - * Returns the effective topic configuration. - * - * @param topic the topic - * @return the configuration entries - */ - Map configuration(String topic); - - /** - * Returns the topics that exist. - * - * @return the topic names - */ - List topics(); -} diff --git a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java index 941abf3..f5580a5 100644 --- a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java +++ b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java @@ -4,6 +4,9 @@ import static org.assertj.core.api.Assertions.assertThat; import dev.caskeleton.messaging.api.MessageId; import dev.caskeleton.messaging.api.header.ReservedHeaders; +import dev.caskeleton.messaging.policy.OrderingImpact; +import dev.caskeleton.messaging.policy.RetryMode; +import dev.caskeleton.messaging.policy.RetryPolicy; import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; import dev.caskeleton.messaging.transport.TransportConsumerSpec; import dev.caskeleton.messaging.transport.TransportDelivery; @@ -154,51 +157,231 @@ class KafkaConsumerRegistrarTest { } @Test - void handlerTimeoutCancelsTheOutstandingHandlerBeforeThePartitionPermitIsReleased() { - CompletableFuture handler = new CompletableFuture<>(); - dev.caskeleton.messaging.policy.DestinationProfile base = KafkaContractHarness.profile(); - dev.caskeleton.messaging.policy.DestinationProfile fastTimeout = - new dev.caskeleton.messaging.policy.DestinationProfile( - base.name(), - base.broker(), - base.kind(), - base.physical(), - base.schema(), - base.deliveryGuarantee(), - base.orderingScope(), - base.externalSideEffectGuarantee(), - base.producer(), - new dev.caskeleton.messaging.policy.ConsumerPolicy( - base.consumer().group(), - base.consumer().concurrency(), - base.consumer().maxInFlightPerOrderingUnit(), - base.consumer().prefetch(), - Duration.ofMillis(5), - base.consumer().manualSettlement()), - base.retry(), - base.deadLetter(), - base.payload(), - base.tier(), - base.production(), - base.keyResolverConfigured(), - base.topologyAutoCreate()); - + void aTimedOutHandlerStillRunningIsNotDispatchedASecondTime() { + // given + Instant start = Instant.now(); + CompletableFuture stillRunning = new CompletableFuture<>(); KafkaConsumerRegistrar registrar = - new KafkaConsumerRegistrar( - new TransportConsumerSpec(fastTimeout, delivery -> handler), - consumer, - handlers, - new GracefulShutdownCoordinator(Duration.ofSeconds(30)), - Duration.ofMillis(1)); + registrar( + profile(Duration.ofMillis(5), RetryPolicy.none(), false), + delivery -> { + handled.add(delivery); + return stillRunning; + }, + KafkaQuarantinePublisher.unavailable(), + () -> start); + assign(); + addRecord(10); + registrar.pollOnce(start); + handlers.runAll(); + // What the broker serves again if the consumer seeks back to the timed-out offset. + addRecord(10); + // when + registrar.pollOnce(start.plusSeconds(60)); + handlers.runAll(); + + // then + assertThat(handled) + .as("cancelling a CompletableFuture does not stop the handler behind it") + .hasSize(1); + } + + @Test + void aTimedOutHandlerIsRedeliveredOnceItFinishesWithoutSettling() { + // given + CompletableFuture stillRunning = new CompletableFuture<>(); + KafkaConsumerRegistrar registrar = + registrar( + profile(Duration.ofMillis(5), RetryPolicy.none(), false), + delivery -> { + handled.add(delivery); + return stillRunning; + }, + KafkaQuarantinePublisher.unavailable(), + () -> NOW); + assign(); + addRecord(10); + registrar.pollOnce(NOW); + handlers.runAll(); + stillRunning.complete(null); + addRecord(10); + + // when + registrar.pollOnce(NOW.plusSeconds(60)); + handlers.runAll(); + + // then + assertThat(handled).hasSize(2); + } + + @Test + void aFailedHandlerWaitsOutItsBackoffBeforeThePartitionIsReadAgain() { + // given + Instant start = Instant.now(); + RetryPolicy threeAttempts = + new RetryPolicy( + RetryMode.PAUSE_PARTITION, + 3, + Duration.ofSeconds(1), + Duration.ofSeconds(10), + 2.0, + false, + OrderingImpact.PRESERVE, + java.util.Set.of(), + java.util.Set.of()); + KafkaConsumerRegistrar registrar = + registrar( + profile(Duration.ofSeconds(1), threeAttempts, false), + this::fail, + KafkaQuarantinePublisher.unavailable(), + () -> start); + assign(); + addRecord(10); + registrar.pollOnce(start); + handlers.runAll(); + + // when + registrar.pollOnce(start.plusMillis(500)); + + // then + assertThat(consumer.paused()) + .as("a failure retried with no delay re-reads the same record as fast as the loop polls") + .contains(PARTITION); + } + + @Test + void aHandlerThatSpendsItsAttemptBudgetIsQuarantinedAndCommittedPast() { + // given + List quarantined = new ArrayList<>(); + KafkaConsumerRegistrar registrar = + registrar( + KafkaContractHarness.profile(), + this::fail, + (record, reason) -> quarantined.add(record.offset()), + () -> NOW); assign(); addRecord(10); registrar.pollOnce(NOW); handlers.runAll(); - assertThat(handler) - .as("a timed-out handler must be cancelled before the same partition can be redelivered") - .isCancelled(); + // when + registrar.pollOnce(NOW); + + // then + assertThat(registrar.committedOffset(PARTITION)) + .as("a poison message retried forever stops its partition for good") + .hasValue(11L); + } + + @Test + @DisplayName("handler가 요청한 requeue도 시도 횟수를 세어 예산을 쓰면 quarantine 후 commit된다") + void aDeliveryItsHandlerKeepsRequeueingSpendsTheBudgetAndIsQuarantined() { + // given + List quarantined = new ArrayList<>(); + KafkaConsumerRegistrar registrar = + registrar( + KafkaContractHarness.profile(), + this::requeue, + (record, reason) -> quarantined.add(record.offset()), + () -> NOW); + assign(); + addRecord(10); + registrar.pollOnce(NOW); + handlers.runAll(); + + // when + registrar.pollOnce(NOW); + + // then + assertThat(quarantined) + .as("a Retry answer that is never counted is a poison message retried forever") + .containsExactly(10L); + assertThat(registrar.committedOffset(PARTITION)).hasValue(11L); + } + + @Test + @DisplayName("다시 읽힌 delivery는 몇 번째 시도인지 metadata로 handler에 알려진다") + void aReReadDeliveryTellsTheHandlerWhichAttemptItIs() { + // given + Instant start = NOW; + RetryPolicy threeAttempts = + new RetryPolicy( + RetryMode.PAUSE_PARTITION, + 3, + Duration.ofSeconds(1), + Duration.ofSeconds(1), + 1.0, + false, + OrderingImpact.PRESERVE, + java.util.Set.of(), + java.util.Set.of()); + KafkaConsumerRegistrar registrar = + registrar( + profile(Duration.ofSeconds(1), threeAttempts, false), + this::requeue, + KafkaQuarantinePublisher.unavailable(), + () -> start); + assign(); + addRecord(10); + registrar.pollOnce(start); + handlers.runAll(); + registrar.pollOnce(start); + addRecord(10); + + // when + registrar.pollOnce(start.plusSeconds(2)); + handlers.runAll(); + + // then + assertThat(handled) + .extracting(delivery -> delivery.metadata().deliveryAttempt()) + .containsExactly(1, 2); + assertThat(handled.get(1).metadata().redelivered()).isTrue(); + } + + @Test + void aProductionDestinationWithoutAQuarantineIsRefused() { + // given + dev.caskeleton.messaging.policy.DestinationProfile production = + profile(Duration.ofSeconds(30), RetryPolicy.none(), true); + + // when + Throwable thrown = + org.assertj.core.api.Assertions.catchThrowable( + () -> registrar(production, this::settleImmediately)); + + // then + assertThat(thrown) + .isInstanceOf(dev.caskeleton.messaging.api.error.MessagingConfigurationException.class) + .extracting( + failure -> + ((dev.caskeleton.messaging.api.error.MessagingConfigurationException) failure) + .failure() + .code()) + .isEqualTo("KAFKA_QUARANTINE_REQUIRED"); + } + + @Test + void aPartitionPausedAtItsCeilingResumesOnceItsDeliveryFinishes() { + // given + KafkaConsumerRegistrar registrar = registrar(this::settleImmediately); + assign(); + addRecord(10); + addRecord(11); + registrar.pollOnce(NOW); + handlers.runAll(); + // The second record was returned by the first poll and handed back with a seek. + addRecord(11); + + // when + registrar.pollOnce(NOW); + handlers.runAll(); + + // then + assertThat(handled) + .as("pausing a full partition is backpressure only if something resumes it") + .hasSize(2); } @Test @@ -495,6 +678,75 @@ class KafkaConsumerRegistrarTest { Duration.ofMillis(1)); } + private KafkaConsumerRegistrar registrar( + dev.caskeleton.messaging.policy.DestinationProfile profile, + java.util.function.Function> sink) { + return new KafkaConsumerRegistrar( + new TransportConsumerSpec(profile, sink), + consumer, + handlers, + new GracefulShutdownCoordinator(Duration.ofSeconds(30)), + Duration.ofMillis(1)); + } + + private KafkaConsumerRegistrar registrar( + dev.caskeleton.messaging.policy.DestinationProfile profile, + java.util.function.Function> sink, + KafkaQuarantinePublisher quarantine, + java.time.InstantSource clock) { + return new KafkaConsumerRegistrar( + new TransportConsumerSpec(profile, sink), + consumer, + handlers, + new GracefulShutdownCoordinator(Duration.ofSeconds(30)), + Duration.ofMillis(1), + quarantine, + clock); + } + + /** The harness profile with the three settings these tests vary. */ + private static dev.caskeleton.messaging.policy.DestinationProfile profile( + Duration handlerTimeout, RetryPolicy retry, boolean production) { + dev.caskeleton.messaging.policy.DestinationProfile base = KafkaContractHarness.profile(); + return new dev.caskeleton.messaging.policy.DestinationProfile( + base.name(), + base.broker(), + base.kind(), + base.physical(), + base.schema(), + base.deliveryGuarantee(), + base.orderingScope(), + base.externalSideEffectGuarantee(), + base.producer(), + new dev.caskeleton.messaging.policy.ConsumerPolicy( + base.consumer().group(), + base.consumer().concurrency(), + base.consumer().maxInFlightPerOrderingUnit(), + base.consumer().prefetch(), + handlerTimeout, + base.consumer().manualSettlement()), + retry, + base.deadLetter(), + base.payload(), + base.tier(), + production, + base.keyResolverConfigured(), + base.topologyAutoCreate()); + } + + /** A handler that fails every delivery it is given. */ + private CompletionStage fail(TransportDelivery delivery) { + handled.add(delivery); + throw new IllegalStateException("the downstream this handler writes to is down"); + } + + /** Asks for another attempt through the settlement, as the processor does for a Retry answer. */ + private CompletionStage requeue(TransportDelivery delivery) { + handled.add(delivery); + delivery.settlement().requeue(Duration.ZERO).toCompletableFuture().join(); + return CompletableFuture.completedFuture(null); + } + /** Acknowledges inside the handler, as a successful M1 delivery does. */ private CompletionStage settleImmediately(TransportDelivery delivery) { handled.add(delivery); diff --git a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurerTest.java b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurerTest.java index e0aea61..5ce3ac2 100644 --- a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurerTest.java +++ b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaSecurityConfigurerTest.java @@ -2,80 +2,106 @@ package dev.caskeleton.messaging.kafka; import static org.assertj.core.api.Assertions.assertThat; import static org.assertj.core.api.Assertions.assertThatThrownBy; +import static org.assertj.core.api.Assertions.catchThrowable; import dev.caskeleton.messaging.api.error.MessagingConfigurationException; import dev.caskeleton.messaging.security.BrokerCredentialProfile; import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import dev.caskeleton.messaging.security.BrokerTlsPolicy; +import dev.caskeleton.messaging.security.BrokerTrustStore; import dev.caskeleton.messaging.security.CredentialProvider; import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; +import dev.caskeleton.messaging.security.DestinationAccessPolicy; import java.time.Duration; import java.time.Instant; +import java.util.List; import java.util.Map; import java.util.Optional; -import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; /** - * What reaches a JAAS entry, and what is refused outright. + * What reaches the Kafka client configuration, and what is refused outright. * - *

The password was interpolated into a quoted string with no escaping. A generated secret - * containing a quote or a backslash produced a config that fails to parse — an authentication error - * naming nothing useful — and one containing a quote and a semicolon terminated the option early - * and injected another. + *

The JAAS entry used to carry the credential id as the user name and the resolved secret as an + * immutable string, fixed for the life of the producer. It now names the login module and nothing + * else: the user name and password are answered by {@link KafkaCredentialCallbackHandler} at each + * authentication. */ class KafkaSecurityConfigurerTest { private static final Instant NOW = Instant.parse("2026-08-14T00:00:00Z"); - @Test - @DisplayName("a secret containing a quote and a backslash is escaped, not injected") - void aSecretWithQuotesIsEscaped() { - Map properties = configure("pa\"ss\\word;"); + private static final BrokerCredentialProfile SCRAM = + new BrokerCredentialProfile.SaslScram("kafka-app"); - String jaas = String.valueOf(properties.get("sasl.jaas.config")); - assertThat(jaas) - .as("an unescaped quote closes the option and everything after it becomes new options") - .contains("password=\"pa\\\"ss\\\\word;\";"); - assertThat(jaas) - .as("the raw form would close the password option and make the rest new options") - .doesNotContain("password=\"pa\""); + @Test + void theJaasEntryCarriesNeitherTheUserNameNorTheSecret() { + Map properties = configure(registry(Optional.of("svc-orders")), tls(SCRAM)); + + assertThat(properties.get("sasl.jaas.config")) + .isEqualTo("org.apache.kafka.common.security.scram.ScramLoginModule required;"); } @Test - @DisplayName("a secret containing a control character is refused") - void aSecretWithAControlCharacterIsRefused() { - assertThatThrownBy(() -> configure("pass\nword")) + void aSaslCredentialWhoseProviderNamesNoUserIsRefused() { + // given + CredentialRuntimeRegistry anonymous = registry(Optional.empty()); + + // when + Throwable thrown = catchThrowable(() -> configure(anonymous, tls(SCRAM))); + + // then + assertThat(thrown) .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("control character"); + .extracting(failure -> ((MessagingConfigurationException) failure).failure().code()) + .isEqualTo("SASL_USERNAME_MISSING"); + } + + @Test + void aTrustStoreOnABrokerWithoutTlsIsRefused() { + // given + BrokerSecurityProfile plaintext = + new BrokerSecurityProfile( + "kafka-primary", + false, + false, + true, + SCRAM, + SCRAM, + Optional.empty(), + DestinationAccessPolicy.denyAll(), + Optional.of(new BrokerTrustStore("/etc/kafka/ca.p12", "PKCS12", Optional.empty()))); + + // when + Throwable thrown = + catchThrowable(() -> configure(registry(Optional.of("svc-orders")), plaintext)); + + // then + assertThat(thrown) + .isInstanceOf(MessagingConfigurationException.class) + .extracting(failure -> ((MessagingConfigurationException) failure).failure().code()) + .isEqualTo("TRUST_STORE_REQUIRES_TLS"); } @Test - @DisplayName("an OAuth profile is refused rather than half-configured") void anOAuthProfileIsRefused() { - CredentialRuntimeRegistry credentials = registry("secret"); + CredentialRuntimeRegistry credentials = registry(Optional.of("svc-orders")); BrokerCredentialProfile oauth = new BrokerCredentialProfile.OAuth2("kafka-oauth"); - assertThatThrownBy( - () -> - new KafkaSecurityConfigurer( - credentials, new dev.caskeleton.messaging.security.BrokerTlsPolicy()) - .configure(profile(oauth), oauth, java.util.List.of("TLSv1.3"), NOW)) + // when & then + assertThatThrownBy(() -> configure(credentials, tls(oauth))) .as("naming the mechanism without a callback handler fails at connect time instead") .isInstanceOf(MessagingConfigurationException.class) .hasMessageContaining("callback handler"); } - private static Map configure(String secret) { - return new KafkaSecurityConfigurer( - registry(secret), new dev.caskeleton.messaging.security.BrokerTlsPolicy()) - .configure( - profile(new BrokerCredentialProfile.SaslScram("kafka-app")), - new BrokerCredentialProfile.SaslScram("kafka-app"), - java.util.List.of("TLSv1.3"), - NOW); + private static Map configure( + CredentialRuntimeRegistry credentials, BrokerSecurityProfile profile) { + return new KafkaSecurityConfigurer(credentials, new BrokerTlsPolicy()) + .configure(profile, profile.producer(), List.of("TLSv1.3"), NOW); } - private static BrokerSecurityProfile profile(BrokerCredentialProfile credential) { + private static BrokerSecurityProfile tls(BrokerCredentialProfile credential) { return new BrokerSecurityProfile( "kafka-primary", true, @@ -84,21 +110,26 @@ class KafkaSecurityConfigurerTest { credential, credential, Optional.empty(), - dev.caskeleton.messaging.security.DestinationAccessPolicy.denyAll()); + DestinationAccessPolicy.denyAll()); } - private static CredentialRuntimeRegistry registry(String secret) { + private static CredentialRuntimeRegistry registry(Optional username) { return new CredentialRuntimeRegistry( new CredentialProvider() { @Override public char[] resolve(String credentialId) { - return secret.toCharArray(); + return "secret".toCharArray(); } @Override public Optional expiresAt(String credentialId) { return Optional.of(NOW.plus(Duration.ofHours(1))); } + + @Override + public Optional username(String credentialId) { + return username; + } }, Duration.ofMinutes(5)); } diff --git a/messaging/messaging-kafka/src/testFixtures/java/dev/caskeleton/messaging/kafka/KafkaContractHarness.java b/messaging/messaging-kafka/src/testFixtures/java/dev/caskeleton/messaging/kafka/KafkaContractHarness.java index fb2e3ae..7d05669 100644 --- a/messaging/messaging-kafka/src/testFixtures/java/dev/caskeleton/messaging/kafka/KafkaContractHarness.java +++ b/messaging/messaging-kafka/src/testFixtures/java/dev/caskeleton/messaging/kafka/KafkaContractHarness.java @@ -22,7 +22,7 @@ import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.ConsumerPolicy; import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; import dev.caskeleton.messaging.policy.DeadLetterPolicy; @@ -263,7 +263,7 @@ final class KafkaContractHarness implements MessagingAdapterHarness { RetryPolicy.none(), DeadLetterPolicy.to(DEAD_LETTER), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, false, false, false); diff --git a/messaging/messaging-nats-experimental/build.gradle b/messaging/messaging-nats-experimental/build.gradle deleted file mode 100644 index 929bf94..0000000 --- a/messaging/messaging-nats-experimental/build.gradle +++ /dev/null @@ -1,14 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-transport-spi') - api project(':messaging:messaging-observability') - api project(':messaging:messaging-security') - api project(':messaging:messaging-admin-api') - -} diff --git a/messaging/messaging-nats-experimental/gradle.lockfile b/messaging/messaging-nats-experimental/gradle.lockfile deleted file mode 100644 index f5ad1bb..0000000 --- a/messaging/messaging-nats-experimental/gradle.lockfile +++ /dev/null @@ -1,89 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=runtimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=runtimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty= diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsAckMode.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsAckMode.java deleted file mode 100644 index 2a8b865..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsAckMode.java +++ /dev/null @@ -1,57 +0,0 @@ -package dev.caskeleton.messaging.nats; - -/** - * How a JetStream consumer acknowledges what it has handled. - * - *

Only {@link #EXPLICIT} is usable by the platform, and the other two are modelled anyway so the - * refusal can name what was configured instead of failing with a generic message. - * - *

{@link #NONE} and {@link #ALL} both break at-least-once in ways that look like success. {@code - * AckNone} never acknowledges, so JetStream considers every message delivered the moment it is sent - * — a handler that throws has already had its message forgotten. {@code AckAll} acknowledges every - * message up to the one being acknowledged, so a consumer processing messages concurrently silently - * acknowledges work that is still in flight. - */ -public enum NatsAckMode { - - /** Every message is acknowledged individually. The only mode the platform supports. */ - EXPLICIT(true), - - /** Acknowledging one message acknowledges every earlier one. */ - ALL(false), - - /** No acknowledgement at all; JetStream treats delivery as completion. */ - NONE(false); - - private final boolean supportsAtLeastOnce; - - NatsAckMode(boolean supportsAtLeastOnce) { - this.supportsAtLeastOnce = supportsAtLeastOnce; - } - - /** - * Reports whether this mode can carry at-least-once delivery. - * - * @return true only for {@link #EXPLICIT} - */ - public boolean supportsAtLeastOnce() { - return supportsAtLeastOnce; - } - - /** - * Returns why a mode cannot be used, for the refusal message. - * - * @return the sanitized reason, empty for {@link #EXPLICIT} - */ - public String rejectionReason() { - return switch (this) { - case EXPLICIT -> ""; - case ALL -> - "AckAll acknowledges every earlier message, so concurrent handling silently settles work " - + "that is still in flight"; - case NONE -> - "AckNone treats delivery as completion, so a message whose handler throws has already " - + "been forgotten by the broker"; - }; - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfile.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfile.java deleted file mode 100644 index 2713ab2..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfile.java +++ /dev/null @@ -1,103 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; - -/** - * The JetStream settings for one destination. - * - *

{@code maxDeliver} is the setting that makes the platform's dead-lettering necessary. When a - * message reaches it, JetStream terminates the message — it stops redelivering and routes - * it nowhere. Without the platform parking it first, the message is simply gone, which is why - * {@link NatsMaxDeliverParkingWorkflow} has to run strictly before the limit is reached rather than - * in response to it. - * - *

{@code ackWait} is the redelivery trigger. A handler slower than {@code ackWait} gets its - * message redelivered while it is still working on it, so this is the one JetStream number that has - * to be set from the handler's real latency rather than from a default. - * - * @param subject the JetStream subject - * @param stream the stream capturing that subject - * @param durableName the durable consumer name - * @param ackMode how the consumer acknowledges - * @param ackWait how long JetStream waits before redelivering - * @param maxDeliver how many deliveries before JetStream terminates the message - * @param deduplicationWindow the publish deduplication window, when enabled - */ -public record NatsJetStreamProfile( - String subject, - String stream, - String durableName, - NatsAckMode ackMode, - Duration ackWait, - int maxDeliver, - Optional deduplicationWindow) { - - /** How many deliveries the platform leaves as headroom before JetStream's terminate. */ - public static final int PARKING_HEADROOM = 1; - - public NatsJetStreamProfile { - Objects.requireNonNull(ackMode, "ackMode must not be null"); - Objects.requireNonNull(ackWait, "ackWait must not be null"); - Objects.requireNonNull(deduplicationWindow, "deduplicationWindow must not be null"); - requireText(subject, "subject"); - requireText(stream, "stream"); - requireText(durableName, "durableName"); - - if (!ackMode.supportsAtLeastOnce()) { - throw new IllegalArgumentException( - "%s cannot carry at-least-once delivery: %s" - .formatted(ackMode, ackMode.rejectionReason())); - } - if (ackWait.isNegative() || ackWait.isZero()) { - throw new IllegalArgumentException("ackWait must be positive"); - } - if (maxDeliver < 2) { - throw new IllegalArgumentException( - "maxDeliver must leave at least one delivery of headroom for the platform to park the " - + "message before JetStream terminates it"); - } - if (deduplicationWindow.isPresent() - && (deduplicationWindow.get().isNegative() || deduplicationWindow.get().isZero())) { - throw new IllegalArgumentException("deduplicationWindow must be positive when set"); - } - } - - /** - * Returns the delivery count at which the platform must park the message. - * - *

Strictly before {@code maxDeliver}: acting on the limit itself is too late, because the - * delivery that reaches it is the one JetStream terminates. - * - * @return the parking threshold - */ - public int parkAtDelivery() { - return maxDeliver - PARKING_HEADROOM; - } - - /** - * Returns a profile with the platform defaults. - * - * @param subject the JetStream subject - * @param stream the stream name - * @param durableName the durable consumer name - * @return the profile - */ - public static NatsJetStreamProfile durable(String subject, String stream, String durableName) { - return new NatsJetStreamProfile( - subject, - stream, - durableName, - NatsAckMode.EXPLICIT, - Duration.ofSeconds(30), - 5, - Optional.of(Duration.ofMinutes(2))); - } - - private static void requireText(String value, String field) { - if (value == null || value.isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfileValidator.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfileValidator.java deleted file mode 100644 index e5aaf17..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamProfileValidator.java +++ /dev/null @@ -1,75 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.util.Objects; - -/** - * Guards the NATS JetStream Experimental adapter. - * - *

Core NATS is refused for any at-least-once destination. Core NATS is fire-and-forget with no - * persistence and no acknowledgement, so a durable destination configured against it would report - * success for messages that were never stored anywhere — the failure mode is total and silent. - * - *

An ordered JetStream consumer and a competing work queue are mutually exclusive: the ordered - * consumer is a single-reader construct, and pointing several workers at it does not distribute the - * load, it breaks the ordering the consumer exists to provide. - */ -public final class NatsJetStreamProfileValidator { - - /** - * Validates a destination against a JetStream consumer configuration. - * - * @param profile the destination profile - * @param jetStreamEnabled whether JetStream persistence is in use - * @param orderedConsumer whether an ordered consumer is configured - * @param competingWorkers how many workers share the consumer - * @param enabled whether the experimental adapter is switched on - */ - public void validate( - DestinationProfile profile, - boolean jetStreamEnabled, - boolean orderedConsumer, - int competingWorkers, - boolean enabled) { - Objects.requireNonNull(profile, "profile must not be null"); - - if (!enabled) { - throw new MessagingCapabilityUnavailableException( - "NATS_DISABLED", - "the NATS JetStream adapter is experimental and disabled unless " - + "app.messaging.experimental.nats=true"); - } - if (profile.deliveryGuarantee() == DeliveryGuarantee.AT_LEAST_ONCE && !jetStreamEnabled) { - throw new IllegalArgumentException( - "core NATS cannot provide at-least-once delivery; JetStream is required: " - + profile.name().value()); - } - if (orderedConsumer && competingWorkers > 1) { - throw new IllegalArgumentException( - "an ordered JetStream consumer cannot be shared by competing workers: " - + profile.name().value()); - } - if (profile.orderingScope() == OrderingScope.KEY) { - throw new IllegalArgumentException( - "the NATS adapter does not offer per-key ordering: " + profile.name().value()); - } - } - - /** - * Returns what the JetStream adapter can prove. - * - *

Native dead lettering is reported as unavailable. JetStream terminates a message after its - * delivery limit rather than routing it anywhere, so the platform supplies dead lettering itself; - * claiming a native one would leave operators looking for a queue that does not exist. - * - * @return the capabilities - */ - public MessagingCapabilities capabilities() { - return new MessagingCapabilities( - true, true, true, true, true, false, true, false, false, true, false, true); - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamTransport.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamTransport.java deleted file mode 100644 index bf8f5c8..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsJetStreamTransport.java +++ /dev/null @@ -1,295 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import dev.caskeleton.messaging.api.destination.DestinationCapabilities; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import dev.caskeleton.messaging.transport.MessagingTransport; -import dev.caskeleton.messaging.transport.TransportConsumerRegistration; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import dev.caskeleton.messaging.transport.TransportPublishResult; -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionException; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicBoolean; -import java.util.function.Function; - -/** - * The Experimental NATS JetStream adapter. - * - *

Publishes go through JetStream, never core NATS. A core publish returns as soon as the bytes - * are written to the socket, with no persistence and no acknowledgement, so an adapter using it - * would report success for messages that were never stored — the failure is total and silent, which - * is why {@link NatsJetStreamProfileValidator} refuses the combination at startup. - * - *

A JetStream publish acknowledgement names the stream and sequence the message landed in, which - * is real persistence evidence. A publish that times out is {@code AMBIGUOUS}: JetStream may have - * stored it and lost only the acknowledgement, and the deduplication window is what makes retrying - * it safe when the profile enables one. - * - *

This is an Experimental contract seam, not a Stable adapter. It exercises the transport SPI - * against a publish operation the application supplies; it does not ship a NATS client bridge, - * connection management, or reconnection. Only a failure the operation reports as a {@link - * NatsPreSendRejection} is treated as definitely not transmitted — every other failure, recognised - * or not, is ambiguous, because a caller that reads {@code REJECTED} may republish under a new - * identity and duplicate a message the server already stored. - * - *

The adapter reports {@code nativeDeadLetter} as false. JetStream terminates a message at its - * delivery limit rather than routing it anywhere, so the platform parks it via {@link - * NatsMaxDeliverParkingWorkflow}; claiming a native dead-letter would send operators looking for a - * queue that does not exist. - */ -public final class NatsJetStreamTransport implements MessagingTransport { - - private static final MessagingCapabilities CAPABILITIES = - new MessagingCapabilities( - true, true, true, true, true, false, true, false, false, true, false, true); - - private final String brokerName; - private final long generation; - private final NatsJetStreamProfile profile; - private final JetStreamPublishOperation publish; - private final Function consumerFactory; - private final AtomicBoolean closed = new AtomicBoolean(); - - /** - * Creates a publish-only transport. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param profile the JetStream settings - * @param publish the JetStream publish operation - */ - public NatsJetStreamTransport( - String brokerName, - long generation, - NatsJetStreamProfile profile, - JetStreamPublishOperation publish) { - this( - brokerName, - generation, - profile, - publish, - spec -> { - throw new MessagingCapabilityUnavailableException( - "NATS_CONSUMER_NOT_CONFIGURED", - "this NATS transport was created without a consumer factory"); - }); - } - - /** - * Creates a transport with a consumer factory. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param profile the JetStream settings - * @param publish the JetStream publish operation - * @param consumerFactory builds a consumer registration for a spec - */ - public NatsJetStreamTransport( - String brokerName, - long generation, - NatsJetStreamProfile profile, - JetStreamPublishOperation publish, - Function consumerFactory) { - this.brokerName = Objects.requireNonNull(brokerName, "brokerName must not be null"); - this.generation = generation; - this.profile = Objects.requireNonNull(profile, "profile must not be null"); - this.publish = Objects.requireNonNull(publish, "publish must not be null"); - this.consumerFactory = Objects.requireNonNull(consumerFactory, "consumerFactory is required"); - } - - @Override - public CompletionStage publish(TransportPublishRequest request) { - Objects.requireNonNull(request, "request must not be null"); - - if (closed.get()) { - return completed(rejectedLocally("NATS_TRANSPORT_CLOSED", "the transport is shutting down")); - } - int size = request.envelope().payload().size(); - int limit = request.profile().payload().maxBytes(); - if (size > limit) { - return completed( - rejectedLocally( - "PAYLOAD_TOO_LARGE", "encoded payload is " + size + " bytes, limit is " + limit)); - } - - // The caller's own deadline, honoured rather than ignored. Without it a stalled JetStream - // publish holds the caller for as long as the SDK's default takes, and the timeout the caller - // set to bound its own latency budget means nothing. - long startedAt = System.nanoTime(); - return publish - .publish(profile.subject(), deduplicationId(request), request) - .toCompletableFuture() - .orTimeout( - request.options().timeout().toMillis(), java.util.concurrent.TimeUnit.MILLISECONDS) - .handle( - (position, failure) -> - failure == null - ? confirmed(position, elapsedSince(startedAt)) - : classify(failure, elapsedSince(startedAt))) - .thenApply(TransportPublishResult::new); - } - - @Override - public TransportConsumerRegistration register(TransportConsumerSpec spec) { - Objects.requireNonNull(spec, "spec must not be null"); - if (closed.get()) { - throw new MessagingCapabilityUnavailableException( - "NATS_TRANSPORT_CLOSED", "the transport is shutting down"); - } - return consumerFactory.apply(spec); - } - - @Override - public DestinationCapabilities capabilities(DestinationName destination) { - return new DestinationCapabilities(destination, brokerName, CAPABILITIES); - } - - @Override - public String brokerName() { - return brokerName; - } - - @Override - public long generation() { - return generation; - } - - @Override - public void close() { - closed.set(true); - } - - /** - * Reports whether the transport is still accepting work. - * - * @return true until close - */ - public boolean isAcceptingWork() { - return !closed.get(); - } - - /** - * Returns the deduplication id for a publish, when the profile enables deduplication. - * - *

The logical message id, so a retry of an ambiguous publish is recognised as the same message - * rather than accepted as a new one. Minting a fresh id per attempt would make the deduplication - * window useless in exactly the situation it exists for. - * - * @param request the publish request - * @return the deduplication id, or empty when the profile has no window - */ - private Optional deduplicationId(TransportPublishRequest request) { - return profile - .deduplicationWindow() - .map(window -> request.envelope().messageId().value().toString()); - } - - private static Duration elapsedSince(long startedAtNanos) { - return Duration.ofNanos(System.nanoTime() - startedAtNanos); - } - - private static PublishResult confirmed(NatsStreamPosition position, Duration elapsed) { - return new PublishResult( - PublishCompletion.CONFIRMED, - // The publish acknowledgement names the stream and sequence the message landed in, which is - // persistence evidence rather than a receipt for bytes on a socket. - new PublishEvidence( - true, - TransmissionEvidence.TRANSMITTED, - true, - ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK), - RoutingOutcome.ROUTED, - Optional.of(position), - 1, - elapsed, - Optional.empty()); - } - - /** - * Decides what a publish failure means, from typed signals only. - * - *

Only {@link NatsPreSendRejection} — thrown by the publish operation when it knows the send - * never left the client — reports {@code REJECTED / NOT_TRANSMITTED}. Everything else is {@code - * AMBIGUOUS}, including failures this adapter does not recognise, because the alternative is - * telling a caller nothing was transmitted when the server may already hold the message. - */ - private static PublishResult classify(Throwable failure, Duration elapsed) { - Throwable cause = failure instanceof CompletionException ? failure.getCause() : failure; - if (cause instanceof NatsPreSendRejection rejection) { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.CONFIGURATION, rejection.code(), rejection.getMessage()))); - } - boolean timedOut = cause instanceof java.util.concurrent.TimeoutException; - return new PublishResult( - PublishCompletion.AMBIGUOUS, - new PublishEvidence( - true, TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.TRANSIENT_INFRASTRUCTURE, - timedOut ? "NATS_PUBLISH_TIMEOUT" : "NATS_PUBLISH_OUTCOME_UNKNOWN", - timedOut - ? "the publish acknowledgement did not arrive; the stream may already hold it" - : "the publish failed in a way this adapter does not classify; the stream may" - + " already hold the message"))); - } - - private static TransportPublishResult rejectedLocally(String code, String message) { - return new TransportPublishResult( - new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - // A local refusal: nothing was attempted, so no time was spent on the wire. - Duration.ZERO, - Optional.of(FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message)))); - } - - private static CompletionStage completed(TransportPublishResult result) { - return CompletableFuture.completedFuture(result); - } - - /** The JetStream publish, isolated so the adapter is testable without a server. */ - @FunctionalInterface - public interface JetStreamPublishOperation { - - /** - * Publishes one encoded message to JetStream. - * - * @param subject the JetStream subject - * @param deduplicationId the {@code Nats-Msg-Id} value, when deduplication is enabled - * @param request the publish request - * @return a stage completing with the stream position the message landed at - */ - CompletionStage publish( - String subject, Optional deduplicationId, TransportPublishRequest request); - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingWorkflow.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingWorkflow.java deleted file mode 100644 index d13ee0c..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingWorkflow.java +++ /dev/null @@ -1,85 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import java.util.Objects; - -/** - * Decides what to do with a JetStream delivery before its {@code maxDeliver} limit destroys it. - * - *

JetStream has no dead-letter queue. When a message hits {@code maxDeliver} the server - * terminates it: no redelivery, no routing, no record beyond an advisory. Every other broker in - * this platform parks a poison message somewhere an operator can find it, and this workflow is what - * makes NATS behave the same way. - * - *

The parking therefore happens on the delivery before the limit, not on the limit - * itself. Acting at {@code maxDeliver} would mean acting on the delivery JetStream is about to - * discard, so any failure in the dead-letter publish would lose the message outright — precisely - * the case the DLQ-confirm-before-settle invariant exists to prevent. - */ -public final class NatsMaxDeliverParkingWorkflow { - - /** What the platform should do with one delivery. */ - public enum Action { - /** Hand the message to the handler as normal. */ - DELIVER_TO_HANDLER, - /** Publish to the platform dead-letter destination, then terminate the source. */ - PARK_TO_DEAD_LETTER, - /** - * The message has already passed the limit and JetStream has terminated it. - * - *

Reachable only if the profile changed under a live consumer. Nothing can be recovered - * here; the outcome exists so the situation is reported rather than mistaken for a redelivery. - */ - ALREADY_TERMINATED - } - - private final NatsJetStreamProfile profile; - - /** - * Creates a workflow for one destination. - * - * @param profile the JetStream settings - */ - public NatsMaxDeliverParkingWorkflow(NatsJetStreamProfile profile) { - this.profile = Objects.requireNonNull(profile, "profile must not be null"); - } - - /** - * Decides what to do with a delivery. - * - * @param deliveryCount how many times this message has been delivered, counting this one - * @return the action to take - */ - public Action decide(long deliveryCount) { - if (deliveryCount < 1) { - throw new IllegalArgumentException("the first delivery is delivery 1"); - } - if (deliveryCount > profile.maxDeliver()) { - return Action.ALREADY_TERMINATED; - } - return deliveryCount >= profile.parkAtDelivery() - ? Action.PARK_TO_DEAD_LETTER - : Action.DELIVER_TO_HANDLER; - } - - /** - * Reports whether the source may be terminated after a dead-letter publish. - * - *

Only on a confirmed publish. Terminating first would discard the message on a broker that - * cannot redeliver it, which is the one irreversible mistake available here. - * - * @param deadLetterConfirmed whether the dead-letter publish was confirmed - * @return true when the source may be terminated - */ - public boolean maySettleSource(boolean deadLetterConfirmed) { - return deadLetterConfirmed; - } - - /** - * Returns the delivery count at which parking begins. - * - * @return the parking threshold - */ - public int parkingThreshold() { - return profile.parkAtDelivery(); - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsPreSendRejection.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsPreSendRejection.java deleted file mode 100644 index 039addc..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsPreSendRejection.java +++ /dev/null @@ -1,65 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import java.util.Objects; - -/** - * The one signal that lets this adapter report {@code REJECTED / NOT_TRANSMITTED}. - * - *

Classification used to work by reading the exception's class simple name and matching - * substrings — {@code "Timeout"} meant ambiguous, anything else meant rejected. Two things are - * wrong with that. A class name is not part of any SDK's contract: it changes between versions, and - * an unrelated exception whose name happens to contain the substring is misread. And defaulting the - * unknown case to {@code REJECTED} tells the caller nothing was transmitted, so a caller that - * republishes under a new identity duplicates a message the server may well have stored. - * - *

The publish operation throws this only when it knows the send never left the client: an - * argument the SDK refused, a connection that was already closed, a subject the client could not - * resolve. Everything else — including anything the adapter does not recognise — is ambiguous, and - * ambiguity is the honest answer when the write may have landed. - */ -public final class NatsPreSendRejection extends RuntimeException { - - private static final long serialVersionUID = 1L; - - private final String code; - - /** - * Creates a pre-send rejection. - * - * @param code the sanitised platform code for the refusal - * @param message a sanitised description - */ - public NatsPreSendRejection(String code, String message) { - super(message); - this.code = requireCode(code); - } - - /** - * Creates a pre-send rejection wrapping the SDK's own exception. - * - * @param code the sanitised platform code for the refusal - * @param message a sanitised description - * @param cause the SDK exception that proves the send never started - */ - public NatsPreSendRejection(String code, String message, Throwable cause) { - super(message, cause); - this.code = requireCode(code); - } - - private static String requireCode(String code) { - Objects.requireNonNull(code, "code must not be null"); - if (code.isBlank()) { - throw new IllegalArgumentException("code must not be blank"); - } - return code; - } - - /** - * The sanitised code reported to the caller. - * - * @return the code - */ - public String code() { - return code; - } -} diff --git a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsStreamPosition.java b/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsStreamPosition.java deleted file mode 100644 index 7463b69..0000000 --- a/messaging/messaging-nats-experimental/src/main/java/dev/caskeleton/messaging/nats/NatsStreamPosition.java +++ /dev/null @@ -1,75 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import dev.caskeleton.messaging.api.publish.BrokerPosition; -import java.util.Map; -import java.util.Objects; - -/** - * A JetStream message's coordinate: its stream, its sequence in that stream, and the consumer's own - * sequence. - * - *

Two sequences, because they answer different questions. The stream sequence identifies the - * message and is what a replay seeks to. The consumer sequence counts deliveries to this - * consumer, so it advances on every redelivery while the stream sequence does not — comparing them - * is how a redelivery is recognised at all. - * - * @param stream the stream holding the message - * @param streamSequence the message's position in the stream - * @param consumerSequence how many messages this consumer has been delivered - * @param deliveryCount how many times this message has been delivered - */ -public record NatsStreamPosition( - String stream, long streamSequence, long consumerSequence, long deliveryCount) - implements BrokerPosition { - - public NatsStreamPosition { - if (stream == null || stream.isBlank()) { - throw new IllegalArgumentException("stream must not be blank"); - } - if (streamSequence < 1) { - throw new IllegalArgumentException("JetStream sequences start at 1"); - } - if (deliveryCount < 1) { - throw new IllegalArgumentException("a delivered message has been delivered at least once"); - } - } - - @Override - public String broker() { - return "nats"; - } - - @Override - public Map diagnosticAttributes() { - return Map.of( - "stream", stream, - "streamSequence", Long.toString(streamSequence), - "consumerSequence", Long.toString(consumerSequence), - "deliveryCount", Long.toString(deliveryCount)); - } - - /** - * Reports whether this delivery is a redelivery. - * - * @return true when the message has been delivered before - */ - public boolean isRedelivery() { - return deliveryCount > 1; - } - - /** - * Returns the stream sequence a replay should resume from to re-read this message. - * - * @return the resume sequence - */ - public long replayFrom() { - return streamSequence; - } - - @Override - public String toString() { - return Objects.toString( - "NatsStreamPosition[stream=%s, streamSequence=%d, deliveryCount=%d]" - .formatted(stream, streamSequence, deliveryCount)); - } -} diff --git a/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsAdapterContractTest.java b/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsAdapterContractTest.java deleted file mode 100644 index 6e9dd33..0000000 --- a/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsAdapterContractTest.java +++ /dev/null @@ -1,337 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; -import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import dev.caskeleton.messaging.policy.CapabilityTier; -import dev.caskeleton.messaging.policy.ConsumerPolicy; -import dev.caskeleton.messaging.policy.DeadLetterPolicy; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.PayloadPolicy; -import dev.caskeleton.messaging.policy.PhysicalDestination; -import dev.caskeleton.messaging.policy.ProducerPolicy; -import dev.caskeleton.messaging.policy.RetryPolicy; -import dev.caskeleton.messaging.policy.SchemaPolicy; -import dev.caskeleton.messaging.schema.EncodedMessage; -import dev.caskeleton.messaging.schema.SchemaCompatibility; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import dev.caskeleton.messaging.transport.TransportPublishResult; -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Optional; -import java.util.Set; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.TimeoutException; -import org.junit.jupiter.api.Test; - -class NatsAdapterContractTest { - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - private static final NatsStreamPosition POSITION = new NatsStreamPosition("ORDERS", 42, 42, 1); - - private final List> capturedDeduplicationIds = new ArrayList<>(); - - private NatsJetStreamTransport confirming(Optional deduplicationWindow) { - return new NatsJetStreamTransport( - "nats-primary", - 1, - profile(deduplicationWindow), - (subject, deduplicationId, request) -> { - capturedDeduplicationIds.add(deduplicationId); - return CompletableFuture.completedFuture(POSITION); - }); - } - - private NatsJetStreamTransport failingWith(Throwable failure) { - return new NatsJetStreamTransport( - "nats-primary", - 1, - profile(Optional.of(Duration.ofMinutes(2))), - (subject, deduplicationId, request) -> CompletableFuture.failedFuture(failure)); - } - - private static PublishResult await(CompletionStage stage) { - return stage.toCompletableFuture().join().result(); - } - - @Test - void aStoredMessageReportsPersistenceEvidence() { - PublishResult result = await(confirming(Optional.empty()).publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.evidence().confirmationLevel()) - .as("the acknowledgement names the stream and sequence, not just bytes on a socket") - .isEqualTo(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK); - } - - @Test - void theStreamPositionIsCarriedBack() { - assertThat(await(confirming(Optional.empty()).publish(request(64))).position()) - .hasValue(POSITION); - } - - @Test - void aPublishTimeoutIsAmbiguousBecauseTheStreamMayHoldIt() { - PublishResult result = await(failingWith(new TimeoutException("no ack")).publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.evidence().transmission()) - .isEqualTo(TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED); - } - - @Test - void aFailureNamedLikeAKnownOneIsStillAmbiguous() { - // The class name used to be the classifier: an exception whose simple name contained - // "NoRespond" was reported REJECTED / NOT_TRANSMITTED. A name is not an SDK contract, and a - // caller that reads NOT_TRANSMITTED republishes under a new identity. - PublishResult result = - await(failingWith(new IllegalStateException("NoRespondersException")).publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("NATS_PUBLISH_OUTCOME_UNKNOWN")); - } - - @Test - void onlyATypedPreSendRejectionReportsNothingTransmitted() { - PublishResult result = - await( - failingWith( - new NatsPreSendRejection( - "NATS_CONNECTION_CLOSED", "the client was closed before the send")) - .publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()) - .isEqualTo(dev.caskeleton.messaging.api.publish.TransmissionEvidence.NOT_TRANSMITTED); - assertThat(result.failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("NATS_CONNECTION_CLOSED")); - } - - @Test - void aWrappedUnknownFailureIsAmbiguousRatherThanRejected() { - PublishResult result = - await( - failingWith( - new java.util.concurrent.CompletionException( - new RuntimeException("something the adapter has never seen"))) - .publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - } - - @Test - void aPublishThatNeverCompletesIsBoundedByTheCallersTimeout() { - try (NatsJetStreamTransport transport = - new NatsJetStreamTransport( - "nats-primary", - 1L, - profile(Optional.empty()), - (subject, deduplicationId, request) -> - new java.util.concurrent.CompletableFuture<>())) { - - PublishResult result = await(transport.publish(requestWithTimeout(Duration.ofMillis(50)))); - - assertThat(result.completion()) - .as("the caller's deadline used to be ignored entirely") - .isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("NATS_PUBLISH_TIMEOUT")); - } - } - - @Test - void theReportedElapsedTimeIsMeasuredRatherThanZero() { - PublishResult result = - await( - failingWith(new java.util.concurrent.TimeoutException("no ack")).publish(request(64))); - - assertThat(result.elapsed()) - .as("every outcome reported Duration.ZERO, so latency evidence was fiction") - .isGreaterThanOrEqualTo(Duration.ZERO); - } - - @Test - void aDeduplicationWindowSendsTheLogicalMessageIdSoARetryIsRecognised() { - confirming(Optional.of(Duration.ofMinutes(2))) - .publish(request(64)) - .toCompletableFuture() - .join(); - - assertThat(capturedDeduplicationIds) - .as("a fresh id per attempt would make the window useless exactly when it is needed") - .singleElement() - .satisfies(id -> assertThat(id).isPresent()); - } - - @Test - void noDeduplicationWindowSendsNoDeduplicationId() { - confirming(Optional.empty()).publish(request(64)).toCompletableFuture().join(); - - assertThat(capturedDeduplicationIds).singleElement().satisfies(id -> assertThat(id).isEmpty()); - } - - @Test - void anOversizedPayloadIsRejectedBeforeTheStream() { - PublishResult result = - await(confirming(Optional.empty()).publish(request(PayloadPolicy.DEFAULT_MAX_BYTES + 1))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.NOT_TRANSMITTED); - } - - @Test - void theAdapterDoesNotClaimANativeDeadLetter() { - assertThat( - confirming(Optional.empty()) - .capabilities(new DestinationName("orders.v1")) - .capabilities() - .nativeDeadLetter()) - .as("JetStream terminates at the delivery limit; it routes nowhere") - .isFalse(); - } - - @Test - void theAdapterAdvertisesDeduplicatedPublish() { - assertThat( - confirming(Optional.empty()) - .capabilities(new DestinationName("orders.v1")) - .capabilities() - .deduplicatedPublish()) - .isTrue(); - } - - @Test - void aClosedTransportStopsAcceptingWork() { - NatsJetStreamTransport transport = confirming(Optional.empty()); - transport.close(); - - assertThat(transport.isAcceptingWork()).isFalse(); - assertThat(await(transport.publish(request(64))).completion()) - .isEqualTo(PublishCompletion.REJECTED); - } - - @Test - void aRedeliveryIsRecognisedFromTheDeliveryCountNotTheSequence() { - NatsStreamPosition redelivered = new NatsStreamPosition("ORDERS", 42, 43, 2); - - assertThat(POSITION.isRedelivery()).isFalse(); - assertThat(redelivered.isRedelivery()).isTrue(); - assertThat(redelivered.replayFrom()) - .as("the stream sequence does not advance on a redelivery; the consumer sequence does") - .isEqualTo(POSITION.replayFrom()); - } - - @Test - void aSequenceBelowOneIsRejectedBecauseJetStreamCountsFromOne() { - assertThatThrownBy(() -> new NatsStreamPosition("ORDERS", 0, 1, 1)) - .isInstanceOf(IllegalArgumentException.class); - } - - @Test - void aFailureCategoryIsAlwaysAttachedToAFailure() { - PublishResult result = - await(failingWith(new IllegalStateException("boom")).publish(request(64))); - - assertThat(result.failure()) - .hasValueSatisfying( - failure -> - assertThat(failure.category()) - .as("an unclassified failure is transient, not a permanent business rejection") - .isEqualTo(FailureCategory.TRANSIENT_INFRASTRUCTURE)); - } - - private static NatsJetStreamProfile profile(Optional deduplicationWindow) { - return new NatsJetStreamProfile( - "orders.created", - "ORDERS", - "orders-worker", - NatsAckMode.EXPLICIT, - Duration.ofSeconds(30), - 5, - deduplicationWindow); - } - - private static TransportPublishRequest request(int payloadBytes) { - return new TransportPublishRequest( - destinationProfile(), envelope(payloadBytes), PublishOptions.defaults()); - } - - private static TransportPublishRequest requestWithTimeout(Duration timeout) { - PublishOptions defaults = PublishOptions.defaults(); - return new TransportPublishRequest( - destinationProfile(), - envelope(64), - new PublishOptions(timeout, defaults.confirmation(), defaults.deduplication())); - } - - private static DestinationProfile destinationProfile() { - return new DestinationProfile( - new DestinationName("orders.v1"), - "nats-primary", - DestinationKind.EVENT_STREAM, - PhysicalDestination.natsStream("orders.created", "ORDERS"), - new SchemaPolicy( - ContentType.JSON, - SchemaCompatibility.BACKWARD, - Set.of(new MessageType("order.created"))), - DeliveryGuarantee.AT_LEAST_ONCE, - OrderingScope.NONE, - ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, - ProducerPolicy.defaults(), - ConsumerPolicy.defaults("orders"), - RetryPolicy.none(), - DeadLetterPolicy.to(new DestinationName("orders.v1.dlq")), - PayloadPolicy.defaults(), - CapabilityTier.M1, - false, - false, - false); - } - - private static MessageEnvelope envelope(int payloadBytes) { - byte[] payload = new byte[payloadBytes]; - java.util.Arrays.fill(payload, (byte) 'x'); - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - new EncodedMessage(payload, ContentType.JSON, Optional.empty())); - } -} diff --git a/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingTest.java b/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingTest.java deleted file mode 100644 index 93d0a0c..0000000 --- a/messaging/messaging-nats-experimental/src/test/java/dev/caskeleton/messaging/nats/NatsMaxDeliverParkingTest.java +++ /dev/null @@ -1,123 +0,0 @@ -package dev.caskeleton.messaging.nats; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.nats.NatsMaxDeliverParkingWorkflow.Action; -import java.time.Duration; -import java.util.Optional; -import org.junit.jupiter.api.Test; - -class NatsMaxDeliverParkingTest { - - private static NatsJetStreamProfile profile(int maxDeliver) { - return new NatsJetStreamProfile( - "orders.created", - "ORDERS", - "orders-worker", - NatsAckMode.EXPLICIT, - Duration.ofSeconds(30), - maxDeliver, - Optional.of(Duration.ofMinutes(2))); - } - - private static NatsMaxDeliverParkingWorkflow workflow(int maxDeliver) { - return new NatsMaxDeliverParkingWorkflow(profile(maxDeliver)); - } - - @Test - void earlyDeliveriesGoToTheHandler() { - NatsMaxDeliverParkingWorkflow workflow = workflow(5); - - assertThat(workflow.decide(1)).isEqualTo(Action.DELIVER_TO_HANDLER); - assertThat(workflow.decide(3)).isEqualTo(Action.DELIVER_TO_HANDLER); - } - - @Test - void parkingHappensBeforeTheLimitNotAtIt() { - NatsMaxDeliverParkingWorkflow workflow = workflow(5); - - assertThat(workflow.decide(4)) - .as("the delivery that reaches maxDeliver is the one JetStream discards") - .isEqualTo(Action.PARK_TO_DEAD_LETTER); - } - - @Test - void theLimitDeliveryItselfStillParksRatherThanBeingHandled() { - assertThat(workflow(5).decide(5)).isEqualTo(Action.PARK_TO_DEAD_LETTER); - } - - @Test - void aDeliveryPastTheLimitIsReportedRatherThanMistakenForARedelivery() { - assertThat(workflow(5).decide(6)) - .as("only reachable if the profile changed under a live consumer; nothing is recoverable") - .isEqualTo(Action.ALREADY_TERMINATED); - } - - @Test - void theSourceIsOnlyTerminatedAfterTheDeadLetterPublishConfirms() { - NatsMaxDeliverParkingWorkflow workflow = workflow(5); - - assertThat(workflow.maySettleSource(true)).isTrue(); - assertThat(workflow.maySettleSource(false)) - .as("terminating first discards the message on a broker that cannot redeliver it") - .isFalse(); - } - - @Test - void aProfileWithNoParkingHeadroomIsRefused() { - assertThatThrownBy(() -> profile(1)) - .as("maxDeliver=1 leaves no delivery on which the platform could park the message") - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("headroom"); - } - - @Test - void aDeliveryCountBelowOneIsRejected() { - assertThatThrownBy(() -> workflow(5).decide(0)).isInstanceOf(IllegalArgumentException.class); - } - - @Test - void theParkingThresholdIsOneBelowTheLimit() { - assertThat(workflow(7).parkingThreshold()).isEqualTo(6); - } - - @Test - void ackNoneIsRefusedBecauseDeliveryIsTreatedAsCompletion() { - assertThatThrownBy( - () -> - new NatsJetStreamProfile( - "orders.created", - "ORDERS", - "orders-worker", - NatsAckMode.NONE, - Duration.ofSeconds(30), - 5, - Optional.empty())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("forgotten"); - } - - @Test - void ackAllIsRefusedBecauseItSettlesWorkStillInFlight() { - assertThatThrownBy( - () -> - new NatsJetStreamProfile( - "orders.created", - "ORDERS", - "orders-worker", - NatsAckMode.ALL, - Duration.ofSeconds(30), - 5, - Optional.empty())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("still in flight"); - } - - @Test - void onlyExplicitAcknowledgementCarriesAtLeastOnce() { - assertThat(NatsAckMode.EXPLICIT.supportsAtLeastOnce()).isTrue(); - assertThat(NatsAckMode.ALL.supportsAtLeastOnce()).isFalse(); - assertThat(NatsAckMode.NONE.supportsAtLeastOnce()).isFalse(); - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumMappedRecord.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumMappedRecord.java deleted file mode 100644 index cd53419..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumMappedRecord.java +++ /dev/null @@ -1,54 +0,0 @@ -package dev.caskeleton.messaging.outbox; - -import java.util.Arrays; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; - -/** - * What one outbox row looks like after Debezium's Event Router has transformed it. - * - *

Modelled explicitly so the CDC path can be tested without running Debezium. The property that - * matters is not that the transform is invoked but that its output carries the same identity, - * destination, and reserved headers the polling relay would have produced — and that is asserted by - * comparing this record against one built from the same row. - * - * @param topic the topic the row was routed to - * @param key the broker partition key, absent when the row is unkeyed - * @param payload the encoded payload - * @param headers the reserved headers carried alongside - */ -@SuppressWarnings("ArrayRecordComponent") -public record DebeziumMappedRecord( - String topic, Optional key, byte[] payload, Map headers) { - - public DebeziumMappedRecord { - Objects.requireNonNull(key, "key must not be null"); - Objects.requireNonNull(payload, "payload must not be null"); - Objects.requireNonNull(headers, "headers must not be null"); - if (topic == null || topic.isBlank()) { - throw new IllegalArgumentException("topic must not be blank"); - } - payload = payload.clone(); - headers = Map.copyOf(headers); - } - - @Override - public byte[] payload() { - return payload.clone(); - } - - @Override - public boolean equals(Object other) { - return other instanceof DebeziumMappedRecord record - && topic.equals(record.topic) - && key.equals(record.key) - && headers.equals(record.headers) - && Arrays.equals(payload, record.payload); - } - - @Override - public int hashCode() { - return Objects.hash(topic, key, headers, Arrays.hashCode(payload)); - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxEventRouter.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxEventRouter.java deleted file mode 100644 index dbb64a8..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxEventRouter.java +++ /dev/null @@ -1,151 +0,0 @@ -package dev.caskeleton.messaging.outbox; - -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.reliability.OutboxCanonicalMetadata; -import dev.caskeleton.messaging.reliability.OutboxRecord; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Objects; - -/** - * Maps outbox columns onto the headers Debezium's Outbox Event Router expects. - * - *

An optional alternative to the polling relay. CDC reads the database's write-ahead log instead - * of querying the table, which removes the polling interval and the lease contention — at the cost - * of an extra piece of infrastructure and its own failure modes. - * - *

The platform contract does not change with the transport. The router emits the same reserved - * headers the polling relay does, so a consumer cannot tell which one published a message, and - * switching between them is a deployment decision rather than a contract change. - * - *

The message key and the topic are two different questions and used to be answered by one - * column. {@code destination} routed the topic and became the key, so every message on a - * topic carried the same key, landed on one partition, and made keyed ordering a claim with no - * mechanism behind it. The key now comes from {@code routing_key}, the stored column that states - * the partition-key-or-message-id fallback once for both relays, and {@code destination} stays what - * it always was: routing metadata. - * - *

One canonical field has no header here: the schema subject. There is no reserved header name - * for it, and inventing one that only this path wrote would replace a symmetric gap with an - * asymmetric one — the polling relay would carry a schema reference on the envelope and the CDC - * relay a header no adapter reads. It stays a column that the polling relay resolves, and closing - * it means adding the name to {@code ReservedHeaders} and teaching both broker header mappers to - * write and read it. - */ -public final class DebeziumOutboxEventRouter { - - /** The outbox column Debezium routes the topic by. */ - public static final String ROUTE_BY_COLUMN = "destination"; - - /** The outbox column Debezium uses as the message key. */ - public static final String KEY_COLUMN = "routing_key"; - - /** The outbox column carrying the logical message id. */ - public static final String ID_COLUMN = "message_id"; - - /** The outbox column carrying the payload. */ - public static final String PAYLOAD_COLUMN = "payload"; - - /** - * Returns the router configuration for a Debezium connector. - * - * @param topicPrefix the prefix routed topics are created under - * @return the connector properties - */ - public Map connectorConfiguration(String topicPrefix) { - Objects.requireNonNull(topicPrefix, "topicPrefix must not be null"); - Map configuration = new LinkedHashMap<>(); - configuration.put("transforms.outbox.type", "io.debezium.transforms.outbox.EventRouter"); - configuration.put("transforms.outbox.table.field.event.id", ID_COLUMN); - configuration.put("transforms.outbox.table.field.event.key", KEY_COLUMN); - configuration.put("transforms.outbox.table.field.event.payload", PAYLOAD_COLUMN); - configuration.put("transforms.outbox.route.by.field", ROUTE_BY_COLUMN); - configuration.put( - "transforms.outbox.route.topic.replacement", topicPrefix + "${routedByValue}"); - configuration.put( - "transforms.outbox.table.fields.additional.placement", - String.join( - ",", - "message_type:header:" + ReservedHeaders.MESSAGE_TYPE, - "schema_version:header:" + ReservedHeaders.SCHEMA_VERSION, - "content_type:header:" + ReservedHeaders.CONTENT_TYPE, - "created_at:header:" + ReservedHeaders.PRODUCED_AT, - "destination:header:" + ReservedHeaders.ORIGIN_DESTINATION, - "producer:header:" + ReservedHeaders.PRODUCER, - "occurred_at:header:" + ReservedHeaders.OCCURRED_AT, - "correlation_id:header:" + ReservedHeaders.CORRELATION_ID, - "causation_id:header:" + ReservedHeaders.CAUSATION_ID, - "tenant:header:" + ReservedHeaders.TENANT, - "partition_key:header:" + ReservedHeaders.PARTITION_KEY, - "ordering_key:header:" + ReservedHeaders.ORDERING_KEY, - "traceparent:header:" + ReservedHeaders.TRACEPARENT, - "tracestate:header:" + ReservedHeaders.TRACESTATE, - "baggage:header:" + ReservedHeaders.BAGGAGE)); - return Map.copyOf(configuration); - } - - /** - * Returns the reserved headers a routed record must carry. - * - *

Used to assert that the CDC path and the polling path produce the same wire contract. - * - * @param record the outbox row - * @return the reserved headers - */ - public Map reservedHeaders(OutboxRecord record) { - Objects.requireNonNull(record, "record must not be null"); - Map headers = new LinkedHashMap<>(); - headers.put(ReservedHeaders.MESSAGE_ID, record.messageId().value().toString()); - headers.put(ReservedHeaders.MESSAGE_TYPE, record.messageType().value()); - headers.put(ReservedHeaders.SCHEMA_VERSION, Integer.toString(record.schemaVersion().value())); - headers.put(ReservedHeaders.CONTENT_TYPE, record.contentType().value()); - headers.put(ReservedHeaders.PRODUCED_AT, record.createdAt().toString()); - headers.put(ReservedHeaders.ORIGIN_DESTINATION, record.destination().value()); - - // Absent stays absent. Writing an empty string for a column the row does not have would make a - // message that carries no tenant indistinguishable from one whose tenant is the empty string, - // and the consumer would have to know which of the two this platform meant. - OutboxCanonicalMetadata metadata = record.metadata(); - metadata.producer().ifPresent(value -> headers.put(ReservedHeaders.PRODUCER, value.value())); - metadata - .occurredAt() - .ifPresent(value -> headers.put(ReservedHeaders.OCCURRED_AT, value.toString())); - metadata - .correlationId() - .ifPresent(value -> headers.put(ReservedHeaders.CORRELATION_ID, value.value())); - metadata - .causationId() - .ifPresent( - value -> headers.put(ReservedHeaders.CAUSATION_ID, value.value().value().toString())); - metadata.tenant().ifPresent(value -> headers.put(ReservedHeaders.TENANT, value.tenantId())); - metadata.partitionKey().ifPresent(value -> headers.put(ReservedHeaders.PARTITION_KEY, value)); - metadata.orderingKey().ifPresent(value -> headers.put(ReservedHeaders.ORDERING_KEY, value)); - metadata - .traceContext() - .traceparent() - .ifPresent(value -> headers.put(ReservedHeaders.TRACEPARENT, value)); - metadata - .traceContext() - .tracestate() - .ifPresent(value -> headers.put(ReservedHeaders.TRACESTATE, value)); - metadata - .traceContext() - .baggage() - .ifPresent(value -> headers.put(ReservedHeaders.BAGGAGE, value)); - return Map.copyOf(headers); - } - - /** - * Returns the key a routed record carries. - * - *

The same value the {@code routing_key} column computes, stated here so a test can compare - * the two rather than trusting that the SQL and the Java agree. - * - * @param record the outbox row - * @return the broker message key - */ - public String routingKey(OutboxRecord record) { - Objects.requireNonNull(record, "record must not be null"); - return record.metadata().partitionKey().orElseGet(() -> record.messageId().value().toString()); - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxProfile.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxProfile.java deleted file mode 100644 index 4c08572..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxProfile.java +++ /dev/null @@ -1,67 +0,0 @@ -package dev.caskeleton.messaging.outbox; - -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import java.util.Objects; - -/** - * Which relay a deployment uses, and the rule that only one of them may be active. - * - *

Running the polling relay and the CDC relay together publishes every row twice. Both are - * correct in isolation and neither can see the other: the poller leases and marks rows, while - * Debezium reads the write-ahead log and never observes the lease at all. The duplicate is - * therefore invisible in the outbox table and only shows up downstream, which is why the - * incompatibility is enforced at startup instead of documented. - * - * @param mode which relay publishes outbox rows - * @param topicPrefix the prefix CDC-routed topics are created under - * @param aggregateIdAsPartitionKey whether the aggregate id becomes the broker partition key - */ -public record DebeziumOutboxProfile( - RelayMode mode, String topicPrefix, boolean aggregateIdAsPartitionKey) { - - /** How outbox rows reach the broker. */ - public enum RelayMode { - /** The in-process relay leases and publishes rows. */ - POLLING, - /** Debezium reads the write-ahead log and routes rows. */ - CHANGE_DATA_CAPTURE - } - - public DebeziumOutboxProfile { - Objects.requireNonNull(mode, "mode must not be null"); - Objects.requireNonNull(topicPrefix, "topicPrefix must not be null"); - if (mode == RelayMode.CHANGE_DATA_CAPTURE && topicPrefix.isBlank()) { - throw new IllegalArgumentException("a CDC profile must name a topic prefix"); - } - } - - /** - * Returns the default polling profile. - * - * @return the polling profile - */ - public static DebeziumOutboxProfile polling() { - return new DebeziumOutboxProfile(RelayMode.POLLING, "", false); - } - - /** - * Refuses a configuration that would run both relays at once. - * - * @param pollingRelayEnabled whether the in-process relay bean is active - * @throws MessagingConfigurationException when both relays would run - */ - public void requireExactlyOneRelay(boolean pollingRelayEnabled) { - if (mode == RelayMode.CHANGE_DATA_CAPTURE && pollingRelayEnabled) { - throw new MessagingConfigurationException( - "DUPLICATE_OUTBOX_RELAY", - "the CDC relay and the polling relay are both enabled; Debezium never observes the " - + "polling lease, so every row would be published twice"); - } - if (mode == RelayMode.POLLING && !pollingRelayEnabled) { - throw new MessagingConfigurationException( - "NO_OUTBOX_RELAY", - "the profile selects the polling relay but no relay is enabled, so outbox rows would " - + "accumulate without ever being published"); - } - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapper.java b/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapper.java deleted file mode 100644 index 513f246..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapper.java +++ /dev/null @@ -1,79 +0,0 @@ -package dev.caskeleton.messaging.outbox; - -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.reliability.OutboxRecord; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; - -/** - * Produces what Debezium's Event Router will emit for an outbox row. - * - *

Exists so the CDC path is testable without Debezium running. The property worth proving is - * that a consumer cannot tell which relay published a message: same logical id, same destination, - * same reserved headers. Building the expected record here and comparing it against the polling - * relay's output is what makes "switching relays is a deployment decision, not a contract change" - * an assertion rather than a claim. - * - *

The message id comes from the row's {@code message_id} column, never from Debezium's own event - * id. Debezium mints a fresh event id per change record, so using it would break identity across a - * CDC re-snapshot — exactly the scenario where identity matters most. - * - *

The reserved headers come from {@link DebeziumOutboxEventRouter} rather than being rebuilt - * here. Two lists of the same headers is how the connector's configuration and the expectation this - * class encodes come to disagree, and the disagreement would be invisible: the test compares this - * output against the polling relay's, so it would pass while the connector emitted something else. - */ -public final class DebeziumOutboxRecordMapper { - - private final String topicPrefix; - private final boolean aggregateIdAsPartitionKey; - private final DebeziumOutboxEventRouter router = new DebeziumOutboxEventRouter(); - - /** - * Creates a mapper from a CDC profile. - * - * @param profile the outbox relay profile - */ - public DebeziumOutboxRecordMapper(DebeziumOutboxProfile profile) { - Objects.requireNonNull(profile, "profile must not be null"); - this.topicPrefix = profile.topicPrefix(); - this.aggregateIdAsPartitionKey = profile.aggregateIdAsPartitionKey(); - } - - /** - * Maps one outbox row to its routed record. - * - * @param record the outbox row - * @return the record a CDC relay would emit - */ - public DebeziumMappedRecord map(OutboxRecord record) { - Objects.requireNonNull(record, "record must not be null"); - - // Application headers first, canonical headers second, and the canonical ones win. The row's - // header map is application data; a row that wrote `msg.id` into it would otherwise overwrite - // the identity the platform put there, and the consumer trusts the last such header it sees. - // The write path refuses reserved names outright — this order is the second lock on the same - // door, because a CDC connector reading the table directly never passes through that check. - Map headers = new LinkedHashMap<>(record.headers()); - headers.keySet().removeIf(ReservedHeaders::isReserved); - headers.putAll(router.reservedHeaders(record)); - - return new DebeziumMappedRecord( - topicPrefix + record.destination().value(), - partitionKey(record), - record.payload(), - headers); - } - - private Optional partitionKey(OutboxRecord record) { - if (!aggregateIdAsPartitionKey) { - return Optional.empty(); - } - // The same value the polling relay would use and the same value the `routing_key` column - // computes, so both relays land the same message on the same partition and neither reorders a - // keyed stream. - return Optional.of(router.routingKey(record)); - } -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/debezium/outbox-event-router.properties b/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/debezium/outbox-event-router.properties deleted file mode 100644 index b28a254..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/debezium/outbox-event-router.properties +++ /dev/null @@ -1,43 +0,0 @@ -# Debezium Outbox Event Router configuration for the messaging_outbox table. -# -# Optional. Enable this OR the in-process polling relay, never both: Debezium reads the -# write-ahead log and never observes the polling lease, so with both running every row is -# published twice and the duplicate is invisible in the outbox table itself. -# -# Operating the Debezium connector — its offsets, its snapshots, its slot — is out of scope for -# this module. What is in scope is that a message routed by CDC is indistinguishable from one -# published by the polling relay: same logical id, same destination, same reserved headers. - -name=messaging-outbox-connector -connector.class=io.debezium.connector.postgresql.PostgresConnector - -# Only the outbox table. A wider include list turns every business table change into a message. -table.include.list=public.messaging_outbox - -# The row is deleted or marked after routing; the tombstone would otherwise reach the topic as a -# null-payload record that consumers must special-case. -tombstones.on.delete=false - -transforms=outbox -transforms.outbox.type=io.debezium.transforms.outbox.EventRouter - -# Identity comes from the row's own message_id, not from Debezium's per-change event id. A CDC -# re-snapshot mints new event ids for rows that already published, which would break identity in -# exactly the situation where it matters most. -transforms.outbox.table.field.event.id=message_id -transforms.outbox.table.field.event.key=destination -transforms.outbox.table.field.event.payload=payload -transforms.outbox.table.field.event.timestamp=created_at - -transforms.outbox.route.by.field=destination -transforms.outbox.route.topic.replacement=${routedByValue} - -# The reserved headers a consumer relies on, carried through the transform so the CDC path emits -# the same envelope the polling relay does. -transforms.outbox.table.fields.additional.placement=message_type:header:msg.type,schema_version:header:msg.schema-version,content_type:header:msg.content-type,message_id:header:msg.id - -# Publisher idempotence and full acknowledgement: a CDC relay that loses a record cannot replay it -# without re-reading the log from an earlier offset, which would republish everything after it. -producer.override.enable.idempotence=true -producer.override.acks=all -producer.override.max.in.flight.requests.per.connection=5 diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapperTest.java b/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapperTest.java deleted file mode 100644 index e49e162..0000000 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/DebeziumOutboxRecordMapperTest.java +++ /dev/null @@ -1,199 +0,0 @@ -package dev.caskeleton.messaging.outbox; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.outbox.DebeziumOutboxProfile.RelayMode; -import dev.caskeleton.messaging.reliability.OutboxCanonicalMetadata; -import dev.caskeleton.messaging.reliability.OutboxRecord; -import dev.caskeleton.messaging.reliability.OutboxStatus; -import java.nio.charset.StandardCharsets; -import java.time.Instant; -import java.util.Map; -import java.util.Optional; -import org.junit.jupiter.api.Test; - -class DebeziumOutboxRecordMapperTest { - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - private static final MessageId MESSAGE_ID = MessageId.newId(); - - private static OutboxRecord record(Map headers) { - return record(headers, OutboxCanonicalMetadata.none()); - } - - private static OutboxRecord record( - Map headers, OutboxCanonicalMetadata metadata) { - return new OutboxRecord( - MESSAGE_ID, - new DestinationName("orders.v1"), - new MessageType("order.created"), - new SchemaVersion(1), - ContentType.JSON, - "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), - headers, - metadata, - NOW, - OutboxStatus.PENDING, - 0, - Optional.empty(), - Optional.empty()); - } - - private static DebeziumOutboxRecordMapper mapper(boolean aggregateIdAsPartitionKey) { - return new DebeziumOutboxRecordMapper( - new DebeziumOutboxProfile( - RelayMode.CHANGE_DATA_CAPTURE, "prod.", aggregateIdAsPartitionKey)); - } - - @Test - void theRoutedTopicIsThePrefixedDestination() { - assertThat(mapper(false).map(record(Map.of())).topic()).isEqualTo("prod.orders.v1"); - } - - @Test - void theLogicalMessageIdSurvivesTheCdcPath() { - DebeziumMappedRecord mapped = mapper(false).map(record(Map.of())); - - assertThat(mapped.headers().get(ReservedHeaders.MESSAGE_ID)) - .as("Debezium mints a fresh event id per change record; using it would break identity") - .isEqualTo(MESSAGE_ID.value().toString()); - } - - @Test - void everyReservedHeaderAConsumerNeedsIsCarried() { - Map headers = mapper(false).map(record(Map.of())).headers(); - - assertThat(headers) - .containsKeys( - ReservedHeaders.MESSAGE_ID, - ReservedHeaders.MESSAGE_TYPE, - ReservedHeaders.SCHEMA_VERSION, - ReservedHeaders.CONTENT_TYPE, - ReservedHeaders.PRODUCED_AT); - } - - @Test - void applicationHeadersOnTheRowArePreserved() { - DebeziumMappedRecord mapped = mapper(false).map(record(Map.of("x-tenant-tier", "gold"))); - - assertThat(mapped.headers()).containsEntry("x-tenant-tier", "gold"); - } - - @Test - void anUnkeyedProfileEmitsNoPartitionKey() { - assertThat(mapper(false).map(record(Map.of())).key()).isEmpty(); - } - - @Test - void aKeyedProfileUsesTheSamePartitionKeyThePollingRelayWouldHave() { - DebeziumMappedRecord mapped = - mapper(true) - .map( - record( - Map.of(), - OutboxCanonicalMetadata.none().withRoutingKeys("customer-42", "customer-42"))); - - assertThat(mapped.key()) - .as("a different key would send the two relays' messages to different partitions") - .hasValue("customer-42"); - } - - @Test - void aKeyedProfileFallsBackToTheMessageIdWhenTheRowHasNoPartitionKey() { - assertThat(mapper(true).map(record(Map.of())).key()) - .as("the fallback the routing_key column computes, stated in one place for both relays") - .hasValue(MESSAGE_ID.value().toString()); - } - - @Test - void theCanonicalColumnsBecomeReservedHeaders() { - OutboxCanonicalMetadata metadata = - OutboxCanonicalMetadata.none() - .withProducer(new dev.caskeleton.messaging.api.ProducerId("order-api")) - .withTenant(new dev.caskeleton.messaging.api.TenantContext("acme")) - .withRoutingKeys("customer-42", "customer-42"); - - Map headers = mapper(false).map(record(Map.of(), metadata)).headers(); - - assertThat(headers) - .as("a relay that drops these makes the publish path part of the message's meaning") - .containsEntry(ReservedHeaders.PRODUCER, "order-api") - .containsEntry(ReservedHeaders.TENANT, "acme") - .containsEntry(ReservedHeaders.PARTITION_KEY, "customer-42") - .containsEntry(ReservedHeaders.ORDERING_KEY, "customer-42") - .containsEntry(ReservedHeaders.ORIGIN_DESTINATION, "orders.v1"); - } - - @Test - void aColumnTheRowDoesNotCarryProducesNoHeaderAtAll() { - assertThat(mapper(false).map(record(Map.of())).headers()) - .as("an empty string would be indistinguishable from a tenant literally named ''") - .doesNotContainKeys( - ReservedHeaders.TENANT, ReservedHeaders.CORRELATION_ID, ReservedHeaders.TRACEPARENT); - } - - @Test - void aForgedReservedHeaderOnTheRowCannotOverwriteTheCanonicalOne() { - Map forged = - Map.of(ReservedHeaders.MESSAGE_ID, "00000000-0000-7000-8000-000000000000"); - - assertThat(mapper(false).map(record(forged)).headers()) - .as("a CDC connector reads the table directly and never passes the write-path check") - .containsEntry(ReservedHeaders.MESSAGE_ID, MESSAGE_ID.value().toString()); - } - - @Test - void theRoutedKeyIsNotTheTopicName() { - // Literals, not the class's own constants: comparing a configuration value against the constant - // that produced it asserts that the router agrees with itself, which it always will. - assertThat(new DebeziumOutboxEventRouter().connectorConfiguration("prod.")) - .as("keying by destination puts every message on a topic onto one partition") - .containsEntry("transforms.outbox.table.field.event.key", "routing_key") - .containsEntry("transforms.outbox.route.by.field", "destination"); - } - - @Test - void thePayloadIsUnchangedByTheTransform() { - assertThat(mapper(false).map(record(Map.of())).payload()) - .isEqualTo("{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8)); - } - - @Test - void runningBothRelaysAtOnceIsRefused() { - DebeziumOutboxProfile cdc = - new DebeziumOutboxProfile(RelayMode.CHANGE_DATA_CAPTURE, "p.", false); - - assertThatThrownBy(() -> cdc.requireExactlyOneRelay(true)) - .as("Debezium never sees the polling lease, so every row would publish twice") - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("twice"); - } - - @Test - void runningNoRelayAtAllIsAlsoRefused() { - assertThatThrownBy(() -> DebeziumOutboxProfile.polling().requireExactlyOneRelay(false)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("accumulate"); - } - - @Test - void exactlyOneRelayIsAccepted() { - assertThatCode(() -> DebeziumOutboxProfile.polling().requireExactlyOneRelay(true)) - .doesNotThrowAnyException(); - } - - @Test - void aCdcProfileMustNameATopicPrefix() { - assertThatThrownBy(() -> new DebeziumOutboxProfile(RelayMode.CHANGE_DATA_CAPTURE, " ", false)) - .isInstanceOf(IllegalArgumentException.class); - } -} diff --git a/messaging/messaging-policy/build.gradle b/messaging/messaging-policy/build.gradle index f2ca78e..fac7b20 100644 --- a/messaging/messaging-policy/build.gradle +++ b/messaging/messaging-policy/build.gradle @@ -4,5 +4,4 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') } diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/ApiTier.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/ApiTier.java new file mode 100644 index 0000000..c48fb24 --- /dev/null +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/ApiTier.java @@ -0,0 +1,30 @@ +package dev.caskeleton.messaging.policy; + +/** + * The API tier a destination is approved for. + * + *

"Tier" names this one thing in the platform: which API surface a destination may use. How far + * an adapter or capability has been proven is a "grade" ({@code CompatibilityMatrix.SupportGrade}), + * the same word the gRPC family uses for its capability and evidence grades. It was {@code + * CapabilityTier}, which put a third meaning on "capability" beside {@code MessagingCapabilities} + * and the gRPC capability grade (S7-#14). + * + *

Recorded on the profile so that the tier boundary is enforced by configuration review rather + * than by whichever handler happens to be wired in. An M1 destination that silently accepted a + * manual settlement handler would move the acknowledge-after-success ordering out of the platform + * and into application code. + */ +public enum ApiTier { + + /** Typed publish and handler with platform-owned settlement. */ + M1, + + /** + * Surfaces beyond M1. None ships: the batch, manual-settlement and pause/resume contracts had no + * implementation and were removed (M-9). + */ + M2, + + /** Broker-native capabilities exposed as typed interfaces. */ + M3 +} diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/CapabilityTier.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/CapabilityTier.java deleted file mode 100644 index 3760b8a..0000000 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/CapabilityTier.java +++ /dev/null @@ -1,21 +0,0 @@ -package dev.caskeleton.messaging.policy; - -/** - * The API tier a destination is approved for. - * - *

Recorded on the profile so that the tier boundary is enforced by configuration review rather - * than by whichever handler happens to be wired in. An M1 destination that silently accepted a - * manual settlement handler would move the acknowledge-after-success ordering out of the platform - * and into application code. - */ -public enum CapabilityTier { - - /** Typed publish and handler with platform-owned settlement. */ - M1, - - /** Batch, manual settlement, pause and resume, delayed delivery, replay request. */ - M2, - - /** Broker-native capabilities exposed as typed interfaces. */ - M3 -} diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DefaultRetryDecisionEngine.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DefaultRetryDecisionEngine.java index 943d498..5708bd5 100644 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DefaultRetryDecisionEngine.java +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DefaultRetryDecisionEngine.java @@ -46,7 +46,11 @@ public final class DefaultRetryDecisionEngine implements RetryDecisionEngine { if (policy.orderingImpact() == OrderingImpact.PRESERVE && profile.isOrdered() - && context.capabilities().orderedStream()) { + && context + .capabilities() + .supports( + dev.caskeleton.messaging.api.destination.MessagingCapabilities.Capability + .ORDERED_STREAM)) { return new RetryDecision.PauseAndRetry(delay); } if (policy.mode() == RetryMode.PAUSE_PARTITION) { @@ -61,7 +65,12 @@ public final class DefaultRetryDecisionEngine implements RetryDecisionEngine { if (policy.mode() == RetryMode.INLINE || policy.mode() == RetryMode.BLOCKING) { return new RetryDecision.RetryInline(delay); } - if (policy.mode() == RetryMode.BROKER_DELAYED && context.capabilities().delayedDelivery()) { + if (policy.mode() == RetryMode.BROKER_DELAYED + && context + .capabilities() + .supports( + dev.caskeleton.messaging.api.destination.MessagingCapabilities.Capability + .DELAYED_DELIVERY)) { return new RetryDecision.PublishToRetryDestination( policy.retryDestination().orElse(profile.name()), delay); } diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfile.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfile.java index 69b3938..11da9cb 100644 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfile.java +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfile.java @@ -47,7 +47,7 @@ public record DestinationProfile( RetryPolicy retry, DeadLetterPolicy deadLetter, PayloadPolicy payload, - CapabilityTier tier, + ApiTier tier, boolean production, boolean keyResolverConfigured, boolean topologyAutoCreate) { diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfileValidator.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfileValidator.java index 4757611..b7bebba 100644 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfileValidator.java +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/DestinationProfileValidator.java @@ -62,7 +62,7 @@ public final class DestinationProfileValidator { throw new IllegalArgumentException( "ordering scope KEY requires a key resolver for " + profile.name().value()); } - if (profile.tier() == CapabilityTier.M1 && profile.consumer().manualSettlement()) { + if (profile.tier() == ApiTier.M1 && profile.consumer().manualSettlement()) { throw new IllegalArgumentException( "an M1 destination cannot configure manual settlement: " + profile.name().value()); } diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/InFlightLimiter.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/InFlightLimiter.java index 30ef288..e15b7e6 100644 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/InFlightLimiter.java +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/InFlightLimiter.java @@ -50,7 +50,20 @@ public final class InFlightLimiter { * @throws InterruptedException when the waiting thread is interrupted */ public boolean tryAcquire() throws InterruptedException { - return permits.tryAcquire(maxWait.toNanos(), TimeUnit.NANOSECONDS); + return tryAcquire(maxWait); + } + + /** + * Tries to acquire one permit, waiting no longer than the configured bound or the given budget. + * + * @param budget what the caller has left to spend on the wait + * @return true when a permit was acquired + * @throws InterruptedException when the waiting thread is interrupted + */ + public boolean tryAcquire(Duration budget) throws InterruptedException { + Objects.requireNonNull(budget, "budget must not be null"); + long wait = Math.max(0, Math.min(maxWait.toNanos(), budget.toNanos())); + return permits.tryAcquire(wait, TimeUnit.NANOSECONDS); } /** Returns one permit to the pool. */ diff --git a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/MessagingAdmissionController.java b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/MessagingAdmissionController.java index 4b2cc98..118f998 100644 --- a/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/MessagingAdmissionController.java +++ b/messaging/messaging-policy/src/main/java/dev/caskeleton/messaging/policy/MessagingAdmissionController.java @@ -73,7 +73,24 @@ public final class MessagingAdmissionController { * @throws MessageBackpressureException when shutting down or no permit became available */ public void admit(String destination, int payloadBytes) { + admit(destination, payloadBytes, java.time.Duration.ofNanos(Long.MAX_VALUE)); + } + + /** + * Admits one publish, waiting for a permit no longer than the caller has left. + * + *

The limiter's own bound was the only one: a publish with a 100ms deadline could block its + * caller for the limiter's full wait before the deadline was even consulted. + * + * @param destination the logical destination name + * @param payloadBytes the encoded payload size + * @param budget the most this call may wait for a permit + * @throws MessageTooLargeException when the payload exceeds the destination limit + * @throws MessageBackpressureException when shutting down or no permit became available + */ + public void admit(String destination, int payloadBytes, java.time.Duration budget) { Objects.requireNonNull(destination, "destination must not be null"); + Objects.requireNonNull(budget, "budget must not be null"); payloadGuard.checkPayload(destination, payloadBytes); if (!acceptingNewWork) { @@ -91,7 +108,7 @@ public final class MessagingAdmissionController { boolean acquired; try { - acquired = limiter.tryAcquire(); + acquired = limiter.tryAcquire(budget); } catch (InterruptedException interrupted) { // The destination slot was taken a moment ago and no publish will use it, so it goes back // here: a slot leaked per interruption shrinks the destination's ceiling until it is zero. diff --git a/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/DestinationProfileValidatorTest.java b/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/DestinationProfileValidatorTest.java index f2c16a9..7beb6be 100644 --- a/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/DestinationProfileValidatorTest.java +++ b/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/DestinationProfileValidatorTest.java @@ -188,7 +188,7 @@ final class DestinationProfileFixtures { RetryPolicy.none(), DeadLetterPolicy.to(new DestinationName("order-events-dlq")), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, false, false, false); @@ -298,7 +298,7 @@ final class DestinationProfileFixtures { base.retry(), base.deadLetter(), base.payload(), - CapabilityTier.M1, + ApiTier.M1, base.production(), base.keyResolverConfigured(), base.topologyAutoCreate()); diff --git a/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/RetryDecisionEngineTest.java b/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/RetryDecisionEngineTest.java index 53fc29c..6a8dc24 100644 --- a/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/RetryDecisionEngineTest.java +++ b/messaging/messaging-policy/src/test/java/dev/caskeleton/messaging/policy/RetryDecisionEngineTest.java @@ -131,12 +131,11 @@ final class RetryContextFixtures { private RetryContextFixtures() {} private static final MessagingCapabilities ORDERED_KAFKA = - new MessagingCapabilities( - true, true, true, true, true, true, true, false, true, true, false, true); + MessagingCapabilities.of( + MessagingCapabilities.Capability.ORDERED_STREAM, + MessagingCapabilities.Capability.DEDUPLICATED_PUBLISH); - private static final MessagingCapabilities PLAIN_QUEUE = - new MessagingCapabilities( - true, true, true, false, false, false, false, false, false, false, true, true); + private static final MessagingCapabilities PLAIN_QUEUE = MessagingCapabilities.none(); static DeliveryMetadata delivery(int attempt) { return new DeliveryMetadata( diff --git a/messaging/messaging-pulsar-experimental/build.gradle b/messaging/messaging-pulsar-experimental/build.gradle deleted file mode 100644 index 929bf94..0000000 --- a/messaging/messaging-pulsar-experimental/build.gradle +++ /dev/null @@ -1,14 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-transport-spi') - api project(':messaging:messaging-observability') - api project(':messaging:messaging-security') - api project(':messaging:messaging-admin-api') - -} diff --git a/messaging/messaging-pulsar-experimental/gradle.lockfile b/messaging/messaging-pulsar-experimental/gradle.lockfile deleted file mode 100644 index f5ad1bb..0000000 --- a/messaging/messaging-pulsar-experimental/gradle.lockfile +++ /dev/null @@ -1,89 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=runtimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=runtimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty= diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagePosition.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagePosition.java deleted file mode 100644 index 7271ac0..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagePosition.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import dev.caskeleton.messaging.api.publish.BrokerPosition; -import java.util.Map; - -/** - * A Pulsar message id, decomposed into its parts. - * - *

Kept as four fields rather than the opaque {@code MessageId} string so a diagnostic can be - * read without a Pulsar client to parse it. {@code batchIndex} is the part that surprises people: a - * batched message's id is shared by every message in the batch, so the index is what makes an - * individual message addressable at all. - * - * @param ledgerId the ledger holding the entry - * @param entryId the entry within the ledger - * @param partition the partition index, or -1 for a non-partitioned topic - * @param batchIndex the index within a batch, or -1 when the message was not batched - */ -public record PulsarMessagePosition(long ledgerId, long entryId, int partition, int batchIndex) - implements BrokerPosition { - - /** The value used where a topic is not partitioned or a message is not batched. */ - public static final int NOT_APPLICABLE = -1; - - @Override - public String broker() { - return "pulsar"; - } - - @Override - public Map diagnosticAttributes() { - return Map.of( - "ledgerId", Long.toString(ledgerId), - "entryId", Long.toString(entryId), - "partition", Integer.toString(partition), - "batchIndex", Integer.toString(batchIndex)); - } - - /** - * Returns the canonical Pulsar message id rendering. - * - * @return the message id as Pulsar prints it - */ - public String toMessageId() { - return batchIndex == NOT_APPLICABLE - ? "%d:%d:%d".formatted(ledgerId, entryId, partition) - : "%d:%d:%d:%d".formatted(ledgerId, entryId, partition, batchIndex); - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagingTransport.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagingTransport.java deleted file mode 100644 index 1884d57..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarMessagingTransport.java +++ /dev/null @@ -1,275 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import dev.caskeleton.messaging.api.destination.DestinationCapabilities; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import dev.caskeleton.messaging.transport.MessagingTransport; -import dev.caskeleton.messaging.transport.TransportConsumerRegistration; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import dev.caskeleton.messaging.transport.TransportPublishResult; -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicBoolean; -import java.util.function.Function; - -/** - * The Experimental Pulsar adapter. - * - *

This is an Experimental contract seam, not a Stable adapter. It exercises the transport SPI - * against a send operation the application supplies; it does not ship a Pulsar client bridge, - * producer lifecycle, or reconnection. Only a failure the operation reports as a {@link - * PulsarPreSendRejection} is treated as definitely not transmitted — every other failure, - * recognised or not, is ambiguous. - * - *

Disabled unless explicitly enabled. Experimental means the contract suite has been run but the - * live-broker certification has not, so the adapter must never become load-bearing because a - * default switched it on. - * - *

Pulsar's send future resolves with a {@code MessageId} once the broker has persisted the - * message to its configured number of bookies, which is genuine replication evidence rather than a - * receipt. A send that fails with a timeout is reported {@code AMBIGUOUS}: the write may have - * reached the bookies and only the acknowledgement been lost, and treating that as a rejection is - * how a message gets published twice. - */ -public final class PulsarMessagingTransport implements MessagingTransport { - - /** What this adapter can prove today. Transactions stay false while Experimental. */ - private static final MessagingCapabilities SHARED_CAPABILITIES = - new MessagingCapabilities( - true, true, true, true, false, false, true, true, false, false, true, true); - - private static final MessagingCapabilities KEY_SHARED_CAPABILITIES = - new MessagingCapabilities( - true, true, true, true, false, true, true, true, false, false, true, true); - - private final String brokerName; - private final long generation; - private final PulsarProfile profile; - private final PulsarSendOperation send; - private final Function consumerFactory; - private final AtomicBoolean closed = new AtomicBoolean(); - - /** - * Creates a publish-only transport. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param profile the Pulsar destination settings - * @param send the producer send operation - */ - public PulsarMessagingTransport( - String brokerName, long generation, PulsarProfile profile, PulsarSendOperation send) { - this( - brokerName, - generation, - profile, - send, - spec -> { - throw new MessagingCapabilityUnavailableException( - "PULSAR_CONSUMER_NOT_CONFIGURED", - "this Pulsar transport was created without a consumer factory"); - }); - } - - /** - * Creates a transport with a consumer factory. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param profile the Pulsar destination settings - * @param send the producer send operation - * @param consumerFactory builds a consumer registration for a spec - */ - public PulsarMessagingTransport( - String brokerName, - long generation, - PulsarProfile profile, - PulsarSendOperation send, - Function consumerFactory) { - this.brokerName = Objects.requireNonNull(brokerName, "brokerName must not be null"); - this.generation = generation; - this.profile = Objects.requireNonNull(profile, "profile must not be null"); - this.send = Objects.requireNonNull(send, "send must not be null"); - this.consumerFactory = Objects.requireNonNull(consumerFactory, "consumerFactory is required"); - } - - @Override - public CompletionStage publish(TransportPublishRequest request) { - Objects.requireNonNull(request, "request must not be null"); - - if (closed.get()) { - return completed( - rejectedLocally("PULSAR_TRANSPORT_CLOSED", "the transport is shutting down")); - } - int size = request.envelope().payload().size(); - int limit = request.profile().payload().maxBytes(); - if (size > limit) { - return completed( - rejectedLocally( - "PAYLOAD_TOO_LARGE", "encoded payload is " + size + " bytes, limit is " + limit)); - } - - // The caller's own deadline, honoured rather than ignored: a stalled send used to hold the - // caller for whatever the SDK's default was, whatever the caller had asked for. - long startedAt = System.nanoTime(); - return send.send(profile.topic(), request) - .toCompletableFuture() - .orTimeout( - request.options().timeout().toMillis(), java.util.concurrent.TimeUnit.MILLISECONDS) - .handle( - (position, failure) -> - failure == null - ? confirmed(position, elapsedSince(startedAt)) - : classify(failure, elapsedSince(startedAt))) - .thenApply(TransportPublishResult::new); - } - - @Override - public TransportConsumerRegistration register(TransportConsumerSpec spec) { - Objects.requireNonNull(spec, "spec must not be null"); - if (closed.get()) { - throw new MessagingCapabilityUnavailableException( - "PULSAR_TRANSPORT_CLOSED", "the transport is shutting down"); - } - return consumerFactory.apply(spec); - } - - @Override - public DestinationCapabilities capabilities(DestinationName destination) { - MessagingCapabilities capabilities = - profile.mode().subscriptionType() == PulsarSubscriptionType.KEY_SHARED - ? KEY_SHARED_CAPABILITIES - : SHARED_CAPABILITIES; - return new DestinationCapabilities(destination, brokerName, capabilities); - } - - @Override - public String brokerName() { - return brokerName; - } - - @Override - public long generation() { - return generation; - } - - @Override - public void close() { - closed.set(true); - } - - /** - * Reports whether the transport is still accepting work. - * - * @return true until close - */ - public boolean isAcceptingWork() { - return !closed.get(); - } - - private static Duration elapsedSince(long startedAtNanos) { - return Duration.ofNanos(System.nanoTime() - startedAtNanos); - } - - private static PublishResult confirmed(PulsarMessagePosition position, Duration elapsed) { - return new PublishResult( - PublishCompletion.CONFIRMED, - // The send future resolves only after the entry is persisted to the configured bookies, so - // this is replication evidence rather than a broker receipt. - new PublishEvidence( - true, - TransmissionEvidence.TRANSMITTED, - true, - ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK), - RoutingOutcome.ROUTED, - Optional.of(position), - 1, - elapsed, - Optional.empty()); - } - - /** - * Decides what a send failure means, from typed signals only. - * - *

Classification used to read the exception's class simple name: {@code "Timeout"} meant - * ambiguous, anything else meant rejected. A class name is not part of Pulsar's contract — it - * changes between client versions — and defaulting the unknown case to {@code REJECTED} tells the - * caller nothing was transmitted, which is how the same entry is published to the bookies twice. - */ - private static PublishResult classify(Throwable failure, Duration elapsed) { - Throwable cause = - failure instanceof java.util.concurrent.CompletionException ? failure.getCause() : failure; - if (cause instanceof PulsarPreSendRejection rejection) { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.CONFIGURATION, rejection.code(), rejection.getMessage()))); - } - boolean timedOut = cause instanceof java.util.concurrent.TimeoutException; - return new PublishResult( - PublishCompletion.AMBIGUOUS, - new PublishEvidence( - true, TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.TRANSIENT_INFRASTRUCTURE, - timedOut ? "PULSAR_SEND_TIMEOUT" : "PULSAR_SEND_OUTCOME_UNKNOWN", - timedOut - ? "the send timed out; the broker may already hold the message" - : "the send failed in a way this adapter does not classify; the broker may" - + " already hold the message"))); - } - - private static TransportPublishResult rejectedLocally(String code, String message) { - return new TransportPublishResult( - new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of(FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message)))); - } - - private static CompletionStage completed(TransportPublishResult result) { - return CompletableFuture.completedFuture(result); - } - - /** The producer send, isolated so the adapter is testable without a broker. */ - @FunctionalInterface - public interface PulsarSendOperation { - - /** - * Sends one encoded message. - * - * @param topic the Pulsar topic - * @param request the publish request - * @return a stage completing with the broker position - */ - CompletionStage send(String topic, TransportPublishRequest request); - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarPreSendRejection.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarPreSendRejection.java deleted file mode 100644 index 6cf5f62..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarPreSendRejection.java +++ /dev/null @@ -1,65 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import java.util.Objects; - -/** - * The one signal that lets this adapter report {@code REJECTED / NOT_TRANSMITTED}. - * - *

Classification used to work by reading the exception's class simple name and matching - * substrings — {@code "Timeout"} meant ambiguous, anything else meant rejected. Two things are - * wrong with that. A class name is not part of any SDK's contract: it changes between versions, and - * an unrelated exception whose name happens to contain the substring is misread. And defaulting the - * unknown case to {@code REJECTED} tells the caller nothing was transmitted, so a caller that - * republishes under a new identity duplicates a message the server may well have stored. - * - *

The send operation throws this only when it knows the send never left the client: an argument - * the SDK refused, a producer that was already closed, a topic the client could not resolve. - * Everything else — including anything the adapter does not recognise — is ambiguous, and ambiguity - * is the honest answer when the write may have landed. - */ -public final class PulsarPreSendRejection extends RuntimeException { - - private static final long serialVersionUID = 1L; - - private final String code; - - /** - * Creates a pre-send rejection. - * - * @param code the sanitised platform code for the refusal - * @param message a sanitised description - */ - public PulsarPreSendRejection(String code, String message) { - super(message); - this.code = requireCode(code); - } - - /** - * Creates a pre-send rejection wrapping the SDK's own exception. - * - * @param code the sanitised platform code for the refusal - * @param message a sanitised description - * @param cause the SDK exception that proves the send never started - */ - public PulsarPreSendRejection(String code, String message, Throwable cause) { - super(message, cause); - this.code = requireCode(code); - } - - private static String requireCode(String code) { - Objects.requireNonNull(code, "code must not be null"); - if (code.isBlank()) { - throw new IllegalArgumentException("code must not be blank"); - } - return code; - } - - /** - * The sanitised code reported to the caller. - * - * @return the code - */ - public String code() { - return code; - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfile.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfile.java deleted file mode 100644 index 154cce1..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfile.java +++ /dev/null @@ -1,80 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; - -/** - * The Pulsar-side settings for one destination. - * - *

{@code negativeAckRedeliveryDelay} is the platform's retry hook on Pulsar. Pulsar has no retry - * topic of its own in this adapter, so a failed handler negatively acknowledges and the broker - * redelivers after this delay — which means the delay is the backoff, and setting it to - * zero converts a failing handler into a spin loop against the broker. - * - *

{@code ackTimeout} is deliberately optional and off by default. It redelivers any message the - * consumer has not acknowledged in time, which looks like a safety net and behaves like a duplicate - * generator for handlers that are simply slow. - * - * @param topic the fully-qualified Pulsar topic - * @param subscription the durable subscription name - * @param mode the subscription type and acknowledgement style - * @param negativeAckRedeliveryDelay how long the broker waits after a negative acknowledgement - * @param ackTimeout redelivers unacknowledged messages after this long, when set - * @param maxRedeliverCount how many redeliveries before the message is parked - */ -public record PulsarProfile( - String topic, - String subscription, - PulsarSubscriptionMode mode, - Duration negativeAckRedeliveryDelay, - Optional ackTimeout, - int maxRedeliverCount) { - - /** Pulsar's own floor for an acknowledgement timeout. */ - public static final Duration MINIMUM_ACK_TIMEOUT = Duration.ofSeconds(10); - - public PulsarProfile { - Objects.requireNonNull(mode, "mode must not be null"); - Objects.requireNonNull(negativeAckRedeliveryDelay, "negativeAckRedeliveryDelay is required"); - Objects.requireNonNull(ackTimeout, "ackTimeout must not be null"); - requireText(topic, "topic"); - requireText(subscription, "subscription"); - - if (negativeAckRedeliveryDelay.isNegative() || negativeAckRedeliveryDelay.isZero()) { - throw new IllegalArgumentException( - "negativeAckRedeliveryDelay must be positive; zero turns a failing handler into a spin " - + "loop against the broker"); - } - if (maxRedeliverCount < 1) { - throw new IllegalArgumentException("maxRedeliverCount must be at least 1"); - } - if (ackTimeout.isPresent() && ackTimeout.get().compareTo(MINIMUM_ACK_TIMEOUT) < 0) { - throw new IllegalArgumentException( - "ackTimeout below %s is rejected by Pulsar itself".formatted(MINIMUM_ACK_TIMEOUT)); - } - } - - /** - * Returns a shared-subscription profile with the platform defaults. - * - * @param topic the Pulsar topic - * @param subscription the subscription name - * @return the profile - */ - public static PulsarProfile shared(String topic, String subscription) { - return new PulsarProfile( - topic, - subscription, - PulsarSubscriptionMode.shared(), - Duration.ofSeconds(30), - Optional.empty(), - 5); - } - - private static void requireText(String value, String field) { - if (value == null || value.isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfileValidator.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfileValidator.java deleted file mode 100644 index b0e0db3..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarProfileValidator.java +++ /dev/null @@ -1,66 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.util.Objects; - -/** - * Guards the Pulsar Experimental adapter. - * - *

The subscription type and the destination's ordering scope have to agree, and the platform - * checks rather than assumes. A {@code Shared} subscription distributes messages round-robin across - * consumers, so a destination that declares key ordering on one is advertising a guarantee Pulsar - * is not providing — the messages arrive, in an order nobody promised. - * - *

Experimental means off by default. An adapter whose contract suite is still being proven must - * not become load-bearing because someone's configuration defaulted it on. - */ -public final class PulsarProfileValidator { - - /** - * Validates a destination against a Pulsar subscription. - * - * @param profile the destination profile - * @param subscriptionType the Pulsar subscription type - * @param enabled whether the experimental adapter is switched on - */ - public void validate( - DestinationProfile profile, PulsarSubscriptionType subscriptionType, boolean enabled) { - Objects.requireNonNull(profile, "profile must not be null"); - Objects.requireNonNull(subscriptionType, "subscriptionType must not be null"); - - if (!enabled) { - throw new MessagingCapabilityUnavailableException( - "PULSAR_DISABLED", - "the Pulsar adapter is experimental and disabled unless " - + "app.messaging.experimental.pulsar=true"); - } - if (profile.orderingScope() == OrderingScope.KEY - && subscriptionType != PulsarSubscriptionType.KEY_SHARED) { - throw new IllegalArgumentException( - "keyed ordering requires a Key_Shared subscription: " + profile.name().value()); - } - if (profile.orderingScope() == OrderingScope.DESTINATION) { - throw new IllegalArgumentException( - "the Pulsar adapter does not offer destination-wide ordering: " + profile.name().value()); - } - } - - /** - * Returns what the Pulsar adapter can prove for a subscription type. - * - *

Transactions are reported as unavailable while the adapter is Experimental: Pulsar has them, - * but the platform has not yet proven them against its own contract suite, and advertising an - * unverified guarantee is exactly what the Experimental tier exists to prevent. - * - * @param subscriptionType the subscription type - * @return the capabilities - */ - public MessagingCapabilities capabilities(PulsarSubscriptionType subscriptionType) { - boolean keyed = subscriptionType == PulsarSubscriptionType.KEY_SHARED; - return new MessagingCapabilities( - true, true, true, true, keyed, keyed, true, true, false, false, true, true); - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionMode.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionMode.java deleted file mode 100644 index 4d2c53e..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionMode.java +++ /dev/null @@ -1,62 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import java.util.Objects; - -/** - * How a Pulsar subscription is configured, and what ordering it can actually deliver. - * - *

Pairs the subscription type with its acknowledgement style because the two together determine - * the guarantee. A {@code Key_Shared} subscription with cumulative acknowledgement is not keyed - * ordering with a faster ack — cumulative ack over interleaved keys acknowledges messages from keys - * the consumer has not finished, so the combination silently loses the property the subscription - * type was chosen for. - * - * @param subscriptionType the Pulsar subscription type - * @param cumulativeAcknowledgement whether acknowledgements are cumulative rather than individual - */ -public record PulsarSubscriptionMode( - PulsarSubscriptionType subscriptionType, boolean cumulativeAcknowledgement) { - - public PulsarSubscriptionMode { - Objects.requireNonNull(subscriptionType, "subscriptionType must not be null"); - if (cumulativeAcknowledgement && subscriptionType == PulsarSubscriptionType.KEY_SHARED) { - throw new IllegalArgumentException( - "Key_Shared with cumulative acknowledgement acknowledges keys the consumer has not " - + "finished, which discards the per-key ordering the subscription type provides"); - } - if (cumulativeAcknowledgement && subscriptionType == PulsarSubscriptionType.SHARED) { - throw new IllegalArgumentException( - "Shared subscriptions do not support cumulative acknowledgement"); - } - } - - /** - * Returns a shared subscription with individual acknowledgement. - * - * @return the shared mode - */ - public static PulsarSubscriptionMode shared() { - return new PulsarSubscriptionMode(PulsarSubscriptionType.SHARED, false); - } - - /** - * Returns a key-shared subscription with individual acknowledgement. - * - * @return the key-shared mode - */ - public static PulsarSubscriptionMode keyShared() { - return new PulsarSubscriptionMode(PulsarSubscriptionType.KEY_SHARED, false); - } - - /** - * Returns the strongest ordering this mode can deliver. - * - * @return the ordering scope - */ - public OrderingScope orderingScope() { - return subscriptionType == PulsarSubscriptionType.KEY_SHARED - ? OrderingScope.KEY - : OrderingScope.NONE; - } -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionType.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionType.java deleted file mode 100644 index 4a88a32..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionType.java +++ /dev/null @@ -1,17 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -/** - * The Pulsar subscription types the Experimental adapter exposes. - * - *

Only the two that map cleanly onto the platform's ordering model. {@code Exclusive} and {@code - * Failover} are omitted deliberately: they encode a topology decision the destination profile - * already owns, and exposing them would give an application two places to configure the same thing. - */ -public enum PulsarSubscriptionType { - - /** Competing consumers, no ordering. */ - SHARED, - - /** Competing consumers with per-key ordering. */ - KEY_SHARED -} diff --git a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarTransactionCapability.java b/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarTransactionCapability.java deleted file mode 100644 index 104caf4..0000000 --- a/messaging/messaging-pulsar-experimental/src/main/java/dev/caskeleton/messaging/pulsar/PulsarTransactionCapability.java +++ /dev/null @@ -1,49 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; - -/** - * States, in code, that Pulsar transactions are not promoted. - * - *

Pulsar has transactions. The platform does not offer them, and the distinction matters enough - * to be a class rather than a comment: an operator reading the capability matrix needs to know the - * answer is "not proven here", not "the broker cannot do it". - * - *

What is missing is the proof. The platform's contract suite has not been run against Pulsar - * transactions across broker restart, client fencing, and partial-commit recovery, and the - * Experimental tier exists precisely so that an unproven guarantee is never advertised as one. A - * capability that reports {@code true} on the strength of the broker's documentation is the failure - * mode this class prevents. - */ -public final class PulsarTransactionCapability { - - private PulsarTransactionCapability() {} - - /** Whether the adapter advertises broker transactions. */ - public static final boolean PROMOTED = false; - - /** - * Reports whether transactions may be used. - * - * @return false while the adapter is Experimental - */ - public static boolean isAvailable() { - return PROMOTED; - } - - /** - * Returns the refusal for a transactional request, for the caller to throw. - * - *

Returned rather than thrown so the refusal reads as a value at the call site: {@code throw - * PulsarTransactionCapability.rejection();} makes it obvious that control leaves there, which a - * method that throws on the caller's behalf hides. - * - * @return the exception explaining why transactions are unavailable - */ - public static MessagingCapabilityUnavailableException rejection() { - return new MessagingCapabilityUnavailableException( - "PULSAR_TRANSACTION_NOT_PROMOTED", - "Pulsar supports transactions but this adapter has not certified them against the " - + "platform contract suite, so the capability is not advertised"); - } -} diff --git a/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarAdapterContractTest.java b/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarAdapterContractTest.java deleted file mode 100644 index 0ec8223..0000000 --- a/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarAdapterContractTest.java +++ /dev/null @@ -1,289 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; -import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import dev.caskeleton.messaging.policy.CapabilityTier; -import dev.caskeleton.messaging.policy.ConsumerPolicy; -import dev.caskeleton.messaging.policy.DeadLetterPolicy; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.PayloadPolicy; -import dev.caskeleton.messaging.policy.PhysicalDestination; -import dev.caskeleton.messaging.policy.ProducerPolicy; -import dev.caskeleton.messaging.policy.RetryPolicy; -import dev.caskeleton.messaging.policy.SchemaPolicy; -import dev.caskeleton.messaging.schema.EncodedMessage; -import dev.caskeleton.messaging.schema.SchemaCompatibility; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import java.time.Instant; -import java.util.Optional; -import java.util.Set; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.TimeoutException; -import org.junit.jupiter.api.Test; - -class PulsarAdapterContractTest { - - private static final String TOPIC = "persistent://public/default/orders"; - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - private static final PulsarMessagePosition POSITION = new PulsarMessagePosition(5, 9, 0, -1); - - private static PulsarMessagingTransport transport( - PulsarMessagingTransport.PulsarSendOperation send, PulsarSubscriptionMode mode) { - PulsarProfile profile = - new PulsarProfile( - TOPIC, "orders-sub", mode, java.time.Duration.ofSeconds(30), Optional.empty(), 5); - return new PulsarMessagingTransport("pulsar-primary", 1, profile, send); - } - - private static PulsarMessagingTransport confirming() { - return transport( - (topic, request) -> CompletableFuture.completedFuture(POSITION), - PulsarSubscriptionMode.shared()); - } - - private static PulsarMessagingTransport failingWith(Throwable failure) { - return transport( - (topic, request) -> CompletableFuture.failedFuture(failure), - PulsarSubscriptionMode.shared()); - } - - private static PublishResult await(CompletionStage stage) { - Object value = stage.toCompletableFuture().join(); - return ((dev.caskeleton.messaging.transport.TransportPublishResult) value).result(); - } - - @Test - void aSuccessfulSendReportsReplicationEvidenceNotAMereReceipt() { - PublishResult result = await(confirming().publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.evidence().confirmationLevel()) - .as("the send future resolves only once the entry is persisted to the bookies") - .isEqualTo(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.ROUTED); - } - - @Test - void theBrokerPositionIsCarriedBackForDiagnostics() { - assertThat(await(confirming().publish(request(64))).position()).hasValue(POSITION); - } - - @Test - void aSendTimeoutIsAmbiguousRatherThanRejected() { - PublishResult result = - await(failingWith(new TimeoutException("send timed out")).publish(request(64))); - - assertThat(result.completion()) - .as("the entry may be on the bookies with only the acknowledgement lost") - .isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.evidence().transmission()) - .isEqualTo(TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED); - assertThat(result.evidence().brokerAccepted()).isFalse(); - } - - @Test - void onlyATypedPreSendRejectionIsReportedAsNotTransmitted() { - PublishResult result = - await( - failingWith( - new PulsarPreSendRejection( - "PULSAR_PRODUCER_FENCED", "the producer was fenced before the send")) - .publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.NOT_TRANSMITTED); - assertThat(result.failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("PULSAR_PRODUCER_FENCED")); - } - - @Test - void anUnrecognisedFailureIsAmbiguousRatherThanRejected() { - // Any exception the adapter cannot type is a send whose outcome it does not know. Reporting - // REJECTED invites the caller to republish an entry the bookies may already hold. - PublishResult result = - await(failingWith(new IllegalStateException("producer is fenced")).publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.evidence().transmission()) - .isEqualTo(TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED); - assertThat(result.failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("PULSAR_SEND_OUTCOME_UNKNOWN")); - } - - @Test - void aWrappedUnknownFailureIsAlsoAmbiguous() { - PublishResult result = - await( - failingWith( - new java.util.concurrent.CompletionException( - new RuntimeException("something the adapter has never seen"))) - .publish(request(64))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - } - - @Test - void aSendThatNeverCompletesIsBoundedByTheCallersTimeout() { - PulsarMessagingTransport stalled = - transport((topic, request) -> new CompletableFuture<>(), PulsarSubscriptionMode.shared()); - - PublishResult result = - await(stalled.publish(requestWithTimeout(java.time.Duration.ofMillis(50)))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.failure()) - .hasValueSatisfying(failure -> assertThat(failure.code()).isEqualTo("PULSAR_SEND_TIMEOUT")); - } - - @Test - void anOversizedPayloadIsRejectedBeforeItReachesTheBroker() { - PublishResult result = - await(confirming().publish(request(PayloadPolicy.DEFAULT_MAX_BYTES + 1))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.NOT_TRANSMITTED); - assertThat(result.failure()) - .hasValueSatisfying(failure -> assertThat(failure.code()).isEqualTo("PAYLOAD_TOO_LARGE")); - } - - @Test - void aClosedTransportStopsAcceptingNewWork() { - PulsarMessagingTransport transport = confirming(); - transport.close(); - - assertThat(transport.isAcceptingWork()).isFalse(); - assertThat(await(transport.publish(request(64))).completion()) - .isEqualTo(PublishCompletion.REJECTED); - } - - @Test - void aTransportWithoutAConsumerFactoryRefusesToRegisterRatherThanReturningNothing() { - assertThatThrownBy(() -> confirming().register(null)).isInstanceOf(NullPointerException.class); - } - - @Test - void aSharedSubscriptionAdvertisesNoKeyedOrdering() { - var capabilities = - transport( - (topic, request) -> CompletableFuture.completedFuture(POSITION), - PulsarSubscriptionMode.shared()) - .capabilities(new DestinationName("orders.v1")) - .capabilities(); - - assertThat(capabilities.keyedOrdering()).isFalse(); - } - - @Test - void aKeySharedSubscriptionAdvertisesKeyedOrdering() { - var capabilities = - transport( - (topic, request) -> CompletableFuture.completedFuture(POSITION), - PulsarSubscriptionMode.keyShared()) - .capabilities(new DestinationName("orders.v1")) - .capabilities(); - - assertThat(capabilities.keyedOrdering()).isTrue(); - } - - @Test - void noSubscriptionModeAdvertisesBrokerTransactions() { - assertThat( - transport( - (topic, request) -> CompletableFuture.completedFuture(POSITION), - PulsarSubscriptionMode.keyShared()) - .capabilities(new DestinationName("orders.v1")) - .capabilities() - .brokerTransaction()) - .as("Pulsar has transactions; this adapter has not certified them") - .isFalse(); - } - - @Test - void requestingATransactionFailsWithAnExplanation() { - assertThat(PulsarTransactionCapability.rejection()) - .isInstanceOf(MessagingCapabilityUnavailableException.class) - .hasMessageContaining("not advertised"); - } - - private static TransportPublishRequest requestWithTimeout(java.time.Duration timeout) { - PublishOptions defaults = PublishOptions.defaults(); - return new TransportPublishRequest( - profile(), - envelope(64), - new PublishOptions(timeout, defaults.confirmation(), defaults.deduplication())); - } - - private static TransportPublishRequest request(int payloadBytes) { - return new TransportPublishRequest( - profile(), envelope(payloadBytes), PublishOptions.defaults()); - } - - private static DestinationProfile profile() { - return new DestinationProfile( - new DestinationName("orders.v1"), - "pulsar-primary", - DestinationKind.EVENT_STREAM, - PhysicalDestination.pulsarTopic(TOPIC, "orders-sub"), - new SchemaPolicy( - ContentType.JSON, - SchemaCompatibility.BACKWARD, - Set.of(new MessageType("order.created"))), - DeliveryGuarantee.AT_LEAST_ONCE, - OrderingScope.NONE, - ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, - ProducerPolicy.defaults(), - ConsumerPolicy.defaults("orders"), - RetryPolicy.none(), - DeadLetterPolicy.to(new DestinationName("orders.v1.dlq")), - PayloadPolicy.defaults(), - CapabilityTier.M1, - false, - false, - false); - } - - private static MessageEnvelope envelope(int payloadBytes) { - byte[] payload = new byte[payloadBytes]; - java.util.Arrays.fill(payload, (byte) 'x'); - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - new EncodedMessage(payload, ContentType.JSON, Optional.empty())); - } -} diff --git a/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionGuardTest.java b/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionGuardTest.java deleted file mode 100644 index a7dea10..0000000 --- a/messaging/messaging-pulsar-experimental/src/test/java/dev/caskeleton/messaging/pulsar/PulsarSubscriptionGuardTest.java +++ /dev/null @@ -1,125 +0,0 @@ -package dev.caskeleton.messaging.pulsar; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import java.time.Duration; -import java.util.Optional; -import org.junit.jupiter.api.Test; - -class PulsarSubscriptionGuardTest { - - @Test - void keySharedWithCumulativeAcknowledgementIsRefused() { - assertThatThrownBy(() -> new PulsarSubscriptionMode(PulsarSubscriptionType.KEY_SHARED, true)) - .as("cumulative ack over interleaved keys discards the ordering the type provides") - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("ordering"); - } - - @Test - void sharedWithCumulativeAcknowledgementIsRefused() { - assertThatThrownBy(() -> new PulsarSubscriptionMode(PulsarSubscriptionType.SHARED, true)) - .isInstanceOf(IllegalArgumentException.class); - } - - @Test - void aSharedSubscriptionPromisesNoOrdering() { - assertThat(PulsarSubscriptionMode.shared().orderingScope()).isEqualTo(OrderingScope.NONE); - } - - @Test - void aKeySharedSubscriptionPromisesKeyOrdering() { - assertThat(PulsarSubscriptionMode.keyShared().orderingScope()).isEqualTo(OrderingScope.KEY); - } - - @Test - void aZeroNegativeAckDelayIsRefused() { - assertThatThrownBy( - () -> - new PulsarProfile( - "persistent://public/default/orders", - "orders-sub", - PulsarSubscriptionMode.shared(), - Duration.ZERO, - Optional.empty(), - 5)) - .as("a zero redelivery delay turns a failing handler into a spin loop") - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("spin loop"); - } - - @Test - void anAckTimeoutBelowPulsarsOwnFloorIsRefused() { - assertThatThrownBy( - () -> - new PulsarProfile( - "persistent://public/default/orders", - "orders-sub", - PulsarSubscriptionMode.shared(), - Duration.ofSeconds(30), - Optional.of(Duration.ofSeconds(1)), - 5)) - .isInstanceOf(IllegalArgumentException.class); - } - - @Test - void theDefaultProfileLeavesAckTimeoutOff() { - assertThat( - PulsarProfile.shared("persistent://public/default/orders", "orders-sub").ackTimeout()) - .as("an ack timeout redelivers messages from handlers that are merely slow") - .isEmpty(); - } - - @Test - void transactionsAreNotAdvertisedWhileTheAdapterIsExperimental() { - assertThat(PulsarTransactionCapability.isAvailable()).isFalse(); - assertThat(PulsarTransactionCapability.rejection()) - .isInstanceOf(MessagingCapabilityUnavailableException.class); - } - - @Test - void theRefusalExplainsThatTheGapIsProofNotBrokerSupport() { - assertThat(PulsarTransactionCapability.rejection()) - .hasMessageContaining("certified") - .hasMessageContaining("contract suite"); - } - - @Test - void aNonBatchedPositionRendersWithoutABatchIndex() { - PulsarMessagePosition position = - new PulsarMessagePosition(12, 34, 2, PulsarMessagePosition.NOT_APPLICABLE); - - assertThat(position.toMessageId()).isEqualTo("12:34:2"); - assertThat(position.broker()).isEqualTo("pulsar"); - } - - @Test - void aBatchedPositionKeepsItsIndexBecauseTheIdAloneIsShared() { - assertThat(new PulsarMessagePosition(12, 34, 2, 7).toMessageId()).isEqualTo("12:34:2:7"); - } - - @Test - void thePositionIsReadableAsDiagnosticAttributes() { - assertThat(new PulsarMessagePosition(12, 34, 2, 7).diagnosticAttributes()) - .containsEntry("ledgerId", "12") - .containsEntry("batchIndex", "7"); - } - - @Test - void theValidatorAcceptsAKeyedProfileOnKeyShared() { - assertThatCode( - () -> - new PulsarProfile( - "persistent://public/default/orders", - "orders-sub", - PulsarSubscriptionMode.keyShared(), - Duration.ofSeconds(30), - Optional.empty(), - 5)) - .doesNotThrowAnyException(); - } -} diff --git a/messaging/messaging-rabbit/build.gradle b/messaging/messaging-rabbit/build.gradle deleted file mode 100644 index f346996..0000000 --- a/messaging/messaging-rabbit/build.gradle +++ /dev/null @@ -1,43 +0,0 @@ -plugins { - id 'java-test-fixtures' - id 'ca.auxiliary-source-set' - id 'ca.jmh-benchmarks' -} - -auxiliarySourceSets { - sourceSet('integrationTest') { - compilesAgainst 'main', 'testFixtures' - } -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-transport-spi') - api project(':messaging:messaging-observability') - api project(':messaging:messaging-security') - api project(':messaging:messaging-admin-api') - - // api: RabbitChannelPublisher and the mappers expose Spring AMQP's Message on public - // signatures, so a consumer implementing the seam needs the type on its compile classpath. - api 'org.springframework.amqp:spring-rabbit' - implementation 'org.springframework:spring-context' - - // Test-only. The same adapter contract Kafka runs, so a guarantee can only be weakened by - // editing the contract rather than by an adapter quietly not implementing it. - testImplementation project(':messaging:messaging-testkit') - - // Live-broker certification. The confirm-and-return ordering the adapter depends on is a - // property of AMQP, and only a real broker exhibits it. - integrationTestImplementation 'org.testcontainers:testcontainers-rabbitmq' - integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' -} - -strictTestLanes { - lane('integrationTest') { - integration() - sourceSet = 'integrationTest' - description = 'Runs RabbitMQ broker integration contracts against a real container.' - } -} diff --git a/messaging/messaging-rabbit/gradle.lockfile b/messaging/messaging-rabbit/gradle.lockfile deleted file mode 100644 index e5000c2..0000000 --- a/messaging/messaging-rabbit/gradle.lockfile +++ /dev/null @@ -1,128 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.fasterxml.jackson.core:jackson-annotations:2.21=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.github.docker-java:docker-java-api:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.docker-java:docker-java-transport-zerodep:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.docker-java:docker-java-transport:3.7.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -com.rabbitmq:amqp-client:5.27.1=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-codec:commons-codec:1.19.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.20.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -commons-io:commons-io:2.21.0=spotbugs -commons-logging:commons-logging:1.3.6=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-buffer:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-codec-base:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-codec-compression:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-codec-marshalling:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-codec-protobuf:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-codec:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-common:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-handler:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-resolver:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-transport-native-unix-common:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -io.netty:netty-transport:4.2.17.Final=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -net.java.dev.jna:jna:5.18.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath -net.sf.jopt-simple:jopt-simple:5.0.4=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-compress:1.28.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.apache.commons:commons-lang3:3.20.0=checkstyle,integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs -org.apache.commons:commons-math3:3.6.1=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,jmhCompileClasspath,testCompileClasspath -org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=integrationTestRuntimeClasspath,jmhRuntimeClasspath,runtimeClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jetbrains:annotations:17.0.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,integrationTestAnnotationProcessor,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testFixturesAnnotationProcessor,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,testRuntimeClasspath -org.junit:junit-bom:6.0.3=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=integrationTestRuntimeClasspath,jmhRuntimeClasspath,runtimeClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.openjdk.jmh:jmh-core:1.37=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath -org.openjdk.jmh:jmh-generator-annprocess:1.37=jmhAnnotationProcessor -org.opentest4j:opentest4j:1.3.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,integrationTestAnnotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor,testFixturesAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.rnorth.duct-tape:duct-tape:1.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.slf4j:slf4j-api:2.0.18=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,spotbugs,spotbugsSlf4j,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.springframework.amqp:spring-amqp:4.0.5=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework.amqp:spring-rabbit:4.0.5=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-aop:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-beans:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-context:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-core:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-expression:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-messaging:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.springframework:spring-tx:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testFixturesCompileClasspath,testFixturesRuntimeClasspath,testRuntimeClasspath -org.testcontainers:testcontainers-junit-jupiter:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers-rabbitmq:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.testcontainers:testcontainers:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty= diff --git a/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitBrokerIT.java b/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitBrokerIT.java deleted file mode 100644 index 7937f52..0000000 --- a/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitBrokerIT.java +++ /dev/null @@ -1,232 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; - -import com.rabbitmq.client.AMQP; -import com.rabbitmq.client.Channel; -import com.rabbitmq.client.Connection; -import com.rabbitmq.client.ConnectionFactory; -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.schema.EncodedMessage; -import dev.caskeleton.messaging.testkit.DockerAvailability; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import java.io.IOException; -import java.nio.charset.StandardCharsets; -import java.time.Duration; -import java.time.Instant; -import java.util.Map; -import java.util.Optional; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.TimeoutException; -import org.junit.jupiter.api.AfterEach; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.condition.EnabledIf; -import org.springframework.amqp.core.MessageProperties; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; -import org.testcontainers.rabbitmq.RabbitMQContainer; - -/** - * Certifies the RabbitMQ adapter against a live broker. - * - *

The test that matters is the unroutable one. AMQP delivers a {@code basic.return} - * before the confirm for a message no queue was bound to receive, and an adapter that - * completes on the confirm alone reports success for a message the broker threw away. No mock - * proves the ordering is handled correctly, because the ordering is the broker's behaviour. - */ -@Testcontainers -@EnabledIf("dockerAvailable") -class RabbitBrokerIT { - - private static final String EXCHANGE = "notification.work"; - private static final String BOUND_KEY = "email"; - private static final String UNBOUND_KEY = "nothing.is.bound.here"; - private static final String QUEUE = "notification.email.q"; - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - - @Container - private static final RabbitMQContainer RABBIT = new RabbitMQContainer("rabbitmq:4.3-management"); - - private Connection connection; - private Channel channel; - private RabbitMessagingTransport transport; - - static boolean dockerAvailable() { - return DockerAvailability.isAvailable(); - } - - @BeforeEach - void declareTopology() throws IOException, TimeoutException { - ConnectionFactory factory = new ConnectionFactory(); - factory.setHost(RABBIT.getHost()); - factory.setPort(RABBIT.getAmqpPort()); - factory.setUsername(RABBIT.getAdminUsername()); - factory.setPassword(RABBIT.getAdminPassword()); - connection = factory.newConnection(); - channel = connection.createChannel(); - - channel.exchangeDeclare(EXCHANGE, "direct", true); - channel.queueDeclare(QUEUE, true, false, false, Map.of("x-queue-type", "quorum")); - channel.queueBind(QUEUE, EXCHANGE, BOUND_KEY); - // The container is shared across the class, so a message left by an earlier test would make a - // later queue-depth assertion pass or fail for the wrong reason. - channel.queuePurge(QUEUE); - channel.confirmSelect(); - - transport = - new RabbitMessagingTransport( - "rabbit-primary", - 1, - new RabbitChannelPublisher() { - @Override - public long nextPublishSequence() { - return channel.getNextPublishSeqNo(); - } - - @Override - public void publish( - long sequence, - String exchange, - String routingKey, - org.springframework.amqp.core.Message message, - boolean mandatory) { - RabbitBrokerIT.this.publish(sequence, exchange, routingKey, message, mandatory); - } - }); - - channel.addConfirmListener( - (deliveryTag, multiple) -> transport.onConfirm(deliveryTag, true, Duration.ofMillis(1)), - (deliveryTag, multiple) -> transport.onConfirm(deliveryTag, false, Duration.ofMillis(1))); - channel.addReturnListener( - returned -> - transport.onReturn( - Long.parseLong(returned.getProperties().getHeaders().get("x-seq").toString()))); - } - - @AfterEach - void close() throws IOException, TimeoutException { - if (channel != null && channel.isOpen()) { - channel.close(); - } - if (connection != null && connection.isOpen()) { - connection.close(); - } - } - - @Test - void aRoutablePublishConfirmsWithReplicationEvidence() { - PublishResult result = await(transport.publish(request(BOUND_KEY))); - - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.ROUTED); - assertThat(result.evidence().confirmationLevel()) - .isEqualTo(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK); - } - - @Test - void anUnroutablePublishIsRejectedEvenThoughTheExchangeConfirmedIt() { - PublishResult result = await(transport.publish(request(UNBOUND_KEY))); - - assertThat(result.completion()) - .as("the exchange accepted it and the broker threw it away; that is not success") - .isEqualTo(PublishCompletion.REJECTED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.UNROUTABLE); - assertThat(result.evidence().confirmationLevel()).isEqualTo(ConfirmationLevel.NONE); - } - - @Test - void aRoutablePublishReachesTheQueue() throws IOException { - await(transport.publish(request(BOUND_KEY))); - - assertThat(channel.messageCount(QUEUE)).isEqualTo(1); - } - - @Test - void anUnroutablePublishReachesNoQueue() throws IOException { - await(transport.publish(request(UNBOUND_KEY))); - - assertThat(channel.messageCount(QUEUE)).isZero(); - } - - /** - * Publishes and records the sequence number in a header. - * - *

A returned message arrives without its publish sequence number, so the adapter has to carry - * one itself to correlate the return with the pending publish. - */ - private void publish( - long sequence, - String exchange, - String routingKey, - org.springframework.amqp.core.Message message, - boolean mandatory) { - try { - MessageProperties source = message.getMessageProperties(); - AMQP.BasicProperties properties = - new AMQP.BasicProperties.Builder() - .messageId(source.getMessageId()) - .contentType(source.getContentType()) - .deliveryMode(2) - .headers(withSequence(source, sequence)) - .build(); - // No waitForConfirms: the coordinator completes the publish from the confirm listener, and - // blocking here would hold the calling thread through the round trip for no benefit. - channel.basicPublish(exchange, routingKey, mandatory, properties, message.getBody()); - } catch (IOException exception) { - throw new IllegalStateException("rabbit publish failed", exception); - } - } - - private static Map withSequence(MessageProperties source, long sequence) { - java.util.Map headers = new java.util.LinkedHashMap<>(source.getHeaders()); - headers.put("x-seq", Long.toString(sequence)); - return headers; - } - - private static PublishResult await( - CompletionStage stage) { - return stage.toCompletableFuture().join().result(); - } - - private TransportPublishRequest request(String routingKey) { - return new TransportPublishRequest( - RabbitFixtureProfiles.routingTo(EXCHANGE, routingKey, QUEUE), - envelope(), - PublishOptions.defaults()); - } - - private static MessageEnvelope envelope() { - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - new EncodedMessage( - "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), - ContentType.JSON, - Optional.empty())); - } -} diff --git a/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitFixtureProfiles.java b/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitFixtureProfiles.java deleted file mode 100644 index da40530..0000000 --- a/messaging/messaging-rabbit/src/integrationTest/java/dev/caskeleton/messaging/rabbit/RabbitFixtureProfiles.java +++ /dev/null @@ -1,32 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.PhysicalDestination; - -/** Builds destination profiles that vary only in their physical mapping. */ -final class RabbitFixtureProfiles { - - private RabbitFixtureProfiles() {} - - static DestinationProfile routingTo(String exchange, String routingKey, String queue) { - DestinationProfile base = RabbitProfileFixtures.workQueueDestination(); - return new DestinationProfile( - base.name(), - base.broker(), - base.kind(), - PhysicalDestination.rabbitQueue(exchange, routingKey, queue), - base.schema(), - base.deliveryGuarantee(), - base.orderingScope(), - base.externalSideEffectGuarantee(), - base.producer(), - base.consumer(), - base.retry(), - base.deadLetter(), - base.payload(), - base.tier(), - base.production(), - base.keyResolverConfigured(), - base.topologyAutoCreate()); - } -} diff --git a/messaging/messaging-rabbit/src/jmh/java/dev/caskeleton/messaging/rabbit/RabbitPublishBenchmark.java b/messaging/messaging-rabbit/src/jmh/java/dev/caskeleton/messaging/rabbit/RabbitPublishBenchmark.java deleted file mode 100644 index e8e2d46..0000000 --- a/messaging/messaging-rabbit/src/jmh/java/dev/caskeleton/messaging/rabbit/RabbitPublishBenchmark.java +++ /dev/null @@ -1,156 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.schema.EncodedMessage; -import java.time.Duration; -import java.time.Instant; -import java.util.Optional; -import java.util.concurrent.TimeUnit; -import org.openjdk.jmh.annotations.Benchmark; -import org.openjdk.jmh.annotations.BenchmarkMode; -import org.openjdk.jmh.annotations.Fork; -import org.openjdk.jmh.annotations.Level; -import org.openjdk.jmh.annotations.Measurement; -import org.openjdk.jmh.annotations.Mode; -import org.openjdk.jmh.annotations.OutputTimeUnit; -import org.openjdk.jmh.annotations.Param; -import org.openjdk.jmh.annotations.Scope; -import org.openjdk.jmh.annotations.Setup; -import org.openjdk.jmh.annotations.State; -import org.openjdk.jmh.annotations.Warmup; -import org.openjdk.jmh.infra.Blackhole; - -/** - * Measures the RabbitMQ adapter's own cost per publish, with no broker in the loop. - * - *

The confirm coordinator is the interesting one here. Every publish registers a pending entry - * and every confirm resolves it, so its map operations run twice per message on the hot path — and - * because a confirm arrives on the connection thread while publishes arrive on caller threads, a - * regression there shows up as contention rather than as a slower method. - * - *

No channel and no network, for the same reason the Kafka benchmark has no producer: measuring - * the broker measures the environment. - */ -@BenchmarkMode(Mode.AverageTime) -@OutputTimeUnit(TimeUnit.NANOSECONDS) -@Warmup(iterations = 3, time = 1) -@Measurement(iterations = 5, time = 1) -@Fork(1) -@State(Scope.Benchmark) -public class RabbitPublishBenchmark { - - /** Payload sizes spanning the range a portable destination allows. */ - @Param({"256", "4096", "65536"}) - public int payloadBytes; - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - private static final String EXCHANGE = "notification.work"; - private static final String ROUTING_KEY = "email"; - - private RabbitHeaderMapper headerMapper; - private RabbitPublishFailureClassifier classifier; - private MessageEnvelope envelope; - private RabbitConfirmCoordinator coordinator; - private long sequence; - - /** Builds the fixtures once so the measured methods allocate only what a publish allocates. */ - @Setup(Level.Trial) - public void setUp() { - headerMapper = new RabbitHeaderMapper(); - classifier = new RabbitPublishFailureClassifier(); - byte[] payload = new byte[payloadBytes]; - java.util.Arrays.fill(payload, (byte) 'x'); - envelope = - new MessageEnvelope<>( - MessageId.newId(), - new MessageType("email.requested"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("notification-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - new EncodedMessage(payload, ContentType.JSON, Optional.empty())); - } - - /** Resets the coordinator per iteration so pending entries do not accumulate across runs. */ - @Setup(Level.Iteration) - public void resetCoordinator() { - coordinator = new RabbitConfirmCoordinator(); - sequence = 0; - } - - /** - * Measures mapping an envelope's reserved headers onto AMQP properties. - * - * @param blackhole consumes the result - */ - @Benchmark - public void mapHeaders(Blackhole blackhole) { - blackhole.consume(headerMapper.toProperties(envelope)); - } - - /** - * Measures one full register-then-confirm round trip through the coordinator. - * - *

Both halves together, because a benchmark of the registration alone would leave the pending - * map growing and report a cost that no real publish pays. - * - * @param blackhole consumes the result - */ - @Benchmark - public void registerAndConfirm(Blackhole blackhole) { - long current = ++sequence; - blackhole.consume( - coordinator.awaiting( - current, - EXCHANGE, - ROUTING_KEY, - ConfirmationRequirement.REPLICATION_OR_PERSISTENCE_ACK)); - coordinator.confirmed(current, true, Duration.ofMillis(1)); - } - - /** - * Measures the unroutable path, which resolves through a return before the confirm. - * - * @param blackhole consumes the result - */ - @Benchmark - public void registerReturnAndConfirm(Blackhole blackhole) { - long current = ++sequence; - blackhole.consume( - coordinator.awaiting( - current, - EXCHANGE, - ROUTING_KEY, - ConfirmationRequirement.REPLICATION_OR_PERSISTENCE_ACK)); - coordinator.returned(current); - coordinator.confirmed(current, true, Duration.ofMillis(1)); - } - - /** - * Measures classifying a lost confirm, which runs on every ambiguous publish. - * - * @param blackhole consumes the result - */ - @Benchmark - public void classifyLostConfirm(Blackhole blackhole) { - blackhole.consume( - classifier.classify( - new java.io.IOException("connection reset"), true, Duration.ofMillis(5))); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBatchConsumerRegistrar.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBatchConsumerRegistrar.java deleted file mode 100644 index 68ab87a..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBatchConsumerRegistrar.java +++ /dev/null @@ -1,165 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.delivery.BatchDeliveryMetadata; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Objects; -import java.util.Optional; - -/** - * Accumulates AMQP deliveries into batches, bounded by size and by age. - * - *

Unlike Kafka, RabbitMQ delivers one message at a time, so a batch here is built by the client - * rather than returned by a poll. That makes the age bound essential: without it, a queue that goes - * quiet leaves the last few messages sitting in the accumulator indefinitely, unacknowledged, until - * enough traffic arrives to fill a batch — which on a low-volume queue may be hours. - * - *

{@code settlableAsBatch} is reported false. AMQP's multiple-ack acknowledges every delivery up - * to a tag, which on a channel handling concurrent work settles messages the handler has not - * finished; the platform therefore acknowledges each delivery individually even when the handler - * saw them as a batch. - * - *

The accumulator is not thread-safe by design. It is driven from the single consumer callback - * thread for its channel, and adding synchronisation here would suggest it can be shared, which - * would interleave deliveries from different channels into one batch and lose their ordering. - */ -public final class RabbitBatchConsumerRegistrar { - - private final DestinationProfile profile; - private final int maxBatchSize; - private final Duration maxBatchAge; - private final List deliveryTags = new ArrayList<>(); - private Instant oldestArrival; - - /** - * Creates a batch accumulator for a destination. - * - * @param profile the validated destination profile - * @param maxBatchSize the largest batch handed to a handler - * @param maxBatchAge how long the oldest delivery may wait before the batch is released - */ - public RabbitBatchConsumerRegistrar( - DestinationProfile profile, int maxBatchSize, Duration maxBatchAge) { - this.profile = Objects.requireNonNull(profile, "profile must not be null"); - Objects.requireNonNull(maxBatchAge, "maxBatchAge must not be null"); - if (maxBatchSize < 1) { - throw new IllegalArgumentException("maxBatchSize must be at least 1"); - } - if (maxBatchAge.isNegative() || maxBatchAge.isZero()) { - throw new IllegalArgumentException( - "maxBatchAge must be positive; without it a quiet queue holds its last messages " - + "unacknowledged indefinitely"); - } - this.maxBatchSize = maxBatchSize; - this.maxBatchAge = maxBatchAge; - } - - /** - * Adds one delivery to the current batch. - * - * @param deliveryTag the AMQP delivery tag - * @param now the current instant - * @return the batch when it is ready to release, otherwise empty - */ - public Optional add(long deliveryTag, Instant now) { - Objects.requireNonNull(now, "now must not be null"); - if (deliveryTags.isEmpty()) { - oldestArrival = now; - } - deliveryTags.add(deliveryTag); - return deliveryTags.size() >= maxBatchSize ? Optional.of(release(now)) : Optional.empty(); - } - - /** - * Releases the current batch if its oldest delivery has waited long enough. - * - * @param now the current instant - * @return the batch when the age bound was reached, otherwise empty - */ - public Optional releaseIfStale(Instant now) { - Objects.requireNonNull(now, "now must not be null"); - if (deliveryTags.isEmpty()) { - return Optional.empty(); - } - return !now.isBefore(oldestArrival.plus(maxBatchAge)) - ? Optional.of(release(now)) - : Optional.empty(); - } - - /** - * Releases whatever has accumulated, for shutdown. - * - * @param now the current instant - * @return the batch when anything was pending - */ - public Optional drain(Instant now) { - Objects.requireNonNull(now, "now must not be null"); - return deliveryTags.isEmpty() ? Optional.empty() : Optional.of(release(now)); - } - - /** - * Returns how many deliveries are waiting in the accumulator. - * - * @return the pending delivery count - */ - public int pending() { - return deliveryTags.size(); - } - - /** - * Refuses batch consumption where the prefetch cannot hold a whole batch. - * - *

A prefetch below the batch size deadlocks: the accumulator waits for more deliveries the - * broker will not send until something is acknowledged, and nothing is acknowledged until the - * batch is released. - * - * @param prefetch the channel prefetch count - * @throws MessagingConfigurationException when the prefetch is too small - */ - public void requirePrefetchFor(int prefetch) { - if (prefetch < maxBatchSize) { - throw new MessagingConfigurationException( - "PREFETCH_BELOW_BATCH_SIZE", - "destination %s batches %d deliveries but the channel prefetch is %d; the accumulator " - .formatted(profile.name().value(), maxBatchSize, prefetch) - + "would wait for deliveries the broker will not send until something is acked"); - } - } - - private AmqpBatch release(Instant now) { - List released = List.copyOf(deliveryTags); - deliveryTags.clear(); - oldestArrival = null; - return new AmqpBatch( - released, - new BatchDeliveryMetadata( - profile.name(), - released.size(), - // One queue is one ordering unit, so an accumulated batch never spans two. - Optional.of(profile.physical().queue().orElse(profile.name().value())), - // AMQP multiple-ack settles everything up to a tag, including work still in flight. - false, - now)); - } - - /** - * One accumulated batch of AMQP deliveries. - * - * @param deliveryTags the delivery tags, in arrival order - * @param metadata the batch-wide metadata handed to the handler - */ - public record AmqpBatch(List deliveryTags, BatchDeliveryMetadata metadata) { - - public AmqpBatch { - Objects.requireNonNull(metadata, "metadata must not be null"); - deliveryTags = List.copyOf(deliveryTags); - if (deliveryTags.isEmpty()) { - throw new IllegalArgumentException("a released batch is never empty"); - } - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBrokerProfile.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBrokerProfile.java deleted file mode 100644 index e53d377..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitBrokerProfile.java +++ /dev/null @@ -1,50 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import java.time.Duration; -import java.util.List; -import java.util.Objects; - -/** - * The RabbitMQ client settings the platform is willing to run with. - * - * @param broker the logical broker name - * @param stable whether this profile claims the Stable guarantees - * @param production whether this profile is used in production - * @param addresses the broker addresses - * @param publisherConfirms whether correlated publisher confirms are on - * @param publisherReturns whether unroutable returns are delivered - * @param mandatory whether publishes are sent with the mandatory flag - * @param confirmTimeout how long to wait for a confirm - * @param autoAck whether the consumer acknowledges on delivery - * @param prefetch the consumer prefetch window - * @param quorumQueues whether durable work queues are declared as quorum queues - * @param tlsEnabled whether transport encryption is on - * @param authenticationEnabled whether broker authentication is on - */ -public record RabbitBrokerProfile( - String broker, - boolean stable, - boolean production, - List addresses, - boolean publisherConfirms, - boolean publisherReturns, - boolean mandatory, - Duration confirmTimeout, - boolean autoAck, - int prefetch, - boolean quorumQueues, - boolean tlsEnabled, - boolean authenticationEnabled) { - - public RabbitBrokerProfile { - Objects.requireNonNull(addresses, "addresses must not be null"); - Objects.requireNonNull(confirmTimeout, "confirmTimeout must not be null"); - if (broker == null || broker.isBlank()) { - throw new IllegalArgumentException("broker must not be blank"); - } - if (addresses.isEmpty()) { - throw new IllegalArgumentException("addresses must not be empty"); - } - addresses = List.copyOf(addresses); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitChannelPublisher.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitChannelPublisher.java deleted file mode 100644 index 8fe08f2..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitChannelPublisher.java +++ /dev/null @@ -1,42 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import org.springframework.amqp.core.Message; - -/** - * The channel operations the transport needs, split so the confirm cannot outrun its registration. - * - *

Reserving the sequence number is a separate call from publishing on purpose. A confirm can - * arrive on the connection thread before {@code basicPublish} has even returned, so a transport - * that registered its pending publish afterwards would drop that confirm and wait forever for one - * that already came. Reserving first closes that race. - */ -public interface RabbitChannelPublisher { - - /** - * Reserves the next publish sequence number without publishing. - * - * @return the sequence number the next publish will carry - */ - long nextPublishSequence(); - - /** - * Publishes a message under a previously reserved sequence number. - * - * @param sequence the reserved sequence number - * @param exchange the exchange to publish to - * @param routingKey the routing key - * @param message the AMQP message - * @param mandatory whether the broker must return an unroutable message - */ - void publish( - long sequence, String exchange, String routingKey, Message message, boolean mandatory); - - /** - * Closes the underlying channel. - * - *

A default no-op so an existing publisher keeps compiling, but the transport calls it: a - * transport whose {@code close} only flipped a boolean left the native channel open and every - * pending stage waiting for a confirm that would never arrive. - */ - default void close() {} -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinator.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinator.java deleted file mode 100644 index f65c985..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinator.java +++ /dev/null @@ -1,279 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; - -/** - * Correlates a RabbitMQ publish with its confirm and its possible return. - * - *

This is the piece that makes the Rabbit adapter honest. AMQP delivers a return before - * the confirm for an unroutable message, so a naive adapter that completes on the confirm reports - * success for a message the broker threw away. The coordinator therefore keeps each publish pending - * until the confirm arrives, and remembers whether a return was seen first. - * - *

The resulting outcomes are: confirmed and not returned is {@code CONFIRMED}; confirmed and - * returned is {@code REJECTED} with {@code UNROUTABLE}; nacked is {@code REJECTED}; and a confirm - * that never arrives is {@code AMBIGUOUS}, because the broker may still have stored it. - * - *

Implemented as a pure state machine over sequence numbers so the whole ordering, including the - * return-before-confirm case, is testable without a broker. - */ -public final class RabbitConfirmCoordinator { - - /** - * Pending publishes, ordered by sequence. - * - *

Ordered because a Rabbit confirm carries a {@code multiple} flag meaning "everything up to - * and including this tag". Resolving one sequence per confirm — which is what a hash map forces — - * leaves every earlier publish pending forever: the caller's stage never completes and the entry - * is never removed, so the map grows for the life of the connection. - */ - private final java.util.concurrent.ConcurrentSkipListMap pending = - new java.util.concurrent.ConcurrentSkipListMap<>(); - - /** - * Registers a publish awaiting its confirm. - * - * @param sequence the channel publish sequence number - * @param exchange the exchange published to - * @param routingKey the routing key used - * @param requirement the confirmation the profile demanded - * @return a stage completing when the confirm, return, or timeout resolves it - */ - public CompletionStage awaiting( - long sequence, String exchange, String routingKey, ConfirmationRequirement requirement) { - Objects.requireNonNull(exchange, "exchange must not be null"); - Objects.requireNonNull(routingKey, "routingKey must not be null"); - Objects.requireNonNull(requirement, "requirement must not be null"); - - PendingPublish publish = new PendingPublish(exchange, routingKey, requirement); - pending.put(sequence, publish); - return publish.completion; - } - - /** - * Records that the broker returned a message as unroutable. - * - *

The return alone does not complete the publish: the confirm still follows, and only the pair - * establishes that the broker both accepted and discarded it. - * - * @param sequence the publish sequence number - */ - public void returned(long sequence) { - PendingPublish publish = pending.get(sequence); - if (publish != null) { - publish.returned = true; - } - } - - /** - * Records a broker confirm. - * - * @param sequence the publish sequence number - * @param acknowledged true for an ack, false for a nack - * @param elapsed how long the publish took - */ - public void confirmed(long sequence, boolean acknowledged, Duration elapsed) { - confirmed(sequence, false, acknowledged, elapsed); - } - - /** - * Records a broker confirm, honouring the {@code multiple} flag. - * - * @param multiple true when the broker is confirming every sequence up to and including this one - */ - public void confirmed(long sequence, boolean multiple, boolean acknowledged, Duration elapsed) { - if (!multiple) { - PendingPublish publish = pending.remove(sequence); - if (publish != null) { - publish.completion.complete(publish.resolve(sequence, acknowledged, elapsed)); - } - return; - } - // headMap(tag, inclusive) is the whole point of the ordered map: the broker is confirming a - // range, and every entry in it must be resolved with the same verdict. - java.util.NavigableMap confirmed = pending.headMap(sequence, true); - for (Long pendingSequence : java.util.List.copyOf(confirmed.keySet())) { - PendingPublish publish = pending.remove(pendingSequence); - if (publish != null) { - publish.completion.complete(publish.resolve(pendingSequence, acknowledged, elapsed)); - } - } - } - - /** - * Completes a publish whose send failed synchronously. - * - *

The classifier's verdict is used as-is: it is the only thing that knows whether the frame - * can have reached the broker, and guessing here would either invent a rejection for a message - * that was stored or an ambiguity for one that never left. - * - * @param sequence the publish sequence number - * @param result the classified outcome - */ - public void sendFailed(long sequence, PublishResult result) { - PendingPublish publish = pending.remove(sequence); - if (publish != null) { - publish.completion.complete(result); - } - } - - /** - * Completes every pending publish because the channel is gone. - * - *

Ambiguous, not rejected: the frames were written, and a channel that closes afterwards says - * nothing about whether the broker stored them. Leaving them pending — which is what close used - * to do — hangs every caller holding a stage. - * - * @param elapsed how long the publishes had been waiting - * @return how many stages were completed - */ - public int channelClosed(Duration elapsed) { - int drained = 0; - for (Long sequence : java.util.List.copyOf(pending.keySet())) { - PendingPublish publish = pending.remove(sequence); - if (publish != null) { - publish.completion.complete(publish.resolveChannelClosed(sequence, elapsed)); - drained++; - } - } - return drained; - } - - /** - * Records that the confirm never arrived within the deadline. - * - * @param sequence the publish sequence number - * @param elapsed how long was waited - */ - public void timedOut(long sequence, Duration elapsed) { - PendingPublish publish = pending.remove(sequence); - if (publish == null) { - return; - } - publish.completion.complete( - new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "RABBIT_CONFIRM_TIMEOUT", - "no publisher confirm arrived before the deadline")))); - } - - /** - * Returns how many publishes are awaiting a confirm. - * - * @return the pending count - */ - public int pendingCount() { - return pending.size(); - } - - /** One publish waiting for its confirm, and whether a return arrived first. */ - private static final class PendingPublish { - - private final String exchange; - private final String routingKey; - private final ConfirmationRequirement requirement; - private final CompletableFuture completion = new CompletableFuture<>(); - - private volatile boolean returned; - - private PendingPublish( - String exchange, String routingKey, ConfirmationRequirement requirement) { - this.exchange = exchange; - this.routingKey = routingKey; - this.requirement = requirement; - } - - /** The channel closed with this publish still awaiting its confirm. */ - private PublishResult resolveChannelClosed(long sequence, Duration elapsed) { - return new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.of(new RabbitPublishReference(exchange, routingKey, sequence)), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "RABBIT_CHANNEL_CLOSED", - "the channel closed before the broker confirmed this publish"))); - } - - private PublishResult resolve(long sequence, boolean acknowledged, Duration elapsed) { - RabbitPublishReference position = new RabbitPublishReference(exchange, routingKey, sequence); - - if (!acknowledged) { - return new PublishResult( - PublishCompletion.REJECTED, - // Transmitted, not "not transmitted": a NACK is the broker's answer to a frame it - // received. Recording it as never sent contradicts the very evidence that produced it, - // and a caller reading the evidence would conclude the message can be re-sent freely. - new PublishEvidence( - true, - dev.caskeleton.messaging.api.publish.TransmissionEvidence.TRANSMITTED, - false, - ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.of(position), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.PERMANENT_BUSINESS, - "RABBIT_NACK", - "the broker negatively acknowledged the publish"))); - } - if (returned) { - return new PublishResult( - PublishCompletion.REJECTED, - new PublishEvidence( - true, - dev.caskeleton.messaging.api.publish.TransmissionEvidence.TRANSMITTED, - false, - ConfirmationLevel.NONE), - RoutingOutcome.UNROUTABLE, - Optional.of(position), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.PERMANENT_BUSINESS, - "RABBIT_UNROUTABLE", - "the exchange accepted the publish but no queue was bound"))); - } - - ConfirmationLevel level = - requirement == ConfirmationRequirement.REPLICATION_OR_PERSISTENCE_ACK - ? ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK - : ConfirmationLevel.BROKER_ACK; - return new PublishResult( - PublishCompletion.CONFIRMED, - PublishEvidence.confirmed(level), - RoutingOutcome.ROUTED, - Optional.of(position), - 1, - elapsed, - Optional.empty()); - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConsumerRegistrar.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConsumerRegistrar.java deleted file mode 100644 index 1852966..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitConsumerRegistrar.java +++ /dev/null @@ -1,238 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.settlement.SettlementCompletion; -import dev.caskeleton.messaging.api.settlement.SettlementEvidence; -import dev.caskeleton.messaging.api.settlement.SettlementResult; -import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; -import dev.caskeleton.messaging.transport.TransportConsumerRegistration; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import dev.caskeleton.messaging.transport.TransportDelivery; -import dev.caskeleton.messaging.transport.TransportSettlement; -import java.time.Duration; -import java.time.Instant; -import java.util.Objects; -import java.util.Optional; -import java.util.Set; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.atomic.AtomicBoolean; -import org.springframework.amqp.core.Message; - -/** - * The RabbitMQ consumer runtime. - * - *

Deliveries arrive from the listener container and are handed to the platform with a settlement - * handle that is bound to one delivery tag and refuses a second terminal call. Duplicating a {@code - * basic.ack} on a reused tag acknowledges a different message, which shows up as - * unexplained loss under load, far from its cause. - * - *

Consumption stops the moment a drain begins, but deliveries already in flight are allowed to - * finish. Rejecting an in-flight delivery at shutdown would either lose it or duplicate its side - * effect, depending on where the handler had got to. - */ -public final class RabbitConsumerRegistrar implements TransportConsumerRegistration { - - private final TransportConsumerSpec spec; - private final RabbitDeliveryMapper deliveryMapper; - private final RabbitSettlementOperations operations; - private final GracefulShutdownCoordinator shutdown; - private final Set pausedScopes = ConcurrentHashMap.newKeySet(); - private final AtomicBoolean active = new AtomicBoolean(true); - - /** - * Creates a consumer runtime. - * - * @param spec what to consume and where to deliver it - * @param operations the channel operations a settlement drives - * @param shutdown the drain coordinator - */ - public RabbitConsumerRegistrar( - TransportConsumerSpec spec, - RabbitSettlementOperations operations, - GracefulShutdownCoordinator shutdown) { - this(spec, operations, shutdown, new RabbitDeliveryMapper()); - } - - /** - * Creates a consumer runtime with an explicit delivery mapper. - * - * @param spec what to consume and where to deliver it - * @param operations the channel operations a settlement drives - * @param shutdown the drain coordinator - * @param deliveryMapper rebuilds envelopes from AMQP messages - */ - public RabbitConsumerRegistrar( - TransportConsumerSpec spec, - RabbitSettlementOperations operations, - GracefulShutdownCoordinator shutdown, - RabbitDeliveryMapper deliveryMapper) { - this.spec = Objects.requireNonNull(spec, "spec must not be null"); - this.operations = Objects.requireNonNull(operations, "operations must not be null"); - this.shutdown = Objects.requireNonNull(shutdown, "shutdown must not be null"); - this.deliveryMapper = Objects.requireNonNull(deliveryMapper, "deliveryMapper is required"); - } - - /** - * Handles one delivery pushed from the listener container. - * - *

Returns whether the delivery was accepted. A refusal means the runtime is draining or the - * queue is paused, and the caller must leave the message unacknowledged so the broker redelivers - * it. - * - * @param message the AMQP message - * @param now the current instant - * @return true when the delivery was dispatched - */ - public boolean onMessage(Message message, Instant now) { - Objects.requireNonNull(message, "message must not be null"); - Objects.requireNonNull(now, "now must not be null"); - - String queue = - Optional.ofNullable(message.getMessageProperties().getConsumerQueue()).orElse(""); - if (!active.get()) { - // close() only flipped a flag and onMessage never read it, so a delivery arriving after - // close was accepted, dispatched, and settled against a controller that was shutting down. - return false; - } - if (pausedScopes.contains(queue) || pausedScopes.contains("")) { - return false; - } - if (!shutdown.tryBeginWork()) { - return false; - } - - long tag = message.getMessageProperties().getDeliveryTag(); - try { - TransportDelivery delivery; - try { - // The narrow catch. One catch around decode, the handler and the settlement meant a - // business failure or an ACK that could not be written was recorded as an undecodable - // payload and discarded — a message that should have been retried, deleted instead. - delivery = - new TransportDelivery( - deliveryMapper.toEnvelope(message), - deliveryMapper.toMetadata(spec.profile().name(), message, now), - new ControllerBackedSettlement(tag)); - } catch (RuntimeException undecodable) { - // Genuinely undecodable: it will fail the same way on every redelivery, so requeueing is a - // hot loop. Discarding is right here and only here. - operations - .discard( - tag, - FailureDescriptor.of( - FailureCategory.DESERIALIZATION, - "RABBIT_UNDECODABLE", - "the message could not be decoded into an envelope")) - .toCompletableFuture() - .join(); - return true; - } - - ControllerBackedSettlement settlement = (ControllerBackedSettlement) delivery.settlement(); - try { - spec.sink().apply(delivery).toCompletableFuture().join(); - } catch (RuntimeException handlerFailed) { - // A handler failure is a retry, not a discard. - requeueQuietly(settlement); - return true; - } - if (!settlement.wasSettled()) { - // A handler that completed without settling has not said what to do with the message. - // Acknowledging on its behalf would silently drop it; the contract violation is surfaced - // as a requeue so the message survives while somebody fixes the handler. - requeueQuietly(settlement); - } - return true; - } finally { - shutdown.endWork(); - } - } - - /** Requeues without letting a settlement failure become the caller's problem. */ - private void requeueQuietly(ControllerBackedSettlement settlement) { - try { - settlement.requeue(Duration.ZERO).toCompletableFuture().join(); - } catch (RuntimeException settlementFailed) { - // The broker connection is the thing that failed. There is nothing left to try here, and - // the delivery stays unsettled — which is what redelivery is for. - return; - } - } - - @Override - public CompletionStage pause(String scope) { - pausedScopes.add(scope == null ? "" : scope); - return CompletableFuture.completedFuture(null); - } - - @Override - public CompletionStage resume(String scope) { - pausedScopes.remove(scope == null ? "" : scope); - return CompletableFuture.completedFuture(null); - } - - /** - * Reports whether a scope is currently paused. - * - * @param scope the queue, or an empty string for all - * @return true while paused - */ - public boolean isPaused(String scope) { - return pausedScopes.contains(scope == null ? "" : scope); - } - - @Override - public boolean isActive() { - return active.get(); - } - - @Override - public void close() { - active.set(false); - } - - /** Adapts the transport settlement SPI onto the one-terminal-call controller. */ - private final class ControllerBackedSettlement implements TransportSettlement { - - private final RabbitSettlementController controller; - - private ControllerBackedSettlement(long deliveryTag) { - this.controller = new RabbitSettlementController(deliveryTag, operations); - } - - private boolean wasSettled() { - return controller.isSettled(); - } - - @Override - public CompletionStage acknowledge() { - return controller.ack(); - } - - @Override - public CompletionStage requeue(Duration delay) { - if (!shutdown.mayCreateRetryAttempt()) { - return CompletableFuture.completedFuture( - new SettlementResult( - SettlementCompletion.UNKNOWN, - SettlementEvidence.unknown(), - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "RABBIT_DRAINING", - "no retry attempt is created while the consumer is draining")))); - } - return controller.retry(delay); - } - - @Override - public CompletionStage discard() { - return controller.reject( - FailureDescriptor.of( - FailureCategory.PERMANENT_BUSINESS, "RABBIT_DISCARDED", "the message was discarded")); - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeadLetterPublisher.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeadLetterPublisher.java deleted file mode 100644 index e534a91..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeadLetterPublisher.java +++ /dev/null @@ -1,130 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.delivery.MessageDelivery; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; -import dev.caskeleton.messaging.policy.DeadLetterResult; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.SourceSettlement; -import dev.caskeleton.messaging.schema.EncodedMessage; -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletionStage; - -/** - * The RabbitMQ entry point to dead lettering. - * - *

Delegates to the shared orchestrator, exactly as the Kafka adapter does, even though RabbitMQ - * has native dead-lettering available. The native path is only taken where {@link - * RabbitNativeDeadLetterCapability} has verified a queue is bound to the dead-letter exchange — - * everywhere else the broker would discard the message silently while the reject still succeeded, - * and the platform's publish-then-settle ordering is what prevents that. - * - *

Which path is taken is decided here rather than by each call site, so a handler cannot - * accidentally reject a message on a destination whose dead-letter exchange goes nowhere. - */ -public final class RabbitDeadLetterPublisher { - - private final DeadLetterOrchestrator orchestrator; - private final RabbitNativeDeadLetterCapability nativeCapability; - private final NativeReject nativeReject; - - /** - * Creates a Rabbit dead letter publisher. - * - * @param orchestrator the shared dead letter orchestrator - * @param nativeCapability decides whether the broker's own dead-lettering may be trusted - * @param nativeReject performs a broker-side reject - */ - public RabbitDeadLetterPublisher( - DeadLetterOrchestrator orchestrator, - RabbitNativeDeadLetterCapability nativeCapability, - NativeReject nativeReject) { - this.orchestrator = Objects.requireNonNull(orchestrator, "orchestrator must not be null"); - this.nativeCapability = Objects.requireNonNull(nativeCapability, "nativeCapability required"); - this.nativeReject = Objects.requireNonNull(nativeReject, "nativeReject must not be null"); - } - - /** - * Dead letters one delivery, by whichever path is safe for this destination. - * - * @param profile the source destination profile - * @param delivery the failed delivery - * @param failure the sanitized failure - * @param deliveryTag the AMQP delivery tag to reject, when taking the native path - * @param settlement the source settlement callback - * @return a stage completing with the dead letter outcome - */ - public CompletionStage deadLetter( - DestinationProfile profile, - MessageDelivery delivery, - FailureDescriptor failure, - long deliveryTag, - SourceSettlement settlement) { - Objects.requireNonNull(profile, "profile must not be null"); - Objects.requireNonNull(delivery, "delivery must not be null"); - Objects.requireNonNull(failure, "failure must not be null"); - - if (nativeCapability.requiresPlatformPublish()) { - return orchestrator.deadLetter(profile, delivery, failure, settlement); - } - // The broker moves the message and settles the source in one operation. Safe only because the - // capability check confirmed a queue is bound to the dead letter exchange. - return nativeReject.reject(deliveryTag).thenApply(moved -> nativelyRouted()); - } - - /** - * Describes what a broker-side reject actually proves. - * - *

{@link ConfirmationLevel#BROKER_ACK}, not replication. RabbitMQ issues no publisher confirm - * for the internal hop to the dead-letter exchange, so the evidence available is that the broker - * accepted the reject — reporting it as a replication acknowledgement would claim a guarantee no - * part of this path provides. - * - * @return the dead letter outcome for the native path - */ - private static DeadLetterResult nativelyRouted() { - return new DeadLetterResult( - new PublishResult( - PublishCompletion.CONFIRMED, - new PublishEvidence( - true, TransmissionEvidence.TRANSMITTED, true, ConfirmationLevel.BROKER_ACK), - RoutingOutcome.ROUTED, - Optional.empty(), - 1, - Duration.ZERO, - Optional.empty()), - // The reject settles the source as part of the same operation, so there is no window in - // which the message is neither parked nor deliverable. - true); - } - - /** - * Returns why the platform is publishing rather than letting the broker route. - * - * @return the sanitized reason, empty when the native path is in use - */ - public String pathExplanation() { - return nativeCapability.reasonForPlatformPublish(); - } - - /** A broker-side reject that routes the message to the dead letter exchange. */ - @FunctionalInterface - public interface NativeReject { - - /** - * Rejects one delivery without requeueing it. - * - * @param deliveryTag the AMQP delivery tag - * @return a stage completing once the broker has taken the reject - */ - CompletionStage reject(long deliveryTag); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeliveryMapper.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeliveryMapper.java deleted file mode 100644 index 0572c18..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitDeliveryMapper.java +++ /dev/null @@ -1,195 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.CausationId; -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.CorrelationId; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TenantContext; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.delivery.DeliveryMetadata; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessageValidationException; -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.schema.EncodedMessage; -import java.time.Instant; -import java.util.Objects; -import java.util.Optional; -import java.util.UUID; -import org.springframework.amqp.core.Message; -import org.springframework.amqp.core.MessageProperties; - -/** - * Rebuilds a platform envelope from an AMQP message. - * - *

The payload stays encoded so a codec failure is classified by the platform and parked, rather - * than surfacing as an AMQP listener exception the container would requeue in a hot loop. - * - *

The delivery attempt comes from the platform's own retry header rather than from AMQP's {@code - * redelivered} flag. That flag only says "this message was delivered before", not how many times, - * and it resets when a message travels through a retry queue. - */ -public final class RabbitDeliveryMapper { - - private final RabbitHeaderMapper headerMapper; - - /** Creates a mapper with the default header mapper. */ - public RabbitDeliveryMapper() { - this(new RabbitHeaderMapper()); - } - - /** - * Creates a mapper with an explicit header mapper. - * - * @param headerMapper the header mapper - */ - public RabbitDeliveryMapper(RabbitHeaderMapper headerMapper) { - this.headerMapper = Objects.requireNonNull(headerMapper, "headerMapper must not be null"); - } - - /** - * Converts an AMQP message into an envelope. - * - * @param message the AMQP message - * @return the still-encoded envelope - */ - public MessageEnvelope toEnvelope(Message message) { - Objects.requireNonNull(message, "message must not be null"); - MessageProperties properties = message.getMessageProperties(); - - ContentType contentType = - new ContentType( - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.CONTENT_TYPE)) - .or(() -> Optional.ofNullable(properties.getContentType())) - .orElse(ContentType.JSON.value())); - - byte[] payload = message.getBody() == null ? new byte[0] : message.getBody(); - Instant producedAt = instant(properties, ReservedHeaders.PRODUCED_AT); - - return new MessageEnvelope<>( - new MessageId(uuid(properties, ReservedHeaders.MESSAGE_ID)), - new MessageType(required(properties, ReservedHeaders.MESSAGE_TYPE)), - new SchemaVersion(intValue(properties, ReservedHeaders.SCHEMA_VERSION)), - producedAt, - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.OCCURRED_AT)) - .map(Instant::parse), - new ProducerId( - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.PRODUCER)) - .orElse("unknown")), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.CORRELATION_ID)) - .map(CorrelationId::new), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.CAUSATION_ID)) - .map(value -> new CausationId(new MessageId(UUID.fromString(value)))), - contentType, - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.PARTITION_KEY)), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.ORDERING_KEY)), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.TENANT)) - .map(TenantContext::new), - new TraceContext( - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.TRACEPARENT)), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.TRACESTATE)), - Optional.ofNullable(headerMapper.value(properties, ReservedHeaders.BAGGAGE))), - headerMapper.carriedForward(properties), - new EncodedMessage(payload, contentType, Optional.empty())); - } - - /** - * Builds the delivery metadata for an AMQP message. - * - * @param destination the logical destination - * @param message the AMQP message - * @param receivedAt when the consumer received it - * @return the delivery metadata - */ - public DeliveryMetadata toMetadata( - DestinationName destination, Message message, Instant receivedAt) { - MessageProperties properties = message.getMessageProperties(); - int attempt = attemptOf(properties); - - return new DeliveryMetadata( - destination, - attempt, - attempt > 1, - Optional.of( - new RabbitPublishReference( - Optional.ofNullable(properties.getReceivedExchange()).orElse(""), - Optional.ofNullable(properties.getReceivedRoutingKey()).orElse(""), - Math.max(0, properties.getDeliveryTag()))), - Optional.ofNullable(properties.getConsumerQueue()), - Optional.ofNullable(properties.getConsumerTag()), - receivedAt); - } - - /** - * Returns the attempt a message carries, counting the first delivery as one. - * - * @param properties the message properties - * @return the attempt number - */ - public int attemptOf(MessageProperties properties) { - String value = headerMapper.value(properties, ReservedHeaders.RETRY_ATTEMPT); - if (value == null) { - return 1; - } - try { - int attempt = Integer.parseInt(value); - if (attempt < 1) { - throw new NumberFormatException("attempt " + attempt); - } - return attempt; - } catch (NumberFormatException malformed) { - // Fail closed. Resetting to 1 restarts the retry budget on every redelivery, so a message - // that should have been dead-lettered is retried forever — and the header is - // caller-influenced, which turns "unreadable" into a way to defeat the cap. - throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( - "RETRY_ATTEMPT_MALFORMED", - "the retry attempt header is not a positive integer; the message is quarantined rather" - + " than restarting its retry budget"); - } - } - - private String required(MessageProperties properties, String name) { - String value = headerMapper.value(properties, name); - if (value == null || value.isBlank()) { - throw new MessageValidationException( - "RABBIT_HEADER_MISSING", "message is missing the required header " + name); - } - return value; - } - - private UUID uuid(MessageProperties properties, String name) { - try { - return UUID.fromString(required(properties, name)); - } catch (IllegalArgumentException exception) { - throw new MessageValidationException( - "RABBIT_HEADER_MALFORMED", "header " + name + " is not a UUID", exception); - } - } - - private int intValue(MessageProperties properties, String name) { - try { - return Integer.parseInt(required(properties, name)); - } catch (NumberFormatException exception) { - throw new MessageValidationException( - "RABBIT_HEADER_MALFORMED", "header " + name + " is not an integer", exception); - } - } - - private Instant instant(MessageProperties properties, String name) { - String value = headerMapper.value(properties, name); - if (value == null || value.isBlank()) { - return Optional.ofNullable(properties.getTimestamp()) - .map(java.util.Date::toInstant) - .orElse(Instant.EPOCH); - } - try { - return Instant.parse(value); - } catch (java.time.format.DateTimeParseException exception) { - throw new MessageValidationException( - "RABBIT_HEADER_MALFORMED", "header " + name + " is not an instant", exception); - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitHeaderMapper.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitHeaderMapper.java deleted file mode 100644 index dc5124c..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitHeaderMapper.java +++ /dev/null @@ -1,180 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.header.CanonicalEnvelopeHeaders; -import dev.caskeleton.messaging.api.header.HeaderName; -import dev.caskeleton.messaging.api.header.HeaderValue; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Objects; -import org.springframework.amqp.core.MessageProperties; - -/** - * Maps envelope identity onto AMQP message properties and back. - * - *

Identity goes in headers, not in the payload, so a consumer can classify and park a message it - * cannot deserialize. The AMQP {@code message-id} and {@code correlation-id} properties are also - * populated, because management UIs and tracing tools read those and not the platform's headers — - * but the platform's own headers stay authoritative, since AMQP's properties have no schema version - * or producer field. - */ -public final class RabbitHeaderMapper { - - /** - * Writes an envelope's identity into AMQP message properties. - * - * @param envelope the envelope being published - * @return the message properties - */ - public MessageProperties toProperties(MessageEnvelope envelope) { - Objects.requireNonNull(envelope, "envelope must not be null"); - MessageProperties properties = new MessageProperties(); - - properties.setMessageId(envelope.messageId().value().toString()); - properties.setContentType(envelope.contentType().value()); - properties.setTimestamp(java.util.Date.from(envelope.producedAt())); - properties.setDeliveryMode(org.springframework.amqp.core.MessageDeliveryMode.PERSISTENT); - - setHeader(properties, ReservedHeaders.MESSAGE_ID, envelope.messageId().value().toString()); - setHeader(properties, ReservedHeaders.MESSAGE_TYPE, envelope.messageType().value()); - setHeader( - properties, - ReservedHeaders.SCHEMA_VERSION, - Integer.toString(envelope.schemaVersion().value())); - setHeader(properties, ReservedHeaders.PRODUCER, envelope.producer().value()); - setHeader(properties, ReservedHeaders.PRODUCED_AT, envelope.producedAt().toString()); - setHeader(properties, ReservedHeaders.CONTENT_TYPE, envelope.contentType().value()); - - envelope - .occurredAt() - .ifPresent(value -> setHeader(properties, ReservedHeaders.OCCURRED_AT, value.toString())); - envelope - .correlationId() - .ifPresent( - value -> { - properties.setCorrelationId(value.value()); - setHeader(properties, ReservedHeaders.CORRELATION_ID, value.value()); - }); - envelope - .causationId() - .ifPresent( - value -> - setHeader( - properties, ReservedHeaders.CAUSATION_ID, value.value().value().toString())); - envelope - .partitionKey() - .ifPresent(value -> setHeader(properties, ReservedHeaders.PARTITION_KEY, value)); - envelope - .orderingKey() - .ifPresent(value -> setHeader(properties, ReservedHeaders.ORDERING_KEY, value)); - // Written for the same reason as on the other broker: an AMQP message carries no ambient - // tenant, so a tenant left off the wire is a tenant the consumed envelope does not have. - envelope - .tenantContext() - .ifPresent(value -> setHeader(properties, ReservedHeaders.TENANT, value.tenantId())); - envelope - .traceContext() - .traceparent() - .ifPresent(value -> setHeader(properties, ReservedHeaders.TRACEPARENT, value)); - envelope - .traceContext() - .tracestate() - .ifPresent(value -> setHeader(properties, ReservedHeaders.TRACESTATE, value)); - envelope - .traceContext() - .baggage() - .ifPresent(value -> setHeader(properties, ReservedHeaders.BAGGAGE, value)); - - // The envelope's header map is written last, so an unchecked name would overwrite the canonical - // reserved value the platform just wrote. A forged `msg.id` corrupts another message's inbox - // deduplication, not merely its own delivery. - // - // Only names that restate an envelope field are refused. Retry attempt, failure and redrive - // bookkeeping have no envelope field to be read back from, so refusing them would make every - // retry-queue and dead-letter re-publish fail on the guard meant to protect identity. - envelope - .headers() - .asMap() - .forEach( - (name, value) -> { - if (CanonicalEnvelopeHeaders.restatesEnvelopeField(name.value())) { - throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( - "RESERVED_HEADER_FORGED", - "header '" - + name.value() - + "' restates a value the envelope already carries; the canonical value is" - + " written by the platform and must not be overridable by message content"); - } - setHeader(properties, name.value(), value.value()); - }); - - return properties; - } - - /** - * Reads AMQP headers back into a platform header map. - * - * @param properties the message properties - * @return the platform headers, including reserved names - */ - public MessageHeaders fromProperties(MessageProperties properties) { - Objects.requireNonNull(properties, "properties must not be null"); - Map values = new LinkedHashMap<>(); - properties - .getHeaders() - .forEach( - (name, value) -> { - if (value != null) { - values.put(new HeaderName(name), new HeaderValue(String.valueOf(value))); - } - }); - return MessageHeaders.platform(values); - } - - /** - * Reads back the headers a rebuilt envelope keeps. - * - *

Everything except the names that were hoisted into envelope fields: the application's own - * headers, which were dropped entirely and so never survived a broker hop, and the platform's - * retry and redrive bookkeeping, whose loss restarted the attempt counter at one on every - * redelivery and left the retry budget capping nothing. - * - *

The hoisted names are excluded rather than kept alongside their fields, because a duplicate - * that can drift from the field is how the same message ends up with two different ids depending - * on which one the reader trusts. - * - * @param properties the message properties - * @return the headers to carry on the rebuilt envelope - */ - public MessageHeaders carriedForward(MessageProperties properties) { - Objects.requireNonNull(properties, "properties must not be null"); - Map values = new LinkedHashMap<>(); - properties - .getHeaders() - .forEach( - (name, value) -> { - if (value != null && !CanonicalEnvelopeHeaders.restatesEnvelopeField(name)) { - values.put(new HeaderName(name), new HeaderValue(String.valueOf(value))); - } - }); - return MessageHeaders.platform(values); - } - - /** - * Reads one header value. - * - * @param properties the message properties - * @param name the header name - * @return the value, or null when absent - */ - public String value(MessageProperties properties, String name) { - Object value = properties.getHeaders().get(name); - return value == null ? null : String.valueOf(value); - } - - private static void setHeader(MessageProperties properties, String name, String value) { - properties.setHeader(name, value); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitMessagingTransport.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitMessagingTransport.java deleted file mode 100644 index b6d5862..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitMessagingTransport.java +++ /dev/null @@ -1,286 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.destination.DestinationCapabilities; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessagingCapabilities; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessagingCapabilityUnavailableException; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.transport.MessagingTransport; -import dev.caskeleton.messaging.transport.TransportConsumerRegistration; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import dev.caskeleton.messaging.transport.TransportPublishResult; -import java.time.Duration; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.Executors; -import java.util.concurrent.ScheduledExecutorService; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicBoolean; -import java.util.function.Function; - -/** - * The Stable RabbitMQ adapter. - * - *

Publishes are always sent with {@code mandatory} set and correlated through {@link - * RabbitConfirmCoordinator}, because in AMQP a confirm and a return answer different questions. The - * confirm says the exchange accepted the message; the return says no queue was bound to receive it. - * An adapter that completed on the confirm alone would report success for a message the broker - * discarded, which is the single most damaging thing a Rabbit adapter can get wrong. - * - *

Payload size and shutdown state are checked before anything reaches the channel, so both fail - * with {@code NOT_TRANSMITTED} evidence rather than as a broker error whose ambiguity the caller - * then has to reason about. - */ -public final class RabbitMessagingTransport implements MessagingTransport { - - private static final MessagingCapabilities CAPABILITIES = - new MessagingCapabilities( - true, true, true, false, false, false, false, true, false, false, true, true); - - private final String brokerName; - private final long generation; - private final RabbitChannelPublisher channel; - private final RabbitPublishMapper mapper; - private final RabbitConfirmCoordinator coordinator; - private final Function consumerFactory; - private final Duration confirmTimeout; - private final ScheduledExecutorService confirmTimeouts; - private final AtomicBoolean closed = new AtomicBoolean(); - - /** - * Creates a publish-only transport. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param channel the channel publish operation - */ - public RabbitMessagingTransport( - String brokerName, long generation, RabbitChannelPublisher channel) { - this(brokerName, generation, channel, Duration.ofSeconds(5)); - } - - /** Creates a publish-only transport with an explicit confirm timeout. */ - public RabbitMessagingTransport( - String brokerName, long generation, RabbitChannelPublisher channel, Duration confirmTimeout) { - this( - brokerName, - generation, - channel, - spec -> { - throw new MessagingCapabilityUnavailableException( - "RABBIT_CONSUMER_NOT_CONFIGURED", - "this Rabbit transport was created without a consumer factory"); - }, - confirmTimeout); - } - - /** - * Creates a transport with a consumer factory. - * - * @param brokerName the logical broker name - * @param generation the runtime generation - * @param channel the channel publish operation - * @param consumerFactory builds a consumer registration for a spec - */ - public RabbitMessagingTransport( - String brokerName, - long generation, - RabbitChannelPublisher channel, - Function consumerFactory) { - this(brokerName, generation, channel, consumerFactory, Duration.ofSeconds(5)); - } - - /** Creates a transport with explicit consumer factory and confirm deadline. */ - public RabbitMessagingTransport( - String brokerName, - long generation, - RabbitChannelPublisher channel, - Function consumerFactory, - Duration confirmTimeout) { - this.brokerName = Objects.requireNonNull(brokerName, "brokerName must not be null"); - this.generation = generation; - this.channel = Objects.requireNonNull(channel, "channel must not be null"); - this.consumerFactory = Objects.requireNonNull(consumerFactory, "consumerFactory is required"); - this.confirmTimeout = Objects.requireNonNull(confirmTimeout, "confirmTimeout must not be null"); - if (confirmTimeout.isNegative() || confirmTimeout.isZero()) { - throw new IllegalArgumentException("confirmTimeout must be positive"); - } - this.mapper = new RabbitPublishMapper(); - this.coordinator = new RabbitConfirmCoordinator(); - this.confirmTimeouts = - Executors.newSingleThreadScheduledExecutor( - runnable -> { - Thread thread = new Thread(runnable, "messaging-rabbit-confirm-timeouts"); - thread.setDaemon(true); - return thread; - }); - } - - @Override - public CompletionStage publish(TransportPublishRequest request) { - Objects.requireNonNull(request, "request must not be null"); - - if (closed.get()) { - return completed( - rejectedLocally("RABBIT_TRANSPORT_CLOSED", "the transport is shutting down")); - } - int size = request.envelope().payload().size(); - int limit = request.profile().payload().maxBytes(); - if (size > limit) { - return completed( - rejectedLocally( - "PAYLOAD_TOO_LARGE", "encoded payload is " + size + " bytes, limit is " + limit)); - } - - String exchange = mapper.exchange(request.profile()); - String routingKey = mapper.routingKey(request.profile()); - - // Register before publishing. A confirm can land on the connection thread while basicPublish is - // still returning; registering afterwards would drop it and leave the caller waiting forever. - long sequence = channel.nextPublishSequence(); - CompletionStage pending = - coordinator - .awaiting(sequence, exchange, routingKey, request.profile().producer().confirmation()) - .thenApply(TransportPublishResult::new); - var timeout = - confirmTimeouts.schedule( - () -> coordinator.timedOut(sequence, confirmTimeout), - confirmTimeout.toNanos(), - TimeUnit.NANOSECONDS); - pending.whenComplete((result, failure) -> timeout.cancel(false)); - - try { - channel.publish(sequence, exchange, routingKey, mapper.toMessage(request), true); - } catch (RuntimeException sendFailed) { - // A synchronous failure from basicPublish used to propagate to the caller as a thrown - // exception while the pending entry stayed in the map: the caller got an exception instead - // of the stage the contract promises, and the entry waited for a confirm that could never - // arrive. The classifier decides whether the frame can have left; anything it cannot rule - // out is ambiguous rather than rejected. - coordinator.sendFailed( - sequence, new RabbitPublishFailureClassifier().classify(sendFailed, true, Duration.ZERO)); - } - return pending; - } - - /** - * Records a broker confirm. Called from the channel's confirm callback. - * - * @param sequence the publish sequence number - * @param acknowledged true for an ack, false for a nack - * @param elapsed how long the publish took - */ - public void onConfirm(long sequence, boolean acknowledged, Duration elapsed) { - onConfirm(sequence, false, acknowledged, elapsed); - } - - /** - * Records a broker confirm carrying the {@code multiple} flag. - * - *

The flag was dropped, so a broker confirming a range resolved one publish and left every - * earlier one pending forever. - * - * @param multiple true when every sequence up to and including this one is confirmed - */ - public void onConfirm(long sequence, boolean multiple, boolean acknowledged, Duration elapsed) { - coordinator.confirmed(sequence, multiple, acknowledged, elapsed); - } - - /** - * Records an unroutable return. Called from the channel's return callback. - * - * @param sequence the publish sequence number - */ - public void onReturn(long sequence) { - coordinator.returned(sequence); - } - - /** - * Records that a confirm never arrived. - * - * @param sequence the publish sequence number - * @param elapsed how long was waited - */ - public void onConfirmTimeout(long sequence, Duration elapsed) { - coordinator.timedOut(sequence, elapsed); - } - - /** - * Returns how many publishes are awaiting a confirm. - * - * @return the pending count - */ - public int pendingConfirms() { - return coordinator.pendingCount(); - } - - @Override - public TransportConsumerRegistration register(TransportConsumerSpec spec) { - Objects.requireNonNull(spec, "spec must not be null"); - if (closed.get()) { - throw new MessagingCapabilityUnavailableException( - "RABBIT_TRANSPORT_CLOSED", "the transport is shutting down"); - } - return consumerFactory.apply(spec); - } - - @Override - public DestinationCapabilities capabilities(DestinationName destination) { - return new DestinationCapabilities(destination, brokerName, CAPABILITIES); - } - - @Override - public String brokerName() { - return brokerName; - } - - @Override - public long generation() { - return generation; - } - - @Override - public void close() { - if (!closed.compareAndSet(false, true)) { - return; - } - // Drain what is still waiting. Leaving the stages pending hangs every caller holding one, and - // a closed channel will never deliver their confirms. - coordinator.channelClosed(Duration.ZERO); - confirmTimeouts.shutdownNow(); - channel.close(); - } - - /** - * Reports whether the transport is still accepting work. - * - * @return true until close - */ - public boolean isAcceptingWork() { - return !closed.get(); - } - - private static TransportPublishResult rejectedLocally(String code, String message) { - return new TransportPublishResult( - new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of(FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message)))); - } - - private static CompletionStage completed(TransportPublishResult result) { - return CompletableFuture.completedFuture(result); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitNativeDeadLetterCapability.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitNativeDeadLetterCapability.java deleted file mode 100644 index 84d5e88..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitNativeDeadLetterCapability.java +++ /dev/null @@ -1,70 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import java.util.Objects; - -/** - * Decides whether the broker's own dead-lettering may be used, or the platform must publish. - * - *

RabbitMQ is the one Stable broker with native dead-lettering, and using it is tempting: a - * {@code basic.reject} with {@code requeue=false} moves the message with no publish at all. The - * catch is that the move is not confirmed to the publisher. If the dead-letter exchange is - * unroutable — nobody bound a queue to it, or the binding was removed — the broker discards the - * message silently and the reject still succeeds. - * - *

So native dead-lettering is only used where the dead-letter path has been verified to have a - * bound queue. Everywhere else the platform publishes and waits for its own confirm, which is the - * same invariant every other broker gets: the source is settled only after the dead-letter - * publish is confirmed. - */ -public final class RabbitNativeDeadLetterCapability { - - private final RabbitTopologyProfile topology; - private final boolean deadLetterQueueBound; - - /** - * Creates a capability check for one destination. - * - * @param topology the declared topology - * @param deadLetterQueueBound whether a queue is verified bound to the dead letter exchange - */ - public RabbitNativeDeadLetterCapability( - RabbitTopologyProfile topology, boolean deadLetterQueueBound) { - this.topology = Objects.requireNonNull(topology, "topology must not be null"); - this.deadLetterQueueBound = deadLetterQueueBound; - } - - /** - * Reports whether a rejected message may be left to the broker. - * - * @return true only when a dead letter exchange is declared and verified bound - */ - public boolean mayUseNativeDeadLetter() { - return topology.hasNativeDeadLetter() && deadLetterQueueBound; - } - - /** - * Reports whether the platform must publish the dead letter itself. - * - * @return true when native dead-lettering cannot be trusted for this destination - */ - public boolean requiresPlatformPublish() { - return !mayUseNativeDeadLetter(); - } - - /** - * Returns why native dead-lettering is not being used, for a startup diagnostic. - * - * @return the sanitized reason, empty when native dead-lettering is in use - */ - public String reasonForPlatformPublish() { - if (mayUseNativeDeadLetter()) { - return ""; - } - if (!topology.hasNativeDeadLetter()) { - return "queue %s was declared without a dead letter exchange".formatted(topology.queue()); - } - return "no queue is bound to dead letter exchange %s, so a broker-side reject would discard " - .formatted(topology.deadLetterExchange().orElse("(none)")) - + "the message silently while still reporting success"; - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidator.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidator.java deleted file mode 100644 index 3463b5c..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidator.java +++ /dev/null @@ -1,83 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.util.Objects; - -/** - * Startup validation of the RabbitMQ Stable profile. - * - *

Confirms and returns are both required, and they answer different questions. A confirm says - * the exchange accepted the message; a return says no queue was bound to receive it. Enabling - * confirms without returns produces the worst possible outcome: a publish that reports success - * while the message was silently discarded. - * - *

Durable work queues default to quorum queues because classic mirrored queues can lose - * acknowledged messages during a partition — which is precisely the guarantee a durable work queue - * exists to provide. - */ -public final class RabbitProfileValidator { - - /** - * Validates one RabbitMQ broker profile. - * - * @param profile the profile to validate - * @throws IllegalArgumentException when the profile cannot honour its declared guarantees - */ - public void validate(RabbitBrokerProfile profile) { - Objects.requireNonNull(profile, "profile must not be null"); - - if (profile.stable() && !profile.publisherConfirms()) { - throw new IllegalArgumentException( - "stable Rabbit producer requires publisher confirms: " + profile.broker()); - } - if (profile.stable() && !profile.publisherReturns()) { - throw new IllegalArgumentException( - "stable Rabbit producer requires publisher returns: " + profile.broker()); - } - if (profile.stable() && !profile.mandatory()) { - throw new IllegalArgumentException( - "stable Rabbit producer requires mandatory routing: " + profile.broker()); - } - if (profile.autoAck()) { - throw new IllegalArgumentException( - "consumer auto ack is forbidden; the platform acknowledges after handler success: " - + profile.broker()); - } - if (profile.prefetch() < 1) { - throw new IllegalArgumentException("prefetch must be at least 1: " + profile.broker()); - } - if (profile.confirmTimeout().isNegative() || profile.confirmTimeout().isZero()) { - throw new IllegalArgumentException("confirm timeout must be positive: " + profile.broker()); - } - if (profile.production() && !profile.tlsEnabled()) { - throw new IllegalArgumentException( - "a production Rabbit connection requires TLS: " + profile.broker()); - } - if (profile.production() && !profile.authenticationEnabled()) { - throw new IllegalArgumentException( - "a production Rabbit connection requires broker authentication: " + profile.broker()); - } - } - - /** - * Validates a destination against the broker profile. - * - * @param destination the destination profile - * @param broker the broker profile - * @throws IllegalArgumentException when the destination cannot be served safely - */ - public void validateDestination(DestinationProfile destination, RabbitBrokerProfile broker) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(broker, "broker must not be null"); - - if (destination.kind() == DestinationKind.WORK_QUEUE && !broker.quorumQueues()) { - throw new IllegalArgumentException( - "a durable Rabbit work queue must use a quorum queue: " + destination.name().value()); - } - if (destination.physical().queue().isEmpty() && destination.physical().exchange().isEmpty()) { - throw new IllegalArgumentException( - "a Rabbit destination requires an exchange or a queue: " + destination.name().value()); - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishFailureClassifier.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishFailureClassifier.java deleted file mode 100644 index c9408b9..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishFailureClassifier.java +++ /dev/null @@ -1,133 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.io.IOException; -import java.time.Duration; -import java.util.Optional; - -/** - * Turns an AMQP publish failure into a completion the caller can act on. - * - *

The distinction that matters is where the failure happened. A channel-level exception raised - * before {@code basicPublish} returned means nothing was written: rejected, with no - * ambiguity. A connection that drops after the frame was written proves nothing — the - * broker may have the message and only the confirm was lost. - * - *

Unknown failures default to ambiguous. Guessing "rejected" on an unrecognised error is what - * turns one lost confirm into two of the same order, and the cost of an unnecessary ambiguous is a - * retry the inbox absorbs. - */ -public final class RabbitPublishFailureClassifier { - - /** - * Classifies a failure raised while publishing. - * - * @param error the throwable the channel reported - * @param framePossiblyWritten whether the publish frame may already have reached the broker - * @param elapsed how long the attempt took - * @return the publish outcome with its evidence - */ - public PublishResult classify(Throwable error, boolean framePossiblyWritten, Duration elapsed) { - Throwable cause = unwrap(error); - - if (!framePossiblyWritten) { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - categoryOf(cause), - "RABBIT_PUBLISH_REJECTED", - "the channel refused the publish before writing it: " + name(cause)))); - } - - return new PublishResult( - PublishCompletion.AMBIGUOUS, - new PublishEvidence( - true, TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "RABBIT_CONFIRM_LOST", - "the frame was written but no confirm arrived: " + name(cause)))); - } - - /** - * Classifies a broker nack, which is always a definitive rejection. - * - * @param elapsed how long the attempt took - * @return the publish outcome - */ - public PublishResult nacked(Duration elapsed) { - return new PublishResult( - PublishCompletion.REJECTED, - // A nack means the broker took the frame and decided against it, so the bytes were - // transmitted even though nothing was stored. - new PublishEvidence(true, TransmissionEvidence.TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - elapsed, - Optional.of( - FailureDescriptor.of( - FailureCategory.TRANSIENT_INFRASTRUCTURE, - "RABBIT_NACKED", - "the broker negatively acknowledged the publish"))); - } - - private static FailureCategory categoryOf(Throwable cause) { - if (cause instanceof com.rabbitmq.client.PossibleAuthenticationFailureException) { - return FailureCategory.AUTHENTICATION; - } - if (cause instanceof com.rabbitmq.client.ShutdownSignalException shutdown) { - int replyCode = replyCode(shutdown.getReason()); - if (replyCode == 403) { - return FailureCategory.AUTHORIZATION; - } - if (replyCode >= 500) { - return FailureCategory.TRANSIENT_INFRASTRUCTURE; - } - return FailureCategory.PERMANENT_BUSINESS; - } - if (cause instanceof IOException) { - return FailureCategory.TRANSIENT_INFRASTRUCTURE; - } - return FailureCategory.PERMANENT_BUSINESS; - } - - private static int replyCode(com.rabbitmq.client.Method reason) { - if (reason instanceof com.rabbitmq.client.AMQP.Channel.Close close) { - return close.getReplyCode(); - } - if (reason instanceof com.rabbitmq.client.AMQP.Connection.Close close) { - return close.getReplyCode(); - } - return -1; - } - - private static Throwable unwrap(Throwable error) { - if (error instanceof java.util.concurrent.CompletionException && error.getCause() != null) { - return error.getCause(); - } - return error; - } - - private static String name(Throwable cause) { - return cause == null ? "unknown" : cause.getClass().getSimpleName(); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishMapper.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishMapper.java deleted file mode 100644 index ec03aae..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishMapper.java +++ /dev/null @@ -1,72 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import java.util.Objects; -import org.springframework.amqp.core.Message; - -/** - * Builds AMQP messages from platform publish requests. - * - *

The exchange and routing key come from the destination profile, never from the caller. A - * publisher that could choose its own routing key would be able to reach queues the destination's - * access policy never granted. - */ -public final class RabbitPublishMapper { - - private final RabbitHeaderMapper headerMapper; - - /** Creates a mapper with the default header mapper. */ - public RabbitPublishMapper() { - this(new RabbitHeaderMapper()); - } - - /** - * Creates a mapper with an explicit header mapper. - * - * @param headerMapper the header mapper - */ - public RabbitPublishMapper(RabbitHeaderMapper headerMapper) { - this.headerMapper = Objects.requireNonNull(headerMapper, "headerMapper must not be null"); - } - - /** - * Converts a publish request into an AMQP message. - * - * @param request the publish request - * @return the AMQP message - */ - public Message toMessage(TransportPublishRequest request) { - Objects.requireNonNull(request, "request must not be null"); - return new Message( - request.envelope().payload().bytes(), headerMapper.toProperties(request.envelope())); - } - - /** - * Returns the exchange a destination publishes to. - * - * @param profile the destination profile - * @return the exchange name - */ - public String exchange(DestinationProfile profile) { - return profile - .physical() - .exchange() - .orElseThrow( - () -> - new MessagingConfigurationException( - "RABBIT_EXCHANGE_MISSING", - "destination has no Rabbit exchange: " + profile.name().value())); - } - - /** - * Returns the routing key a destination publishes with. - * - * @param profile the destination profile - * @return the routing key - */ - public String routingKey(DestinationProfile profile) { - return profile.physical().routingKey().orElse(""); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishReference.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishReference.java deleted file mode 100644 index aee6d79..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitPublishReference.java +++ /dev/null @@ -1,37 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.publish.BrokerPosition; -import java.util.Map; -import java.util.Objects; - -/** - * A RabbitMQ exchange, routing key, and publish sequence coordinate. - * - * @param exchange the exchange published to - * @param routingKey the routing key used - * @param sequence the channel publish sequence number - */ -public record RabbitPublishReference(String exchange, String routingKey, long sequence) - implements BrokerPosition { - - public RabbitPublishReference { - Objects.requireNonNull(exchange, "exchange must not be null"); - Objects.requireNonNull(routingKey, "routingKey must not be null"); - if (sequence < 0) { - throw new IllegalArgumentException("sequence must not be negative"); - } - } - - @Override - public String broker() { - return "rabbitmq"; - } - - @Override - public Map diagnosticAttributes() { - return Map.of( - "exchange", exchange, - "routingKey", routingKey, - "sequence", Long.toString(sequence)); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRequestReply.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRequestReply.java deleted file mode 100644 index 2854b41..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRequestReply.java +++ /dev/null @@ -1,32 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.schema.EncodedMessage; -import java.time.Duration; -import java.util.concurrent.CompletionStage; - -/** - * The M2 request-reply capability. - * - *

Deliberately limited. Request-reply over a broker is a synchronous call wearing an - * asynchronous costume: it holds a correlation entry and a caller for the whole round trip, and it - * fails in ways a plain publish does not — a lost reply is indistinguishable from a slow one. It is - * offered with a mandatory deadline and no automatic retry, because retrying a request whose reply - * was merely late duplicates the work on the other side. - */ -public interface RabbitRequestReply { - - /** - * Sends a request and awaits its correlated reply. - * - * @param destination the request destination - * @param request the encoded request envelope - * @param timeout how long to wait for the reply - * @return a stage completing with the reply, or failing on timeout - */ - CompletionStage> request( - MessageDestination destination, - MessageEnvelope request, - Duration timeout); -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRetryQueueTopology.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRetryQueueTopology.java deleted file mode 100644 index ca42dfb..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitRetryQueueTopology.java +++ /dev/null @@ -1,57 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import java.time.Duration; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Objects; - -/** - * Describes the delayed retry queue a destination uses. - * - *

RabbitMQ has no native per-message delay in the core broker, so a retry queue is a queue with - * a message TTL whose dead letter exchange points back at the work queue: the message sits until - * the TTL expires and is then re-routed. Modelling this explicitly, rather than hiding it behind a - * plugin, keeps the behaviour reviewable — including its main caveat, that TTL expiry is evaluated - * at the head of the queue, so mixed delays in one retry queue do not expire independently. - * - * @param retryQueue the queue messages wait in - * @param deadLetterExchange the exchange expired messages are routed back through - * @param deadLetterRoutingKey the routing key expired messages are re-published with - * @param messageTtl how long messages wait - */ -public record RabbitRetryQueueTopology( - String retryQueue, - String deadLetterExchange, - String deadLetterRoutingKey, - Duration messageTtl) { - - public RabbitRetryQueueTopology { - Objects.requireNonNull(messageTtl, "messageTtl must not be null"); - if (retryQueue == null || retryQueue.isBlank()) { - throw new IllegalArgumentException("retryQueue must not be blank"); - } - if (deadLetterExchange == null) { - throw new IllegalArgumentException("deadLetterExchange must not be null"); - } - if (deadLetterRoutingKey == null) { - throw new IllegalArgumentException("deadLetterRoutingKey must not be null"); - } - if (messageTtl.isNegative() || messageTtl.isZero()) { - throw new IllegalArgumentException("messageTtl must be positive"); - } - } - - /** - * Returns the queue arguments this topology requires. - * - * @return the AMQP queue arguments - */ - public Map queueArguments() { - Map arguments = new LinkedHashMap<>(); - arguments.put("x-queue-type", "quorum"); - arguments.put("x-message-ttl", messageTtl.toMillis()); - arguments.put("x-dead-letter-exchange", deadLetterExchange); - arguments.put("x-dead-letter-routing-key", deadLetterRoutingKey); - return Map.copyOf(arguments); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSecurityConfigurer.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSecurityConfigurer.java deleted file mode 100644 index 9841916..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSecurityConfigurer.java +++ /dev/null @@ -1,179 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.security.BrokerCredentialProfile; -import dev.caskeleton.messaging.security.BrokerSecurityProfile; -import dev.caskeleton.messaging.security.BrokerTlsPolicy; -import dev.caskeleton.messaging.security.CredentialRuntime; -import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; -import java.time.Instant; -import java.util.List; -import java.util.Objects; - -/** - * Builds the AMQP connection settings from a validated security profile. - * - *

Credentials are resolved per connection attempt rather than cached in the factory. RabbitMQ - * client connections are long-lived and reconnect on their own, so a factory holding a credential - * from startup will happily reconnect with a revoked one for as long as the process runs — the - * reconnect is exactly the moment a rotated credential should take effect. - * - *

The password is returned inside a short-lived {@link AmqpCredentials} rather than assigned to - * a connection factory field, so the caller controls when it is cleared. - */ -public final class RabbitSecurityConfigurer { - - private final CredentialRuntimeRegistry credentials; - private final BrokerTlsPolicy tlsPolicy; - - /** - * Creates a configurer. - * - * @param credentials resolves and rotates credential material - * @param tlsPolicy validates the transport security posture - */ - public RabbitSecurityConfigurer( - CredentialRuntimeRegistry credentials, BrokerTlsPolicy tlsPolicy) { - this.credentials = Objects.requireNonNull(credentials, "credentials must not be null"); - this.tlsPolicy = Objects.requireNonNull(tlsPolicy, "tlsPolicy must not be null"); - } - - /** - * Resolves the connection credentials for one role. - * - * @param profile the broker security profile - * @param credential the role credential to configure - * @param enabledProtocols the TLS protocol versions to enable - * @param now the current instant - * @return the AMQP connection credentials - */ - public AmqpCredentials configure( - BrokerSecurityProfile profile, - BrokerCredentialProfile credential, - List enabledProtocols, - Instant now) { - Objects.requireNonNull(profile, "profile must not be null"); - Objects.requireNonNull(credential, "credential must not be null"); - Objects.requireNonNull(enabledProtocols, "enabledProtocols must not be null"); - - tlsPolicy.validate(profile, enabledProtocols); - - return switch (credential) { - case BrokerCredentialProfile.UsernamePassword plain -> { - CredentialRuntime resolved = credentials.resolve(plain.credentialId(), now); - yield new AmqpCredentials( - "PLAIN", plain.credentialId(), resolved.material(), profile.tlsEnabled()); - } - case BrokerCredentialProfile.MutualTls mutual -> { - // The certificate is the credential; there is no password to carry. - credentials.resolve(mutual.credentialId(), now); - yield new AmqpCredentials("EXTERNAL", mutual.credentialId(), new char[0], true); - } - case BrokerCredentialProfile.OAuth2 oauth -> { - CredentialRuntime resolved = credentials.resolve(oauth.credentialId(), now); - // RabbitMQ's OAuth 2 plugin takes the token in the password field of a PLAIN exchange. - yield new AmqpCredentials( - "PLAIN", oauth.credentialId(), resolved.material(), profile.tlsEnabled()); - } - case BrokerCredentialProfile.SaslScram scram -> { - CredentialRuntime resolved = credentials.resolve(scram.credentialId(), now); - yield new AmqpCredentials( - "RABBIT-CR-DEMO", scram.credentialId(), resolved.material(), profile.tlsEnabled()); - } - case BrokerCredentialProfile.Nkey ignored -> - throw new IllegalArgumentException( - "NKey credentials are a NATS concept, not an AMQP one"); - }; - } - - /** - * One connection's resolved credentials. - * - *

A class rather than a record because the secret has to stay mutable to be cleared, and a - * record carrying a {@code char[]} would compare and hash it by identity — two credentials with - * the same material would be unequal, which is a surprise waiting in any future collection use. - * - *

{@link #clear()} overwrites the secret. The caller is expected to call it once the - * connection is established, so the material's lifetime is the handshake rather than the process. - */ - public static final class AmqpCredentials { - - private final String mechanism; - private final String username; - private final char[] secret; - private final boolean tls; - - /** - * Creates connection credentials. - * - * @param mechanism the SASL mechanism name - * @param username the AMQP username - * @param secret the password or token, taken by reference and cleared by {@link #clear()} - * @param tls whether the connection uses TLS - */ - public AmqpCredentials(String mechanism, String username, char[] secret, boolean tls) { - Objects.requireNonNull(secret, "secret must not be null"); - if (mechanism == null || mechanism.isBlank()) { - throw new IllegalArgumentException("mechanism must not be blank"); - } - if (username == null || username.isBlank()) { - throw new IllegalArgumentException("username must not be blank"); - } - this.mechanism = mechanism; - this.username = username; - this.secret = secret; - this.tls = tls; - } - - /** - * Returns the SASL mechanism name. - * - * @return the mechanism - */ - public String mechanism() { - return mechanism; - } - - /** - * Returns the AMQP username. - * - * @return the username - */ - public String username() { - return username; - } - - /** - * Returns a copy of the secret. - * - * @return the password or token - */ - public char[] secret() { - return secret.clone(); - } - - /** - * Reports whether the connection uses TLS. - * - * @return true when TLS is enabled - */ - public boolean tls() { - return tls; - } - - /** Overwrites the secret in place. */ - public void clear() { - java.util.Arrays.fill(secret, '\0'); - } - - /** - * Returns a representation that names the connection without revealing it. - * - * @return a safe description - */ - @Override - public String toString() { - return "AmqpCredentials[mechanism=%s, username=%s, tls=%s]" - .formatted(mechanism, username, tls); - } - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementController.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementController.java deleted file mode 100644 index 3f0a248..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementController.java +++ /dev/null @@ -1,84 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessageSettlementException; -import dev.caskeleton.messaging.api.settlement.SettlementController; -import dev.caskeleton.messaging.api.settlement.SettlementResult; -import java.time.Duration; -import java.util.Objects; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicBoolean; - -/** - * Manual settlement for one RabbitMQ delivery. - * - *

Exactly one terminal call is allowed. A second one fails loudly rather than racing the first, - * because a duplicated {@code basic.ack} on a reused delivery tag acknowledges a different - * message — a failure mode that shows up as unexplained message loss under load, far from its - * cause. - * - *

{@code retry} does not use {@code basic.nack} with requeue. Requeueing puts the message back - * at the head of the queue with no delay, which turns a transient downstream failure into a hot - * loop; the delayed retry queue is used instead. - */ -public final class RabbitSettlementController implements SettlementController { - - private final long deliveryTag; - private final RabbitSettlementOperations operations; - private final AtomicBoolean settled = new AtomicBoolean(); - - /** - * Creates a controller for one delivery. - * - * @param deliveryTag the AMQP delivery tag - * @param operations the channel operations, invoked on the container thread - */ - public RabbitSettlementController(long deliveryTag, RabbitSettlementOperations operations) { - this.deliveryTag = deliveryTag; - this.operations = Objects.requireNonNull(operations, "operations must not be null"); - } - - @Override - public CompletionStage ack() { - return terminal(() -> operations.ack(deliveryTag)); - } - - @Override - public CompletionStage retry(Duration delay) { - Objects.requireNonNull(delay, "delay must not be null"); - return terminal(() -> operations.publishToRetryQueue(deliveryTag, delay)); - } - - @Override - public CompletionStage deadLetter(FailureDescriptor failure) { - Objects.requireNonNull(failure, "failure must not be null"); - return terminal(() -> operations.deadLetter(deliveryTag, failure)); - } - - @Override - public CompletionStage reject(FailureDescriptor failure) { - Objects.requireNonNull(failure, "failure must not be null"); - return terminal(() -> operations.discard(deliveryTag, failure)); - } - - /** - * Reports whether this delivery has already been settled. - * - * @return true once a terminal call has been made - */ - public boolean isSettled() { - return settled.get(); - } - - private CompletionStage terminal( - java.util.function.Supplier> operation) { - if (!settled.compareAndSet(false, true)) { - return CompletableFuture.failedFuture( - new MessageSettlementException( - "SETTLEMENT_ALREADY_COMPLETED", - "delivery tag " + deliveryTag + " has already been settled")); - } - return operation.get(); - } -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementOperations.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementOperations.java deleted file mode 100644 index 6f2c975..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitSettlementOperations.java +++ /dev/null @@ -1,50 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.settlement.SettlementResult; -import java.time.Duration; -import java.util.concurrent.CompletionStage; - -/** - * The channel-side operations a settlement controller may perform. - * - *

Separated from the controller so the one-terminal-call guard is testable without an AMQP - * channel, and so the controller cannot reach anything the platform has not exposed. - */ -public interface RabbitSettlementOperations { - - /** - * Acknowledges a delivery. - * - * @param deliveryTag the AMQP delivery tag - * @return a stage completing with the settlement outcome - */ - CompletionStage ack(long deliveryTag); - - /** - * Publishes to the delayed retry queue, then acknowledges the source once confirmed. - * - * @param deliveryTag the AMQP delivery tag - * @param delay how long the message should wait - * @return a stage completing with the settlement outcome - */ - CompletionStage publishToRetryQueue(long deliveryTag, Duration delay); - - /** - * Publishes to the dead letter destination, then acknowledges the source once confirmed. - * - * @param deliveryTag the AMQP delivery tag - * @param failure the sanitized failure - * @return a stage completing with the settlement outcome - */ - CompletionStage deadLetter(long deliveryTag, FailureDescriptor failure); - - /** - * Discards a delivery without requeueing or dead lettering. - * - * @param deliveryTag the AMQP delivery tag - * @param failure the sanitized failure - * @return a stage completing with the settlement outcome - */ - CompletionStage discard(long deliveryTag, FailureDescriptor failure); -} diff --git a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitTopologyProfile.java b/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitTopologyProfile.java deleted file mode 100644 index 66633bb..0000000 --- a/messaging/messaging-rabbit/src/main/java/dev/caskeleton/messaging/rabbit/RabbitTopologyProfile.java +++ /dev/null @@ -1,137 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; - -/** - * The declared shape of one RabbitMQ destination: exchange, queue, and the arguments that give the - * queue its guarantees. - * - *

Quorum queues are required for anything durable. A classic mirrored queue can lose - * acknowledged messages during a partition — the promotion of a stale mirror is exactly the case - * where "the broker confirmed it" stops being true — and RabbitMQ itself has deprecated them. - * - *

The dead-letter exchange is part of the topology rather than a runtime decision, because a - * queue's {@code x-dead-letter-exchange} can only be set at declaration. A destination that gains a - * dead-letter policy later cannot acquire one without deleting and recreating the queue, so the - * mismatch is caught here instead of surfacing when the first message needs parking. - * - * @param exchange the exchange to publish to - * @param exchangeType the exchange type - * @param routingKey the routing key - * @param queue the queue that receives the messages - * @param quorum whether the queue is a quorum queue - * @param deadLetterExchange the dead letter exchange, when the destination has one - * @param deadLetterRoutingKey the dead letter routing key, when the destination has one - * @param maxLength the queue length limit, when one is set - */ -public record RabbitTopologyProfile( - String exchange, - String exchangeType, - String routingKey, - String queue, - boolean quorum, - Optional deadLetterExchange, - Optional deadLetterRoutingKey, - Optional maxLength) { - - /** RabbitMQ's argument key for a queue's type. */ - public static final String QUEUE_TYPE = "x-queue-type"; - - /** RabbitMQ's argument key for a queue's dead letter exchange. */ - public static final String DEAD_LETTER_EXCHANGE = "x-dead-letter-exchange"; - - /** RabbitMQ's argument key for a queue's dead letter routing key. */ - public static final String DEAD_LETTER_ROUTING_KEY = "x-dead-letter-routing-key"; - - /** RabbitMQ's argument key for a queue's length limit. */ - public static final String MAX_LENGTH = "x-max-length"; - - public RabbitTopologyProfile { - Objects.requireNonNull(deadLetterExchange, "deadLetterExchange must not be null"); - Objects.requireNonNull(deadLetterRoutingKey, "deadLetterRoutingKey must not be null"); - Objects.requireNonNull(maxLength, "maxLength must not be null"); - requireText(exchange, "exchange"); - requireText(exchangeType, "exchangeType"); - requireText(routingKey, "routingKey"); - requireText(queue, "queue"); - - if (deadLetterRoutingKey.isPresent() && deadLetterExchange.isEmpty()) { - throw new IllegalArgumentException( - "a dead letter routing key without a dead letter exchange routes nowhere"); - } - if (maxLength.isPresent() && maxLength.get() < 1) { - throw new IllegalArgumentException("maxLength must be positive when set"); - } - } - - /** - * Refuses a topology that cannot back a durable destination. - * - * @param durable whether the destination promises durability - * @throws MessagingConfigurationException when a durable destination lacks a quorum queue - */ - public void requireSafeFor(boolean durable) { - if (durable && !quorum) { - throw new MessagingConfigurationException( - "CLASSIC_QUEUE_ON_DURABLE_DESTINATION", - "queue %s is not a quorum queue; a classic mirrored queue can lose acknowledged messages " - .formatted(queue) - + "when a stale mirror is promoted during a partition"); - } - } - - /** - * Returns the arguments this queue must be declared with. - * - * @return the declaration arguments - */ - public Map queueArguments() { - Map arguments = new LinkedHashMap<>(); - arguments.put(QUEUE_TYPE, quorum ? "quorum" : "classic"); - deadLetterExchange.ifPresent(value -> arguments.put(DEAD_LETTER_EXCHANGE, value)); - deadLetterRoutingKey.ifPresent(value -> arguments.put(DEAD_LETTER_ROUTING_KEY, value)); - maxLength.ifPresent(value -> arguments.put(MAX_LENGTH, value)); - return Map.copyOf(arguments); - } - - /** - * Reports whether the broker will route a rejected message itself. - * - * @return true when the queue was declared with a dead letter exchange - */ - public boolean hasNativeDeadLetter() { - return deadLetterExchange.isPresent(); - } - - /** - * Returns a durable work-queue topology with a dead letter exchange. - * - * @param exchange the exchange - * @param routingKey the routing key - * @param queue the queue - * @param deadLetterExchange the dead letter exchange - * @return the topology profile - */ - public static RabbitTopologyProfile workQueue( - String exchange, String routingKey, String queue, String deadLetterExchange) { - return new RabbitTopologyProfile( - exchange, - "direct", - routingKey, - queue, - true, - Optional.of(deadLetterExchange), - Optional.of(routingKey), - Optional.empty()); - } - - private static void requireText(String value, String field) { - if (value == null || value.isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitAdapterContractTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitAdapterContractTest.java deleted file mode 100644 index 60d8e24..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitAdapterContractTest.java +++ /dev/null @@ -1,24 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.testkit.MessagingAdapterContract; -import dev.caskeleton.messaging.testkit.MessagingAdapterHarness; -import org.junit.jupiter.api.Nested; - -/** - * Runs the shared adapter contract against the RabbitMQ adapter. - * - *

Identical to the Kafka run. Two brokers with completely different machinery — offsets and - * commits versus delivery tags and confirms — answering the same seven questions the same way is - * what makes the logical destination abstraction real rather than aspirational. - */ -class RabbitAdapterContractTest { - - @Nested - class Contract extends MessagingAdapterContract { - - @Override - protected MessagingAdapterHarness harness() { - return RabbitContractHarness.create(); - } - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinatorTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinatorTest.java deleted file mode 100644 index c9bd1ec..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitConfirmCoordinatorTest.java +++ /dev/null @@ -1,167 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; - -import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import java.time.Duration; -import java.util.concurrent.CompletionStage; -import org.junit.jupiter.api.Test; - -class RabbitConfirmCoordinatorTest { - - private static final ConfirmationRequirement REPLICATION = - ConfirmationRequirement.REPLICATION_OR_PERSISTENCE_ACK; - - private final RabbitConfirmCoordinator coordinator = new RabbitConfirmCoordinator(); - - @Test - void aPlainConfirmIsAConfirmedPublish() { - CompletionStage stage = - coordinator.awaiting(1L, "notification.work", "email", REPLICATION); - - coordinator.confirmed(1L, true, Duration.ofMillis(4)); - - PublishResult result = stage.toCompletableFuture().join(); - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.ROUTED); - assertThat(result.evidence().confirmationLevel()) - .isEqualTo(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK); - } - - @Test - void aReturnFollowedByAConfirmIsRejectedAsUnroutable() { - CompletionStage stage = - coordinator.awaiting(1L, "notification.work", "missing", REPLICATION); - - coordinator.returned(1L); - coordinator.confirmed(1L, true, Duration.ofMillis(4)); - - PublishResult result = stage.toCompletableFuture().join(); - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.UNROUTABLE); - assertThat(result.evidence().confirmationLevel()).isEqualTo(ConfirmationLevel.NONE); - } - - @Test - void aReturnAloneDoesNotCompleteThePublish() { - CompletionStage stage = - coordinator.awaiting(1L, "notification.work", "missing", REPLICATION); - - coordinator.returned(1L); - - assertThat(stage.toCompletableFuture()).isNotDone(); - assertThat(coordinator.pendingCount()).isEqualTo(1); - } - - @Test - void aNackIsARejectedPublish() { - CompletionStage stage = - coordinator.awaiting(2L, "notification.work", "email", REPLICATION); - - coordinator.confirmed(2L, false, Duration.ofMillis(9)); - - PublishResult result = stage.toCompletableFuture().join(); - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.failure()).isPresent(); - } - - @Test - void aMissingConfirmIsAmbiguousRatherThanFailed() { - CompletionStage stage = - coordinator.awaiting(3L, "notification.work", "email", REPLICATION); - - coordinator.timedOut(3L, Duration.ofSeconds(5)); - - PublishResult result = stage.toCompletableFuture().join(); - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.mayHaveBeenStored()).isTrue(); - assertThat(result.evidence().confirmationLevel()).isEqualTo(ConfirmationLevel.NONE); - } - - @Test - void aConfirmedPublishCarriesItsExchangeAndSequence() { - CompletionStage stage = - coordinator.awaiting(7L, "notification.work", "email", REPLICATION); - - coordinator.confirmed(7L, true, Duration.ofMillis(1)); - - PublishResult result = stage.toCompletableFuture().join(); - assertThat(result.position()).isPresent(); - assertThat(result.position().orElseThrow().diagnosticAttributes()) - .containsEntry("exchange", "notification.work") - .containsEntry("routingKey", "email") - .containsEntry("sequence", "7"); - } - - @Test - void resolvingAPublishRemovesItFromThePendingSet() { - coordinator.awaiting(1L, "notification.work", "email", REPLICATION); - coordinator.confirmed(1L, true, Duration.ofMillis(1)); - - assertThat(coordinator.pendingCount()).isZero(); - } - - @Test - void anUnknownSequenceIsIgnoredRatherThanFailingTheChannel() { - coordinator.confirmed(99L, true, Duration.ofMillis(1)); - coordinator.returned(99L); - coordinator.timedOut(99L, Duration.ofMillis(1)); - - assertThat(coordinator.pendingCount()).isZero(); - } - - @Test - @org.junit.jupiter.api.DisplayName("a multiple ack resolves every pending publish up to the tag") - void aMultipleAckResolvesEveryPendingPublish() { - RabbitConfirmCoordinator coordinator = new RabbitConfirmCoordinator(); - java.util.concurrent.CompletableFuture - first = coordinator.awaiting(1L, "orders", "created", REPLICATION).toCompletableFuture(); - java.util.concurrent.CompletableFuture - second = coordinator.awaiting(2L, "orders", "created", REPLICATION).toCompletableFuture(); - java.util.concurrent.CompletableFuture - third = coordinator.awaiting(3L, "orders", "created", REPLICATION).toCompletableFuture(); - - coordinator.confirmed(2L, true, true, java.time.Duration.ofMillis(5)); - - assertThat(first) - .as("the multiple flag was dropped, so everything below the tag waited forever") - .isCompleted(); - assertThat(second).isCompleted(); - assertThat(third).isNotCompleted(); - assertThat(coordinator.pendingCount()).isEqualTo(1); - } - - @Test - @org.junit.jupiter.api.DisplayName("a nack records the frame as transmitted") - void aNackRecordsTheFrameAsTransmitted() { - RabbitConfirmCoordinator coordinator = new RabbitConfirmCoordinator(); - java.util.concurrent.CompletableFuture - pending = coordinator.awaiting(1L, "orders", "created", REPLICATION).toCompletableFuture(); - - coordinator.confirmed(1L, false, java.time.Duration.ofMillis(5)); - - assertThat(pending.join().evidence().transmission()) - .as("a NACK is the broker's answer to a frame it received") - .isEqualTo(dev.caskeleton.messaging.api.publish.TransmissionEvidence.TRANSMITTED); - } - - @Test - @org.junit.jupiter.api.DisplayName( - "a closed channel completes every pending publish as ambiguous") - void aClosedChannelCompletesEveryPendingPublish() { - RabbitConfirmCoordinator coordinator = new RabbitConfirmCoordinator(); - java.util.concurrent.CompletableFuture - pending = coordinator.awaiting(1L, "orders", "created", REPLICATION).toCompletableFuture(); - - assertThat(coordinator.channelClosed(java.time.Duration.ofMillis(5))).isEqualTo(1); - - assertThat(pending.join().completion()) - .as("the frames were written; a channel close says nothing about whether they were stored") - .isEqualTo(dev.caskeleton.messaging.api.publish.PublishCompletion.AMBIGUOUS); - assertThat(coordinator.pendingCount()).isZero(); - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitContractHarness.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitContractHarness.java deleted file mode 100644 index 54394d5..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitContractHarness.java +++ /dev/null @@ -1,304 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.testkit.ContractMessage; -import dev.caskeleton.messaging.testkit.FaultController; -import dev.caskeleton.messaging.testkit.HandleOutcome; -import dev.caskeleton.messaging.testkit.MessagingAdapterHarness; -import dev.caskeleton.messaging.testkit.ObservedDelivery; -import java.time.Duration; -import java.util.ArrayDeque; -import java.util.ArrayList; -import java.util.Deque; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Optional; -import java.util.Set; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicLong; - -/** - * Runs the shared adapter contract against the real RabbitMQ confirm model. - * - *

Publishes go through the production {@link RabbitConfirmCoordinator}, which is the piece that - * has to get AMQP's confirm and return ordering right. Driving it from the harness is the only way - * to exercise a lost confirm deterministically — a live broker will not drop one on request. - * - *

Settlement goes through the production {@link RabbitSettlementController}, so the - * one-terminal-call guard and the DLQ-before-ack ordering are the real ones. - */ -final class RabbitContractHarness implements MessagingAdapterHarness { - - private static final int MAX_PAYLOAD_BYTES = 1_048_576; - private static final String EXCHANGE = "order.events"; - private static final String ROUTING_KEY = "created"; - - private final RabbitConfirmCoordinator coordinator = new RabbitConfirmCoordinator(); - private final AtomicLong sequence = new AtomicLong(); - private final AtomicLong deliveryTag = new AtomicLong(); - private final Faults faults = new Faults(); - private final Deque pending = new ArrayDeque<>(); - private final List deadLetters = new ArrayList<>(); - private final Set unsettled = new LinkedHashSet<>(); - - private boolean shuttingDown; - - static RabbitContractHarness create() { - return new RabbitContractHarness(); - } - - @Override - public String brokerName() { - return "rabbitmq"; - } - - @Override - public FaultController faults() { - return faults; - } - - @Override - public CompletionStage publish(ContractMessage message) { - if (shuttingDown) { - return CompletableFuture.completedFuture( - rejectedLocally("SHUTTING_DOWN", "the consumer is draining and accepts no new work")); - } - if (message.envelope().payload().size() > MAX_PAYLOAD_BYTES) { - return CompletableFuture.completedFuture( - rejectedLocally( - "PAYLOAD_TOO_LARGE", "encoded payload exceeds " + MAX_PAYLOAD_BYTES + " bytes")); - } - - long next = sequence.incrementAndGet(); - CompletionStage stage = - coordinator.awaiting( - next, EXCHANGE, ROUTING_KEY, ConfirmationRequirement.REPLICATION_OR_PERSISTENCE_ACK); - - if (faults.consumeRejectPublish()) { - coordinator.confirmed(next, false, Duration.ofMillis(2)); - } else if (faults.consumeDropPublishConfirmation()) { - coordinator.timedOut(next, Duration.ofSeconds(5)); - } else { - coordinator.confirmed(next, true, Duration.ofMillis(2)); - } - - return stage.thenApply( - result -> { - if (result.completion() == PublishCompletion.CONFIRMED) { - pending.addLast( - new Pending(message.messageId(), deliveryTag.incrementAndGet(), 1, false)); - } - return result; - }); - } - - @Override - public List drain(HandleOutcome outcome) { - List batch = new ArrayList<>(pending); - pending.clear(); - - List observed = new ArrayList<>(); - for (Pending delivery : batch) { - RabbitSettlementController controller = - new RabbitSettlementController(delivery.deliveryTag(), new HarnessOperations(delivery)); - boolean settled = - switch (outcome) { - case SUCCESS -> join(controller.ack()); - case RETRY -> join(controller.retry(Duration.ofSeconds(1))); - case DEAD_LETTER -> - join( - controller.deadLetter( - FailureDescriptor.of( - FailureCategory.PERMANENT_BUSINESS, - "CONTRACT_FAILURE", - "permanent failure"))); - }; - observed.add( - new ObservedDelivery( - delivery.messageId(), delivery.attempt(), delivery.redelivered(), settled)); - } - return List.copyOf(observed); - } - - private static boolean join( - CompletionStage stage) { - return stage.toCompletableFuture().join().completion() - == dev.caskeleton.messaging.api.settlement.SettlementCompletion.SETTLED; - } - - private void redeliver(Pending delivery) { - pending.addLast( - new Pending( - delivery.messageId(), deliveryTag.incrementAndGet(), delivery.attempt() + 1, true)); - } - - @Override - public List deadLettered() { - return List.copyOf(deadLetters); - } - - @Override - public List unsettled() { - return List.copyOf(unsettled); - } - - @Override - public void beginShutdown() { - shuttingDown = true; - } - - @Override - public boolean isAcceptingWork() { - return !shuttingDown; - } - - @Override - public void close() { - pending.clear(); - } - - private static PublishResult rejectedLocally(String code, String message) { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of(FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, code, message))); - } - - /** One message in flight on the channel. */ - private record Pending(MessageId messageId, long deliveryTag, int attempt, boolean redelivered) {} - - /** The channel operations the settlement controller drives. */ - private final class HarnessOperations implements RabbitSettlementOperations { - - private final Pending delivery; - - private HarnessOperations(Pending delivery) { - this.delivery = delivery; - } - - @Override - public CompletionStage ack(long tag) { - if (faults.consumeDropSettlementConfirmation()) { - unsettled.add(delivery.messageId()); - redeliver(delivery); - return CompletableFuture.completedFuture(unknownSettlement()); - } - unsettled.remove(delivery.messageId()); - return CompletableFuture.completedFuture( - dev.caskeleton.messaging.api.settlement.SettlementResult.settled()); - } - - @Override - public CompletionStage - publishToRetryQueue(long tag, Duration delay) { - unsettled.add(delivery.messageId()); - redeliver(delivery); - return CompletableFuture.completedFuture(unknownSettlement()); - } - - /** Publishes to the dead letter destination first, and only then acknowledges the source. */ - @Override - public CompletionStage deadLetter( - long tag, FailureDescriptor failure) { - if (faults.deadLetterPublishFails()) { - unsettled.add(delivery.messageId()); - redeliver(delivery); - return CompletableFuture.completedFuture(unknownSettlement()); - } - deadLetters.add(delivery.messageId()); - unsettled.remove(delivery.messageId()); - return CompletableFuture.completedFuture( - dev.caskeleton.messaging.api.settlement.SettlementResult.settled()); - } - - @Override - public CompletionStage discard( - long tag, FailureDescriptor failure) { - unsettled.remove(delivery.messageId()); - return CompletableFuture.completedFuture( - dev.caskeleton.messaging.api.settlement.SettlementResult.settled()); - } - - private dev.caskeleton.messaging.api.settlement.SettlementResult unknownSettlement() { - return new dev.caskeleton.messaging.api.settlement.SettlementResult( - dev.caskeleton.messaging.api.settlement.SettlementCompletion.UNKNOWN, - dev.caskeleton.messaging.api.settlement.SettlementEvidence.unknown(), - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "SETTLEMENT_UNKNOWN", - "the settlement was not confirmed"))); - } - } - - /** One-shot fault flags. */ - private static final class Faults implements FaultController { - - private boolean dropPublishConfirmation; - private boolean dropSettlementConfirmation; - private boolean failDeadLetterPublish; - private boolean rejectPublish; - - @Override - public void dropPublishConfirmation() { - dropPublishConfirmation = true; - } - - @Override - public void dropSettlementConfirmation() { - dropSettlementConfirmation = true; - } - - @Override - public void failDeadLetterPublish() { - failDeadLetterPublish = true; - } - - @Override - public void rejectPublish() { - rejectPublish = true; - } - - @Override - public void reset() { - dropPublishConfirmation = false; - dropSettlementConfirmation = false; - failDeadLetterPublish = false; - rejectPublish = false; - } - - boolean consumeDropPublishConfirmation() { - boolean active = dropPublishConfirmation; - dropPublishConfirmation = false; - return active; - } - - boolean consumeDropSettlementConfirmation() { - boolean active = dropSettlementConfirmation; - dropSettlementConfirmation = false; - return active; - } - - boolean consumeRejectPublish() { - boolean active = rejectPublish; - rejectPublish = false; - return active; - } - - boolean deadLetterPublishFails() { - return failDeadLetterPublish; - } - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitEnvelopeRoundTripTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitEnvelopeRoundTripTest.java deleted file mode 100644 index 4a6f105..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitEnvelopeRoundTripTest.java +++ /dev/null @@ -1,152 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.CorrelationId; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.api.header.HeaderName; -import dev.caskeleton.messaging.api.header.HeaderValue; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.schema.EncodedMessage; -import java.nio.charset.StandardCharsets; -import java.time.Instant; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Optional; -import org.junit.jupiter.api.DisplayName; -import org.junit.jupiter.api.Test; -import org.springframework.amqp.core.Message; -import org.springframework.amqp.core.MessageProperties; - -/** - * What an envelope still holds after it has been through RabbitMQ. - * - *

The publish and delivery mappers were written against each other's headers but never checked - * as one trip, so the application's own headers were dropped on the way back and the retry - * bookkeeping went with them — an attempt counter that restarts on every hop caps no redeliveries - * at all. - */ -class RabbitEnvelopeRoundTripTest { - - private static final Instant NOW = Instant.parse("2026-08-15T10:00:00Z"); - - private final RabbitHeaderMapper headerMapper = new RabbitHeaderMapper(); - - private final RabbitDeliveryMapper deliveryMapper = new RabbitDeliveryMapper(headerMapper); - - @Test - @DisplayName("an application header survives the trip through the broker") - void anApplicationHeaderSurvivesTheTrip() { - MessageEnvelope published = - envelope( - MessageHeaders.application( - Map.of(new HeaderName("x-source-system"), new HeaderValue("erp")))); - - MessageEnvelope delivered = roundTrip(published); - - assertThat(delivered.headers().find("x-source-system")) - .map(HeaderValue::value) - .as("a header the producer wrote and the consumer cannot read is a header nobody has") - .hasValue("erp"); - } - - @Test - @DisplayName("the tenant survives the trip instead of arriving as no tenant at all") - void theTenantSurvivesTheTrip() { - MessageEnvelope published = - envelope(MessageHeaders.empty()) - .withTenant(new dev.caskeleton.messaging.api.TenantContext("acme")); - - MessageEnvelope delivered = roundTrip(published); - - assertThat(delivered.tenantContext()) - .as("an AMQP message carries no ambient tenant, so an unwritten tenant is an absent one") - .contains(new dev.caskeleton.messaging.api.TenantContext("acme")); - } - - @Test - @DisplayName("the retry attempt survives the trip instead of restarting at one") - void theRetryAttemptSurvivesTheTrip() { - MessageEnvelope delivered = roundTrip(retryStamped(3)); - - assertThat(delivered.headers().find(ReservedHeaders.RETRY_ATTEMPT)) - .map(HeaderValue::value) - .as("an attempt counter that resets on every hop caps nothing") - .hasValue("3"); - } - - @Test - @DisplayName("a retry-stamped envelope can be published to its retry queue") - void aRetryStampedEnvelopeCanBePublished() { - assertThatCode(() -> headerMapper.toProperties(retryStamped(2))) - .as("the platform's own bookkeeping is not message content forging an identity") - .doesNotThrowAnyException(); - } - - @Test - @DisplayName("a header restating the message id is still refused") - void aForgedMessageIdIsStillRefused() { - MessageEnvelope forged = - envelope( - MessageHeaders.platform( - Map.of(new HeaderName(ReservedHeaders.MESSAGE_ID), new HeaderValue("forged")))); - - assertThatThrownBy(() -> headerMapper.toProperties(forged)) - .isInstanceOf(MessagingConfigurationException.class); - } - - @Test - @DisplayName("identity is read from its field, not duplicated into the header map") - void identityIsNotDuplicatedIntoTheHeaderMap() { - MessageEnvelope delivered = roundTrip(envelope(MessageHeaders.empty())); - - assertThat(delivered.headers().find(ReservedHeaders.MESSAGE_ID)) - .as("a copy that can drift from the field gives the same message two identities") - .isEmpty(); - } - - private MessageEnvelope roundTrip(MessageEnvelope published) { - MessageProperties properties = headerMapper.toProperties(published); - return deliveryMapper.toEnvelope(new Message(published.payload().bytes(), properties)); - } - - private static MessageEnvelope retryStamped(int attempt) { - Map headers = new LinkedHashMap<>(); - headers.put( - new HeaderName(ReservedHeaders.RETRY_ATTEMPT), new HeaderValue(Integer.toString(attempt))); - headers.put(new HeaderName(ReservedHeaders.LAST_FAILURE_AT), new HeaderValue(NOW.toString())); - return envelope(MessageHeaders.platform(headers)); - } - - private static MessageEnvelope envelope(MessageHeaders headers) { - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.of(new CorrelationId("wf-1")), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - headers, - new EncodedMessage( - "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), - ContentType.JSON, - Optional.empty())); - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidatorTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidatorTest.java deleted file mode 100644 index ba0f14f..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitProfileValidatorTest.java +++ /dev/null @@ -1,68 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import org.junit.jupiter.api.Test; - -class RabbitProfileValidatorTest { - - private final RabbitProfileValidator validator = new RabbitProfileValidator(); - - @Test - void stableProducerRequiresConfirms() { - assertThatThrownBy(() -> validator.validate(RabbitProfileFixtures.withoutConfirms())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("publisher confirms"); - } - - @Test - void stableProducerRequiresReturnsBecauseAConfirmDoesNotProveRouting() { - assertThatThrownBy(() -> validator.validate(RabbitProfileFixtures.withoutReturns())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("publisher returns"); - } - - @Test - void stableProducerRequiresMandatoryRouting() { - assertThatThrownBy(() -> validator.validate(RabbitProfileFixtures.withoutMandatory())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("mandatory"); - } - - @Test - void consumerAutoAckIsForbidden() { - assertThatThrownBy(() -> validator.validate(RabbitProfileFixtures.withAutoAck())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("auto ack"); - } - - @Test - void productionRequiresTlsAndAuthentication() { - assertThatThrownBy(() -> validator.validate(RabbitProfileFixtures.productionWithoutTls())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("TLS"); - } - - @Test - void aDurableWorkQueueMustBeAQuorumQueue() { - assertThatThrownBy( - () -> - validator.validateDestination( - RabbitProfileFixtures.workQueueDestination(), - RabbitProfileFixtures.withoutQuorumQueues())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("quorum"); - } - - @Test - void aStableProfileAndDestinationValidate() { - assertThatCode( - () -> { - validator.validate(RabbitProfileFixtures.stable()); - validator.validateDestination( - RabbitProfileFixtures.workQueueDestination(), RabbitProfileFixtures.stable()); - }) - .doesNotThrowAnyException(); - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitRuntimeTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitRuntimeTest.java deleted file mode 100644 index 903beae..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitRuntimeTest.java +++ /dev/null @@ -1,341 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.CorrelationId; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.header.ReservedHeaders; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.schema.EncodedMessage; -import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; -import dev.caskeleton.messaging.transport.TransportConsumerSpec; -import dev.caskeleton.messaging.transport.TransportDelivery; -import dev.caskeleton.messaging.transport.TransportPublishRequest; -import dev.caskeleton.messaging.transport.TransportPublishResult; -import java.nio.charset.StandardCharsets; -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicLong; -import org.junit.jupiter.api.Test; -import org.springframework.amqp.core.Message; - -class RabbitRuntimeTest { - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - - private final AtomicLong sequence = new AtomicLong(); - private final List published = new ArrayList<>(); - private final List handled = new ArrayList<>(); - - private RabbitMessagingTransport transport() { - return new RabbitMessagingTransport( - "rabbit-primary", - 1, - new RabbitChannelPublisher() { - @Override - public long nextPublishSequence() { - return sequence.incrementAndGet(); - } - - @Override - public void publish( - long sequenceNumber, - String exchange, - String routingKey, - Message message, - boolean mandatory) { - published.add(message); - } - }); - } - - @Test - void aConfirmedPublishReportsRoutedWithReplicationEvidence() { - RabbitMessagingTransport transport = transport(); - - CompletionStage stage = transport.publish(request()); - transport.onConfirm(1L, true, Duration.ofMillis(3)); - - var result = stage.toCompletableFuture().join().result(); - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.ROUTED); - } - - @Test - void aReturnedThenConfirmedPublishIsRejectedAsUnroutable() { - RabbitMessagingTransport transport = transport(); - - CompletionStage stage = transport.publish(request()); - transport.onReturn(1L); - transport.onConfirm(1L, true, Duration.ofMillis(3)); - - var result = stage.toCompletableFuture().join().result(); - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.routingOutcome()).isEqualTo(RoutingOutcome.UNROUTABLE); - } - - @Test - void aMissingConfirmExpiresWithoutAnExternalTimeoutCallback() throws Exception { - try (RabbitMessagingTransport timed = - new RabbitMessagingTransport( - "rabbit-primary", - 1, - new RabbitChannelPublisher() { - @Override - public long nextPublishSequence() { - return sequence.incrementAndGet(); - } - - @Override - public void publish( - long sequenceNumber, - String exchange, - String routingKey, - Message message, - boolean mandatory) { - published.add(message); - } - }, - Duration.ofMillis(20))) { - CompletionStage stage = timed.publish(request()); - - var result = - stage.toCompletableFuture().get(1, java.util.concurrent.TimeUnit.SECONDS).result(); - assertThat(result.completion()).isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(timed.pendingConfirms()).isZero(); - } - } - - @Test - void aMissingConfirmIsAmbiguous() { - RabbitMessagingTransport transport = transport(); - - CompletionStage stage = transport.publish(request()); - transport.onConfirmTimeout(1L, Duration.ofSeconds(5)); - - assertThat(stage.toCompletableFuture().join().result().mayHaveBeenStored()).isTrue(); - } - - @Test - void anOversizedPayloadNeverReachesTheChannel() { - RabbitMessagingTransport transport = transport(); - - var result = - transport.publish(request(new byte[1_048_577])).toCompletableFuture().join().result(); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(published).isEmpty(); - } - - @Test - void aClosedTransportRefusesPublishesLocally() { - RabbitMessagingTransport transport = transport(); - transport.close(); - - var result = transport.publish(request()).toCompletableFuture().join().result(); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(transport.isAcceptingWork()).isFalse(); - assertThat(published).isEmpty(); - } - - @Test - void theHeaderRoundTripPreservesIdentityAndProvenance() { - RabbitHeaderMapper headers = new RabbitHeaderMapper(); - MessageEnvelope envelope = envelope(new byte[] {1, 2, 3}); - - var properties = headers.toProperties(envelope); - - assertThat(headers.value(properties, ReservedHeaders.MESSAGE_ID)) - .isEqualTo(envelope.messageId().value().toString()); - assertThat(headers.value(properties, ReservedHeaders.MESSAGE_TYPE)).isEqualTo("order.created"); - assertThat(headers.value(properties, ReservedHeaders.CORRELATION_ID)).isEqualTo("wf-1"); - assertThat(properties.getCorrelationId()).isEqualTo("wf-1"); - } - - @Test - void aDeliveryIsRebuiltFromItsHeaders() { - RabbitHeaderMapper headers = new RabbitHeaderMapper(); - MessageEnvelope original = envelope(new byte[] {1, 2, 3}); - Message message = new Message(new byte[] {1, 2, 3}, headers.toProperties(original)); - - var restored = new RabbitDeliveryMapper().toEnvelope(message); - - assertThat(restored.messageId()).isEqualTo(original.messageId()); - assertThat(restored.messageType()).isEqualTo(original.messageType()); - assertThat(restored.schemaVersion()).isEqualTo(original.schemaVersion()); - assertThat(restored.payload().bytes()).containsExactly(1, 2, 3); - } - - @Test - void theConsumerDispatchesAndSettlesThroughTheController() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = registrar(operations, this::acknowledge); - - boolean accepted = registrar.onMessage(deliveredMessage(7L), NOW); - - assertThat(accepted).isTrue(); - assertThat(handled).hasSize(1); - assertThat(operations.calls).containsExactly("ack:7"); - } - - @Test - void aPausedQueueRefusesDeliveriesSoTheBrokerRedeliversThem() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = registrar(operations, this::acknowledge); - registrar.pause("").toCompletableFuture().join(); - - boolean accepted = registrar.onMessage(deliveredMessage(7L), NOW); - - assertThat(accepted).isFalse(); - assertThat(handled).isEmpty(); - assertThat(operations.calls).isEmpty(); - } - - @Test - void drainingRefusesNewDeliveries() { - RecordingOperations operations = new RecordingOperations(); - GracefulShutdownCoordinator shutdown = new GracefulShutdownCoordinator(Duration.ofSeconds(30)); - try (RabbitConsumerRegistrar registrar = - new RabbitConsumerRegistrar( - new TransportConsumerSpec( - RabbitProfileFixtures.workQueueDestination(), this::acknowledge), - operations, - shutdown)) { - - shutdown.beginDrain(NOW); - boolean accepted = registrar.onMessage(deliveredMessage(7L), NOW); - - assertThat(accepted).isFalse(); - assertThat(operations.calls).isEmpty(); - } - } - - @Test - void anUndecodableMessageIsDiscardedRatherThanRequeued() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = registrar(operations, this::acknowledge); - - Message malformed = - new Message(new byte[] {1}, new org.springframework.amqp.core.MessageProperties()); - malformed.getMessageProperties().setDeliveryTag(9L); - - boolean accepted = registrar.onMessage(malformed, NOW); - - assertThat(accepted).isTrue(); - assertThat(handled).isEmpty(); - assertThat(operations.calls).containsExactly("discard:9"); - } - - private RabbitConsumerRegistrar registrar( - RabbitSettlementOperations operations, - java.util.function.Function> sink) { - return new RabbitConsumerRegistrar( - new TransportConsumerSpec(RabbitProfileFixtures.workQueueDestination(), sink), - operations, - new GracefulShutdownCoordinator(Duration.ofSeconds(30))); - } - - private CompletionStage acknowledge(TransportDelivery delivery) { - handled.add(delivery); - delivery.settlement().acknowledge().toCompletableFuture().join(); - return CompletableFuture.completedFuture(null); - } - - private Message deliveredMessage(long deliveryTag) { - Message message = - new Message( - "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), - new RabbitHeaderMapper().toProperties(envelope(new byte[] {1}))); - message.getMessageProperties().setDeliveryTag(deliveryTag); - message.getMessageProperties().setConsumerQueue("notification.email.q"); - return message; - } - - private TransportPublishRequest request() { - return request("{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8)); - } - - private TransportPublishRequest request(byte[] payload) { - return new TransportPublishRequest( - RabbitProfileFixtures.workQueueDestination(), envelope(payload), PublishOptions.defaults()); - } - - private MessageEnvelope envelope(byte[] payload) { - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.of(new CorrelationId("wf-1")), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - new EncodedMessage(payload, ContentType.JSON, Optional.empty())); - } - - @Test - @org.junit.jupiter.api.DisplayName("a handler failure is retried, not discarded as undecodable") - void aHandlerFailureIsRetriedNotDiscarded() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = - registrar( - operations, - delivery -> { - throw new IllegalStateException("the projection is down"); - }); - - registrar.onMessage(deliveredMessage(11), NOW); - - assertThat(operations.calls) - .as("one catch covered decode, handler and settlement, so a retryable failure was deleted") - .containsExactly("retry:11:PT0S"); - } - - @Test - @org.junit.jupiter.api.DisplayName("a handler that never settles does not acknowledge") - void aHandlerThatNeverSettlesDoesNotAcknowledge() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = - registrar(operations, delivery -> CompletableFuture.completedFuture(null)); - - registrar.onMessage(deliveredMessage(12), NOW); - - assertThat(operations.calls) - .as("acknowledging on the handler's behalf silently drops a message nobody handled") - .containsExactly("retry:12:PT0S"); - } - - @Test - @org.junit.jupiter.api.DisplayName("a delivery after close is refused") - void aDeliveryAfterCloseIsRefused() { - RecordingOperations operations = new RecordingOperations(); - RabbitConsumerRegistrar registrar = - registrar(operations, delivery -> CompletableFuture.completedFuture(null)); - registrar.close(); - - assertThat(registrar.onMessage(deliveredMessage(13), NOW)) - .as("close only flipped a flag that onMessage never read") - .isFalse(); - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitSettlementControllerTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitSettlementControllerTest.java deleted file mode 100644 index f0cba51..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitSettlementControllerTest.java +++ /dev/null @@ -1,105 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; - -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.settlement.SettlementResult; -import java.time.Duration; -import java.util.ArrayList; -import java.util.List; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import org.junit.jupiter.api.Test; - -class RabbitSettlementControllerTest { - - private static final FailureDescriptor FAILURE = - FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, "REJECTED", "rejected"); - - @Test - void acknowledgesExactlyOnce() { - RecordingOperations operations = new RecordingOperations(); - RabbitSettlementController controller = new RabbitSettlementController(7L, operations); - - controller.ack().toCompletableFuture().join(); - - assertThat(operations.calls).containsExactly("ack:7"); - assertThat(controller.isSettled()).isTrue(); - } - - @Test - void aSecondTerminalCallFailsRatherThanRacingTheFirst() { - RecordingOperations operations = new RecordingOperations(); - RabbitSettlementController controller = new RabbitSettlementController(7L, operations); - - controller.ack().toCompletableFuture().join(); - - assertThat(controller.ack().toCompletableFuture()).isCompletedExceptionally(); - assertThat(operations.calls).containsExactly("ack:7"); - } - - @Test - void mixingTerminalOperationsIsAlsoRefused() { - RecordingOperations operations = new RecordingOperations(); - RabbitSettlementController controller = new RabbitSettlementController(7L, operations); - - controller.deadLetter(FAILURE).toCompletableFuture().join(); - - assertThat(controller.ack().toCompletableFuture()).isCompletedExceptionally(); - assertThat(operations.calls).containsExactly("deadLetter:7"); - } - - @Test - void retryGoesToTheDelayedQueueRatherThanRequeueingImmediately() { - RecordingOperations operations = new RecordingOperations(); - RabbitSettlementController controller = new RabbitSettlementController(7L, operations); - - controller.retry(Duration.ofSeconds(30)).toCompletableFuture().join(); - - assertThat(operations.calls).containsExactly("retry:7:PT30S"); - } - - @Test - void aRetryQueueCarriesTtlAndADeadLetterRouteBackToTheWorkQueue() { - RabbitRetryQueueTopology topology = - new RabbitRetryQueueTopology( - "notification.email.retry.q", "notification.work", "email", Duration.ofSeconds(30)); - - assertThat(topology.queueArguments()) - .containsEntry("x-queue-type", "quorum") - .containsEntry("x-message-ttl", 30_000L) - .containsEntry("x-dead-letter-exchange", "notification.work") - .containsEntry("x-dead-letter-routing-key", "email"); - } -} - -/** Records which channel operation the controller invoked. */ -final class RecordingOperations implements RabbitSettlementOperations { - - final List calls = new ArrayList<>(); - - @Override - public CompletionStage ack(long deliveryTag) { - calls.add("ack:" + deliveryTag); - return CompletableFuture.completedFuture(SettlementResult.settled()); - } - - @Override - public CompletionStage publishToRetryQueue(long deliveryTag, Duration delay) { - calls.add("retry:" + deliveryTag + ":" + delay); - return CompletableFuture.completedFuture(SettlementResult.settled()); - } - - @Override - public CompletionStage deadLetter(long deliveryTag, FailureDescriptor failure) { - calls.add("deadLetter:" + deliveryTag); - return CompletableFuture.completedFuture(SettlementResult.settled()); - } - - @Override - public CompletionStage discard(long deliveryTag, FailureDescriptor failure) { - calls.add("discard:" + deliveryTag); - return CompletableFuture.completedFuture(SettlementResult.settled()); - } -} diff --git a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitTopologyAndBatchTest.java b/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitTopologyAndBatchTest.java deleted file mode 100644 index 040513e..0000000 --- a/messaging/messaging-rabbit/src/test/java/dev/caskeleton/messaging/rabbit/RabbitTopologyAndBatchTest.java +++ /dev/null @@ -1,253 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.io.IOException; -import java.time.Duration; -import java.time.Instant; -import java.util.Optional; -import org.junit.jupiter.api.Test; - -class RabbitTopologyAndBatchTest { - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - - private static RabbitTopologyProfile quorumWithDlx() { - return RabbitTopologyProfile.workQueue( - "notification.work", "email", "notification.email.q", "notification.dlx"); - } - - private static RabbitTopologyProfile classic() { - return new RabbitTopologyProfile( - "notification.work", - "direct", - "email", - "notification.email.q", - false, - Optional.empty(), - Optional.empty(), - Optional.empty()); - } - - @Test - void aDurableDestinationRequiresAQuorumQueue() { - assertThatThrownBy(() -> classic().requireSafeFor(true)) - .as("a promoted stale mirror loses messages the broker already confirmed") - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("quorum"); - } - - @Test - void aQuorumQueueSatisfiesADurableDestination() { - assertThatCode(() -> quorumWithDlx().requireSafeFor(true)).doesNotThrowAnyException(); - } - - @Test - void theQueueArgumentsCarryTheDeclaredGuarantees() { - assertThat(quorumWithDlx().queueArguments()) - .containsEntry(RabbitTopologyProfile.QUEUE_TYPE, "quorum") - .containsEntry(RabbitTopologyProfile.DEAD_LETTER_EXCHANGE, "notification.dlx"); - } - - @Test - void aDeadLetterRoutingKeyWithoutAnExchangeIsRejected() { - assertThatThrownBy( - () -> - new RabbitTopologyProfile( - "x", - "direct", - "k", - "q", - true, - Optional.empty(), - Optional.of("k"), - Optional.empty())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("routes nowhere"); - } - - @Test - void nativeDeadLetterNeedsBothADeclarationAndAVerifiedBinding() { - assertThat(new RabbitNativeDeadLetterCapability(quorumWithDlx(), true).mayUseNativeDeadLetter()) - .isTrue(); - assertThat( - new RabbitNativeDeadLetterCapability(quorumWithDlx(), false).mayUseNativeDeadLetter()) - .as("an unbound dead letter exchange discards the message while the reject still succeeds") - .isFalse(); - } - - @Test - void aQueueWithoutADeadLetterExchangeFallsBackToThePlatformPublish() { - RabbitNativeDeadLetterCapability capability = - new RabbitNativeDeadLetterCapability(classic(), true); - - assertThat(capability.requiresPlatformPublish()).isTrue(); - assertThat(capability.reasonForPlatformPublish()).contains("without a dead letter exchange"); - } - - @Test - void theUnboundReasonExplainsTheSilentDiscard() { - assertThat( - new RabbitNativeDeadLetterCapability(quorumWithDlx(), false).reasonForPlatformPublish()) - .contains("silently") - .contains("reporting success"); - } - - @Test - void aFailureBeforeTheFrameWasWrittenIsADefinitiveRejection() { - PublishResult result = - new RabbitPublishFailureClassifier() - .classify(new IllegalStateException("channel closed"), false, Duration.ofMillis(3)); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.NOT_TRANSMITTED); - } - - @Test - void aFailureAfterTheFrameWasWrittenIsAmbiguous() { - PublishResult result = - new RabbitPublishFailureClassifier() - .classify(new IOException("connection reset"), true, Duration.ofMillis(3)); - - assertThat(result.completion()) - .as("the broker may hold the message and only the confirm was lost") - .isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.evidence().transmission()) - .isEqualTo(TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED); - } - - @Test - void authenticationFailureUsesTheTypedRabbitSignal() { - PublishResult result = - new RabbitPublishFailureClassifier() - .classify( - new com.rabbitmq.client.AuthenticationFailureException("bad credentials"), - false, - Duration.ofMillis(3)); - - assertThat(result.failure()) - .hasValueSatisfying( - failure -> - assertThat(failure.category()) - .isEqualTo(dev.caskeleton.messaging.api.error.FailureCategory.AUTHENTICATION)); - } - - @Test - void accessRefusedReplyCodeIsAuthorizationRatherThanBusinessRejection() { - com.rabbitmq.client.ShutdownSignalException accessRefused = - new com.rabbitmq.client.ShutdownSignalException( - false, - false, - new com.rabbitmq.client.impl.AMQImpl.Channel.Close(403, "ACCESS_REFUSED", 60, 40), - null); - - PublishResult result = - new RabbitPublishFailureClassifier().classify(accessRefused, false, Duration.ofMillis(3)); - - assertThat(result.failure()) - .hasValueSatisfying( - failure -> - assertThat(failure.category()) - .isEqualTo(dev.caskeleton.messaging.api.error.FailureCategory.AUTHORIZATION)); - } - - @Test - void aNackIsRejectedButStillCountsAsTransmitted() { - PublishResult result = new RabbitPublishFailureClassifier().nacked(Duration.ofMillis(3)); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.TRANSMITTED); - } - - @Test - void aBatchIsReleasedOnceItIsFull() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 3, Duration.ofSeconds(5)); - - assertThat(registrar.add(1, NOW)).isEmpty(); - assertThat(registrar.add(2, NOW)).isEmpty(); - assertThat(registrar.add(3, NOW)) - .hasValueSatisfying(batch -> assertThat(batch.deliveryTags()).containsExactly(1L, 2L, 3L)); - } - - @Test - void aPartialBatchIsReleasedOnceItsOldestDeliveryGoesStale() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 100, Duration.ofSeconds(5)); - registrar.add(1, NOW); - - assertThat(registrar.releaseIfStale(NOW.plusSeconds(1))).isEmpty(); - assertThat(registrar.releaseIfStale(NOW.plusSeconds(6))) - .as("a quiet queue would otherwise hold its last messages unacknowledged indefinitely") - .isPresent(); - } - - @Test - void anAccumulatedBatchIsNotSettlableAsAUnit() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 2, Duration.ofSeconds(5)); - registrar.add(1, NOW); - - assertThat(registrar.add(2, NOW)) - .hasValueSatisfying( - batch -> - assertThat(batch.metadata().settlableAsBatch()) - .as("AMQP multiple-ack settles work that is still in flight") - .isFalse()); - } - - @Test - void anAccumulatedBatchNamesExactlyOneOrderingUnit() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 1, Duration.ofSeconds(5)); - - assertThat(registrar.add(1, NOW)) - .hasValueSatisfying( - batch -> assertThat(batch.metadata().isSafeForOrderedDestination()).isTrue()); - } - - @Test - void aPrefetchBelowTheBatchSizeWouldDeadlockAndIsRefused() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 50, Duration.ofSeconds(5)); - - assertThatThrownBy(() -> registrar.requirePrefetchFor(10)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("prefetch"); - assertThatCode(() -> registrar.requirePrefetchFor(100)).doesNotThrowAnyException(); - } - - @Test - void aZeroBatchAgeIsRefusedBecauseItStrandsQuietQueues() { - assertThatThrownBy( - () -> - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 10, Duration.ZERO)) - .isInstanceOf(IllegalArgumentException.class); - } - - @Test - void drainingReleasesWhateverIsPendingForShutdown() { - RabbitBatchConsumerRegistrar registrar = - new RabbitBatchConsumerRegistrar( - RabbitProfileFixtures.workQueueDestination(), 100, Duration.ofSeconds(5)); - registrar.add(1, NOW); - registrar.add(2, NOW); - - assertThat(registrar.drain(NOW)) - .hasValueSatisfying(batch -> assertThat(batch.deliveryTags()).hasSize(2)); - assertThat(registrar.pending()).isZero(); - assertThat(registrar.drain(NOW)).isEmpty(); - } -} diff --git a/messaging/messaging-rabbit/src/testFixtures/java/dev/caskeleton/messaging/rabbit/RabbitProfileFixtures.java b/messaging/messaging-rabbit/src/testFixtures/java/dev/caskeleton/messaging/rabbit/RabbitProfileFixtures.java deleted file mode 100644 index e505cd2..0000000 --- a/messaging/messaging-rabbit/src/testFixtures/java/dev/caskeleton/messaging/rabbit/RabbitProfileFixtures.java +++ /dev/null @@ -1,118 +0,0 @@ -package dev.caskeleton.messaging.rabbit; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; -import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.policy.CapabilityTier; -import dev.caskeleton.messaging.policy.ConsumerPolicy; -import dev.caskeleton.messaging.policy.DeadLetterPolicy; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.PayloadPolicy; -import dev.caskeleton.messaging.policy.PhysicalDestination; -import dev.caskeleton.messaging.policy.ProducerPolicy; -import dev.caskeleton.messaging.policy.RetryPolicy; -import dev.caskeleton.messaging.policy.SchemaPolicy; -import dev.caskeleton.messaging.schema.SchemaCompatibility; -import java.time.Duration; -import java.util.List; -import java.util.Set; - -/** Builds RabbitMQ profiles for validator tests. */ -final class RabbitProfileFixtures { - - private RabbitProfileFixtures() {} - - static RabbitBrokerProfile stable() { - return new RabbitBrokerProfile( - "rabbit-primary", - true, - false, - List.of("localhost:5672"), - true, - true, - true, - Duration.ofSeconds(5), - false, - 16, - true, - true, - true); - } - - static RabbitBrokerProfile withoutConfirms() { - return copy(stable(), false, true, true, false, true, false, true); - } - - static RabbitBrokerProfile withoutReturns() { - return copy(stable(), true, false, true, false, true, false, true); - } - - static RabbitBrokerProfile withoutMandatory() { - return copy(stable(), true, true, false, false, true, false, true); - } - - static RabbitBrokerProfile withAutoAck() { - return copy(stable(), true, true, true, true, true, false, true); - } - - static RabbitBrokerProfile withoutQuorumQueues() { - return copy(stable(), true, true, true, false, false, false, true); - } - - static RabbitBrokerProfile productionWithoutTls() { - return copy(stable(), true, true, true, false, true, true, false); - } - - private static RabbitBrokerProfile copy( - RabbitBrokerProfile base, - boolean confirms, - boolean returns, - boolean mandatory, - boolean autoAck, - boolean quorumQueues, - boolean production, - boolean tlsEnabled) { - return new RabbitBrokerProfile( - base.broker(), - base.stable(), - production, - base.addresses(), - confirms, - returns, - mandatory, - base.confirmTimeout(), - autoAck, - base.prefetch(), - quorumQueues, - tlsEnabled, - base.authenticationEnabled()); - } - - static DestinationProfile workQueueDestination() { - return new DestinationProfile( - new DestinationName("email-work"), - "rabbit-primary", - DestinationKind.WORK_QUEUE, - PhysicalDestination.rabbitQueue("notification.work", "email", "notification.email.q"), - new SchemaPolicy( - ContentType.JSON, - SchemaCompatibility.BACKWARD, - Set.of(new MessageType("email.requested"))), - DeliveryGuarantee.AT_LEAST_ONCE, - OrderingScope.NONE, - ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, - ProducerPolicy.defaults(), - ConsumerPolicy.defaults("notification"), - RetryPolicy.none(), - DeadLetterPolicy.to(new DestinationName("email-work-dlq")), - PayloadPolicy.defaults(), - CapabilityTier.M1, - false, - false, - false); - } -} diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/IdempotentMessageHandler.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/IdempotentMessageHandler.java deleted file mode 100644 index 478e6ff..0000000 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/IdempotentMessageHandler.java +++ /dev/null @@ -1,33 +0,0 @@ -package dev.caskeleton.messaging.reliability; - -import dev.caskeleton.messaging.api.delivery.HandleResult; -import dev.caskeleton.messaging.api.delivery.MessageDelivery; -import java.util.concurrent.CompletionStage; - -/** - * Runs a side effect at most once per consumer, whatever the broker redelivers. - * - *

{@code consumerName} is a parameter rather than a property of the message because the same - * message legitimately reaches several independent consumers, and each of them is entitled to apply - * its own effect once. Keying the inbox on the message id alone would let whichever consumer ran - * first suppress all the others. - * - *

A duplicate is not a failure. The handler returns success for a message it has already - * applied, so the broker settles it and stops redelivering; treating it as an error would park a - * message whose work is already done. - * - * @param the payload type - */ -public interface IdempotentMessageHandler { - - /** - * Applies an action once for a given consumer. - * - * @param consumerName the consumer identity the reservation is keyed on - * @param delivery the message being handled - * @param action the effect to apply inside the reservation's transaction - * @return a stage completing with the handler outcome - */ - CompletionStage handleOnce( - String consumerName, MessageDelivery delivery, TransactionalMessageAction action); -} diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRecord.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRecord.java deleted file mode 100644 index bfa16dc..0000000 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRecord.java +++ /dev/null @@ -1,27 +0,0 @@ -package dev.caskeleton.messaging.reliability; - -import dev.caskeleton.messaging.api.MessageId; -import java.time.Instant; -import java.util.Objects; - -/** - * One row of the consumer inbox. - * - *

Keyed by message id and consumer id, because two independent consumers of the same - * event must each process it once — deduplicating on the message alone would let the first consumer - * suppress the second. - * - * @param messageId the logical message identity - * @param consumerId the consumer's stable identity - * @param processedAt when the side effect committed - */ -public record InboxRecord(MessageId messageId, String consumerId, Instant processedAt) { - - public InboxRecord { - Objects.requireNonNull(messageId, "messageId must not be null"); - Objects.requireNonNull(processedAt, "processedAt must not be null"); - if (consumerId == null || consumerId.isBlank()) { - throw new IllegalArgumentException("consumerId must not be blank"); - } - } -} diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRepository.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRepository.java index a438805..55846b4 100644 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRepository.java +++ b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxRepository.java @@ -34,19 +34,11 @@ public interface InboxRepository { boolean isProcessed(MessageId messageId, String consumerId); /** - * Deletes records older than a cutoff. + * Deletes at most {@code limit} processed rows older than the cutoff. * *

Retention must outlive the broker's maximum redelivery window, otherwise a late redelivery * arrives after its inbox row was pruned and is processed a second time. * - * @param processedBefore the retention cutoff - * @return how many rows were removed - */ - int purgeProcessedBefore(Instant processedBefore); - - /** - * Deletes at most {@code limit} processed rows older than the cutoff. - * * @return how many rows were deleted; fewer than {@code limit} means the sweep is finished */ int purgeProcessedBefore(Instant processedBefore, int limit); diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxResult.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxResult.java deleted file mode 100644 index 76d0bef..0000000 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/InboxResult.java +++ /dev/null @@ -1,45 +0,0 @@ -package dev.caskeleton.messaging.reliability; - -/** - * What the inbox decided about one delivery. - * - *

Three outcomes, not two. Collapsing {@link #ALREADY_APPLIED} and {@link #CLAIMED_ELSEWHERE} - * into a single "duplicate" would settle a message whose effect is still only half-written by - * another instance: if that instance then rolls back, the effect is lost and the broker will never - * redeliver, because this instance already acknowledged it. - */ -public enum InboxResult { - - /** No reservation existed; the effect ran inside this transaction. */ - APPLIED(true), - - /** - * A committed reservation already exists, so the effect ran before. - * - *

Settle the message: the work is done and redelivering it achieves nothing. - */ - ALREADY_APPLIED(true), - - /** - * Another instance holds an uncommitted reservation for this message. - * - *

Do not settle. The other transaction may still roll back, and this delivery is the - * only remaining copy that could re-apply the effect. - */ - CLAIMED_ELSEWHERE(false); - - private final boolean safeToSettle; - - InboxResult(boolean safeToSettle) { - this.safeToSettle = safeToSettle; - } - - /** - * Reports whether the source message may be settled on this outcome. - * - * @return true when settling cannot lose the effect - */ - public boolean isSafeToSettle() { - return safeToSettle; - } -} diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java index 6552a3b..6765abf 100644 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java +++ b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/OutboxRepository.java @@ -22,38 +22,23 @@ public interface OutboxRepository { void append(OutboxRecord record); /** - * Leases a batch of publishable records. + * Claims a batch and returns the leases, each carrying its owner and fencing token, skipping rows + * that are waiting for their backoff or out of attempts. * - *

Leasing rather than simply selecting is what makes multiple relay instances safe: a record - * claimed by one relay is invisible to the others until its lease expires, so the same message is - * not published concurrently by two processes. - * - * @param batchSize how many records to claim - * @param leaseDuration how long the claim holds - * @param now the current instant - * @return the leased records, oldest first - */ - List leaseBatch(int batchSize, Duration leaseDuration, Instant now); - - /** - * Claims a batch and returns the leases, each carrying its owner and fencing token. - * - *

The token is what a terminal write is checked against. {@link #leaseBatch} returns records - * without one, so its callers cannot prove a write belongs to their claim; it remains for - * inspection paths and is deprecated for the relay's use. - * - * @param owner this relay's stable identity - */ - List claimBatch(String owner, int batchSize, Duration leaseDuration, Instant now); - - /** - * Claims a batch, skipping rows that are waiting for their backoff or out of attempts. + *

Claiming rather than simply selecting is what makes multiple relay instances safe: a record + * claimed by one relay is invisible to the others until its lease expires, and the token is what + * a terminal write is checked against, so a relay that stalled past its lease writes nothing. * *

Eligibility belongs in the row. A relay that decides in memory forgets on restart, and every * replica decides for itself — so the same message is retried by as many workers as there are * replicas, as often as they poll. * + * @param owner this relay's stable identity + * @param batchSize how many records to claim + * @param leaseDuration how long the claim holds + * @param now the current instant * @param maxAttempts the attempt budget; rows at or above it are not claimed + * @return the leases, oldest first */ List claimBatch( String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts); @@ -65,10 +50,10 @@ public interface OutboxRepository { */ OutboxTransitionResult markPublished(OutboxLease lease, Instant now); - /** Records an ambiguous outcome, only if this lease still owns the row. */ - OutboxTransitionResult markAmbiguous(OutboxLease lease, String failureCode, Instant now); - - /** Records an ambiguous outcome and when this row may next be claimed. */ + /** + * Records an ambiguous outcome and when this row may next be claimed, only if this lease still + * owns the row. + */ OutboxTransitionResult markAmbiguous( OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt); @@ -94,22 +79,12 @@ public interface OutboxRepository { */ Optional find(MessageId messageId); - /** - * Deletes records published before a cutoff. - * - * @param publishedBefore the retention cutoff - * @return how many rows were removed - */ - int purgePublishedBefore(Instant publishedBefore); - /** * Deletes at most {@code limit} published rows older than the cutoff. * - *

The unbounded version deletes everything before the cutoff in one statement. On a table that - * has been accumulating published rows since the last sweep that is a single long transaction - * holding locks and generating WAL in proportion to the backlog, which shows up as the relay and - * the business writes stalling behind retention. The cleanup jobs describe themselves as bounded - * by batch size; this is the parameter that makes that true. + *

There is no unbounded version. Deleting everything before the cutoff in one statement is a + * single long transaction holding locks and generating WAL in proportion to the backlog, which + * shows up as the relay and the business writes stalling behind retention. * * @return how many rows were deleted; fewer than {@code limit} means the sweep is finished */ diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/ReliableMessagePublisher.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/ReliableMessagePublisher.java deleted file mode 100644 index 9eac6c7..0000000 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/ReliableMessagePublisher.java +++ /dev/null @@ -1,27 +0,0 @@ -package dev.caskeleton.messaging.reliability; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; - -/** - * Enrols a message in the caller's database transaction instead of publishing it. - * - *

The return type is {@code void}, and that is the contract. There is no publish outcome to - * report yet: the row is written inside the caller's transaction, so if the transaction rolls back - * the message never existed, and if it commits the relay will publish it later. Handing back a - * {@code PublishResult} here would be a lie about work that has not happened. - * - *

This is the answer to the dual-write problem. Writing to the database and publishing to the - * broker in the same method cannot be made atomic; writing both to the database can. - */ -public interface ReliableMessagePublisher { - - /** - * Stages a message for publication when the current transaction commits. - * - * @param the payload type - * @param destination the logical destination - * @param message the envelope to publish, carrying the identity the relay will preserve - */ - void addToOutbox(MessageDestination destination, MessageEnvelope message); -} diff --git a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/TransactionalMessageAction.java b/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/TransactionalMessageAction.java deleted file mode 100644 index 5b5b121..0000000 --- a/messaging/messaging-reliability-api/src/main/java/dev/caskeleton/messaging/reliability/TransactionalMessageAction.java +++ /dev/null @@ -1,29 +0,0 @@ -package dev.caskeleton.messaging.reliability; - -import dev.caskeleton.messaging.api.delivery.MessageDelivery; - -/** - * The business work that runs inside the same transaction as the inbox reservation. - * - *

Sharing one transaction is the entire mechanism. If the effect committed separately from the - * "I have handled this message" marker, a crash between the two would either replay the effect or - * suppress a message that was never handled — and which of those you get would depend on the order - * the two commits happened to be written in. - * - *

Implementations must not settle the message, publish, or start their own transaction. The - * runtime owns the transaction boundary precisely so that the action cannot accidentally commit - * half of it. - * - * @param the payload type - */ -@FunctionalInterface -public interface TransactionalMessageAction { - - /** - * Applies the side effect for one delivery. - * - * @param delivery the message being handled - * @throws Exception when the effect fails, rolling back both it and the inbox reservation - */ - void apply(MessageDelivery delivery) throws Exception; -} diff --git a/messaging/messaging-outbox-jdbc-postgresql/build.gradle b/messaging/messaging-reliability-jdbc-postgresql/build.gradle similarity index 55% rename from messaging/messaging-outbox-jdbc-postgresql/build.gradle rename to messaging/messaging-reliability-jdbc-postgresql/build.gradle index 36beb14..f549b2d 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/build.gradle +++ b/messaging/messaging-reliability-jdbc-postgresql/build.gradle @@ -1,24 +1,33 @@ plugins { - id 'ca.auxiliary-source-set' id 'ca.platform-module' } -auxiliarySourceSets { - sourceSet('integrationTest') { - compilesAgainst 'main' +testing { + suites { + integrationTest(JvmTestSuite) { + dependencies { + implementation project() + } + } } } +configurations { + integrationTestImplementation.extendsFrom testImplementation + integrationTestRuntimeOnly.extendsFrom testRuntimeOnly +} +// The relational implementations of the reliability ports: the outbox repository, the inbox +// repository and the destructive-operation journal. Spring JDBC for the transaction join, and SQL +// that is PostgreSQL's (`jsonb`, `FOR UPDATE SKIP LOCKED`, `ON CONFLICT`, `TIMESTAMPTZ`) — the name +// is the contract. The relay and consumer logic that uses them is messaging-reliability. dependencies { api project(':messaging:messaging-core-api') api project(':messaging:messaging-reliability-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-observability') // The destructive-operation journal lives here because it needs exactly what the outbox needs: // one relational database every replica can see, and a migration lane that already exists. The // contract it implements belongs to the admin API. - api project(':messaging:messaging-admin-api') + api project(':messaging:messaging-admin') implementation 'org.springframework:spring-jdbc' implementation 'org.springframework:spring-tx' @@ -26,6 +35,7 @@ dependencies { // Live-database certification. The reliability patterns are claims about transaction // boundaries and uniqueness constraints, and only a real database can settle them. testImplementation project(':messaging:messaging-testkit') + integrationTestImplementation project(':messaging:messaging-testkit') integrationTestImplementation 'org.testcontainers:testcontainers-postgresql' integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' integrationTestImplementation 'org.postgresql:postgresql' @@ -35,6 +45,6 @@ strictTestLanes { lane('integrationTest') { integration() sourceSet = 'integrationTest' - description = 'Runs outbox and admin-journal contracts against real PostgreSQL.' + description = 'Runs outbox, inbox and admin-journal contracts against real PostgreSQL.' } } diff --git a/messaging/messaging-outbox-jdbc-postgresql/gradle.lockfile b/messaging/messaging-reliability-jdbc-postgresql/gradle.lockfile similarity index 94% rename from messaging/messaging-outbox-jdbc-postgresql/gradle.lockfile rename to messaging/messaging-reliability-jdbc-postgresql/gradle.lockfile index 608e6f7..2eeab5a 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/gradle.lockfile +++ b/messaging/messaging-reliability-jdbc-postgresql/gradle.lockfile @@ -39,9 +39,9 @@ commons-logging:commons-logging:1.3.6=compileClasspath,integrationTestCompileCla info.picocli:picocli:4.7.7=checkstyle io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath +io.micrometer:micrometer-commons:1.16.7=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath +io.micrometer:micrometer-core:1.16.7=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath +io.micrometer:micrometer-observation:1.16.7=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java similarity index 99% rename from messaging/messaging-inbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java rename to messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java index 367a6f5..899707c 100644 --- a/messaging/messaging-inbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java +++ b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/inbox/InboxPostgresIT.java @@ -151,7 +151,7 @@ class InboxPostgresIT { MessageId purged = MessageId.newId(); inTransaction(() -> repository.reserve(purged, "order-projection", NOW)); - assertThat(repository.purgeProcessedBefore(NOW.plusSeconds(1))).isEqualTo(1); + assertThat(repository.purgeProcessedBefore(NOW.plusSeconds(1), 100)).isEqualTo(1); } /** Reserves and writes inside one Spring transaction, exactly as an idempotent consumer must. */ diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/AdminOperationJournalPostgresIT.java b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/AdminOperationJournalPostgresIT.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/AdminOperationJournalPostgresIT.java rename to messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/AdminOperationJournalPostgresIT.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java similarity index 76% rename from messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java rename to messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java index 992ad2f..16becff 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java +++ b/messaging/messaging-reliability-jdbc-postgresql/src/integrationTest/java/dev/caskeleton/messaging/outbox/OutboxPostgresIT.java @@ -71,6 +71,9 @@ class OutboxPostgresIT { try (Connection connection = dataSource.getConnection(); Statement statement = connection.createStatement()) { statement.execute("DROP TABLE IF EXISTS messaging_outbox"); + // The inbox migration ships in the same module now, so "every migration" creates its table + // too. + statement.execute("DROP TABLE IF EXISTS messaging_inbox"); // Every migration in order, not just V1: a test that applies part of the schema certifies a // database nobody deploys. for (String migration : migrations()) { @@ -90,7 +93,7 @@ class OutboxPostgresIT { OutboxRecord record = record(); appendInTransaction(record); - List leased = repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + List leased = claim("relay-a", NOW).stream().map(OutboxLease::record).toList(); assertThat(leased) .singleElement() @@ -109,8 +112,8 @@ class OutboxPostgresIT { void aLeasedRowIsInvisibleToASecondRelayInstance() { appendInTransaction(record()); - List first = repository.leaseBatch(10, Duration.ofSeconds(30), NOW); - List second = repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + List first = claim("relay-a", NOW); + List second = claim("relay-b", NOW); assertThat(first).hasSize(1); assertThat(second).as("SKIP LOCKED plus the lease keeps two relays off the same row").isEmpty(); @@ -119,10 +122,9 @@ class OutboxPostgresIT { @Test void anExpiredLeaseBecomesClaimableAgain() { appendInTransaction(record()); - repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + claim("relay-a", NOW); - List reclaimed = - repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(31)); + List reclaimed = claim("relay-b", NOW.plusSeconds(31)); assertThat(reclaimed).hasSize(1); } @@ -131,18 +133,16 @@ class OutboxPostgresIT { void anAmbiguousRowStaysClaimableUnderItsOriginalIdentity() { OutboxRecord record = record(); appendInTransaction(record); - // Claimed rather than leased: a terminal write is checked against the lease token, so - // `markAmbiguous` takes the lease that owns the row. The identity-keyed overload this test used - // to call is gone, and with it the possibility of one relay writing an outcome onto a row - // another relay currently owns. - OutboxLease lease = - repository.claimBatch("ambiguous-relay", 10, Duration.ofSeconds(30), NOW).getFirst(); + // A terminal write is checked against the lease token, so `markAmbiguous` takes the lease that + // owns the row; no identity-keyed write exists for one relay to put an outcome on a row another + // relay currently owns. + OutboxLease lease = claim("ambiguous-relay", NOW).getFirst(); - repository.markAmbiguous(lease, "CONFIRM_TIMEOUT", NOW); + repository.markAmbiguous(lease, "CONFIRM_TIMEOUT", NOW, NOW); assertThat(repository.find(record.messageId()).orElseThrow().status()) .isEqualTo(OutboxStatus.AMBIGUOUS); - assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW)) + assertThat(claim("relay-b", NOW)) .singleElement() .satisfies(retry -> assertThat(retry.messageId()).isEqualTo(record.messageId())); } @@ -151,24 +151,24 @@ class OutboxPostgresIT { void aPublishedRowIsNotClaimedAgain() { OutboxRecord record = record(); appendInTransaction(record); - repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + OutboxLease lease = claim("relay-a", NOW).getFirst(); - repository.markPublishedUnfenced(record.messageId(), NOW); + repository.markPublished(lease, NOW); assertThat(repository.find(record.messageId()).orElseThrow().status()) .isEqualTo(OutboxStatus.PUBLISHED); - assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(60))).isEmpty(); + assertThat(claim("relay-b", NOW.plusSeconds(60))).isEmpty(); } @Test void aFailedRowIsNotClaimedAgainWithoutIntervention() { OutboxRecord record = record(); appendInTransaction(record); - repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + OutboxLease lease = claim("relay-a", NOW).getFirst(); - repository.markFailedUnfenced(record.messageId(), "INVALID_TOPIC", NOW); + repository.markFailed(lease, "INVALID_TOPIC", NOW); - assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(60))).isEmpty(); + assertThat(claim("relay-b", NOW.plusSeconds(60))).isEmpty(); } @Test @@ -202,22 +202,56 @@ class OutboxPostgresIT { OutboxRecord published = record(); OutboxRecord pending = record(); appendInTransaction(published); + repository.markPublished(claim("relay-a", NOW).getFirst(), NOW); appendInTransaction(pending); - repository.markPublishedUnfenced(published.messageId(), NOW); - int removed = repository.purgePublishedBefore(NOW.plusSeconds(1)); + int removed = repository.purgePublishedBefore(NOW.plusSeconds(1), 100); assertThat(removed).isEqualTo(1); assertThat(repository.find(pending.messageId())).isPresent(); } + /** Claims with an unlimited attempt budget, as a relay does on a row's first attempts. */ + private List claim(String owner, Instant now) { + return repository.claimBatch(owner, 10, Duration.ofSeconds(30), now, Integer.MAX_VALUE); + } + private static OutboxRecord record() { return record( // The routing key is a column now, not a reserved header smuggled through the header map. OutboxCanonicalMetadata.none().withRoutingKeys("acct-1", "acct-1")); } + @Test + void aNewerRowWaitsWhileAnOlderRowWithTheSameOrderingKeyIsUnfinished() { + // given + OutboxRecord older = record(ordered("customer-42"), NOW.minusSeconds(60)); + OutboxRecord newer = record(ordered("customer-42"), NOW.minusSeconds(30)); + OutboxRecord otherKey = record(ordered("customer-7"), NOW.minusSeconds(30)); + appendInTransaction(older); + appendInTransaction(newer); + appendInTransaction(otherKey); + + // when + List claimed = + repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); + + // then + assertThat(claimed) + .extracting(OutboxLease::messageId) + .as("a second relay, or a retry of the older row, would put the newer one first") + .containsExactlyInAnyOrder(older.messageId(), otherKey.messageId()); + } + + private static OutboxCanonicalMetadata ordered(String key) { + return OutboxCanonicalMetadata.none().withRoutingKeys(key, key); + } + private static OutboxRecord record(OutboxCanonicalMetadata metadata) { + return record(metadata, NOW.minusSeconds(60)); + } + + private static OutboxRecord record(OutboxCanonicalMetadata metadata, Instant createdAt) { return new OutboxRecord( MessageId.newId(), new DestinationName("order-events"), @@ -227,7 +261,7 @@ class OutboxPostgresIT { "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), Map.of(), metadata, - NOW.minusSeconds(60), + createdAt, OutboxStatus.PENDING, 0, Optional.empty(), @@ -256,14 +290,31 @@ class OutboxPostgresIT { if (directory == null) { throw new IOException("the messaging migrations are missing from the classpath"); } - java.io.File[] files = - new java.io.File(java.net.URI.create(directory.toString())) - .listFiles((dir, name) -> name.endsWith(".sql")); - if (files == null || files.length == 0) { + // The migrations may sit in a directory or, when the suite runs against the main jar, inside + // it; a jar URL is not a file URI, so list through a filesystem that understands either. + java.net.URI uri; + try { + uri = directory.toURI(); + } catch (java.net.URISyntaxException malformed) { + throw new IOException(malformed); + } + java.util.List names; + if ("jar".equals(uri.getScheme())) { + try (java.nio.file.FileSystem jar = + java.nio.file.FileSystems.newFileSystem(uri, java.util.Map.of()); + java.util.stream.Stream listed = + java.nio.file.Files.list(jar.getPath("db/migration/messaging"))) { + names = sqlNames(listed); + } + } else { + try (java.util.stream.Stream listed = + java.nio.file.Files.list(java.nio.file.Path.of(uri))) { + names = sqlNames(listed); + } + } + if (names.isEmpty()) { throw new IOException("no migration applied; the schema under test would be empty"); } - java.util.List names = - java.util.Arrays.stream(files).map(java.io.File::getName).sorted().toList(); java.util.List statements = new java.util.ArrayList<>(); for (String name : names) { statements.add(migration(name)); @@ -271,6 +322,15 @@ class OutboxPostgresIT { return statements; } + private static java.util.List sqlNames( + java.util.stream.Stream listed) { + return listed + .map(path -> path.getFileName().toString()) + .filter(name -> name.endsWith(".sql")) + .sorted() + .toList(); + } + private int countWhere(String predicate) { try (Connection connection = dataSource.getConnection(); Statement statement = connection.createStatement(); @@ -324,12 +384,13 @@ class OutboxPostgresIT { // Relay A claims and stalls. java.util.List first = - repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW); + repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); assertThat(first).hasSize(1); // The lease expires and relay B takes the row and publishes it. java.util.List second = - repository.claimBatch("relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31)); + repository.claimBatch( + "relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31), Integer.MAX_VALUE); assertThat(second).hasSize(1); assertThat(second.get(0).token()) .as("every claim increments the token, which is what makes the older one detectable") @@ -338,7 +399,9 @@ class OutboxPostgresIT { .isEqualTo(dev.caskeleton.messaging.reliability.OutboxTransitionResult.APPLIED); // Relay A finally wakes up and tries to record its own outcome. - assertThat(repository.markAmbiguous(first.get(0), "CONFIRM_TIMEOUT", NOW.plusSeconds(40))) + assertThat( + repository.markAmbiguous( + first.get(0), "CONFIRM_TIMEOUT", NOW.plusSeconds(40), NOW.plusSeconds(40))) .as("this write used to succeed and made a published message claimable again") .isEqualTo(dev.caskeleton.messaging.reliability.OutboxTransitionResult.STALE_LEASE); assertThat(repository.find(record.messageId()).orElseThrow().status()) @@ -350,7 +413,7 @@ class OutboxPostgresIT { OutboxRecord record = record(); appendInTransaction(record); dev.caskeleton.messaging.reliability.OutboxLease lease = - repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW).get(0); + repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE).get(0); assertThat(repository.markPublished(lease, NOW)) .isEqualTo(dev.caskeleton.messaging.reliability.OutboxTransitionResult.APPLIED); @@ -361,15 +424,15 @@ class OutboxPostgresIT { @Test void twoRelaysClaimingConcurrentlyGetDisjointRowsAndDistinctTokens() { - OutboxRecord one = record(); - OutboxRecord two = record(); + OutboxRecord one = record(OutboxCanonicalMetadata.none().withRoutingKeys("acct-1", "acct-1")); + OutboxRecord two = record(OutboxCanonicalMetadata.none().withRoutingKeys("acct-2", "acct-2")); appendInTransaction(one); appendInTransaction(two); java.util.List a = - repository.claimBatch("relay-a", 1, Duration.ofSeconds(30), NOW); + repository.claimBatch("relay-a", 1, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); java.util.List b = - repository.claimBatch("relay-b", 1, Duration.ofSeconds(30), NOW); + repository.claimBatch("relay-b", 1, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); assertThat(a).hasSize(1); assertThat(b).hasSize(1); @@ -382,9 +445,15 @@ class OutboxPostgresIT { appendInTransaction(record); long firstToken = - repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW).get(0).token(); + repository + .claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE) + .get(0) + .token(); dev.caskeleton.messaging.reliability.OutboxLease reclaimed = - repository.claimBatch("relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31)).get(0); + repository + .claimBatch( + "relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31), Integer.MAX_VALUE) + .get(0); assertThat(reclaimed.messageId()).isEqualTo(record.messageId()); assertThat(reclaimed.token()).isEqualTo(firstToken + 1); @@ -506,22 +575,16 @@ class OutboxPostgresIT { } @Test - void theStoredRoutingKeyIsTheOneBothRelaysWouldUse() { + void theStoredRoutingKeyFallsBackToTheMessageId() { OutboxRecord keyed = record(OutboxCanonicalMetadata.none().withRoutingKeys("customer-42", "customer-42")); OutboxRecord unkeyed = record(OutboxCanonicalMetadata.none()); appendInTransaction(keyed); appendInTransaction(unkeyed); - DebeziumOutboxEventRouter router = new DebeziumOutboxEventRouter(); - assertThat(routingKeyOf(keyed.messageId())) - .as("the CDC key used to be the destination, so every message on a topic shared a key") - .isEqualTo(router.routingKey(keyed)) - .isEqualTo("customer-42"); - assertThat(routingKeyOf(unkeyed.messageId())) - .as("the fallback is stated once, in the column, and the Java agrees with it") - .isEqualTo(router.routingKey(unkeyed)) - .isEqualTo(unkeyed.messageId().value().toString()); + assertThat(List.of(routingKeyOf(keyed.messageId()), routingKeyOf(unkeyed.messageId()))) + .as("the fallback is stated once, in the column") + .containsExactly("customer-42", unkeyed.messageId().value().toString()); } @Test @@ -567,6 +630,38 @@ class OutboxPostgresIT { .containsEntry("x-note", "line one\nline\ttwo"); } + /** + * The hand-rolled reader took a quote preceded by a backslash for an escaped quote, so a value + * ending in an escaped backslash ran on into the next header and both came back wrong. + */ + @Test + void aHeaderValueEndingInABackslashRoundTripsWithoutSwallowingTheNextHeader() { + // given + OutboxRecord record = + new OutboxRecord( + MessageId.newId(), + new DestinationName("order-events"), + new MessageType("order.created"), + new SchemaVersion(1), + ContentType.JSON, + "{}".getBytes(StandardCharsets.UTF_8), + Map.of("x-path", "C:\\exports\\", "x-next", "after"), + OutboxCanonicalMetadata.none(), + NOW.minusSeconds(60), + OutboxStatus.PENDING, + 0, + Optional.empty(), + Optional.empty()); + appendInTransaction(record); + + // when + Map headers = repository.find(record.messageId()).orElseThrow().headers(); + + // then + assertThat(headers) + .containsExactlyInAnyOrderEntriesOf(Map.of("x-path", "C:\\exports\\", "x-next", "after")); + } + private void appendInTransaction(OutboxRecord record) { transactions.executeWithoutResult(status -> repository.append(record)); } diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java b/messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java similarity index 92% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java rename to messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java index 1ac930c..d941d73 100644 --- a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java +++ b/messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/JdbcInboxRepository.java @@ -167,18 +167,4 @@ public final class JdbcInboxRepository implements InboxRepository { "INBOX_PURGE_FAILED", "a bounded inbox purge failed", exception); } } - - @Override - public int purgeProcessedBefore(Instant processedBefore) { - Objects.requireNonNull(processedBefore, "processedBefore must not be null"); - try (Connection connection = dataSource.getConnection(); - PreparedStatement statement = - connection.prepareStatement("DELETE FROM messaging_inbox WHERE processed_at < ?")) { - statement.setTimestamp(1, Timestamp.from(processedBefore)); - return statement.executeUpdate(); - } catch (SQLException exception) { - throw new MessagingConfigurationException( - "INBOX_PURGE_FAILED", "could not purge the inbox", exception); - } - } } diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcAdminOperationJournal.java b/messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcAdminOperationJournal.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcAdminOperationJournal.java rename to messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcAdminOperationJournal.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java b/messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java similarity index 68% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java rename to messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java index 44a986a..49eec64 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java +++ b/messaging/messaging-reliability-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/JdbcOutboxRepository.java @@ -17,6 +17,7 @@ import dev.caskeleton.messaging.reliability.OutboxRecord; import dev.caskeleton.messaging.reliability.OutboxRepository; import dev.caskeleton.messaging.reliability.OutboxStatus; import dev.caskeleton.messaging.reliability.OutboxTransitionResult; +import java.sql.Array; import java.sql.Connection; import java.sql.PreparedStatement; import java.sql.ResultSet; @@ -41,7 +42,7 @@ import org.springframework.transaction.support.TransactionSynchronizationManager * already inside, which is the entire mechanism. An outbox row written on a separate connection * commits independently of the business change and reopens the window the pattern exists to close. * - *

Leasing uses {@code SELECT ... FOR UPDATE SKIP LOCKED}. Two relay instances polling the same + *

Claiming uses {@code SELECT ... FOR UPDATE SKIP LOCKED}. Two relay instances polling the same * table would otherwise both claim the same rows and publish them twice; {@code SKIP LOCKED} lets * the second instance step over what the first already holds instead of blocking behind it. */ @@ -55,7 +56,7 @@ public final class JdbcOutboxRepository implements OutboxRepository { producer, occurred_at, correlation_id, causation_id, tenant, partition_key, ordering_key, schema_subject, schema_uri, traceparent, tracestate, baggage) - VALUES (?, ?, ?, ?, ?, ?, ?::jsonb, ?, ?, ?, + VALUES (?, ?, ?, ?, ?, ?, jsonb_object(?::text[]), ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) @@ -72,38 +73,15 @@ public final class JdbcOutboxRepository implements OutboxRepository { private static final String COLUMNS = """ message_id, destination, message_type, schema_version, content_type, - payload, headers, created_at, status, attempts, lease_expires_at, last_failure_code, + payload, + ARRAY(SELECT h.key FROM jsonb_each_text(headers) h ORDER BY h.key) AS header_keys, + ARRAY(SELECT h.value FROM jsonb_each_text(headers) h ORDER BY h.key) AS header_values, + created_at, status, attempts, lease_expires_at, last_failure_code, producer, occurred_at, correlation_id, causation_id, tenant, partition_key, ordering_key, schema_subject, schema_uri, traceparent, tracestate, baggage\ """; - private static final String LEASE = - """ - WITH claimable AS ( - SELECT message_id - FROM messaging_outbox - -- IN_FLIGHT is claimable too, but only once its lease has expired. A relay that dies - -- mid-publish leaves rows in that state; excluding them would strand those messages - -- forever, which is the one failure mode the outbox exists to prevent. - WHERE status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT') - AND (lease_expires_at IS NULL OR lease_expires_at <= ?) - ORDER BY created_at - LIMIT ? - FOR UPDATE SKIP LOCKED - ) - UPDATE messaging_outbox o - SET status = 'IN_FLIGHT', lease_expires_at = ? - FROM claimable c - WHERE o.message_id = c.message_id - RETURNING o.message_id, o.destination, o.message_type, o.schema_version, o.content_type, - o.payload, o.headers, o.created_at, o.status, o.attempts, - o.lease_expires_at, o.last_failure_code, - o.producer, o.occurred_at, o.correlation_id, o.causation_id, o.tenant, - o.partition_key, o.ordering_key, o.schema_subject, o.schema_uri, - o.traceparent, o.tracestate, o.baggage - """; - /** * The claim that records who holds the row and increments its fencing token. * @@ -113,17 +91,29 @@ public final class JdbcOutboxRepository implements OutboxRepository { private static final String CLAIM = """ WITH claimable AS ( - SELECT message_id - FROM messaging_outbox - WHERE status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT') - AND (lease_expires_at IS NULL OR lease_expires_at <= ?) + SELECT m.message_id + FROM messaging_outbox m + WHERE m.status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT') + AND (m.lease_expires_at IS NULL OR m.lease_expires_at <= ?) -- The retry clock lives in the row, not in the relay's memory. Without these two -- predicates an AMBIGUOUS row became claimable again on the very next pass, so a -- broker outage meant the whole backlog was republished every poll interval and the -- configured attempt budget was a number nothing consulted. - AND (next_attempt_at IS NULL OR next_attempt_at <= ?) - AND attempts < ? - ORDER BY created_at + AND (m.next_attempt_at IS NULL OR m.next_attempt_at <= ?) + AND m.attempts < ? + -- A row with an ordering key waits while an older row with the same key is still + -- unfinished. SKIP LOCKED let a second relay take the newer row while the first still + -- held the older one, and an AMBIGUOUS older row was retried after newer ones had + -- already gone out. md5 matches the partial index, which cannot hold a key that long. + AND (m.ordering_key IS NULL OR NOT EXISTS ( + SELECT 1 + FROM messaging_outbox older + WHERE older.destination = m.destination + AND md5(older.ordering_key) = md5(m.ordering_key) + AND older.ordering_key = m.ordering_key + AND older.status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT') + AND (older.created_at, older.message_id) < (m.created_at, m.message_id))) + ORDER BY m.created_at, m.message_id LIMIT ? FOR UPDATE SKIP LOCKED ) @@ -135,7 +125,11 @@ public final class JdbcOutboxRepository implements OutboxRepository { FROM claimable c WHERE o.message_id = c.message_id RETURNING o.message_id, o.destination, o.message_type, o.schema_version, o.content_type, - o.payload, o.headers, o.created_at, o.status, o.attempts, + o.payload, + ARRAY(SELECT h.key FROM jsonb_each_text(o.headers) h ORDER BY h.key) AS header_keys, + ARRAY(SELECT h.value FROM jsonb_each_text(o.headers) h ORDER BY h.key) + AS header_values, + o.created_at, o.status, o.attempts, o.lease_expires_at, o.last_failure_code, o.lease_owner, o.lease_token, o.producer, o.occurred_at, o.correlation_id, o.causation_id, o.tenant, o.partition_key, o.ordering_key, o.schema_subject, o.schema_uri, @@ -175,7 +169,7 @@ public final class JdbcOutboxRepository implements OutboxRepository { statement.setInt(4, record.schemaVersion().value()); statement.setString(5, record.contentType().value()); statement.setBytes(6, record.payload()); - statement.setString(7, toJson(record.headers())); + statement.setArray(7, connection.createArrayOf("text", flatten(record.headers()))); statement.setTimestamp(8, Timestamp.from(record.createdAt())); statement.setString(9, record.status().name()); statement.setInt(10, record.attempts()); @@ -246,33 +240,6 @@ public final class JdbcOutboxRepository implements OutboxRepository { } } - @Override - public List leaseBatch(int batchSize, Duration leaseDuration, Instant now) { - Objects.requireNonNull(leaseDuration, "leaseDuration must not be null"); - Objects.requireNonNull(now, "now must not be null"); - return withConnection( - connection -> { - List leased = new ArrayList<>(); - try (PreparedStatement statement = connection.prepareStatement(LEASE)) { - statement.setTimestamp(1, Timestamp.from(now)); - statement.setInt(2, batchSize); - statement.setTimestamp(3, Timestamp.from(now.plus(leaseDuration))); - try (ResultSet results = statement.executeQuery()) { - while (results.next()) { - leased.add(read(results)); - } - } - } - return List.copyOf(leased); - }); - } - - @Override - public List claimBatch( - String owner, int batchSize, Duration leaseDuration, Instant now) { - return claimBatch(owner, batchSize, leaseDuration, now, Integer.MAX_VALUE); - } - @Override public List claimBatch( String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts) { @@ -316,11 +283,6 @@ public final class JdbcOutboxRepository implements OutboxRepository { null); } - @Override - public OutboxTransitionResult markAmbiguous(OutboxLease lease, String failureCode, Instant now) { - return markAmbiguous(lease, failureCode, now, now); - } - @Override public OutboxTransitionResult markAmbiguous( OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt) { @@ -423,49 +385,6 @@ public final class JdbcOutboxRepository implements OutboxRepository { }); } - public void markPublishedUnfenced(MessageId messageId, Instant now) { - update( - "UPDATE messaging_outbox SET status = 'PUBLISHED', published_at = ?, " - + "lease_expires_at = NULL, attempts = attempts + 1 WHERE message_id = ?", - messageId, - now, - null); - } - - public void markAmbiguousUnfenced(MessageId messageId, String failureCode, Instant now) { - // The lease is released rather than held: an ambiguous row must become claimable again so the - // relay retries it under the same message id. - update( - "UPDATE messaging_outbox SET status = 'AMBIGUOUS', lease_expires_at = NULL, " - + "last_failure_code = ?, attempts = attempts + 1 WHERE message_id = ?", - messageId, - now, - failureCode); - } - - public void markFailedUnfenced(MessageId messageId, String failureCode, Instant now) { - update( - "UPDATE messaging_outbox SET status = 'FAILED', lease_expires_at = NULL, " - + "last_failure_code = ?, attempts = attempts + 1 WHERE message_id = ?", - messageId, - now, - failureCode); - } - - public void releaseLeaseUnfenced(MessageId messageId) { - withConnection( - connection -> { - try (PreparedStatement statement = - connection.prepareStatement( - "UPDATE messaging_outbox SET status = 'PENDING', lease_expires_at = NULL " - + "WHERE message_id = ? AND status = 'IN_FLIGHT'")) { - statement.setObject(1, messageId.value()); - statement.executeUpdate(); - } - return null; - }); - } - @Override public Optional find(MessageId messageId) { return withConnection( @@ -514,36 +433,6 @@ public final class JdbcOutboxRepository implements OutboxRepository { }); } - @Override - public int purgePublishedBefore(Instant publishedBefore) { - Objects.requireNonNull(publishedBefore, "publishedBefore must not be null"); - return withConnection( - connection -> { - try (PreparedStatement statement = - connection.prepareStatement( - "DELETE FROM messaging_outbox WHERE status = 'PUBLISHED' AND published_at < ?")) { - statement.setTimestamp(1, Timestamp.from(publishedBefore)); - return statement.executeUpdate(); - } - }); - } - - private void update(String sql, MessageId messageId, Instant now, String failureCode) { - withConnection( - connection -> { - try (PreparedStatement statement = connection.prepareStatement(sql)) { - if (failureCode == null) { - statement.setTimestamp(1, Timestamp.from(now)); - } else { - statement.setString(1, failureCode); - } - statement.setObject(2, messageId.value()); - statement.executeUpdate(); - } - return null; - }); - } - private static OutboxRecord read(ResultSet results) throws SQLException { Timestamp lease = results.getTimestamp("lease_expires_at"); return new OutboxRecord( @@ -553,7 +442,7 @@ public final class JdbcOutboxRepository implements OutboxRepository { new SchemaVersion(results.getInt("schema_version")), new ContentType(results.getString("content_type")), results.getBytes("payload"), - fromJson(results.getString("headers")), + headers(results.getArray("header_keys"), results.getArray("header_values")), readMetadata(results), results.getTimestamp("created_at").toInstant(), OutboxStatus.valueOf(results.getString("status")), @@ -599,119 +488,36 @@ public final class JdbcOutboxRepository implements OutboxRepository { } /** - * Serialises headers as a flat JSON object. + * The headers as one flat key, value, key, value array for {@code jsonb_object(text[])}. * - *

Hand-rolled rather than pulled from a JSON library so this module keeps no codec dependency: - * outbox headers are always flat string pairs, validated by {@code MessageHeaders} before they - * ever reach here. + *

PostgreSQL builds and escapes the JSON. The hand-rolled writer and reader this replaces got + * the escaping right one way and wrong the other: the reader took a quote after an escaped + * backslash for an escaped quote, so a value ending in a backslash ran on into the next header. */ - private static String toJson(Map headers) { - StringBuilder json = new StringBuilder("{"); - boolean first = true; + private static String[] flatten(Map headers) { + String[] flat = new String[headers.size() * 2]; + int index = 0; for (Map.Entry entry : headers.entrySet()) { - if (!first) { - json.append(','); - } - first = false; - json.append('"') - .append(escape(entry.getKey())) - .append("\":\"") - .append(escape(entry.getValue())) - .append('"'); + flat[index++] = entry.getKey(); + flat[index++] = entry.getValue(); } - return json.append('}').toString(); + return flat; } - private static Map fromJson(String json) { + /** Rebuilds the headers from the aligned key and value arrays the reads select. */ + private static Map headers(Array keys, Array values) throws SQLException { Map headers = new LinkedHashMap<>(); - if (json == null || json.length() <= 2) { + if (keys == null || values == null) { return headers; } - String body = json.substring(1, json.length() - 1); - int index = 0; - while (index < body.length()) { - int keyStart = body.indexOf('"', index); - if (keyStart < 0) { - break; - } - int keyEnd = findClosingQuote(body, keyStart + 1); - int valueStart = body.indexOf('"', body.indexOf(':', keyEnd)); - int valueEnd = findClosingQuote(body, valueStart + 1); - headers.put( - unescape(body.substring(keyStart + 1, keyEnd)), - unescape(body.substring(valueStart + 1, valueEnd))); - index = valueEnd + 1; + String[] names = (String[]) keys.getArray(); + String[] texts = (String[]) values.getArray(); + for (int index = 0; index < names.length; index++) { + headers.put(names[index], texts[index]); } return headers; } - private static int findClosingQuote(String text, int from) { - for (int index = from; index < text.length(); index++) { - if (text.charAt(index) == '"' && text.charAt(index - 1) != '\\') { - return index; - } - } - return text.length(); - } - - /** - * JSON string escaping, including the control characters. - * - *

Only backslash and quote were escaped. A header value containing a newline or a NUL — which - * the value objects now refuse, but which older rows and other writers may still carry — produced - * a document PostgreSQL rejects as invalid JSONB, so the append failed at the database with an - * error naming neither the header nor the row. Control characters are the part of the JSON - * grammar that is easy to forget and impossible to get away with. - */ - private static String escape(String value) { - StringBuilder escaped = new StringBuilder(value.length() + 8); - for (int index = 0; index < value.length(); index++) { - char character = value.charAt(index); - switch (character) { - case '\\' -> escaped.append("\\\\"); - case '"' -> escaped.append("\\\""); - case '\n' -> escaped.append("\\n"); - case '\r' -> escaped.append("\\r"); - case '\t' -> escaped.append("\\t"); - case '\b' -> escaped.append("\\b"); - case '\f' -> escaped.append("\\f"); - default -> { - if (character < 0x20) { - escaped.append("\\u%04x".formatted((int) character)); - } else { - escaped.append(character); - } - } - } - } - return escaped.toString(); - } - - private static String unescape(String value) { - StringBuilder plain = new StringBuilder(value.length()); - for (int index = 0; index < value.length(); index++) { - char character = value.charAt(index); - if (character != '\\' || index == value.length() - 1) { - plain.append(character); - continue; - } - char next = value.charAt(++index); - switch (next) { - case 'n' -> plain.append('\n'); - case 'r' -> plain.append('\r'); - case 't' -> plain.append('\t'); - case 'b' -> plain.append('\b'); - case 'f' -> plain.append('\f'); - case 'u' -> { - plain.append((char) Integer.parseInt(value.substring(index + 1, index + 5), 16)); - index += 4; - } - default -> plain.append(next); - } - } - return plain.toString(); - } - /** A unit of work against a borrowed connection. */ @FunctionalInterface private interface ConnectionCallback { diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1001__messaging_outbox.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1001__messaging_outbox.sql similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1001__messaging_outbox.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1001__messaging_outbox.sql diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1002__messaging_outbox_lease_fencing.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1002__messaging_outbox_lease_fencing.sql similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1002__messaging_outbox_lease_fencing.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1002__messaging_outbox_lease_fencing.sql diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1003__messaging_admin_operation_journal.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1003__messaging_admin_operation_journal.sql similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1003__messaging_admin_operation_journal.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1003__messaging_admin_operation_journal.sql diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1004__messaging_outbox_canonical_metadata.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1004__messaging_outbox_canonical_metadata.sql similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1004__messaging_outbox_canonical_metadata.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1004__messaging_outbox_canonical_metadata.sql diff --git a/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql new file mode 100644 index 0000000..f62af01 --- /dev/null +++ b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1005__messaging_outbox_ordering_key_index.sql @@ -0,0 +1,9 @@ +-- The claim now holds back a row while an older, unfinished row shares its destination and +-- ordering key. That check runs once per candidate row, so it needs an index or every claim scans +-- the backlog once per row it considers. +-- +-- md5 of the key rather than the key: ordering_key is VARCHAR(1024), and a long multi-byte key +-- would exceed the btree entry limit and fail the business transaction that appends the row. +CREATE INDEX ix_messaging_outbox_unfinished_ordering_key + ON messaging_outbox (destination, md5(ordering_key), created_at) + WHERE ordering_key IS NOT NULL AND status IN ('PENDING', 'AMBIGUOUS', 'IN_FLIGHT'); diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1101__messaging_inbox.sql b/messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1101__messaging_inbox.sql similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/resources/db/migration/messaging/V1101__messaging_inbox.sql rename to messaging/messaging-reliability-jdbc-postgresql/src/main/resources/db/migration/messaging/V1101__messaging_inbox.sql diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/JdbcInboxTransactionRequirementTest.java b/messaging/messaging-reliability-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/JdbcInboxTransactionRequirementTest.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/JdbcInboxTransactionRequirementTest.java rename to messaging/messaging-reliability-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/JdbcInboxTransactionRequirementTest.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/JdbcOutboxTransactionRequirementTest.java b/messaging/messaging-reliability-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/JdbcOutboxTransactionRequirementTest.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/JdbcOutboxTransactionRequirementTest.java rename to messaging/messaging-reliability-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/JdbcOutboxTransactionRequirementTest.java diff --git a/messaging/messaging-reliability/build.gradle b/messaging/messaging-reliability/build.gradle new file mode 100644 index 0000000..d2f6210 --- /dev/null +++ b/messaging/messaging-reliability/build.gradle @@ -0,0 +1,18 @@ +plugins { + id 'ca.platform-module' +} + +// The outbox relay and the inbox consumer logic, framework-free. What they store and claim goes +// through the ports in messaging-reliability-api; the relational implementation of those ports is +// messaging-reliability-jdbc-postgresql. Split so a leaf's name says what it depends on: this one +// is Java stdlib plus the messaging APIs, the other is Spring JDBC and PostgreSQL SQL. +dependencies { + api project(':messaging:messaging-core-api') + api project(':messaging:messaging-reliability-api') + // The relay hands encoded rows to the publisher as EncodedMessage. + + // Test-only. The relay is also driven through the real DefaultMessagePublisher: a fake publisher + // accepted the pre-encoded rows the real one re-encoded and rejected. + testImplementation project(':messaging:messaging-runtime-core') + testImplementation project(':messaging:messaging-policy') +} diff --git a/messaging/messaging-spring-cloud-stream-bridge/gradle.lockfile b/messaging/messaging-reliability/gradle.lockfile similarity index 92% rename from messaging/messaging-spring-cloud-stream-bridge/gradle.lockfile rename to messaging/messaging-reliability/gradle.lockfile index e2c9585..58cc2a9 100644 --- a/messaging/messaging-spring-cloud-stream-bridge/gradle.lockfile +++ b/messaging/messaging-reliability/gradle.lockfile @@ -32,6 +32,9 @@ commons-io:commons-io:2.21.0=spotbugs info.picocli:picocli:4.7.7=checkstyle io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor +io.micrometer:micrometer-commons:1.16.7=testCompileClasspath,testRuntimeClasspath +io.micrometer:micrometer-core:1.16.7=testCompileClasspath,testRuntimeClasspath +io.micrometer:micrometer-observation:1.16.7=testCompileClasspath,testRuntimeClasspath javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath @@ -57,8 +60,9 @@ org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle org.codehaus.plexus:plexus-utils:3.3.0=checkstyle org.dom4j:dom4j:2.2.0=spotbugs +org.hdrhistogram:HdrHistogram:2.2.2=testRuntimeClasspath org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath +org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath @@ -68,6 +72,7 @@ org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath org.junit:junit-bom:6.1.0=spotbugs +org.latencyutils:LatencyUtils:2.0.3=testRuntimeClasspath org.mockito:mockito-core:5.20.0=mockitoAgent org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath org.ow2.asm:asm-analysis:9.10.1=spotbugs diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/IdempotentConsumer.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/IdempotentConsumer.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/IdempotentConsumer.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/IdempotentConsumer.java diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java similarity index 94% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java index fbeda2e..6baceae 100644 --- a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java +++ b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxCleanupJob.java @@ -53,10 +53,10 @@ public final class InboxCleanupJob { int removed = 0; for (int batch = 0; batch < maxBatches; batch++) { - int deleted = inbox.purgeProcessedBefore(cutoff); + int deleted = inbox.purgeProcessedBefore(cutoff, DEFAULT_BATCH_SIZE); removed += deleted; // A short batch means the backlog is drained; continuing would just re-scan an empty range. - if (deleted == 0) { + if (deleted < DEFAULT_BATCH_SIZE) { break; } } diff --git a/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxMessageHandler.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxMessageHandler.java new file mode 100644 index 0000000..23c230f --- /dev/null +++ b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxMessageHandler.java @@ -0,0 +1,129 @@ +package dev.caskeleton.messaging.inbox; + +import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import java.time.Instant; +import java.util.Objects; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionException; +import java.util.concurrent.CompletionStage; +import java.util.function.Supplier; + +/** + * Runs a {@link MessageHandler} at most once per message and consumer, inside the inbox + * reservation's transaction. + * + *

It is a {@code MessageHandler} itself, so the consume path has one contract. It replaced an + * {@code IdempotentMessageHandler} with a different signature that nothing called (S7-#2): the + * platform wraps a handler in this when the destination's profile declares {@code + * INBOX_TRANSACTIONAL}, and the handler does not change. + * + *

The reservation and the handler's side effect commit together or not at all: + * + *

    + *
  • the message was reserved before (a redelivery) — the handler is not called and the result + * is {@code Success}, so the broker settles a message whose work is already done; + *
  • the handler answers {@code Success} — the transaction commits with the reservation; + *
  • the handler answers anything else, throws, or its stage fails — the transaction rolls back, + * the reservation with it, and the answer (or the failure) goes to the platform, which + * retries or dead-letters as it would without an inbox. + *
+ * + *

The handler's stage is awaited on the calling thread, inside the transaction. A handler that + * hands its write to another thread writes outside the transaction, and the inbox cannot protect + * that write. + * + * @param the payload type + */ +public final class InboxMessageHandler implements MessageHandler { + + private final MessageHandler delegate; + private final IdempotentConsumer inbox; + private final String consumerName; + private final Supplier clock; + + /** + * Wraps a handler. + * + * @param delegate the business handler + * @param inbox the reservation-and-effect runner + * @param consumerName the consumer identity the reservation is keyed on + * @param clock supplies the reservation instant + */ + public InboxMessageHandler( + MessageHandler delegate, + IdempotentConsumer inbox, + String consumerName, + Supplier clock) { + this.delegate = Objects.requireNonNull(delegate, "delegate must not be null"); + this.inbox = Objects.requireNonNull(inbox, "inbox must not be null"); + this.clock = Objects.requireNonNull(clock, "clock must not be null"); + if (consumerName == null || consumerName.isBlank()) { + throw new IllegalArgumentException("consumerName must not be blank"); + } + this.consumerName = consumerName; + } + + @Override + public MessageDestination destination() { + return delegate.destination(); + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + Objects.requireNonNull(delivery, "delivery must not be null"); + try { + InboxOutcome outcome = + inbox.runOnce( + delivery.message().messageId(), + consumerName, + clock.get(), + () -> { + HandleResult result = answerOf(delivery); + if (!(result instanceof HandleResult.Success)) { + // Thrown, not returned: the runner rolls back only on a throw, and a reservation + // that commits without its effect suppresses the redelivery that would apply it. + throw new NotAppliedException(result); + } + return result; + }); + return CompletableFuture.completedFuture(outcome.result().orElseGet(HandleResult::success)); + } catch (NotAppliedException notApplied) { + return CompletableFuture.completedFuture(notApplied.result); + } catch (RuntimeException failed) { + return CompletableFuture.failedFuture(failed); + } + } + + private HandleResult answerOf(MessageDelivery delivery) { + CompletionStage stage = delegate.handle(delivery); + if (stage == null) { + throw new IllegalStateException("the handler returned no stage"); + } + try { + HandleResult result = stage.toCompletableFuture().join(); + if (result == null) { + throw new IllegalStateException("the handler returned no result"); + } + return result; + } catch (CompletionException failed) { + throw failed.getCause() instanceof RuntimeException cause ? cause : failed; + } + } + + /** Carries a non-success answer out of the transaction so it rolls back. */ + private static final class NotAppliedException extends RuntimeException { + + private static final long serialVersionUID = 1L; + + @SuppressWarnings("serial") + private final transient HandleResult result; + + private NotAppliedException(HandleResult result) { + super(null, null, false, false); + this.result = result; + } + } +} diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxOutcome.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxOutcome.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxOutcome.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxOutcome.java diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxRetentionPolicy.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxRetentionPolicy.java similarity index 100% rename from messaging/messaging-inbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/inbox/InboxRetentionPolicy.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/inbox/InboxRetentionPolicy.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java similarity index 85% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java index 2bed4fc..5d067bd 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java +++ b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxCleanupJob.java @@ -15,6 +15,9 @@ import java.util.Objects; */ public final class OutboxCleanupJob { + /** How many rows one delete statement removes before yielding. */ + public static final int DEFAULT_BATCH_SIZE = 1_000; + private final OutboxRepository outbox; private final OutboxProperties properties; private final int maxBatches; @@ -47,9 +50,10 @@ public final class OutboxCleanupJob { int removed = 0; for (int batch = 0; batch < maxBatches; batch++) { - int deleted = outbox.purgePublishedBefore(cutoff); + int deleted = outbox.purgePublishedBefore(cutoff, DEFAULT_BATCH_SIZE); removed += deleted; - if (deleted == 0) { + // A short batch means the backlog is drained; asking again would re-scan an empty range. + if (deleted < DEFAULT_BATCH_SIZE) { break; } } diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactory.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactory.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactory.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactory.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxProperties.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxProperties.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxProperties.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxProperties.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java similarity index 81% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java index 64decb8..ae4c3bb 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java +++ b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelay.java @@ -1,6 +1,10 @@ package dev.caskeleton.messaging.outbox; +import dev.caskeleton.messaging.api.MessageEnvelope; import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.error.FailureCategory; +import dev.caskeleton.messaging.api.error.FailureDescriptor; +import dev.caskeleton.messaging.api.error.MessagingException; import dev.caskeleton.messaging.api.publish.MessagePublisher; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; @@ -150,7 +154,7 @@ public final class OutboxRelay { */ public OutboxRelayReport runOnce(Instant now) { Objects.requireNonNull(now, "now must not be null"); - // claimBatch, not leaseBatch: the claim carries an owner and a fencing token, and every write + // The claim carries an owner and a fencing token, and every write // below is conditioned on them. Without that, a relay that stalled past its lease could record // AMBIGUOUS over the PUBLISHED another relay had already written, making a confirmed message // claimable again. @@ -165,7 +169,22 @@ public final class OutboxRelay { for (OutboxLease lease : batch) { OutboxRecord record = lease.record(); - PublishResult result = publish(record); + MessageEnvelope envelope; + try { + envelope = envelopeFactory.toEnvelope(record); + } catch (RuntimeException unpublishable) { + // A row that cannot become an envelope never will. Thrown out of the pass, it took the + // rest of the batch with it: those rows stayed IN_FLIGHT until their lease expired, and the + // same row was claimed and thrown again on the next pass, forever. + if (repository.markFailed(lease, failureCode(unpublishable), now) + == OutboxTransitionResult.APPLIED) { + failed++; + } else { + stale++; + } + continue; + } + PublishResult result = publish(record, envelope); switch (result.completion()) { case CONFIRMED -> { if (repository.markPublished(lease, now) == OutboxTransitionResult.APPLIED) { @@ -220,12 +239,29 @@ public final class OutboxRelay { return new OutboxRelayReport(batch.size(), published, ambiguous, failed, stale, exhausted); } - private PublishResult publish(OutboxRecord record) { + private PublishResult publish(OutboxRecord record, MessageEnvelope envelope) { MessageDestination destination = new MessageDestination<>(record.destination(), record.messageType(), EncodedMessage.class); - return publisher - .publish(destination, envelopeFactory.toEnvelope(record), PublishOptions.defaults()) - .toCompletableFuture() - .join(); + try { + return publisher + .publish(destination, envelope, PublishOptions.defaults()) + .toCompletableFuture() + .join(); + } catch (RuntimeException unknown) { + // An exception says nothing about whether the broker took the message, so the row is retried + // under the same id like any other unknown outcome — not failed, not left IN_FLIGHT. + return PublishResult.ambiguous( + Duration.ZERO, + FailureDescriptor.of( + FailureCategory.AMBIGUOUS, + "PUBLISH_OUTCOME_UNKNOWN", + "the publisher threw " + unknown.getClass().getSimpleName())); + } + } + + private static String failureCode(RuntimeException failure) { + return failure instanceof MessagingException known + ? known.failure().code() + : "OUTBOX_ROW_UNPUBLISHABLE"; } } diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayReport.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayReport.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayReport.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayReport.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java similarity index 94% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java index 5a3ac4c..5de58c6 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java +++ b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRelayWorker.java @@ -34,6 +34,8 @@ public final class OutboxRelayWorker implements AutoCloseable { /** How long {@link #close()} waits for a running pass when no deadline is given. */ public static final Duration DEFAULT_DRAIN_DEADLINE = Duration.ofSeconds(30); + private static final System.Logger LOG = System.getLogger(OutboxRelayWorker.class.getName()); + private final OutboxRelay relay; private final OutboxRetryScheduler scheduler; @@ -184,9 +186,14 @@ public final class OutboxRelayWorker implements AutoCloseable { } catch (RuntimeException passFailed) { // A failed pass must not stop the loop: the scheduled task's own exception would cancel every // future pass, turning one broker error into a relay that never runs again. The failure is - // counted and the next pass backs off as if nothing was published, which is true. - failedPasses.incrementAndGet(); + // counted and the next pass backs off as if nothing was published, which is true. It is also + // logged: counted silently, a relay failing every pass looked exactly like an idle one. + long failures = failedPasses.incrementAndGet(); unproductive = unproductivePasses.incrementAndGet(); + LOG.log( + System.Logger.Level.WARNING, + "outbox relay pass failed (" + failures + " failed so far); retrying with backoff", + passFailed); } completedPasses.incrementAndGet(); schedule(scheduler.backoff(unproductive)); diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRetryScheduler.java b/messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRetryScheduler.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/main/java/dev/caskeleton/messaging/outbox/OutboxRetryScheduler.java rename to messaging/messaging-reliability/src/main/java/dev/caskeleton/messaging/outbox/OutboxRetryScheduler.java diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java similarity index 94% rename from messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java index 013fe84..b4331a6 100644 --- a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/IdempotentConsumerTest.java @@ -82,7 +82,7 @@ class IdempotentConsumerTest { void retentionPruningRemovesOldRows() { consumer.runOnce(MessageId.newId(), "order-projection", NOW, () -> "done"); - assertThat(inbox.purgeProcessedBefore(NOW.plusSeconds(1))).isEqualTo(1); + assertThat(inbox.purgeProcessedBefore(NOW.plusSeconds(1), 100)).isEqualTo(1); assertThat(inbox.size()).isZero(); } } @@ -104,12 +104,7 @@ final class InMemoryInbox implements InboxRepository { @Override public int purgeProcessedBefore(Instant processedBefore, int limit) { - return Math.min(purgeProcessedBefore(processedBefore), limit); - } - - @Override - public int purgeProcessedBefore(Instant processedBefore) { - int removed = reserved.size(); + int removed = Math.min(reserved.size(), limit); reserved.clear(); return removed; } diff --git a/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxMessageHandlerTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxMessageHandlerTest.java new file mode 100644 index 0000000..2b67205 --- /dev/null +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxMessageHandlerTest.java @@ -0,0 +1,179 @@ +package dev.caskeleton.messaging.inbox; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.ContentType; +import dev.caskeleton.messaging.api.MessageEnvelope; +import dev.caskeleton.messaging.api.MessageId; +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.ProducerId; +import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.delivery.DeliveryContext; +import dev.caskeleton.messaging.api.delivery.DeliveryMetadata; +import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.error.FailureCategory; +import dev.caskeleton.messaging.api.error.FailureDescriptor; +import dev.caskeleton.messaging.api.header.MessageHeaders; +import dev.caskeleton.messaging.reliability.InboxRepository; +import java.time.Instant; +import java.util.ArrayList; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Optional; +import java.util.Set; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionStage; +import java.util.function.Function; +import java.util.function.Supplier; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; + +/** The inbox is a {@code MessageHandler} wrapping the business handler in one transaction. */ +class InboxMessageHandlerTest { + + private static final Instant NOW = Instant.parse("2026-08-14T00:00:00Z"); + private static final DestinationName ORDERS = new DestinationName("order-events"); + + private final RollbackInbox inbox = new RollbackInbox(); + private final List effects = new ArrayList<>(); + + @Test + @DisplayName("같은 메시지가 다시 배달되면 handler를 부르지 않고 Success로 답한다") + void aRedeliveredMessageIsAnsweredWithSuccessWithoutCallingTheHandler() { + // given + MessageDelivery delivery = delivery(MessageId.newId()); + MessageHandler handler = wrap(ignored -> HandleResult.success()); + handler.handle(delivery).toCompletableFuture().join(); + + // when + HandleResult redelivered = handler.handle(delivery).toCompletableFuture().join(); + + // then + assertThat(redelivered).isInstanceOf(HandleResult.Success.class); + assertThat(effects).hasSize(1); + } + + @Test + @DisplayName("handler가 Retry로 답하면 예약이 롤백되어 다음 배달에서 다시 처리된다") + void aRetryAnswerRollsTheReservationBackSoTheNextDeliveryIsProcessed() { + // given + MessageDelivery delivery = delivery(MessageId.newId()); + List answers = + new ArrayList<>(List.of(new HandleResult.Retry(failure()), HandleResult.success())); + MessageHandler handler = wrap(ignored -> answers.removeFirst()); + HandleResult first = handler.handle(delivery).toCompletableFuture().join(); + + // when + HandleResult second = handler.handle(delivery).toCompletableFuture().join(); + + // then + assertThat(first).isInstanceOf(HandleResult.Retry.class); + assertThat(second).isInstanceOf(HandleResult.Success.class); + assertThat(effects).hasSize(2); + assertThat(inbox.reserved).hasSize(1); + } + + @Test + @DisplayName("handler가 예외를 던지면 예약이 롤백되고 실패한 stage로 전달된다") + void aThrowingHandlerRollsTheReservationBackAndFailsTheStage() { + // given + MessageHandler handler = + wrap( + ignored -> { + throw new IllegalStateException("database down"); + }); + + // when + CompletableFuture stage = + handler.handle(delivery(MessageId.newId())).toCompletableFuture(); + + // then + assertThat(stage).isCompletedExceptionally(); + assertThat(inbox.reserved).isEmpty(); + } + + private MessageHandler wrap(Function, HandleResult> body) { + MessageHandler business = + new MessageHandler<>() { + @Override + public MessageDestination destination() { + return new MessageDestination<>(ORDERS, new MessageType("order.created"), String.class); + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + effects.add(delivery.message().payload()); + return CompletableFuture.completedFuture(body.apply(delivery)); + } + }; + return new InboxMessageHandler<>( + business, new IdempotentConsumer(inbox, inbox), "order-projection", () -> NOW); + } + + private static MessageDelivery delivery(MessageId id) { + return new MessageDelivery<>( + new MessageEnvelope<>( + id, + new MessageType("order.created"), + new SchemaVersion(1), + NOW, + Optional.empty(), + new ProducerId("orders"), + Optional.empty(), + Optional.empty(), + ContentType.JSON, + Optional.empty(), + Optional.empty(), + Optional.empty(), + TraceContext.none(), + MessageHeaders.empty(), + "o-1"), + new DeliveryMetadata( + ORDERS, 1, false, Optional.empty(), Optional.empty(), Optional.empty(), NOW), + new DeliveryContext(NOW.plusSeconds(30), false, "order-projection")); + } + + private static FailureDescriptor failure() { + return FailureDescriptor.of( + FailureCategory.PROCESSING_TRANSIENT, "DOWNSTREAM_BUSY", "try again later"); + } + + /** An inbox whose reservations disappear when the transaction around them throws. */ + private static final class RollbackInbox + implements InboxRepository, IdempotentConsumer.TransactionRunner { + + private final Set reserved = new LinkedHashSet<>(); + + @Override + public T inTransaction(Supplier work) { + Set before = new LinkedHashSet<>(reserved); + try { + return work.get(); + } catch (RuntimeException rolledBack) { + reserved.clear(); + reserved.addAll(before); + throw rolledBack; + } + } + + @Override + public boolean reserve(MessageId messageId, String consumerId, Instant now) { + return reserved.add(messageId.value() + "|" + consumerId); + } + + @Override + public boolean isProcessed(MessageId messageId, String consumerId) { + return reserved.contains(messageId.value() + "|" + consumerId); + } + + @Override + public int purgeProcessedBefore(Instant processedBefore, int limit) { + return 0; + } + } +} diff --git a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java similarity index 84% rename from messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java index 17409b1..d019b69 100644 --- a/messaging/messaging-inbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/inbox/InboxOperationsTest.java @@ -7,7 +7,6 @@ import static org.assertj.core.api.Assertions.assertThatThrownBy; import dev.caskeleton.messaging.api.MessageId; import dev.caskeleton.messaging.api.error.MessagingConfigurationException; import dev.caskeleton.messaging.reliability.InboxRepository; -import dev.caskeleton.messaging.reliability.InboxResult; import java.time.Duration; import java.time.Instant; import java.util.ArrayList; @@ -42,15 +41,13 @@ class InboxOperationsTest { return reserved.contains(messageId.value() + "|" + consumerId); } - @Override - public int purgeProcessedBefore(Instant processedBefore, int limit) { - return Math.min(purgeProcessedBefore(processedBefore), limit); - } + private final List limits = new ArrayList<>(); @Override - public int purgeProcessedBefore(Instant processedBefore) { + public int purgeProcessedBefore(Instant processedBefore, int limit) { cutoffs.add(processedBefore); - return pass < deletions.size() ? deletions.get(pass++) : 0; + limits.add(limit); + return Math.min(pass < deletions.size() ? deletions.get(pass++) : 0, limit); } } @@ -91,14 +88,17 @@ class InboxOperationsTest { } @Test - void cleanupDeletesInBoundedBatches() { + void cleanupAsksForBoundedBatchesAndStopsAtTheFirstShortOne() { + // given InMemoryInbox inbox = new InMemoryInbox(List.of(1000, 500)); - int removed = - new InboxCleanupJob(inbox, policy(Duration.ofDays(7), Duration.ofDays(1)), 10).runOnce(NOW); + // when + new InboxCleanupJob(inbox, policy(Duration.ofDays(7), Duration.ofDays(1)), 10).runOnce(NOW); - assertThat(removed).isEqualTo(1500); - assertThat(inbox.cutoffs).hasSize(3); + // then + assertThat(inbox.limits) + .as("DEFAULT_BATCH_SIZE was declared and never passed to the repository") + .containsExactly(InboxCleanupJob.DEFAULT_BATCH_SIZE, InboxCleanupJob.DEFAULT_BATCH_SIZE); } @Test @@ -138,15 +138,6 @@ class InboxOperationsTest { assertThat(redelivery.result()).isEmpty(); } - @Test - void anAlreadyAppliedMessageIsSafeToSettleButAClaimedOneIsNot() { - assertThat(InboxResult.APPLIED.isSafeToSettle()).isTrue(); - assertThat(InboxResult.ALREADY_APPLIED.isSafeToSettle()).isTrue(); - assertThat(InboxResult.CLAIMED_ELSEWHERE.isSafeToSettle()) - .as("the other transaction may still roll back, and this is the last copy") - .isFalse(); - } - /** Runs the work inline; the real runner opens a database transaction. */ private static final class DirectRunner implements IdempotentConsumer.TransactionRunner { diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactoryTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactoryTest.java similarity index 100% rename from messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactoryTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxEnvelopeFactoryTest.java diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java similarity index 84% rename from messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java index f7df5cb..2a52e0e 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxOperationsTest.java @@ -35,17 +35,6 @@ class OutboxOperationsTest { throw new UnsupportedOperationException(); } - @Override - public List leaseBatch(int batchSize, Duration leaseDuration, Instant now) { - throw new UnsupportedOperationException(); - } - - @Override - public java.util.List claimBatch( - String owner, int batchSize, Duration leaseDuration, Instant now) { - return java.util.List.of(); - } - @Override public java.util.List claimBatch( String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts) { @@ -58,7 +47,7 @@ class OutboxOperationsTest { String failureCode, Instant now, Instant nextAttemptAt) { - return markAmbiguous(lease, failureCode, now); + throw new UnsupportedOperationException(); } @Override @@ -73,12 +62,6 @@ class OutboxOperationsTest { throw new UnsupportedOperationException(); } - @Override - public dev.caskeleton.messaging.reliability.OutboxTransitionResult markAmbiguous( - dev.caskeleton.messaging.reliability.OutboxLease lease, String failureCode, Instant now) { - throw new UnsupportedOperationException(); - } - @Override public dev.caskeleton.messaging.reliability.OutboxTransitionResult markFailed( dev.caskeleton.messaging.reliability.OutboxLease lease, String failureCode, Instant now) { @@ -96,15 +79,13 @@ class OutboxOperationsTest { throw new UnsupportedOperationException(); } - @Override - public int purgePublishedBefore(Instant publishedBefore, int limit) { - return Math.min(purgePublishedBefore(publishedBefore), limit); - } + private final List limits = new ArrayList<>(); @Override - public int purgePublishedBefore(Instant publishedBefore) { + public int purgePublishedBefore(Instant publishedBefore, int limit) { cutoffs.add(publishedBefore); - return pass < deletions.size() ? deletions.get(pass++) : 0; + limits.add(limit); + return Math.min(pass < deletions.size() ? deletions.get(pass++) : 0, limit); } } @@ -130,13 +111,20 @@ class OutboxOperationsTest { } @Test - void cleanupDeletesInBoundedBatchesRatherThanOneLongStatement() { + void cleanupAsksForBoundedBatchesAndStopsAtTheFirstShortOne() { + // given RecordingRepository repository = new RecordingRepository(List.of(1000, 1000, 250)); - int removed = new OutboxCleanupJob(repository, OutboxProperties.defaults(), 10).runOnce(NOW); + // when + new OutboxCleanupJob(repository, OutboxProperties.defaults(), 10).runOnce(NOW); - assertThat(removed).isEqualTo(2250); - assertThat(repository.cutoffs).hasSize(4); + // then + assertThat(repository.limits) + .as("one DELETE over the whole backlog holds its locks for as long as the backlog is big") + .containsExactly( + OutboxCleanupJob.DEFAULT_BATCH_SIZE, + OutboxCleanupJob.DEFAULT_BATCH_SIZE, + OutboxCleanupJob.DEFAULT_BATCH_SIZE); } @Test diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java similarity index 64% rename from messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java index 471ba56..62d45f8 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayTest.java @@ -8,6 +8,11 @@ import dev.caskeleton.messaging.api.MessageId; import dev.caskeleton.messaging.api.MessageType; import dev.caskeleton.messaging.api.ProducerId; import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; +import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; +import dev.caskeleton.messaging.api.delivery.OrderingScope; +import dev.caskeleton.messaging.api.destination.DestinationCapabilities; +import dev.caskeleton.messaging.api.destination.DestinationKind; import dev.caskeleton.messaging.api.destination.DestinationName; import dev.caskeleton.messaging.api.destination.MessageDestination; import dev.caskeleton.messaging.api.error.FailureCategory; @@ -19,11 +24,37 @@ import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; import dev.caskeleton.messaging.api.publish.RoutingOutcome; +import dev.caskeleton.messaging.policy.ApiTier; +import dev.caskeleton.messaging.policy.ConsumerPolicy; +import dev.caskeleton.messaging.policy.DeadLetterPolicy; +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.policy.InFlightLimiter; +import dev.caskeleton.messaging.policy.MessagingAdmissionController; +import dev.caskeleton.messaging.policy.PayloadLimitGuard; +import dev.caskeleton.messaging.policy.PayloadPolicy; +import dev.caskeleton.messaging.policy.PhysicalDestination; +import dev.caskeleton.messaging.policy.ProducerPolicy; +import dev.caskeleton.messaging.policy.RetryPolicy; +import dev.caskeleton.messaging.policy.SchemaPolicy; import dev.caskeleton.messaging.reliability.OutboxLease; import dev.caskeleton.messaging.reliability.OutboxRecord; import dev.caskeleton.messaging.reliability.OutboxRepository; import dev.caskeleton.messaging.reliability.OutboxStatus; import dev.caskeleton.messaging.reliability.OutboxTransitionResult; +import dev.caskeleton.messaging.runtime.DefaultMessagePublisher; +import dev.caskeleton.messaging.runtime.DestinationProfileRegistry; +import dev.caskeleton.messaging.runtime.TransportMessagingRuntime; +import dev.caskeleton.messaging.schema.MessageCodec; +import dev.caskeleton.messaging.schema.MessageCodecRegistry; +import dev.caskeleton.messaging.schema.RawBytesMessageCodec; +import dev.caskeleton.messaging.schema.SchemaCompatibility; +import dev.caskeleton.messaging.security.DestinationAccessPolicy; +import dev.caskeleton.messaging.transport.DefaultMessagingRuntimeRegistry; +import dev.caskeleton.messaging.transport.MessagingTransport; +import dev.caskeleton.messaging.transport.TransportConsumerRegistration; +import dev.caskeleton.messaging.transport.TransportConsumerSpec; +import dev.caskeleton.messaging.transport.TransportPublishRequest; +import dev.caskeleton.messaging.transport.TransportPublishResult; import java.nio.charset.StandardCharsets; import java.time.Duration; import java.time.Instant; @@ -32,6 +63,7 @@ import java.util.LinkedHashMap; import java.util.List; import java.util.Map; import java.util.Optional; +import java.util.Set; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionStage; import org.junit.jupiter.api.Test; @@ -54,6 +86,49 @@ class OutboxRelayTest { .isEqualTo(OutboxStatus.PUBLISHED); } + @Test + void aRowThatCannotBecomeAnEnvelopeFailsWhileTheRestOfItsBatchPublishes() { + // given + OutboxRecord poison = OutboxFixtures.withHeaders(Map.of("msg.id", "forged")); + OutboxRecord healthy = OutboxFixtures.pending(); + repository.append(poison); + repository.append(healthy); + + // when + relay(RecordingOutboxPublisher.confirming()).runOnce(NOW); + + // then + assertThat(List.of(poison, healthy)) + .extracting(row -> repository.find(row.messageId()).orElseThrow().status()) + .as("one bad row threw out of the pass and held the whole batch until its lease expired") + .containsExactly(OutboxStatus.FAILED, OutboxStatus.PUBLISHED); + } + + @Test + void aPublisherThatThrowsLeavesTheRowAmbiguousRatherThanStrandingTheBatch() { + // given + OutboxRecord first = OutboxFixtures.pending(); + repository.append(first); + MessagePublisher throwing = + new MessagePublisher() { + @Override + public CompletionStage publish( + MessageDestination destination, + MessageEnvelope message, + PublishOptions options) { + throw new IllegalStateException("the broker connection is gone"); + } + }; + + // when + relay(throwing).runOnce(NOW); + + // then + assertThat(repository.find(first.messageId()).orElseThrow().status()) + .as("an exception says nothing about whether the broker took the message") + .isEqualTo(OutboxStatus.AMBIGUOUS); + } + @Test void anAmbiguousPublishStaysRetryableRatherThanFailing() { OutboxRecord record = OutboxFixtures.pending(); @@ -124,15 +199,18 @@ class OutboxRelayTest { assertThat(repository.find(record.messageId()).orElseThrow().status()) .isEqualTo(OutboxStatus.FAILED); - assertThat(repository.leaseBatch(10, Duration.ofSeconds(30), NOW)).isEmpty(); + assertThat(repository.claimBatch("relay", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE)) + .isEmpty(); } @Test void aLeasedRowIsInvisibleToAnotherRelayInstance() { repository.append(OutboxFixtures.pending()); - List first = repository.leaseBatch(10, Duration.ofSeconds(30), NOW); - List second = repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + List first = + repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); + List second = + repository.claimBatch("relay-b", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); assertThat(first).hasSize(1); assertThat(second).isEmpty(); @@ -141,10 +219,11 @@ class OutboxRelayTest { @Test void anExpiredLeaseBecomesClaimableAgain() { repository.append(OutboxFixtures.pending()); - repository.leaseBatch(10, Duration.ofSeconds(30), NOW); + repository.claimBatch("relay-a", 10, Duration.ofSeconds(30), NOW, Integer.MAX_VALUE); - List reclaimed = - repository.leaseBatch(10, Duration.ofSeconds(30), NOW.plusSeconds(31)); + List reclaimed = + repository.claimBatch( + "relay-b", 10, Duration.ofSeconds(30), NOW.plusSeconds(31), Integer.MAX_VALUE); assertThat(reclaimed).hasSize(1); } @@ -162,6 +241,35 @@ class OutboxRelayTest { assertThat(envelope.schemaVersion()).isEqualTo(new SchemaVersion(1)); } + @Test + void aRowRelayedThroughTheRealPublisherIsMarkedPublished() { + // given + OutboxRecord record = OutboxFixtures.pending(); + repository.append(record); + + // when + relay(RealPublisher.over(new CapturingTransport())).runOnce(NOW); + + // then + assertThat(repository.find(record.messageId()).orElseThrow().status()) + .as("re-encoding the stored bytes made the codec refuse them and ended every row FAILED") + .isEqualTo(OutboxStatus.PUBLISHED); + } + + @Test + void aRowRelayedThroughTheRealPublisherReachesTheTransportWithItsStoredBytes() { + // given + OutboxRecord record = OutboxFixtures.pending(); + repository.append(record); + CapturingTransport transport = new CapturingTransport(); + + // when + relay(RealPublisher.over(transport)).runOnce(NOW); + + // then + assertThat(transport.payloads()).containsExactly(record.payload()); + } + private OutboxRelay relay(MessagePublisher publisher) { return new OutboxRelay( repository, @@ -202,6 +310,10 @@ final class OutboxFixtures { private OutboxFixtures() {} static OutboxRecord pending() { + return withHeaders(Map.of()); + } + + static OutboxRecord withHeaders(Map headers) { return new OutboxRecord( MessageId.newId(), new DestinationName("order-events"), @@ -209,7 +321,7 @@ final class OutboxFixtures { new SchemaVersion(1), ContentType.JSON, "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), - Map.of(), + headers, dev.caskeleton.messaging.reliability.OutboxCanonicalMetadata.none(), Instant.parse("2026-08-10T09:14:00Z"), OutboxStatus.PENDING, @@ -244,8 +356,7 @@ final class InMemoryOutboxRepository implements OutboxRepository { * thing it stands in for hides exactly the defects it was written to catch, so this one follows * the same three-state rule. */ - @Override - public List leaseBatch(int batchSize, Duration leaseDuration, Instant now) { + private List lease(int batchSize, Duration leaseDuration, Instant now) { List leased = new ArrayList<>(); for (OutboxRecord record : List.copyOf(rows.values())) { if (leased.size() >= batchSize) { @@ -268,11 +379,10 @@ final class InMemoryOutboxRepository implements OutboxRepository { return List.copyOf(leased); } - @Override - public List claimBatch( + private List claimIgnoringBudget( String owner, int batchSize, Duration leaseDuration, Instant now) { List claimed = new ArrayList<>(); - for (OutboxRecord record : leaseBatch(batchSize, leaseDuration, now)) { + for (OutboxRecord record : lease(batchSize, leaseDuration, now)) { long token = tokens.merge(record.messageId(), 1L, Long::sum); owners.put(record.messageId(), owner); claimed.add(new OutboxLease(record, owner, token, now.plus(leaseDuration))); @@ -300,7 +410,7 @@ final class InMemoryOutboxRepository implements OutboxRepository { public List claimBatch( String owner, int batchSize, Duration leaseDuration, Instant now, int maxAttempts) { List claimed = new ArrayList<>(); - for (OutboxLease lease : claimBatch(owner, batchSize, leaseDuration, now)) { + for (OutboxLease lease : claimIgnoringBudget(owner, batchSize, leaseDuration, now)) { Instant nextAttempt = nextAttempts.get(lease.messageId()); boolean waiting = nextAttempt != null && now.isBefore(nextAttempt); if (waiting || lease.record().attempts() >= maxAttempts) { @@ -316,11 +426,13 @@ final class InMemoryOutboxRepository implements OutboxRepository { @Override public OutboxTransitionResult markAmbiguous( OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt) { - OutboxTransitionResult result = markAmbiguous(lease, failureCode, now); - if (result == OutboxTransitionResult.APPLIED) { - nextAttempts.put(lease.messageId(), nextAttemptAt); + if (!owns(lease)) { + return OutboxTransitionResult.STALE_LEASE; } - return result; + markAmbiguousUnfenced(lease.messageId(), failureCode); + owners.remove(lease.messageId()); + nextAttempts.put(lease.messageId(), nextAttemptAt); + return OutboxTransitionResult.APPLIED; } @Override @@ -344,16 +456,6 @@ final class InMemoryOutboxRepository implements OutboxRepository { return OutboxTransitionResult.APPLIED; } - @Override - public OutboxTransitionResult markAmbiguous(OutboxLease lease, String failureCode, Instant now) { - if (!owns(lease)) { - return OutboxTransitionResult.STALE_LEASE; - } - markAmbiguousUnfenced(lease.messageId(), failureCode); - owners.remove(lease.messageId()); - return OutboxTransitionResult.APPLIED; - } - @Override public OutboxTransitionResult markFailed(OutboxLease lease, String failureCode, Instant now) { if (!owns(lease)) { @@ -394,14 +496,9 @@ final class InMemoryOutboxRepository implements OutboxRepository { @Override public int purgePublishedBefore(Instant publishedBefore, int limit) { - return Math.min(purgePublishedBefore(publishedBefore), limit); - } - - @Override - public int purgePublishedBefore(Instant publishedBefore) { int before = rows.size(); rows.values().removeIf(record -> record.status() == OutboxStatus.PUBLISHED); - return before - rows.size(); + return Math.min(before - rows.size(), limit); } private void transition(MessageId messageId, OutboxStatus status, Optional failureCode) { @@ -480,3 +577,110 @@ final class RecordingOutboxPublisher implements MessagePublisher { }; } } + +/** The platform's own publisher over a stub transport, registered for the fixture's destination. */ +final class RealPublisher { + + private static final DestinationName ORDERS = new DestinationName("order-events"); + + private RealPublisher() {} + + static DefaultMessagePublisher over(MessagingTransport transport) { + DefaultMessagingRuntimeRegistry runtimes = new DefaultMessagingRuntimeRegistry(); + runtimes.install(new TransportMessagingRuntime("fake", 1, transport)); + // A real codec. Like every codec it refuses a payload that is not its own type, which is what + // an already-encoded row is to it. + MessageCodec raw = new RawBytesMessageCodec(); + return new DefaultMessagePublisher( + DestinationProfileRegistry.of(Map.of(ORDERS, profile())), + new DestinationAccessPolicy(Set.of(ORDERS), Set.of(), Set.of()), + new MessageCodecRegistry() { + @Override + public Optional find(ContentType contentType) { + return Optional.of(raw); + } + + @Override + public MessageCodec defaultCodec() { + return raw; + } + }, + new MessagingAdmissionController( + new PayloadLimitGuard(PayloadPolicy.defaults()), + new InFlightLimiter(4, Duration.ofSeconds(1))), + runtimes, + transport); + } + + private static DestinationProfile profile() { + return new DestinationProfile( + ORDERS, + "fake", + DestinationKind.EVENT_STREAM, + PhysicalDestination.kafkaTopic("order.events.v1"), + new SchemaPolicy( + ContentType.JSON, + SchemaCompatibility.BACKWARD_TRANSITIVE, + Set.of(new MessageType("order.created"))), + DeliveryGuarantee.AT_LEAST_ONCE, + OrderingScope.NONE, + ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, + ProducerPolicy.defaults(), + ConsumerPolicy.defaults("order-projection"), + RetryPolicy.none(), + DeadLetterPolicy.disabled(), + PayloadPolicy.defaults(), + ApiTier.M1, + false, + false, + false); + } +} + +/** A transport that confirms every publish and keeps the bytes it was handed. */ +final class CapturingTransport implements MessagingTransport { + + private final List payloads = new ArrayList<>(); + + List payloads() { + return List.copyOf(payloads); + } + + @Override + public CompletionStage publish(TransportPublishRequest request) { + payloads.add(request.envelope().payload().bytes()); + return CompletableFuture.completedFuture( + new TransportPublishResult( + new PublishResult( + PublishCompletion.CONFIRMED, + PublishEvidence.confirmed(ConfirmationLevel.BROKER_ACK), + RoutingOutcome.ROUTED, + Optional.empty(), + 1, + Duration.ZERO, + Optional.empty()))); + } + + @Override + public TransportConsumerRegistration register(TransportConsumerSpec spec) { + throw new UnsupportedOperationException("the relay only publishes"); + } + + @Override + public DestinationCapabilities capabilities(DestinationName destination) { + throw new UnsupportedOperationException("the relay asks for no publish option"); + } + + @Override + public String brokerName() { + return "fake"; + } + + @Override + public long generation() { + return 1; + } + + @Override + public void close() {} +} diff --git a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java similarity index 87% rename from messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java rename to messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java index cadc679..9f71564 100644 --- a/messaging/messaging-outbox-jdbc-postgresql/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java +++ b/messaging/messaging-reliability/src/test/java/dev/caskeleton/messaging/outbox/OutboxRelayWorkerTest.java @@ -79,8 +79,7 @@ class OutboxRelayWorkerTest { @Test @DisplayName("a pass that throws does not stop every pass after it") void aFailedPassDoesNotEndTheLoop() throws InterruptedException { - repository.append(OutboxFixtures.pending()); - OutboxRelayWorker worker = worker(new ThrowingPublisher()); + OutboxRelayWorker worker = worker(unreachableDatabase(), new ThrowingPublisher()); worker.start(); try { @@ -99,9 +98,25 @@ class OutboxRelayWorkerTest { } private OutboxRelayWorker worker(MessagePublisher publisher) { + return worker(repository, publisher); + } + + /** An outbox whose every call fails, as it does while its database is down. */ + private static dev.caskeleton.messaging.reliability.OutboxRepository unreachableDatabase() { + return (dev.caskeleton.messaging.reliability.OutboxRepository) + java.lang.reflect.Proxy.newProxyInstance( + OutboxRelayWorkerTest.class.getClassLoader(), + new Class[] {dev.caskeleton.messaging.reliability.OutboxRepository.class}, + (proxy, method, arguments) -> { + throw new IllegalStateException("the outbox database is unreachable"); + }); + } + + private static OutboxRelayWorker worker( + dev.caskeleton.messaging.reliability.OutboxRepository outbox, MessagePublisher publisher) { OutboxRelay relay = new OutboxRelay( - repository, + outbox, publisher, new OutboxEnvelopeFactory(new ProducerId("order-api")), 100, diff --git a/messaging/messaging-runtime-core/build.gradle b/messaging/messaging-runtime-core/build.gradle index 5fb6829..06c462f 100644 --- a/messaging/messaging-runtime-core/build.gradle +++ b/messaging/messaging-runtime-core/build.gradle @@ -11,7 +11,6 @@ plugins { // with an application-supplied fake would pass a context test while running none of them. dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') api project(':messaging:messaging-policy') api project(':messaging:messaging-transport-spi') api project(':messaging:messaging-security') diff --git a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessor.java b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessor.java index 7c9dfd7..4f1169d 100644 --- a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessor.java +++ b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessor.java @@ -1,110 +1,255 @@ package dev.caskeleton.messaging.runtime; import dev.caskeleton.messaging.api.MessageEnvelope; +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.delivery.DeliveryContext; +import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.publish.PublishCompletion; import dev.caskeleton.messaging.api.settlement.SettlementResult; +import dev.caskeleton.messaging.policy.DestinationProfile; import dev.caskeleton.messaging.schema.EncodedMessage; +import dev.caskeleton.messaging.schema.MessageCodec; +import dev.caskeleton.messaging.schema.MessageCodecRegistry; import dev.caskeleton.messaging.transport.TransportDelivery; import java.time.Duration; +import java.time.InstantSource; +import java.util.HashMap; +import java.util.List; +import java.util.Map; import java.util.Objects; import java.util.Optional; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionStage; import java.util.concurrent.atomic.AtomicBoolean; -import java.util.function.Function; +import java.util.function.Supplier; /** - * Turns one handler outcome into exactly one broker settlement. + * Turns one still-encoded delivery into a typed {@link MessageHandler} call and exactly one broker + * settlement. * - *

Nothing connected {@code HandleResult} to a settlement. Each broker adapter decided for itself - * what a retry or a dead-letter meant, so "the platform decides when and in what order the - * settlement happens" — the comment on {@code TransportSettlement} — described a decision nobody - * made in one place. + *

This is the consume path the platform had three half-contracts for and no assembly (S7-#2): a + * typed {@code MessageHandler} nothing called, a processor that took an untyped function, and an + * inbox handler with its own signature. There is now one handler contract, and this class is the + * one place that decodes for it, calls it, and settles. * - *

Two rules are enforced here rather than left to each adapter: + *

Rules enforced here rather than left to each adapter: * *

    *
  • One terminal call. A delivery is acknowledged, requeued or discarded once. - * A second call is a programming error, not something to let through: acknowledging after a - * requeue tells the broker the message is done while a copy is already in flight. - *
  • Dead-letter before acknowledgement. The source is acknowledged only after - * the dead-letter publish is confirmed. Acknowledging first and publishing afterwards loses - * the message whenever the second step fails, which is exactly when it matters. + * Acknowledging after a requeue tells the broker the message is done while a copy is already + * in flight. + *
  • Acknowledge only after success. {@code Success} acknowledges; a handler + * that throws, fails its stage or returns nothing is requeued, not discarded. + *
  • Dead letter before acknowledgement. The source is acknowledged only after + * the dead-letter publish is confirmed; an unconfirmed one requeues, because a duplicate is + * the recoverable half of that choice. + *
  • Discard only where the profile allows it. {@code Reject} discards on an + * {@code AT_MOST_ONCE} destination and is dead-lettered everywhere else (design §19.2). + *
  • Undecodable is not retryable. A message whose type has no handler, or + * whose payload the codec refuses, is dead-lettered rather than retried forever. *
+ * + *

How many times a requeued delivery comes back, and what happens when the attempts run out, is + * the transport's retry policy — {@code requeue} is a request, and the Kafka consumer counts it. */ public final class DefaultDeliveryProcessor { - private final Function, HandleResult> handler; - + private final DestinationProfile profile; + private final Map> handlers; + private final MessageCodecRegistry codecs; private final DeadLetterPublisher deadLetters; + private final String consumerId; + private final InstantSource clock; - private final Duration retryDelay; - + /** + * Creates a processor for one destination. + * + * @param profile the destination being consumed + * @param handlers the handlers for that destination, at most one per message type + * @param codecs decodes payloads by content type + * @param deadLetters publishes a delivery to the destination's dead-letter destination + * @param consumerId the stable, low-cardinality consumer identity handlers see + * @param clock the clock handler deadlines are measured on + */ public DefaultDeliveryProcessor( - Function, HandleResult> handler, + DestinationProfile profile, + List> handlers, + MessageCodecRegistry codecs, DeadLetterPublisher deadLetters, - Duration retryDelay) { - this.handler = Objects.requireNonNull(handler, "handler must not be null"); + String consumerId, + InstantSource clock) { + this.profile = Objects.requireNonNull(profile, "profile must not be null"); + this.codecs = Objects.requireNonNull(codecs, "codecs must not be null"); this.deadLetters = Objects.requireNonNull(deadLetters, "deadLetters must not be null"); - this.retryDelay = Objects.requireNonNull(retryDelay, "retryDelay must not be null"); - if (retryDelay.isNegative()) { - throw new IllegalArgumentException("a retry delay must not be negative"); + this.clock = Objects.requireNonNull(clock, "clock must not be null"); + if (consumerId == null || consumerId.isBlank()) { + throw new IllegalArgumentException("consumerId must not be blank"); } + this.consumerId = consumerId; + Map> byType = new HashMap<>(); + for (MessageHandler handler : Objects.requireNonNull(handlers, "handlers")) { + if (!handler.destination().name().equals(profile.name())) { + throw new IllegalArgumentException( + "handler for " + + handler.destination().name().value() + + " cannot consume " + + profile.name().value()); + } + if (byType.put(handler.destination().messageType(), handler) != null) { + throw new IllegalArgumentException( + "destination " + + profile.name().value() + + " has two handlers for " + + handler.destination().messageType().value() + + "; which one runs would depend on bean order"); + } + } + if (byType.isEmpty()) { + throw new IllegalArgumentException("a processor needs at least one handler"); + } + this.handlers = Map.copyOf(byType); } /** * Handles one delivery and settles it once. * - * @return a stage completing with the settlement the broker performed + * @param delivery the still-encoded delivery + * @return a stage completing once the settlement call has returned */ - public CompletionStage process(TransportDelivery delivery) { + public CompletionStage process(TransportDelivery delivery) { Objects.requireNonNull(delivery, "delivery must not be null"); OneShotSettlement settlement = new OneShotSettlement(delivery); - HandleResult result; - try { - result = handler.apply(delivery.envelope()); - } catch (RuntimeException handlerFailed) { - // A handler that threw is a retry, not a discard. Treating an exception as "this message is - // undeliverable" is how a transient bug in one consumer silently drops a day of traffic — - // and it is exactly what the Rabbit consumer did by folding handler exceptions into its - // deserialization-failure path. - return settlement.requeue(retryDelay); + MessageEnvelope envelope = delivery.envelope(); + MessageHandler handler = handlers.get(envelope.messageType()); + if (handler == null) { + return deadLetter( + delivery, + settlement, + FailureDescriptor.of( + FailureCategory.DESERIALIZATION, + "NO_HANDLER_FOR_MESSAGE_TYPE", + "no handler is registered for this message type on this destination")); } - if (result == null) { - return settlement.requeue(retryDelay); + return dispatch(handler, delivery, settlement); + } + + private CompletionStage dispatch( + MessageHandler handler, TransportDelivery delivery, OneShotSettlement settlement) { + MessageEnvelope envelope = delivery.envelope(); + T payload; + try { + MessageCodec codec = + codecs + .find(envelope.contentType()) + .orElseThrow( + () -> + new IllegalStateException( + "no codec for content type " + envelope.contentType().value())); + payload = + codec.decode( + envelope.messageType(), + envelope.schemaVersion(), + envelope.payload(), + handler.destination().payloadType()); + } catch (RuntimeException undecodable) { + return deadLetter( + delivery, + settlement, + new FailureDescriptor( + FailureCategory.DESERIALIZATION, + "PAYLOAD_UNDECODABLE", + false, + "the payload could not be decoded into the handler's type", + Optional.of(undecodable.getClass().getSimpleName()))); } + Duration timeout = profile.consumer().handlerTimeout(); + MessageDelivery typed = + new MessageDelivery<>( + envelope.withPayload(payload), + delivery.metadata(), + new DeliveryContext(clock.instant().plus(timeout), false, consumerId)); + + CompletionStage outcome; + try { + outcome = handler.handle(typed); + } catch (RuntimeException handlerFailed) { + // A handler that threw is a retry, not a discard. Treating an exception as "this message is + // undeliverable" is how a transient bug in one consumer silently drops a day of traffic. + return settlement.requeue(); + } + if (outcome == null) { + return settlement.requeue(); + } + return outcome + .handle( + (result, failure) -> + failure == null ? Optional.ofNullable(result) : Optional.empty()) + .thenCompose( + result -> + result.isEmpty() + ? settlement.requeue() + : settle(result.get(), delivery, settlement)); + } + + private CompletionStage settle( + HandleResult result, TransportDelivery delivery, OneShotSettlement settlement) { return switch (result) { case HandleResult.Success ignored -> settlement.acknowledge(); - case HandleResult.Retry ignored -> settlement.requeue(retryDelay); - case HandleResult.Reject ignored -> settlement.discard(); + case HandleResult.Retry ignored -> settlement.requeue(); + case HandleResult.Reject reject -> + profile.deliveryGuarantee() == DeliveryGuarantee.AT_MOST_ONCE + ? settlement.discard() + // Discarding on an at-least-once destination would drop a message the profile + // promised to deliver, so the rejection is kept where someone can look at it. + : deadLetter(delivery, settlement, reject.failure()); case HandleResult.DeadLetter deadLetter -> - deadLetters - .publish(delivery.envelope(), deadLetter.failure()) - .thenCompose( - completion -> - completion == PublishCompletion.CONFIRMED - // Confirmed: the message exists somewhere else, so removing it here is - // safe. - ? settlement.acknowledge() - // Not confirmed — rejected or ambiguous. Requeueing risks a duplicate; - // acknowledging loses the message outright, and a duplicate is the - // recoverable half of that choice. - : settlement.requeue(retryDelay)); + deadLetter(delivery, settlement, deadLetter.failure()); }; } + private CompletionStage deadLetter( + TransportDelivery delivery, OneShotSettlement settlement, FailureDescriptor failure) { + if (!profile.deadLetter().enabled()) { + // Nowhere to put it. Requeueing keeps the only copy; the transport's retry budget decides + // whether the partition then parks it or keeps it. + return settlement.requeue(); + } + MessageDelivery encoded = + new MessageDelivery<>( + delivery.envelope(), + delivery.metadata(), + new DeliveryContext(clock.instant(), false, consumerId)); + CompletionStage published; + try { + published = deadLetters.publish(encoded, failure); + } catch (RuntimeException publishFailed) { + return settlement.requeue(); + } + return published + .handle((completion, error) -> error == null && completion == PublishCompletion.CONFIRMED) + .thenCompose(confirmed -> confirmed ? settlement.acknowledge() : settlement.requeue()); + } + /** Publishes a delivery that cannot be handled to its dead-letter destination. */ @FunctionalInterface public interface DeadLetterPublisher { - /** Publishes and reports what the broker confirmed. */ + /** + * Publishes and reports what the broker confirmed. + * + * @param delivery the delivery, still encoded + * @param failure why it is being dead-lettered + * @return a stage completing with the publish completion + */ CompletionStage publish( - MessageEnvelope envelope, FailureDescriptor failure); + MessageDelivery delivery, FailureDescriptor failure); } /** A settlement that can be used once. */ @@ -118,20 +263,21 @@ public final class DefaultDeliveryProcessor { this.delivery = delivery; } - private CompletionStage acknowledge() { + private CompletionStage acknowledge() { return once("acknowledge", () -> delivery.settlement().acknowledge()); } - private CompletionStage requeue(Duration delay) { - return once("requeue", () -> delivery.settlement().requeue(delay)); + private CompletionStage requeue() { + // The delay is the transport's retry policy; zero asks for "as soon as that policy allows". + return once("requeue", () -> delivery.settlement().requeue(Duration.ZERO)); } - private CompletionStage discard() { + private CompletionStage discard() { return once("discard", () -> delivery.settlement().discard()); } - private CompletionStage once( - String what, java.util.function.Supplier> call) { + private CompletionStage once( + String what, Supplier> call) { if (!settled.compareAndSet(false, true)) { return CompletableFuture.failedFuture( new IllegalStateException( @@ -139,17 +285,7 @@ public final class DefaultDeliveryProcessor { + what + " would tell the broker something the first call already contradicted")); } - return call.get(); + return call.get().thenApply(ignored -> null); } } - - /** A descriptor for a handler that returned nothing, kept out of the hot path. */ - static FailureDescriptor missingResult() { - return new FailureDescriptor( - FailureCategory.PROCESSING_TRANSIENT, - "HANDLER_RETURNED_NOTHING", - true, - "the handler returned no result, which is treated as a retry rather than a discard", - Optional.empty()); - } } diff --git a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java index 4942565..0f6d9e2 100644 --- a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java +++ b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java @@ -5,11 +5,8 @@ import dev.caskeleton.messaging.api.destination.MessageDestination; import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; import dev.caskeleton.messaging.policy.DestinationProfile; import dev.caskeleton.messaging.policy.MessagingAdmissionController; import dev.caskeleton.messaging.schema.EncodedMessage; @@ -224,7 +221,16 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM startedAt); } - admission.admit(destination.name().value(), encoded.payload().bytes().length); + // Refusals are results, not exceptions. They were thrown out of publish() on the caller's + // thread, so a CompletionStage caller — the outbox relay, the reactive facade — saw neither a + // REJECTED result nor a failed stage. The permit wait is bounded by what is left of the + // caller's deadline, not by the limiter's own bound. + try { + admission.admit(destination.name().value(), encoded.payload().bytes().length, remaining); + } catch (dev.caskeleton.messaging.api.error.MessageTooLargeException + | dev.caskeleton.messaging.api.error.MessageBackpressureException refused) { + return refused(refused.failure(), startedAt); + } MessagingRuntimeLease lease = null; try { lease = runtimes.acquire(profile.broker()); @@ -279,7 +285,12 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM */ private void requireSupportedOptions(DestinationProfile profile, PublishOptions options) { if (options.deduplication().isPresent() - && !transport.capabilities(profile.name()).capabilities().deduplicatedPublish()) { + && !transport + .capabilities(profile.name()) + .capabilities() + .supports( + dev.caskeleton.messaging.api.destination.MessagingCapabilities.Capability + .DEDUPLICATED_PUBLISH)) { throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( "PUBLISH_DEDUPLICATION_UNSUPPORTED", "destination '" @@ -301,6 +312,14 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM } private MessageEnvelope encode(MessageEnvelope message) { + // The outbox relay, dead lettering and the retry topic republish bytes a codec already wrote. + // Encoding them again handed the codec a payload that is not its type, so each such publish was + // REJECTED (or, with a lenient codec, sent the envelope's toString). They go out as they are. + if (message.payload() instanceof EncodedMessage alreadyEncoded) { + MessageEnvelope encoded = message.withPayload(alreadyEncoded); + requireEncodedEnvelope(encoded); + return encoded; + } MessageCodec codec = codecs.find(message.contentType()).orElseGet(codecs::defaultCodec); EncodedMessage bytes = codec.encode(message.messageType(), message.schemaVersion(), message.payload()); @@ -308,30 +327,20 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM } private CompletionStage rejected(String code, String reason, long startedAt) { + return refused( + new FailureDescriptor(FailureCategory.CONFIGURATION, code, false, reason, Optional.empty()), + startedAt); + } + + private CompletionStage refused(FailureDescriptor failure, long startedAt) { return CompletableFuture.completedFuture( - new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - elapsedSince(startedAt), - Optional.of( - new FailureDescriptor( - FailureCategory.CONFIGURATION, code, false, reason, Optional.empty())))); + PublishResult.rejected(elapsedSince(startedAt), failure)); } private PublishResult ambiguousResult(String code, String reason, long startedAt) { - return new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, + return PublishResult.ambiguous( elapsedSince(startedAt), - Optional.of( - new FailureDescriptor( - FailureCategory.AMBIGUOUS, code, true, reason, Optional.empty()))); + new FailureDescriptor(FailureCategory.AMBIGUOUS, code, true, reason, Optional.empty())); } /** diff --git a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessorTest.java b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessorTest.java index 9b2d007..8cff2f2 100644 --- a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessorTest.java +++ b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultDeliveryProcessorTest.java @@ -1,6 +1,7 @@ package dev.caskeleton.messaging.runtime; import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.catchThrowable; import dev.caskeleton.messaging.api.ContentType; import dev.caskeleton.messaging.api.MessageEnvelope; @@ -9,123 +10,318 @@ import dev.caskeleton.messaging.api.MessageType; import dev.caskeleton.messaging.api.ProducerId; import dev.caskeleton.messaging.api.SchemaVersion; import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; import dev.caskeleton.messaging.api.delivery.DeliveryMetadata; import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.header.MessageHeaders; import dev.caskeleton.messaging.api.publish.PublishCompletion; import dev.caskeleton.messaging.api.settlement.SettlementResult; +import dev.caskeleton.messaging.policy.DeadLetterPolicy; +import dev.caskeleton.messaging.policy.DestinationProfile; import dev.caskeleton.messaging.schema.EncodedMessage; +import dev.caskeleton.messaging.schema.MessageCodec; import dev.caskeleton.messaging.transport.TransportDelivery; import dev.caskeleton.messaging.transport.TransportSettlement; import java.nio.charset.StandardCharsets; import java.time.Duration; import java.time.Instant; +import java.time.InstantSource; import java.util.ArrayList; import java.util.List; import java.util.Optional; import java.util.concurrent.CompletableFuture; import java.util.concurrent.CompletionStage; +import java.util.function.Function; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; /** - * One handler outcome, one settlement. + * One typed handler, one outcome, one settlement. * - *

Nothing turned a {@code HandleResult} into a settlement: each broker adapter decided for - * itself, and the Rabbit consumer folded handler exceptions into its deserialization-failure path - * and discarded them. + *

The processor took an untyped function and nothing assembled it; the typed {@code + * MessageHandler} had no caller (S7-#2). These tests hold the settlement rules on the one contract + * that is left. */ class DefaultDeliveryProcessorTest { + private static final DestinationName ORDERS = new DestinationName("order-events"); + private static final MessageType CREATED = new MessageType("order.created"); + private static final Instant NOW = Instant.parse("2026-08-14T00:00:00Z"); + private final RecordingSettlement settlement = new RecordingSettlement(); @Test - @DisplayName("success acknowledges once") - void successAcknowledgesOnce() { - process(envelope -> new HandleResult.Success(), confirmedDeadLetters()); + @DisplayName("성공하면 handler가 decode된 payload를 받고 한 번 acknowledge된다") + void aSuccessfulHandlerReceivesTheDecodedPayloadAndIsAcknowledgedOnce() { + // given + List received = new ArrayList<>(); + MessageHandler handler = + handler( + delivery -> { + received.add(delivery.message().payload()); + return HandleResult.success(); + }); + // when + process(handler, confirmedDeadLetters(), RuntimeFixtures.profile(ORDERS)); + + // then + assertThat(received).containsExactly("{\"orderId\":\"o-1\"}"); assertThat(settlement.calls()).containsExactly("acknowledge"); } @Test - @DisplayName("retry requeues with the configured delay") - void retryRequeues() { - process(envelope -> new HandleResult.Retry(failure()), confirmedDeadLetters()); + @DisplayName("Retry 결과는 requeue된다") + void aRetryResultIsRequeued() { + // when + process( + handler(delivery -> new HandleResult.Retry(failure())), + confirmedDeadLetters(), + RuntimeFixtures.profile(ORDERS)); + // then assertThat(settlement.calls()).containsExactly("requeue"); } @Test - @DisplayName("a handler that throws is retried, not discarded") - void aThrowingHandlerIsRetried() { + @DisplayName("예외를 던진 handler는 버려지지 않고 requeue된다") + void aThrowingHandlerIsRequeuedNotDiscarded() { + // when process( - envelope -> { - throw new IllegalStateException("a bug in the projection"); - }, - confirmedDeadLetters()); + handler( + delivery -> { + throw new IllegalStateException("a bug in the projection"); + }), + confirmedDeadLetters(), + RuntimeFixtures.profile(ORDERS)); - assertThat(settlement.calls()) - .as("treating an exception as undeliverable drops a day of traffic over a transient bug") - .containsExactly("requeue"); + // then + assertThat(settlement.calls()).containsExactly("requeue"); } @Test - @DisplayName("a dead letter is published before the source is acknowledged") - void aDeadLetterIsPublishedBeforeAcknowledgement() { + @DisplayName("실패로 끝난 stage를 돌려준 handler는 requeue된다") + void aHandlerWhoseStageFailsIsRequeued() { + // given + MessageHandler handler = + new StringHandler( + delivery -> { + throw new UnsupportedOperationException("unused"); + }) { + @Override + public CompletionStage handle(MessageDelivery delivery) { + return CompletableFuture.failedFuture(new IllegalStateException("database down")); + } + }; + + // when + process(handler, confirmedDeadLetters(), RuntimeFixtures.profile(ORDERS)); + + // then + assertThat(settlement.calls()).containsExactly("requeue"); + } + + @Test + @DisplayName("dead letter는 발행이 확인된 뒤에 source를 acknowledge한다") + void aDeadLetterIsPublishedBeforeTheSourceIsAcknowledged() { + // given List order = new ArrayList<>(); DefaultDeliveryProcessor.DeadLetterPublisher publisher = - (envelope, failure) -> { + (delivery, failure) -> { order.add("dead-letter published"); return CompletableFuture.completedFuture(PublishCompletion.CONFIRMED); }; settlement.recordInto(order); - process(envelope -> new HandleResult.DeadLetter(failure()), publisher); + // when + process( + handler(delivery -> new HandleResult.DeadLetter(failure())), + publisher, + RuntimeFixtures.profile(ORDERS)); + // then assertThat(order).containsExactly("dead-letter published", "acknowledge"); } @Test - @DisplayName("an unconfirmed dead letter requeues rather than acknowledging") - void anUnconfirmedDeadLetterRequeues() { + @DisplayName("확인되지 않은 dead letter는 acknowledge하지 않고 requeue한다") + void anUnconfirmedDeadLetterIsRequeued() { + // when process( - envelope -> new HandleResult.DeadLetter(failure()), - (envelope, failure) -> CompletableFuture.completedFuture(PublishCompletion.AMBIGUOUS)); + handler(delivery -> new HandleResult.DeadLetter(failure())), + (delivery, failure) -> CompletableFuture.completedFuture(PublishCompletion.AMBIGUOUS), + RuntimeFixtures.profile(ORDERS)); - assertThat(settlement.calls()) - .as("acknowledging loses the message outright; a duplicate is the recoverable half") - .containsExactly("requeue"); + // then + assertThat(settlement.calls()).containsExactly("requeue"); } @Test - @DisplayName("a rejected message is discarded") - void aRejectedMessageIsDiscarded() { - process(envelope -> new HandleResult.Reject(failure()), confirmedDeadLetters()); + @DisplayName("at-least-once destination의 Reject는 버리지 않고 dead letter로 보낸다") + void aRejectOnAnAtLeastOnceDestinationIsDeadLetteredNotDiscarded() { + // given + List deadLettered = new ArrayList<>(); + // when + process( + handler(delivery -> new HandleResult.Reject(failure())), + (delivery, failure) -> { + deadLettered.add(failure); + return CompletableFuture.completedFuture(PublishCompletion.CONFIRMED); + }, + RuntimeFixtures.profile(ORDERS)); + + // then + assertThat(deadLettered).containsExactly(failure()); + assertThat(settlement.calls()).containsExactly("acknowledge"); + } + + @Test + @DisplayName("at-most-once destination의 Reject는 discard된다") + void aRejectOnAnAtMostOnceDestinationIsDiscarded() { + // when + process( + handler(delivery -> new HandleResult.Reject(failure())), + confirmedDeadLetters(), + atMostOnce(RuntimeFixtures.profile(ORDERS))); + + // then assertThat(settlement.calls()).containsExactly("discard"); } @Test - @DisplayName("a handler returning nothing is retried") - void aHandlerReturningNothingIsRetried() { - process(envelope -> null, confirmedDeadLetters()); + @DisplayName("handler가 없는 message type은 재시도하지 않고 dead letter로 보낸다") + void aMessageTypeWithNoHandlerIsDeadLetteredInsteadOfRetried() { + // given + List codes = new ArrayList<>(); + MessageHandler otherType = + new StringHandler( + new MessageDestination<>(ORDERS, new MessageType("order.cancelled"), String.class), + delivery -> HandleResult.success()); + // when + process( + otherType, + (delivery, failure) -> { + codes.add(failure.code()); + return CompletableFuture.completedFuture(PublishCompletion.CONFIRMED); + }, + RuntimeFixtures.profile(ORDERS)); + + // then + assertThat(codes).containsExactly("NO_HANDLER_FOR_MESSAGE_TYPE"); + assertThat(settlement.calls()).containsExactly("acknowledge"); + } + + @Test + @DisplayName("dead letter destination이 없으면 dead letter 대신 requeue해 유일한 사본을 지킨다") + void withoutADeadLetterDestinationADeadLetterIsRequeuedToKeepTheOnlyCopy() { + // when + process( + handler(delivery -> new HandleResult.DeadLetter(failure())), + confirmedDeadLetters(), + withoutDeadLetter(RuntimeFixtures.profile(ORDERS))); + + // then assertThat(settlement.calls()).containsExactly("requeue"); } + @Test + @DisplayName("한 message type에 handler가 둘이면 processor를 만들 수 없다") + void twoHandlersForOneMessageTypeAreRefused() { + // given + List> twice = + List.of(handler(delivery -> HandleResult.success()), handler(delivery -> null)); + + // when + Throwable thrown = + catchThrowable( + () -> + new DefaultDeliveryProcessor( + RuntimeFixtures.profile(ORDERS), + twice, + RegisteredMessageCodecs.of(new Utf8Codec()), + confirmedDeadLetters(), + "order-projection", + InstantSource.fixed(NOW))); + + // then + assertThat(thrown) + .isInstanceOf(IllegalArgumentException.class) + .hasMessageContaining("two handlers for order.created"); + } + private void process( - java.util.function.Function, HandleResult> handler, - DefaultDeliveryProcessor.DeadLetterPublisher deadLetters) { - new DefaultDeliveryProcessor(handler, deadLetters, Duration.ofSeconds(5)) + MessageHandler handler, + DefaultDeliveryProcessor.DeadLetterPublisher deadLetters, + DestinationProfile profile) { + new DefaultDeliveryProcessor( + profile, + List.of(handler), + RegisteredMessageCodecs.of(new Utf8Codec()), + deadLetters, + "order-projection", + InstantSource.fixed(NOW)) .process(new TransportDelivery(envelope(), metadata(), settlement)) .toCompletableFuture() .join(); } + private static MessageHandler handler( + Function, HandleResult> body) { + return new StringHandler(body); + } + private static DefaultDeliveryProcessor.DeadLetterPublisher confirmedDeadLetters() { - return (envelope, failure) -> CompletableFuture.completedFuture(PublishCompletion.CONFIRMED); + return (delivery, failure) -> CompletableFuture.completedFuture(PublishCompletion.CONFIRMED); + } + + private static DestinationProfile atMostOnce(DestinationProfile base) { + return new DestinationProfile( + base.name(), + base.broker(), + base.kind(), + base.physical(), + base.schema(), + DeliveryGuarantee.AT_MOST_ONCE, + base.orderingScope(), + base.externalSideEffectGuarantee(), + base.producer(), + base.consumer(), + base.retry(), + base.deadLetter(), + base.payload(), + base.tier(), + base.production(), + base.keyResolverConfigured(), + base.topologyAutoCreate()); + } + + private static DestinationProfile withoutDeadLetter(DestinationProfile base) { + return new DestinationProfile( + base.name(), + base.broker(), + base.kind(), + base.physical(), + base.schema(), + base.deliveryGuarantee(), + base.orderingScope(), + base.externalSideEffectGuarantee(), + base.producer(), + base.consumer(), + base.retry(), + DeadLetterPolicy.disabled(), + base.payload(), + base.tier(), + base.production(), + base.keyResolverConfigured(), + base.topologyAutoCreate()); } private static FailureDescriptor failure() { @@ -139,21 +335,15 @@ class DefaultDeliveryProcessorTest { private static DeliveryMetadata metadata() { return new DeliveryMetadata( - new dev.caskeleton.messaging.api.destination.DestinationName("order-events"), - 1, - false, - Optional.empty(), - Optional.empty(), - Optional.of("order-projection"), - Instant.parse("2026-08-14T00:00:00Z")); + ORDERS, 1, false, Optional.empty(), Optional.empty(), Optional.of("order-projection"), NOW); } private static MessageEnvelope envelope() { return new MessageEnvelope<>( MessageId.newId(), - new MessageType("order.created"), + CREATED, new SchemaVersion(1), - Instant.parse("2026-08-14T00:00:00Z"), + NOW, Optional.empty(), new ProducerId("orders"), Optional.empty(), @@ -165,7 +355,58 @@ class DefaultDeliveryProcessorTest { TraceContext.none(), MessageHeaders.empty(), new EncodedMessage( - "{}".getBytes(StandardCharsets.UTF_8), ContentType.JSON, Optional.empty())); + "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8), + ContentType.JSON, + Optional.empty())); + } + + /** A handler for {@code order.created} whose body is a function. */ + private static class StringHandler implements MessageHandler { + + private final MessageDestination destination; + private final Function, HandleResult> body; + + StringHandler(Function, HandleResult> body) { + this(new MessageDestination<>(ORDERS, CREATED, String.class), body); + } + + StringHandler( + MessageDestination destination, + Function, HandleResult> body) { + this.destination = destination; + this.body = body; + } + + @Override + public MessageDestination destination() { + return destination; + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + return CompletableFuture.completedFuture(body.apply(delivery)); + } + } + + /** Decodes JSON bytes to the raw string, which is all these tests need. */ + private static final class Utf8Codec implements MessageCodec { + + @Override + public ContentType contentType() { + return ContentType.JSON; + } + + @Override + public EncodedMessage encode(MessageType type, SchemaVersion version, Object payload) { + return new EncodedMessage( + payload.toString().getBytes(StandardCharsets.UTF_8), ContentType.JSON, Optional.empty()); + } + + @Override + public T decode( + MessageType type, SchemaVersion version, byte[] encoded, Class payloadType) { + return payloadType.cast(new String(encoded, StandardCharsets.UTF_8)); + } } /** Records which terminal call the processor made, and in what order. */ diff --git a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java index 510c451..4ac4854 100644 --- a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java +++ b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisherTest.java @@ -10,9 +10,14 @@ import dev.caskeleton.messaging.api.MessageType; import dev.caskeleton.messaging.api.ProducerId; import dev.caskeleton.messaging.api.SchemaVersion; import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.delivery.DeliveryContext; +import dev.caskeleton.messaging.api.delivery.DeliveryMetadata; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; import dev.caskeleton.messaging.api.destination.DestinationCapabilities; import dev.caskeleton.messaging.api.destination.DestinationName; import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.error.FailureCategory; +import dev.caskeleton.messaging.api.error.FailureDescriptor; import dev.caskeleton.messaging.api.error.MessagingConfigurationException; import dev.caskeleton.messaging.api.header.MessageHeaders; import dev.caskeleton.messaging.api.publish.ConfirmationLevel; @@ -21,6 +26,7 @@ import dev.caskeleton.messaging.api.publish.PublishEvidence; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; import dev.caskeleton.messaging.api.publish.RoutingOutcome; +import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; import dev.caskeleton.messaging.policy.InFlightLimiter; import dev.caskeleton.messaging.policy.MessagingAdmissionController; import dev.caskeleton.messaging.policy.PayloadLimitGuard; @@ -120,6 +126,55 @@ class DefaultMessagePublisherTest { assertThat(admission.inFlight()).isZero(); } + @Test + void aDeadLetterPublishedThroughThisPublisherReachesTheTransportWithTheOriginalBytes() { + // given + DestinationName deadLetters = new DestinationName("order-events-dlq"); + DefaultMessagePublisher publisher = + new DefaultMessagePublisher( + DestinationProfileRegistry.of( + Map.of( + ORDERS, + RuntimeFixtures.profile(ORDERS), + deadLetters, + RuntimeFixtures.profile(deadLetters))), + new DestinationAccessPolicy( + java.util.Set.of(deadLetters), java.util.Set.of(), java.util.Set.of()), + new SingleCodecRegistry(), + admission, + new FixedRuntimeRegistry(transport), + transport); + byte[] wireBytes = "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8); + MessageDelivery failed = + new MessageDelivery<>( + envelope() + .withPayload(new EncodedMessage(wireBytes, ContentType.JSON, Optional.empty())), + new DeliveryMetadata( + ORDERS, + 3, + true, + Optional.empty(), + Optional.empty(), + Optional.of("order-projection"), + Instant.parse("2026-08-14T00:00:05Z")), + new DeliveryContext(Instant.parse("2026-08-14T00:00:35Z"), false, "order-projection")); + + // when + new DeadLetterOrchestrator(publisher) + .deadLetter( + RuntimeFixtures.profile(ORDERS), + failed, + FailureDescriptor.of(FailureCategory.PERMANENT_BUSINESS, "ORDER_REJECTED", "rejected"), + () -> CompletableFuture.completedFuture(null)) + .toCompletableFuture() + .join(); + + // then + assertThat(transport.lastRequest().envelope().payload().bytes()) + .as("encoding an already-encoded payload again rewrites the bytes or fails the codec") + .containsExactly(wireBytes); + } + @Test @DisplayName("an unregistered destination is rejected before anything is encoded") void anUnregisteredDestinationIsRejected() { @@ -257,6 +312,89 @@ class DefaultMessagePublisherTest { assertThat(admission.inFlight()).isZero(); } + @Test + void anOversizedPayloadIsRejectedAsNotRetryableInsteadOfThrown() { + // given + DefaultMessagePublisher publisher = + new DefaultMessagePublisher( + registry(), + allowOrders(), + new SingleCodecRegistry(), + new MessagingAdmissionController( + new PayloadLimitGuard(new PayloadPolicy(4, 4)), + new InFlightLimiter(4, Duration.ofSeconds(1))), + new FixedRuntimeRegistry(transport), + transport); + + // when + CompletableFuture result = + CompletableFuture.supplyAsync( + () -> + publisher + .publish( + new MessageDestination<>( + ORDERS, new MessageType("order.created"), String.class), + envelope(), + PublishOptions.defaults()) + .toCompletableFuture() + .join()); + + // then + assertThat(result) + .succeedsWithin(Duration.ofSeconds(5)) + .extracting( + PublishResult::completion, outcome -> outcome.failure().orElseThrow().retryable()) + .containsExactly(PublishCompletion.REJECTED, false); + } + + @Test + void aSaturatedPermitPoolRejectsAsRetryableWithinTheCallersDeadline() { + // given + DestinationName audit = new DestinationName("audit-events"); + DefaultMessagePublisher publisher = + new DefaultMessagePublisher( + DestinationProfileRegistry.of( + Map.of( + ORDERS, + RuntimeFixtures.profile(ORDERS), + audit, + RuntimeFixtures.profile(audit))), + new DestinationAccessPolicy( + java.util.Set.of(ORDERS, audit), java.util.Set.of(), java.util.Set.of()), + new SingleCodecRegistry(), + new MessagingAdmissionController( + new PayloadLimitGuard(PayloadPolicy.defaults()), + new InFlightLimiter(1, Duration.ofMinutes(1))), + new FixedRuntimeRegistry(transport), + transport); + transport.stall(); + publisher.publish( + new MessageDestination<>(ORDERS, new MessageType("order.created"), String.class), + envelope(), + withTimeout(Duration.ofMinutes(1))); + + // when + CompletableFuture second = + CompletableFuture.supplyAsync( + () -> + publisher + .publish( + new MessageDestination<>( + audit, new MessageType("order.created"), String.class), + envelope(), + withTimeout(Duration.ofMillis(100))) + .toCompletableFuture() + .join()); + + // then + assertThat(second) + .as("the permit wait was the limiter's own minute, not what was left of 100ms") + .succeedsWithin(Duration.ofSeconds(5)) + .extracting( + PublishResult::completion, outcome -> outcome.failure().orElseThrow().retryable()) + .containsExactly(PublishCompletion.REJECTED, true); + } + @Test @DisplayName("an unavailable runtime releases the permit it took") void anUnavailableRuntimeReleasesThePermit() { diff --git a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/RuntimeFixtures.java b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/RuntimeFixtures.java index 6509b88..d341d54 100644 --- a/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/RuntimeFixtures.java +++ b/messaging/messaging-runtime-core/src/test/java/dev/caskeleton/messaging/runtime/RuntimeFixtures.java @@ -3,7 +3,7 @@ package dev.caskeleton.messaging.runtime; import dev.caskeleton.messaging.api.ContentType; import dev.caskeleton.messaging.api.MessageType; import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.ConsumerPolicy; import dev.caskeleton.messaging.policy.DeadLetterPolicy; import dev.caskeleton.messaging.policy.DestinationProfile; @@ -36,7 +36,7 @@ final class RuntimeFixtures { dev.caskeleton.messaging.policy.RetryPolicy.none(), DeadLetterPolicy.to(new DestinationName("order-events-dlq")), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, false, false, false); diff --git a/messaging/messaging-schema-api/build.gradle b/messaging/messaging-schema-api/build.gradle deleted file mode 100644 index fac7b20..0000000 --- a/messaging/messaging-schema-api/build.gradle +++ /dev/null @@ -1,7 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') -} diff --git a/messaging/messaging-schema-api/gradle.lockfile b/messaging/messaging-schema-api/gradle.lockfile deleted file mode 100644 index e2c9585..0000000 --- a/messaging/messaging-schema-api/gradle.lockfile +++ /dev/null @@ -1,84 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -info.picocli:picocli:4.7.7=checkstyle -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=spotbugs,spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -empty=compileClasspath,runtimeClasspath diff --git a/messaging/messaging-schema-avro/build.gradle b/messaging/messaging-schema-avro/build.gradle index 4044538..b1e8f84 100644 --- a/messaging/messaging-schema-avro/build.gradle +++ b/messaging/messaging-schema-avro/build.gradle @@ -4,7 +4,6 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') // api: AvroMessageCodec's constructors take a registry of org.apache.avro.Schema and // AvroCompatibilityGate.check takes and compares them. A consumer cannot build that diff --git a/messaging/messaging-schema-json/build.gradle b/messaging/messaging-schema-json/build.gradle index aebe0fe..e87b8c5 100644 --- a/messaging/messaging-schema-json/build.gradle +++ b/messaging/messaging-schema-json/build.gradle @@ -4,7 +4,6 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') implementation 'tools.jackson.core:jackson-databind' } diff --git a/messaging/messaging-schema-protobuf/build.gradle b/messaging/messaging-schema-protobuf/build.gradle index 3c431b4..58f78aa 100644 --- a/messaging/messaging-schema-protobuf/build.gradle +++ b/messaging/messaging-schema-protobuf/build.gradle @@ -4,7 +4,6 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') // api: ProtobufMessageContract is a public record over com.google.protobuf.Message and // Parser, and registering a contract is the first thing a consumer of this codec does. diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerCredentialProfile.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerCredentialProfile.java index 6bd1f5b..87a4df6 100644 --- a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerCredentialProfile.java +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerCredentialProfile.java @@ -44,13 +44,28 @@ public sealed interface BrokerCredentialProfile } /** - * Mutual TLS authentication. + * Mutual TLS authentication: the client certificate in a key store, unlocked by the credential. * - * @param credentialId the credential id + *

The key store was not part of the profile, so a {@code MUTUAL_TLS} credential configured a + * TLS connection with no client certificate — one every broker that asks for a certificate + * refuses. The path is not a secret and lives here; the password is, and resolves through the + * credential id like every other secret. + * + * @param credentialId the credential whose material unlocks the key store + * @param keyStoreLocation the key store holding the client certificate and private key + * @param keyStoreType the key store format, such as {@code PKCS12} */ - record MutualTls(String credentialId) implements BrokerCredentialProfile { + record MutualTls(String credentialId, String keyStoreLocation, String keyStoreType) + implements BrokerCredentialProfile { public MutualTls { CredentialIds.require(credentialId); + if (keyStoreLocation == null || keyStoreLocation.isBlank()) { + throw new IllegalArgumentException( + "mutual TLS presents a client certificate, and without a key store there is none"); + } + if (keyStoreType == null || keyStoreType.isBlank()) { + throw new IllegalArgumentException("keyStoreType must not be blank"); + } } } diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerSecurityProfile.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerSecurityProfile.java index ab59d9a..40d386f 100644 --- a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerSecurityProfile.java +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerSecurityProfile.java @@ -18,6 +18,7 @@ import java.util.Optional; * @param consumer the consumer credential * @param admin the admin credential, absent in an application runtime * @param access the destination access policy + * @param trustStore the certificates the connection trusts; empty means the JVM default trust store */ public record BrokerSecurityProfile( String broker, @@ -27,15 +28,39 @@ public record BrokerSecurityProfile( BrokerCredentialProfile producer, BrokerCredentialProfile consumer, Optional admin, - DestinationAccessPolicy access) { + DestinationAccessPolicy access, + Optional trustStore) { public BrokerSecurityProfile { Objects.requireNonNull(producer, "producer credential must not be null"); Objects.requireNonNull(consumer, "consumer credential must not be null"); Objects.requireNonNull(admin, "admin credential must not be null"); Objects.requireNonNull(access, "access policy must not be null"); + Objects.requireNonNull(trustStore, "trust store must not be null"); if (broker == null || broker.isBlank()) { throw new IllegalArgumentException("broker must not be blank"); } } + + /** A profile that trusts what the JVM's default trust store trusts. */ + public BrokerSecurityProfile( + String broker, + boolean production, + boolean tlsEnabled, + boolean hostnameVerification, + BrokerCredentialProfile producer, + BrokerCredentialProfile consumer, + Optional admin, + DestinationAccessPolicy access) { + this( + broker, + production, + tlsEnabled, + hostnameVerification, + producer, + consumer, + admin, + access, + Optional.empty()); + } } diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerTrustStore.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerTrustStore.java new file mode 100644 index 0000000..b5b7db6 --- /dev/null +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/BrokerTrustStore.java @@ -0,0 +1,30 @@ +package dev.caskeleton.messaging.security; + +import java.util.Objects; +import java.util.Optional; + +/** + * The certificates a broker connection trusts, when the JVM's default trust store is not enough. + * + *

A broker signed by a private CA could not be reached at all: there was no setting for a trust + * store, so the client verified the broker against the JVM defaults and refused it. The path is not + * a secret; the store password, when the format has one, resolves through a credential id. + * + * @param location the trust store file + * @param type the trust store format, such as {@code PKCS12} or {@code PEM} + * @param passwordCredentialId the credential whose material unlocks the store, when it is locked + */ +public record BrokerTrustStore( + String location, String type, Optional passwordCredentialId) { + + public BrokerTrustStore { + Objects.requireNonNull(passwordCredentialId, "passwordCredentialId must not be null"); + if (location == null || location.isBlank()) { + throw new IllegalArgumentException("a trust store needs a location"); + } + if (type == null || type.isBlank()) { + throw new IllegalArgumentException("a trust store needs a type"); + } + passwordCredentialId.ifPresent(CredentialIds::require); + } +} diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialProvider.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialProvider.java index 60836f6..24df053 100644 --- a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialProvider.java +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialProvider.java @@ -26,4 +26,21 @@ public interface CredentialProvider { * @return the expiry instant when known */ Optional expiresAt(String credentialId); + + /** + * Returns the user name the secret authenticates, for mechanisms that send one. + * + *

SASL PLAIN and SCRAM send a user name and a password. The credential id is a reference into + * the store, not the principal the broker knows, and sending it as the user name authenticated as + * whoever happened to share the reference's spelling. A store that rotates by alternating between + * two users rotates the name together with the secret, so the name is resolved alongside it. + * + *

Empty by default. A SASL credential whose provider names no user is refused at startup. + * + * @param credentialId the identifier + * @return the user name, when this credential has one + */ + default Optional username(String credentialId) { + return Optional.empty(); + } } diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntime.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntime.java index aefc5bc..5801728 100644 --- a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntime.java +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntime.java @@ -26,12 +26,14 @@ public final class CredentialRuntime { private final Instant resolvedAt; private final Optional expiresAt; private final Duration rotationLead; + private final Optional username; private char[] material; /** * Creates a runtime entry. * * @param credentialId the credential reference + * @param username the user name the secret authenticates, when the mechanism sends one * @param material the resolved secret, taken by reference and cleared by {@link #clear()} * @param resolvedAt when the material was resolved * @param expiresAt when it stops being valid, when known @@ -39,11 +41,13 @@ public final class CredentialRuntime { */ public CredentialRuntime( String credentialId, + Optional username, char[] material, Instant resolvedAt, Optional expiresAt, Duration rotationLead) { CredentialIds.require(credentialId); + Objects.requireNonNull(username, "username must not be null"); Objects.requireNonNull(material, "material must not be null"); Objects.requireNonNull(resolvedAt, "resolvedAt must not be null"); Objects.requireNonNull(expiresAt, "expiresAt must not be null"); @@ -55,6 +59,7 @@ public final class CredentialRuntime { throw new IllegalArgumentException("credential material must not be empty"); } this.credentialId = credentialId; + this.username = username; this.material = material; this.resolvedAt = resolvedAt; this.expiresAt = expiresAt; @@ -70,6 +75,15 @@ public final class CredentialRuntime { return credentialId; } + /** + * Returns the user name this secret authenticates, when the mechanism sends one. + * + * @return the user name + */ + public Optional username() { + return username; + } + /** * Returns a copy of the material. * diff --git a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntimeRegistry.java b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntimeRegistry.java index ef7e898..35a1e77 100644 --- a/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntimeRegistry.java +++ b/messaging/messaging-security/src/main/java/dev/caskeleton/messaging/security/CredentialRuntimeRegistry.java @@ -134,6 +134,7 @@ public final class CredentialRuntimeRegistry { private CredentialRuntime fetch(String credentialId, Instant now) { char[] material = provider.resolve(credentialId); Optional expiresAt = provider.expiresAt(credentialId); - return new CredentialRuntime(credentialId, material, now, expiresAt, rotationLead); + return new CredentialRuntime( + credentialId, provider.username(credentialId), material, now, expiresAt, rotationLead); } } diff --git a/messaging/messaging-spring-boot-starter-kafka/build.gradle b/messaging/messaging-spring-boot-starter-kafka/build.gradle index 9333a84..a985d71 100644 --- a/messaging/messaging-spring-boot-starter-kafka/build.gradle +++ b/messaging/messaging-spring-boot-starter-kafka/build.gradle @@ -8,6 +8,8 @@ dependencies { // Kafka client classes. api project(':messaging:messaging-spring-boot-starter') api project(':messaging:messaging-kafka') + // KafkaConsumerConfiguration's public consumer-factory seam names kafka-clients types. + api 'org.apache.kafka:kafka-clients' implementation 'org.springframework.boot:spring-boot-autoconfigure' } diff --git a/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile b/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile index 53df71c..3e1b642 100644 --- a/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile +++ b/messaging/messaging-spring-boot-starter-kafka/gradle.lockfile @@ -34,14 +34,11 @@ commons-collections:commons-collections:3.2.2=checkstyle commons-io:commons-io:2.21.0=spotbugs commons-logging:commons-logging:1.3.6=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath info.picocli:picocli:4.7.7=checkstyle -io.cloudevents:cloudevents-api:4.0.1=runtimeClasspath,testRuntimeClasspath -io.cloudevents:cloudevents-core:4.0.1=runtimeClasspath,testRuntimeClasspath io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.micrometer:micrometer-core:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.micrometer:micrometer-observation:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.projectreactor:reactor-core:3.8.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath @@ -89,7 +86,6 @@ org.ow2.asm:asm-tree:9.10.1=spotbugs org.ow2.asm:asm-util:9.10.1=spotbugs org.ow2.asm:asm:9.10.1=spotbugs org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reactivestreams:reactive-streams:1.0.4=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.reflections:reflections:0.10.2=checkstyle org.slf4j:slf4j-api:2.0.18=runtimeClasspath,spotbugs,spotbugsSlf4j,testRuntimeClasspath org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerCatalog.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerCatalog.java new file mode 100644 index 0000000..726df06 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerCatalog.java @@ -0,0 +1,37 @@ +package dev.caskeleton.messaging.autoconfigure; + +import java.util.LinkedHashMap; +import java.util.Map; +import java.util.Objects; + +/** + * The Kafka client settings of every {@code app.messaging.brokers} entry of type {@code kafka}. + * + * @param brokers the Kafka settings, keyed by logical broker name, in declaration order + */ +public record KafkaBrokerCatalog(Map brokers) { + + public KafkaBrokerCatalog { + brokers = + java.util.Collections.unmodifiableMap( + new LinkedHashMap<>(Objects.requireNonNull(brokers, "brokers must not be null"))); + } + + /** + * The one Kafka broker the producer and the consumers connect to. + * + * @return its logical name and settings + * @throws IllegalStateException unless exactly one Kafka entry is configured + */ + public Map.Entry single() { + if (brokers.size() != 1) { + throw new IllegalStateException( + "app.messaging.broker=kafka needs exactly one app.messaging.brokers. entry with" + + " type kafka and its bootstrap-servers; found " + + brokers.size() + + ". A producer with no broker address defaults to localhost:9092 and fails at the" + + " first publish rather than at startup"); + } + return brokers.entrySet().iterator().next(); + } +} diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerSettings.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerSettings.java new file mode 100644 index 0000000..5f38035 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaBrokerSettings.java @@ -0,0 +1,86 @@ +package dev.caskeleton.messaging.autoconfigure; + +import java.time.Duration; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Objects; +import org.springframework.boot.context.properties.bind.Bindable; +import org.springframework.boot.context.properties.bind.Binder; +import org.springframework.boot.context.properties.bind.DefaultValue; +import org.springframework.core.env.Environment; + +/** + * The Kafka client part of one {@code app.messaging.brokers.} entry of type {@code kafka}. + * + *

Bound here, by the Kafka starter, from the same entry the broker-neutral {@link + * BrokerSettings} reads its family and posture from (S7-#9b). These fields used to sit in {@code + * BrokerSettings}, so the neutral starter bound Kafka client settings for deployments with no Kafka + * transport. + * + * @param bootstrapServers Kafka broker addresses + * @param enableIdempotence Kafka producer duplicate suppression on internal retry + * @param acks the Kafka acknowledgement level + * @param maxInFlightRequestsPerConnection the Kafka in-flight request ceiling + * @param deliveryTimeout the Kafka producer delivery deadline + * @param enableAutoCommit whether the Kafka consumer commits offsets on a timer; the platform + * consumer commits itself and refuses {@code true} + * @param consumerGroup the consumer group a destination without its own {@code consumer.group} + * joins + */ +public record KafkaBrokerSettings( + @DefaultValue List bootstrapServers, + @DefaultValue("true") boolean enableIdempotence, + @DefaultValue("all") String acks, + @DefaultValue("5") int maxInFlightRequestsPerConnection, + @DefaultValue("30s") Duration deliveryTimeout, + @DefaultValue("false") boolean enableAutoCommit, + @DefaultValue("") String consumerGroup) { + + public KafkaBrokerSettings { + bootstrapServers = List.copyOf(Objects.requireNonNull(bootstrapServers, "bootstrapServers")); + Objects.requireNonNull(acks, "acks must not be null"); + Objects.requireNonNull(deliveryTimeout, "deliveryTimeout must not be null"); + Objects.requireNonNull(consumerGroup, "consumerGroup must not be null"); + } + + /** + * Binds the Kafka settings of every broker entry whose family is {@code kafka}. + * + * @param environment the resolved environment + * @param compiled the compiled configuration, which names the Kafka entries + * @return the Kafka settings, keyed by logical broker name, in declaration order + * @throws dev.caskeleton.messaging.api.error.MessagingConfigurationException when a Kafka entry + * declares no address + */ + public static Map bindAll( + Environment environment, CompiledMessagingConfiguration compiled) { + Binder binder = Binder.get(environment); + Map kafka = new LinkedHashMap<>(); + compiled + .brokers() + .forEach( + (name, broker) -> { + if (broker.type() == null + || !BrokerSettings.KAFKA.equals(broker.type().toLowerCase(Locale.ROOT))) { + return; + } + String prefix = MessagingSettings.PREFIX + ".brokers." + name; + KafkaBrokerSettings settings = + binder + .bind(prefix, Bindable.of(KafkaBrokerSettings.class)) + .orElseGet(() -> binder.bindOrCreate(prefix, KafkaBrokerSettings.class)); + if (settings.bootstrapServers().isEmpty()) { + throw new dev.caskeleton.messaging.api.error.MessagingConfigurationException( + "MESSAGING_CONFIGURATION_INVALID", + prefix + + ".bootstrap-servers: a Kafka broker needs at least one address; a client" + + " with none defaults to localhost:9092 and fails at the first publish" + + " rather than at startup"); + } + kafka.put(name, settings); + }); + return kafka; + } +} diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerConfiguration.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerConfiguration.java new file mode 100644 index 0000000..8b38523 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerConfiguration.java @@ -0,0 +1,165 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.kafka.KafkaQuarantinePublisher; +import dev.caskeleton.messaging.kafka.KafkaSecurityConfigurer; +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import dev.caskeleton.messaging.transport.MessagingTransport; +import dev.caskeleton.messaging.transport.TransportConsumerRegistration; +import dev.caskeleton.messaging.transport.TransportConsumerSpec; +import java.util.HashMap; +import java.util.Map; +import java.util.function.Function; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.producer.Producer; +import org.apache.kafka.common.serialization.ByteArrayDeserializer; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * Opens a Kafka consumer for every destination a {@code MessageHandler} bean names (owner decision + * 10, S7-#2). + * + *

On when present, like the rest of the capability: an application with no handler bean gets no + * consumer, no thread and no connection; {@code app.messaging.enabled=false} removes the platform + * altogether. The broker-neutral pipeline — decode, inbox, handler, dead letter, settlement — is + * {@link MessageConsumers}; this class adds only what is Kafka's, and it reaches the broker through + * {@link MessagingTransport#register}, which refused every subscription with {@code + * KAFKA_CONSUMER_NOT_CONFIGURED} until it had a consumer factory: + * + *

    + *
  • the client, from the one Kafka broker entry and the broker's consumer credential; + *
  • {@code group.id} from the destination's {@code consumer.group}; + *
  • {@code enable.auto.commit=false} always — the platform commits the contiguous watermark of + * settled deliveries, and a broker entry that asks for auto-commit is refused rather than + * overridden; + *
  • {@code auto.offset.reset=earliest}, so a new group reads what was published before it first + * joined instead of starting at the end and skipping it; + *
  • {@code isolation.level=read_committed}, so an aborted transactional write is never handed + * to a handler; + *
  • the dead-letter topic as the parking place for a record that cannot be rebuilt or whose + * attempt budget ({@code retry.max-attempts}) is spent. Without a dead-letter destination the + * record is retried and its partition stops behind it; a production destination without one + * is refused. + *
+ */ +@Configuration(proxyBeanMethods = false) +public class KafkaConsumerConfiguration { + + /** + * Builds a live subscription for a destination; the transport's consumer factory. + * + * @param compiled the compiled configuration + * @param catalog the Kafka client settings + * @param destinations the validated destination registry, for dead-letter topics + * @param producer the platform producer the dead-letter copy is written with, resolved when a + * subscription opens so that the startup validations run before a client is built + * @param declaredSecurity the security profiles the application declared as beans + * @param securityConfigurer the Kafka security configurer, present only with a credential source + * @param settings the bound settings supplying the drain deadline + * @return the factory + */ + @Bean + public KafkaConsumerFactory messagingKafkaConsumerFactory( + CompiledMessagingConfiguration compiled, + KafkaBrokerCatalog catalog, + ValidatedDestinationRegistry destinations, + ObjectProvider> producer, + ObjectProvider declaredSecurity, + ObjectProvider securityConfigurer, + MessagingSettings settings) { + return spec -> { + KafkaBrokerSettings broker = catalog.single().getValue(); + if (broker.enableAutoCommit()) { + throw new IllegalStateException( + "app.messaging.brokers." + + catalog.single().getKey() + + ".enable-auto-commit=true: the platform consumer commits an offset only after" + + " its delivery was settled, and a timer commit would move past unprocessed" + + " messages"); + } + return KafkaPolledRegistration.open( + spec, + consumerConfig(spec.profile(), compiled, catalog, declaredSecurity, securityConfigurer), + quarantine(spec.profile(), destinations, producer.getObject()), + settings.getShutdown().getDrainDeadline()); + }; + } + + /** + * Subscribes the consumed destinations when the context starts; empty without a handler bean. + * + * @param consumers the consume pipelines, present when a handler bean exists + * @param transport the selected transport + * @return the lifecycle + */ + @Bean + public KafkaConsumerLifecycle messagingKafkaConsumers( + ObjectProvider consumers, ObjectProvider transport) { + return new KafkaConsumerLifecycle( + () -> consumers.stream().flatMap(found -> found.consumers().stream()).toList(), + transport::getObject); + } + + /** The consumer's client configuration; package-visible so a test can read what it holds. */ + static Map consumerConfig( + DestinationProfile profile, + CompiledMessagingConfiguration compiled, + KafkaBrokerCatalog catalog, + ObjectProvider declaredSecurity, + ObjectProvider securityConfigurer) { + KafkaBrokerSettings broker = catalog.single().getValue(); + Map config = new HashMap<>(); + config.put( + ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, String.join(",", broker.bootstrapServers())); + config.put( + ConsumerConfig.GROUP_ID_CONFIG, + profile + .consumer() + .group() + .filter(group -> !group.isBlank()) + .orElseThrow( + () -> + new IllegalStateException( + "destination '" + + profile.name().value() + + "' is consumed but declares no consumer.group"))); + config.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false); + config.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); + config.put(ConsumerConfig.ISOLATION_LEVEL_CONFIG, "read_committed"); + config.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, profile.consumer().prefetch()); + config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class); + config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class); + config.putAll( + KafkaMessagingAutoConfiguration.clientSecurity( + compiled, declaredSecurity, securityConfigurer, BrokerSecurityProfile::consumer)); + return config; + } + + private static KafkaQuarantinePublisher quarantine( + DestinationProfile profile, + ValidatedDestinationRegistry destinations, + Producer producer) { + if (!profile.deadLetter().enabled()) { + return KafkaQuarantinePublisher.unavailable(); + } + DestinationName deadLetter = profile.deadLetter().destination().orElseThrow(); + String topic = + destinations + .require(deadLetter) + .physical() + .topic() + .orElseThrow( + () -> + new IllegalStateException( + "dead-letter destination '" + deadLetter.value() + "' has no Kafka topic")); + return new KafkaDeadLetterQuarantine(producer, topic, profile.producer().timeout()); + } + + /** The transport's consumer factory: a spec in, a live subscription out. */ + @FunctionalInterface + public interface KafkaConsumerFactory + extends Function {} +} diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerLifecycle.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerLifecycle.java new file mode 100644 index 0000000..e12e352 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaConsumerLifecycle.java @@ -0,0 +1,69 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.transport.MessagingTransport; +import dev.caskeleton.messaging.transport.TransportConsumerRegistration; +import dev.caskeleton.messaging.transport.TransportConsumerSpec; +import java.util.ArrayList; +import java.util.List; +import java.util.Objects; +import java.util.function.Supplier; +import org.springframework.context.SmartLifecycle; + +/** + * Subscribes every consumed destination through the transport when the context starts, and closes + * the subscriptions when it stops. + * + *

An application with no handler bean has no consumed destination, and this opens nothing. Both + * collaborators are looked up at start rather than injected, so a context that fails a startup + * validation fails on that validation, not on a client this bean asked for too early. + */ +final class KafkaConsumerLifecycle implements SmartLifecycle { + + private final Supplier> consumers; + private final Supplier transport; + private final List registrations = new ArrayList<>(); + + KafkaConsumerLifecycle( + Supplier> consumers, Supplier transport) { + this.consumers = Objects.requireNonNull(consumers, "consumers must not be null"); + this.transport = Objects.requireNonNull(transport, "transport must not be null"); + } + + @Override + public synchronized void start() { + if (!registrations.isEmpty()) { + return; + } + List consumed = consumers.get(); + if (consumed.isEmpty()) { + return; + } + MessagingTransport selected = transport.get(); + for (MessageConsumers.Consumer consumer : consumed) { + registrations.add( + selected.register(new TransportConsumerSpec(consumer.profile(), consumer.sink()))); + } + } + + @Override + public synchronized void stop() { + registrations.forEach(TransportConsumerRegistration::close); + registrations.clear(); + } + + @Override + public synchronized boolean isRunning() { + return !registrations.isEmpty(); + } + + /** + * Stops before the publish drain, because a handler still running may dead-letter through the + * publisher that drain closes. + * + * @return the lifecycle phase + */ + @Override + public int getPhase() { + return Integer.MAX_VALUE - 512; + } +} diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java new file mode 100644 index 0000000..8185ef6 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java @@ -0,0 +1,65 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.api.header.ReservedHeaders; +import dev.caskeleton.messaging.kafka.KafkaQuarantinePublisher; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.Objects; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.producer.Producer; +import org.apache.kafka.clients.producer.ProducerRecord; +import org.apache.kafka.common.header.internals.RecordHeaders; + +/** + * Parks a record the consumer cannot deliver on the destination's dead-letter topic. + * + *

Used by the Kafka consumer for the two cases the typed pipeline never sees: a record that + * cannot even be rebuilt into an envelope, and a delivery whose attempt budget is spent. The record + * is copied as it arrived — key, value and headers — with the failure category and code added, so + * the message a redrive reads is the message that failed. + * + *

It answers {@code true} only once the broker has acknowledged the copy. The consumer commits + * past the source offset on that answer, so an unconfirmed write must say {@code false} and leave + * the source to be read again. + */ +final class KafkaDeadLetterQuarantine implements KafkaQuarantinePublisher { + + private final Producer producer; + private final String topic; + private final Duration timeout; + + KafkaDeadLetterQuarantine(Producer producer, String topic, Duration timeout) { + this.producer = Objects.requireNonNull(producer, "producer must not be null"); + this.topic = Objects.requireNonNull(topic, "topic must not be null"); + this.timeout = Objects.requireNonNull(timeout, "timeout must not be null"); + } + + @Override + public boolean quarantine(ConsumerRecord record, Throwable reason) { + RecordHeaders headers = new RecordHeaders(record.headers().toArray()); + headers.remove(ReservedHeaders.FAILURE_CATEGORY); + headers.remove(ReservedHeaders.FAILURE_CODE); + headers.add(ReservedHeaders.FAILURE_CATEGORY, bytes("POISON_MESSAGE")); + headers.add( + ReservedHeaders.FAILURE_CODE, + bytes(reason == null ? "UNKNOWN" : reason.getClass().getSimpleName())); + try { + producer + .send(new ProducerRecord<>(topic, null, record.key(), record.value(), headers)) + .get(timeout.toMillis(), TimeUnit.MILLISECONDS); + return true; + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + return false; + } catch (ExecutionException | TimeoutException | RuntimeException notConfirmed) { + return false; + } + } + + private static byte[] bytes(String value) { + return value.getBytes(StandardCharsets.UTF_8); + } +} diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java index cc2b247..05eadb6 100644 --- a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaMessagingAutoConfiguration.java @@ -4,6 +4,7 @@ import dev.caskeleton.messaging.kafka.KafkaProfileValidator; import dev.caskeleton.messaging.kafka.KafkaPublishFailureClassifier; import dev.caskeleton.messaging.kafka.KafkaSecurityConfigurer; import dev.caskeleton.messaging.kafka.KafkaTransactionProfileValidator; +import dev.caskeleton.messaging.security.BrokerSecurityProfile; import dev.caskeleton.messaging.security.BrokerTlsPolicy; import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; import org.springframework.beans.factory.ObjectProvider; @@ -24,13 +25,20 @@ import org.springframework.context.annotation.Configuration; * localhost:9092} — which in a test environment usually succeeds against something. * *

Reached only when app.messaging.broker=kafka selected it. The @ConditionalOnClass this used to - * carry could not select: the starter depends on messaging-kafka and messaging-rabbit, each of - * which exposes its client library as `api`, so both conditions were true for every adopter and - * both providers assembled. + * carry could not select: the starter then depended on two broker adapters, each of which exposed + * its client library as `api`, so both conditions were true for every adopter and both providers + * assembled. */ @Configuration(proxyBeanMethods = false) +@org.springframework.context.annotation.Import(KafkaConsumerConfiguration.class) public class KafkaMessagingAutoConfiguration { + /** + * The two versions {@code BrokerTlsPolicy} accepts. No setting names them, so both are offered. + */ + private static final java.util.List TLS_PROTOCOLS = + java.util.List.of("TLSv1.3", "TLSv1.2"); + /** * Returns the Kafka destination profile validator. * @@ -55,15 +63,31 @@ public class KafkaMessagingAutoConfiguration { kafkaProfileStartupValidation( ObjectProvider profiles, CompiledMessagingConfiguration compiled, + KafkaBrokerCatalog catalog, KafkaProfileValidator validator) { return new StartupProfileValidation<>( () -> java.util.stream.Stream.concat( - configuredKafkaProfiles(compiled).stream(), profiles.orderedStream()) + configuredKafkaProfiles(compiled, catalog).stream(), profiles.orderedStream()) .toList(), validator::validate); } + /** + * The Kafka client settings of every Kafka broker entry, bound by this starter (S7-#9b). + * + * @param environment the resolved environment + * @param compiled the compiled configuration naming the Kafka entries + * @return the catalog + */ + @Bean + @ConditionalOnMissingBean + public KafkaBrokerCatalog kafkaBrokerCatalog( + org.springframework.core.env.Environment environment, + CompiledMessagingConfiguration compiled) { + return new KafkaBrokerCatalog(KafkaBrokerSettings.bindAll(environment, compiled)); + } + /** * Returns the Kafka transaction profile validator. * @@ -114,81 +138,182 @@ public class KafkaMessagingAutoConfiguration { /** * The producer this transport publishes through (MSG-INT-003). * - *

Built from Spring Boot's own {@code spring.kafka.bootstrap-servers} rather than from a - * second description on {@code app.messaging.*}. One broker described twice is the defect this - * repository has already paid for in {@code app.jpa-platform.datasource.*}, and the - * operator-facing property for "where is Kafka" is the one they already know. + *

Built from the one {@code app.messaging.brokers.} entry of type {@code kafka}. It used + * to read {@code spring.kafka.bootstrap-servers} while the compiler required and validated the + * entry's own {@code bootstrap-servers}, so one broker was described twice and only one of the + * two descriptions reached the client. * *

Byte-array serializers on purpose: the platform's codec registry decides the wire format, * and a serializer configured here would be a second encoder disagreeing with it. * - * @param environment the resolved environment + * @param compiled the broker and security profiles compiled from {@code app.messaging} + * @param declaredSecurity the security profiles the application declared as beans + * @param securityConfigurer the Kafka security configurer, present only with a credential source * @return the producer, closed with the context */ @Bean(destroyMethod = "close") @ConditionalOnMissingBean public org.apache.kafka.clients.producer.Producer messagingKafkaProducer( - org.springframework.core.env.Environment environment) { - String bootstrapServers = environment.getProperty("spring.kafka.bootstrap-servers", "").trim(); - if (bootstrapServers.isEmpty()) { - throw new IllegalStateException( - "app.messaging.broker=kafka needs spring.kafka.bootstrap-servers; a producer with no " - + "broker address defaults to localhost:9092 and fails at the first publish rather " - + "than at startup"); - } + CompiledMessagingConfiguration compiled, + KafkaBrokerCatalog catalog, + ObjectProvider declaredSecurity, + ObjectProvider securityConfigurer) { + return new org.apache.kafka.clients.producer.KafkaProducer<>( + producerConfig(compiled, catalog, declaredSecurity, securityConfigurer)); + } + + /** The producer's client configuration; package-visible so a test can read what it holds. */ + static java.util.Map producerConfig( + CompiledMessagingConfiguration compiled, + KafkaBrokerCatalog catalog, + ObjectProvider declaredSecurity, + ObjectProvider securityConfigurer) { + KafkaBrokerSettings broker = catalog.single().getValue(); java.util.Map config = new java.util.HashMap<>(); config.put( org.apache.kafka.clients.producer.ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, - bootstrapServers); + String.join(",", broker.bootstrapServers())); config.put( org.apache.kafka.clients.producer.ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, org.apache.kafka.common.serialization.ByteArraySerializer.class); config.put( org.apache.kafka.clients.producer.ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, org.apache.kafka.common.serialization.ByteArraySerializer.class); - // Durability is not a default worth inheriting: acks=1 loses an accepted publish to a leader - // failover, which is exactly the outcome an outbox exists to prevent. - config.put(org.apache.kafka.clients.producer.ProducerConfig.ACKS_CONFIG, "all"); - config.put(org.apache.kafka.clients.producer.ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true); - return new org.apache.kafka.clients.producer.KafkaProducer<>(config); + // The entry's delivery settings, which KafkaProfileValidator checks (a stable broker must keep + // acks=all and idempotence) and which used to be replaced here by hard-coded values. + config.put(org.apache.kafka.clients.producer.ProducerConfig.ACKS_CONFIG, broker.acks()); + config.put( + org.apache.kafka.clients.producer.ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, + broker.enableIdempotence()); + config.put( + org.apache.kafka.clients.producer.ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, + broker.maxInFlightRequestsPerConnection()); + // Kafka refuses delivery.timeout.ms below linger.ms + request.timeout.ms (5ms + 30s by + // default), which made the documented 30s a startup failure. The request timeout follows a + // short delivery deadline down, and records are not held back to fill a batch. + long delivery = broker.deliveryTimeout().toMillis(); + config.put(org.apache.kafka.clients.producer.ProducerConfig.LINGER_MS_CONFIG, 0); + config.put( + org.apache.kafka.clients.producer.ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, + Math.toIntExact(Math.min(30_000L, delivery))); + config.put( + org.apache.kafka.clients.producer.ProducerConfig.DELIVERY_TIMEOUT_MS_CONFIG, + Math.toIntExact(delivery)); + // send() blocks the caller while it waits for metadata or buffer space, for Kafka's default + // 60s — before the publish deadline is even applied. Bounded by the default deadline instead. + config.put( + org.apache.kafka.clients.producer.ProducerConfig.MAX_BLOCK_MS_CONFIG, + dev.caskeleton.messaging.api.publish.PublishOptions.defaults().timeout().toMillis()); + config.putAll( + clientSecurity( + compiled, declaredSecurity, securityConfigurer, BrokerSecurityProfile::producer)); + return config; } /** - * The Kafka transport the publisher writes through. + * A client's security properties, from the one security profile of the Kafka broker. + * + *

{@code KafkaSecurityConfigurer} was a bean nothing called, so a broker that passed the TLS + * and credential checks was reached over plaintext with no authentication. Every case that cannot + * be applied fails startup rather than falling back to that. The producer and the consumers use + * the same rule, each with its own credential. + * + * @param role picks the producer or the consumer credential of the profile + */ + static java.util.Map clientSecurity( + CompiledMessagingConfiguration compiled, + ObjectProvider declaredSecurity, + ObjectProvider securityConfigurer, + java.util.function.Function< + BrokerSecurityProfile, dev.caskeleton.messaging.security.BrokerCredentialProfile> + role) { + java.util.Map kafkaBrokers = new java.util.TreeMap<>(); + compiled.brokers().entrySet().stream() + .filter(entry -> BrokerSettings.KAFKA.equalsIgnoreCase(entry.getValue().type())) + .forEach(entry -> kafkaBrokers.put(entry.getKey(), entry.getValue())); + // The same fallback as the runtime's logical name: bean-declared profiles name the family. + java.util.Set names = + kafkaBrokers.isEmpty() ? java.util.Set.of(BrokerSettings.KAFKA) : kafkaBrokers.keySet(); + java.util.List profiles = + java.util.stream.Stream.concat( + compiled.security().stream(), declaredSecurity.orderedStream()) + .filter(profile -> names.contains(profile.broker())) + .toList(); + + if (profiles.isEmpty()) { + kafkaBrokers.forEach( + (name, broker) -> { + if (broker.tlsEnabled() || broker.authenticationEnabled()) { + throw new IllegalStateException( + "Kafka broker '" + + name + + "' enables TLS or authentication but has no app.messaging.security." + + name + + " entry; a Kafka client cannot apply either without one"); + } + }); + return java.util.Map.of(); + } + if (profiles.size() > 1) { + throw new IllegalStateException( + "a Kafka client has one security posture but " + + profiles.size() + + " security profiles match " + + names); + } + BrokerSecurityProfile profile = profiles.getFirst(); + KafkaSecurityConfigurer configurer = securityConfigurer.getIfAvailable(); + if (configurer == null) { + throw new IllegalStateException( + "app.messaging.security." + + profile.broker() + + " is configured but no CredentialProvider bean exists; a Kafka client cannot" + + " resolve its credential and must not connect without it"); + } + return configurer.configure( + profile, role.apply(profile), TLS_PROTOCOLS, java.time.Instant.now()); + } + + /** + * The Kafka transport the publisher writes through and the consumers subscribe through. + * + *

It was built publish-only, so {@code register} refused every subscription with {@code + * KAFKA_CONSUMER_NOT_CONFIGURED} and the consumer runtime was never constructed (S7-#2). * * @param producer the configured producer + * @param consumers opens a live subscription for a destination * @return the transport */ @Bean @ConditionalOnMissingBean(dev.caskeleton.messaging.transport.MessagingTransport.class) public dev.caskeleton.messaging.kafka.KafkaMessagingTransport messagingKafkaTransport( - org.apache.kafka.clients.producer.Producer producer) { - // Generation 1: the credential generation a rotation increments. Starting elsewhere would make - // the first rotation look like it had already happened. - return new dev.caskeleton.messaging.kafka.KafkaMessagingTransport("kafka", 1L, producer); + org.apache.kafka.clients.producer.Producer producer, + KafkaConsumerConfiguration.KafkaConsumerFactory consumers) { + // Generation 1 for the life of the producer. A SASL credential rotation does not replace the + // producer: KafkaCredentialCallbackHandler reads the rotated secret at the next authentication, + // so there is no second transport generation to install. + return new dev.caskeleton.messaging.kafka.KafkaMessagingTransport( + "kafka", 1L, producer, consumers); } private static java.util.List - configuredKafkaProfiles(CompiledMessagingConfiguration compiled) { - return compiled.brokers().entrySet().stream() - .filter( - entry -> - entry.getValue().type() != null - && entry.getValue().type().equalsIgnoreCase(BrokerSettings.KAFKA)) + configuredKafkaProfiles(CompiledMessagingConfiguration compiled, KafkaBrokerCatalog catalog) { + return catalog.brokers().entrySet().stream() .map( entry -> { - BrokerSettings broker = entry.getValue(); + BrokerSettings broker = compiled.brokers().get(entry.getKey()); + KafkaBrokerSettings client = entry.getValue(); return new dev.caskeleton.messaging.kafka.KafkaBrokerProfile( entry.getKey(), broker.stable(), broker.production(), - broker.bootstrapServers(), - broker.enableIdempotence(), - broker.acks(), - broker.maxInFlightRequestsPerConnection(), - broker.deliveryTimeout(), - broker.enableAutoCommit(), - broker.consumerGroup(), + client.bootstrapServers(), + client.enableIdempotence(), + client.acks(), + client.maxInFlightRequestsPerConnection(), + client.deliveryTimeout(), + client.enableAutoCommit(), + client.consumerGroup(), broker.tlsEnabled(), broker.authenticationEnabled()); }) diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaPolledRegistration.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaPolledRegistration.java new file mode 100644 index 0000000..c309a13 --- /dev/null +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaPolledRegistration.java @@ -0,0 +1,169 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.kafka.KafkaConsumerRegistrar; +import dev.caskeleton.messaging.kafka.KafkaQuarantinePublisher; +import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; +import dev.caskeleton.messaging.transport.TransportConsumerRegistration; +import dev.caskeleton.messaging.transport.TransportConsumerSpec; +import java.time.Duration; +import java.time.Instant; +import java.util.Map; +import java.util.concurrent.CompletionStage; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeUnit; +import org.apache.kafka.clients.consumer.KafkaConsumer; + +/** + * One live Kafka subscription: a {@link KafkaConsumerRegistrar}, the poll thread that drives it and + * the bounded pool its handlers run on. + * + *

The poll thread is the only thread that touches the {@code KafkaConsumer}: the registrar + * polls, commits and closes there, and handler work runs on a pool sized by the destination's + * {@code consumer.concurrency}. Offsets are committed through the contiguous watermark of settled + * deliveries, never on a timer. + * + *

{@link #close()} stops dispatching, lets running handlers finish up to the drain deadline, + * commits what they settled and closes the client. Work still running at the deadline is abandoned + * unsettled, so the broker redelivers it rather than the platform pretending it completed. + */ +final class KafkaPolledRegistration implements TransportConsumerRegistration { + + private static final System.Logger LOG = + System.getLogger(KafkaPolledRegistration.class.getName()); + + private static final Duration POLL_TIMEOUT = Duration.ofMillis(200); + + private final String name; + private final KafkaConsumerRegistrar registrar; + private final GracefulShutdownCoordinator drain; + private final ExecutorService handlers; + private final Duration drainDeadline; + private final Thread poller; + private volatile boolean stopping; + + private KafkaPolledRegistration( + TransportConsumerSpec spec, + Map clientConfig, + KafkaQuarantinePublisher quarantine, + Duration drainDeadline) { + this.name = spec.profile().name().value(); + this.drainDeadline = drainDeadline; + String topic = + spec.profile() + .physical() + .topic() + .orElseThrow( + () -> + new IllegalStateException( + "destination '" + name + "' is consumed from Kafka but has no topic")); + this.drain = new GracefulShutdownCoordinator(drainDeadline); + this.handlers = + Executors.newFixedThreadPool( + spec.profile().consumer().concurrency(), + Thread.ofPlatform().name("messaging-kafka-handler-" + name + "-", 0).factory()); + this.registrar = + new KafkaConsumerRegistrar( + spec, + new KafkaConsumer(clientConfig), + handlers, + drain, + POLL_TIMEOUT, + quarantine); + registrar.subscribe(topic); + this.poller = Thread.ofPlatform().name("messaging-kafka-poll-" + name).unstarted(this::loop); + } + + /** + * Opens the client, subscribes and starts polling. + * + * @param spec the destination and the pipeline its deliveries go through + * @param clientConfig the consumer client configuration + * @param quarantine where a record the consumer cannot deliver is parked + * @param drainDeadline how long {@link #close()} lets running handlers finish + * @return the live registration + */ + static KafkaPolledRegistration open( + TransportConsumerSpec spec, + Map clientConfig, + KafkaQuarantinePublisher quarantine, + Duration drainDeadline) { + KafkaPolledRegistration registration = + new KafkaPolledRegistration(spec, clientConfig, quarantine, drainDeadline); + registration.poller.start(); + return registration; + } + + @Override + public CompletionStage pause(String scope) { + return registrar.pause(scope); + } + + @Override + public CompletionStage resume(String scope) { + return registrar.resume(scope); + } + + @Override + public boolean isActive() { + return registrar.isActive() && !stopping; + } + + @Override + public void close() { + Instant startedAt = Instant.now(); + drain.beginDrain(startedAt); + Instant deadline = startedAt.plus(drainDeadline); + while (!drain.isDrained(Instant.now()) && Instant.now().isBefore(deadline)) { + if (!sleep(Duration.ofMillis(50))) { + break; + } + } + stopping = true; + try { + poller.join(POLL_TIMEOUT.toMillis() * 10); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + } + handlers.shutdownNow(); + try { + handlers.awaitTermination(1, TimeUnit.SECONDS); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + } + } + + private void loop() { + try { + while (!stopping) { + try { + registrar.pollOnce(Instant.now()); + } catch (RuntimeException failure) { + // A failed poll or commit is not a reason to stop consuming for good: nothing past the + // last successful commit is lost, and the next cycle retries it. + LOG.log( + System.Logger.Level.WARNING, + "Kafka consumer for " + name + " failed a poll cycle: " + failure); + sleep(POLL_TIMEOUT); + } + if (!drain.isAcceptingWork()) { + // pollOnce no longer polls once draining begins; it only applies settlements and + // commits. Without a pause the loop would spin. + sleep(Duration.ofMillis(20)); + } + } + } finally { + registrar.close(); + } + } + + private static boolean sleep(Duration duration) { + try { + Thread.sleep(duration.toMillis()); + return true; + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + return false; + } + } +} diff --git a/messaging/messaging-spring-boot-starter-rabbit/build.gradle b/messaging/messaging-spring-boot-starter-rabbit/build.gradle deleted file mode 100644 index 3635e1c..0000000 --- a/messaging/messaging-spring-boot-starter-rabbit/build.gradle +++ /dev/null @@ -1,12 +0,0 @@ -plugins { - id 'ca.platform-module' - id 'ca.spring-config' -} - -dependencies { - // Capability artifact: Rabbit types/client exist only when this artifact is deliberately added. - api project(':messaging:messaging-spring-boot-starter') - api project(':messaging:messaging-rabbit') - - implementation 'org.springframework.boot:spring-boot-autoconfigure' -} diff --git a/messaging/messaging-spring-boot-starter-rabbit/gradle.lockfile b/messaging/messaging-spring-boot-starter-rabbit/gradle.lockfile deleted file mode 100644 index 1d3075d..0000000 --- a/messaging/messaging-spring-boot-starter-rabbit/gradle.lockfile +++ /dev/null @@ -1,124 +0,0 @@ -# This is a Gradle generated file for dependency locking. -# Manual edits can break the build and are not advised. -# This file is expected to be part of source control. -com.fasterxml.jackson.core:jackson-annotations:2.21=runtimeClasspath,testRuntimeClasspath -com.github.ben-manes.caffeine:caffeine:3.2.4=annotationProcessor,testAnnotationProcessor -com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor -com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs -com.github.spotbugs:spotbugs:4.10.2=spotbugs -com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs -com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor -com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor -com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor -com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs -com.google.code.gson:gson:2.13.2=spotbugs -com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_annotations:2.41.0=spotbugs -com.google.errorprone:error_prone_annotations:2.47.0=checkstyle -com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor -com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor -com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor -com.google.guava:guava:33.6.0-jre=checkstyle -com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor -com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor -com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins -com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -com.rabbitmq:amqp-client:5.27.1=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -commons-beanutils:commons-beanutils:1.11.0=checkstyle -commons-collections:commons-collections:3.2.2=checkstyle -commons-io:commons-io:2.21.0=spotbugs -commons-logging:commons-logging:1.3.6=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -info.picocli:picocli:4.7.7=checkstyle -io.cloudevents:cloudevents-api:4.0.1=runtimeClasspath,testRuntimeClasspath -io.cloudevents:cloudevents-core:4.0.1=runtimeClasspath,testRuntimeClasspath -io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor -io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor -io.micrometer:micrometer-commons:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-core:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.micrometer:micrometer-observation:1.16.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-buffer:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-base:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-compression:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-marshalling:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-protobuf:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-common:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-handler:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-resolver:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-transport-native-unix-common:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-transport:4.2.17.Final=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.projectreactor:reactor-core:3.8.7=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor -jaxen:jaxen:2.0.6=spotbugs -net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath -net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs -org.antlr:antlr4-runtime:4.13.2=checkstyle -org.apache.bcel:bcel:6.12.0=spotbugs -org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-text:1.15.0=spotbugs -org.apache.commons:commons-text:1.3=checkstyle -org.apache.httpcomponents:httpclient:4.5.13=checkstyle -org.apache.httpcomponents:httpcore:4.4.16=checkstyle -org.apache.logging.log4j:log4j-api:2.25.5=spotbugs -org.apache.logging.log4j:log4j-core:2.25.5=spotbugs -org.apache.maven.doxia:doxia-core:1.12.0=checkstyle -org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle -org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle -org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle -org.apache.xbean:xbean-reflect:3.7=checkstyle -org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath -org.assertj:assertj-core:3.27.7=testCompileClasspath,testRuntimeClasspath -org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle -org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle -org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle -org.codehaus.plexus:plexus-utils:3.3.0=checkstyle -org.dom4j:dom4j:2.2.0=spotbugs -org.hdrhistogram:HdrHistogram:2.2.2=runtimeClasspath,testRuntimeClasspath -org.javassist:javassist:3.28.0-GA=checkstyle -org.jspecify:jspecify:1.0.1=annotationProcessor,checkstyle,compileClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-api:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter-engine:6.0.3=testRuntimeClasspath -org.junit.jupiter:junit-jupiter-params:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.jupiter:junit-jupiter:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-commons:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit.platform:junit-platform-engine:6.0.3=testRuntimeClasspath -org.junit.platform:junit-platform-launcher:6.0.3=testRuntimeClasspath -org.junit:junit-bom:6.0.3=testCompileClasspath,testRuntimeClasspath -org.junit:junit-bom:6.1.0=spotbugs -org.latencyutils:LatencyUtils:2.0.3=runtimeClasspath,testRuntimeClasspath -org.mockito:mockito-core:5.20.0=mockitoAgent -org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath -org.ow2.asm:asm-analysis:9.10.1=spotbugs -org.ow2.asm:asm-commons:9.10.1=spotbugs -org.ow2.asm:asm-tree:9.10.1=spotbugs -org.ow2.asm:asm-util:9.10.1=spotbugs -org.ow2.asm:asm:9.10.1=spotbugs -org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor -org.reactivestreams:reactive-streams:1.0.4=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.reflections:reflections:0.10.2=checkstyle -org.slf4j:slf4j-api:2.0.18=compileClasspath,runtimeClasspath,spotbugs,spotbugsSlf4j,testCompileClasspath,testRuntimeClasspath -org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j -org.slf4j:slf4j-simple:2.0.18=checkstyle -org.springframework.amqp:spring-amqp:4.0.5=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework.amqp:spring-rabbit:4.0.5=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework.boot:spring-boot-actuator:4.0.8=runtimeClasspath,testRuntimeClasspath -org.springframework.boot:spring-boot-autoconfigure:4.0.8=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework.boot:spring-boot-configuration-processor:4.0.8=annotationProcessor -org.springframework.boot:spring-boot:4.0.8=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-aop:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-beans:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-context:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-core:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-expression:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-jdbc:7.0.9=runtimeClasspath,testRuntimeClasspath -org.springframework:spring-messaging:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-tx:7.0.9=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs -tools.jackson.core:jackson-core:3.1.5=runtimeClasspath,testRuntimeClasspath -tools.jackson.core:jackson-databind:3.1.5=runtimeClasspath,testRuntimeClasspath -tools.jackson:jackson-bom:3.1.5=runtimeClasspath,testRuntimeClasspath -empty= diff --git a/messaging/messaging-spring-boot-starter-rabbit/src/main/java/dev/caskeleton/messaging/autoconfigure/RabbitMessagingAutoConfiguration.java b/messaging/messaging-spring-boot-starter-rabbit/src/main/java/dev/caskeleton/messaging/autoconfigure/RabbitMessagingAutoConfiguration.java deleted file mode 100644 index f64cd42..0000000 --- a/messaging/messaging-spring-boot-starter-rabbit/src/main/java/dev/caskeleton/messaging/autoconfigure/RabbitMessagingAutoConfiguration.java +++ /dev/null @@ -1,126 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.rabbit.RabbitProfileValidator; -import dev.caskeleton.messaging.rabbit.RabbitPublishFailureClassifier; -import dev.caskeleton.messaging.rabbit.RabbitSecurityConfigurer; -import dev.caskeleton.messaging.security.BrokerTlsPolicy; -import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; - -/** - * Wires the RabbitMQ adapter's validators and security configurer. - * - *

Conditional on the AMQP client, for the same reason the Kafka configuration is conditional on - * the Kafka client: a service using one broker must not need the other on its classpath. - * - *

No connection or channel bean is created here. A channel opened before the application has - * declared its security profile would connect with whatever defaults the client carries, which for - * RabbitMQ means {@code guest/guest} against localhost. - * - *

Reached only when app.messaging.broker=rabbit selected it; see MessagingProviderSelection for - * why a classpath condition could not make this choice. - */ -@Configuration(proxyBeanMethods = false) -public class RabbitMessagingAutoConfiguration { - - /** - * Returns the RabbitMQ destination profile validator. - * - * @return the validator - */ - @Bean - @ConditionalOnMissingBean - public RabbitProfileValidator rabbitProfileValidator() { - return new RabbitProfileValidator(); - } - - /** - * Runs the RabbitMQ profile validator over every broker profile, declared or configured. - * - * @param profiles the broker profiles the application declared as beans - * @param compiled the broker profiles compiled from {@code app.messaging.brokers} - * @param validator the RabbitMQ profile validator - * @return the startup validation - */ - @Bean - public StartupProfileValidation - rabbitProfileStartupValidation( - org.springframework.beans.factory.ObjectProvider< - dev.caskeleton.messaging.rabbit.RabbitBrokerProfile> - profiles, - CompiledMessagingConfiguration compiled, - RabbitProfileValidator validator) { - return new StartupProfileValidation<>( - () -> - java.util.stream.Stream.concat( - configuredRabbitProfiles(compiled).stream(), profiles.orderedStream()) - .toList(), - validator::validate); - } - - /** - * Returns the publish failure classifier. - * - * @return the classifier - */ - @Bean - @ConditionalOnMissingBean - public RabbitPublishFailureClassifier rabbitPublishFailureClassifier() { - return new RabbitPublishFailureClassifier(); - } - - /** - * Returns the AMQP security configurer. - * - * @param credentials the credential runtime registry - * @param tlsPolicy the transport security policy - * @return the configurer - */ - /** - * The security configurer, which exists only when a credential source does. - * - *

It took {@code CredentialRuntimeRegistry} as a required parameter, and that registry is - * created only when the application supplies a {@code CredentialProvider}. Without one the - * context failed on a missing bean naming the registry — an error that describes the platform's - * internals rather than the one thing the operator has to do. The condition moves the failure to - * a place that can say so. - */ - @Bean - @ConditionalOnMissingBean - @org.springframework.boot.autoconfigure.condition.ConditionalOnBean( - CredentialRuntimeRegistry.class) - public RabbitSecurityConfigurer rabbitSecurityConfigurer( - CredentialRuntimeRegistry credentials, BrokerTlsPolicy tlsPolicy) { - return new RabbitSecurityConfigurer(credentials, tlsPolicy); - } - - private static java.util.List - configuredRabbitProfiles(CompiledMessagingConfiguration compiled) { - return compiled.brokers().entrySet().stream() - .filter( - entry -> - entry.getValue().type() != null - && entry.getValue().type().equalsIgnoreCase(BrokerSettings.RABBITMQ)) - .map( - entry -> { - BrokerSettings broker = entry.getValue(); - return new dev.caskeleton.messaging.rabbit.RabbitBrokerProfile( - entry.getKey(), - broker.stable(), - broker.production(), - broker.addresses(), - broker.publisherConfirms(), - broker.publisherReturns(), - broker.mandatory(), - broker.confirmTimeout(), - broker.autoAck(), - broker.prefetch(), - broker.quorumQueues(), - broker.tlsEnabled(), - broker.authenticationEnabled()); - }) - .toList(); - } -} diff --git a/messaging/messaging-spring-boot-starter/README.md b/messaging/messaging-spring-boot-starter/README.md index 9580cea..2cb6c3e 100644 --- a/messaging/messaging-spring-boot-starter/README.md +++ b/messaging/messaging-spring-boot-starter/README.md @@ -2,21 +2,21 @@ ## 아티팩트 분리 -기본 `messaging-spring-boot-starter`는 이제 **broker-neutral core**다. Kafka/Rabbit client를 +기본 `messaging-spring-boot-starter`는 이제 **broker-neutral core**다. Kafka client를 production runtime에 직접 넣지 않으며, 브로커 채택은 capability artifact를 composition에 추가하는 build-time 결정이다. ```text messaging-spring-boot-starter -├─ messaging-spring-boot-starter-kafka # Kafka client/transport capability -└─ messaging-spring-boot-starter-rabbit # Rabbit profile/validator capability +└─ messaging-spring-boot-starter-kafka # Kafka client/transport capability ``` -- 기본 app-bootstrap은 core starter만 가진다. 따라서 messaging off/default runtime에는 - `kafka-clients`, `spring-kafka`, `spring-rabbit`, `amqp-client`가 없다. -- Kafka를 채택하는 composition은 `:messaging:messaging-spring-boot-starter-kafka`를 추가한다. -- Rabbit capability도 별도 artifact지만, 아래 설명처럼 아직 production transport가 없어 - `app.messaging.broker=rabbit` 선택은 startup에서 거부된다. +- 기본 app-bootstrap은 messaging을 싣지 않는다. 서비스는 `runtimeOnly + project(':adapter:outbound:messaging-kafka')` 한 줄로 Kafka capability를 고르고, 그 모듈이 + `:messaging:messaging-spring-boot-starter-kafka`(와 이 core starter)를 runtime으로 끌어온다. +- starter는 있으면 켜진다(`app.messaging.enabled` 기본 on). `app.messaging.enabled=false`로만 끈다. +- 등록된 broker 계열은 Kafka 하나다. RabbitMQ 설정 키(`type: rabbitmq`, `addresses`, `prefetch` 등)는 + transport 없이 바인딩만 되던 키라 삭제했고, 지금은 키 이름을 담아 기동이 거부된다. - reliability/admin의 추가 artifact 분리는 별도 구조 개선 여지로 남지만, broker SDK 동시 탑재 문제와 transport 선택 문제는 core/broker capability 분리로 해소했다. - legacy R0 outbound bridge도 같은 원칙을 따른다. broker-neutral @@ -35,13 +35,9 @@ consume을 채택하려면 애플리케이션 composition이 별도로 `KafkaCon 조립해야 한다. leaf 안의 consumer/rebalance/settlement 테스트가 초록이라는 사실은 capability starter가 consumer runtime까지 자동 제공한다는 뜻이 아니다. -### RabbitMQ +### RabbitMQ (삭제됨) -RabbitMQ는 별도 capability artifact에 격리되어 있지만 **아직 선택할 수 없는 transport**다. -`app.messaging.broker=rabbit`을 설정하면 `MessagingProviderSelection`이 startup에서 거부한다. -이 leaf에는 validator, failure classifier, topology/security 부품과 테스트는 있지만 production -`RabbitChannelPublisher` 구현이 없어 `MessagingTransport`를 완성할 수 없다. - -현재 selectable broker는 Kafka뿐이다. Rabbit이 selectable로 승격되는 조건은 production -channel publisher/settlement bridge를 구현하고 starter composition 및 broker integration 증거를 -함께 추가하는 것이다. +production transport가 없어 선택할 수 없던 `messaging-rabbit`·`messaging-spring-boot-starter-rabbit`는 +2026-09-25에 삭제했다(`docs/capability-status.md` 제거 표). 현재 등록된 broker는 Kafka 하나다. +`app.messaging.brokers..type: rabbitmq` 설정 family는 아직 바인딩·검증되지만 소비하는 transport가 +없다. diff --git a/messaging/messaging-spring-boot-starter/build.gradle b/messaging/messaging-spring-boot-starter/build.gradle index 14167fa..efed4b9 100644 --- a/messaging/messaging-spring-boot-starter/build.gradle +++ b/messaging/messaging-spring-boot-starter/build.gradle @@ -1,30 +1,30 @@ plugins { - id 'ca.auxiliary-source-set' id 'ca.platform-module' id 'ca.spring-config' } // Scopes, not a flat list of `api`. // -// Sixteen internal leaves were exposed as `api`, so every adopter of the starter compiled against -// the Kafka adapter, the Rabbit adapter, both JDBC reliability leaves and the admin plane whether -// they used them or not — and both brokers' `@ConditionalOnClass` guards were true at once because -// the starter itself dragged in both clients. -// -// `api` is now the types that genuinely appear in this module's public signatures; everything else -// is `implementation`. Splitting the starter into per-capability artifacts (core / kafka / rabbit / -// reliability / admin) is the structural fix the review asks for and is a registry change of its -// own; this is the part that stops an adopter's compile classpath from containing two broker SDKs. -auxiliarySourceSets { - sourceSet('integrationTest') { - compilesAgainst 'main' +// `api` is the types that genuinely appear in this module's public signatures; everything else is +// `implementation`. Broker transports live in capability starters (messaging-spring-boot-starter-kafka), +// so this starter's compile classpath carries no broker SDK. +testing { + suites { + integrationTest(JvmTestSuite) { + dependencies { + implementation project() + } + } } } +configurations { + integrationTestImplementation.extendsFrom testImplementation + integrationTestRuntimeOnly.extendsFrom testRuntimeOnly +} dependencies { // Types an adopter writes against: envelopes, destinations, publish results, profiles, ports. api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') api project(':messaging:messaging-policy') api project(':messaging:messaging-transport-spi') api project(':messaging:messaging-observability') @@ -33,32 +33,23 @@ dependencies { // Wired by the auto-configurations and never named in an adopter's source. implementation project(':messaging:messaging-schema-json') - implementation project(':messaging:messaging-cloudevents') // The publisher and its collaborators live here. The starter did not depend on it at all, // which is why MessagingCoreAutoConfiguration consumed a MessagePublisher that nothing in // reach could produce (MSG-INT-003). implementation project(':messaging:messaging-runtime-core') - implementation project(':messaging:messaging-outbox-jdbc-postgresql') - implementation project(':messaging:messaging-inbox-jdbc-postgresql') - implementation project(':messaging:messaging-claim-check') - implementation project(':messaging:messaging-admin-api') - implementation project(':messaging:messaging-admin-runtime') + // The relay and consumer logic, and the PostgreSQL stores behind their ports. + implementation project(':messaging:messaging-reliability') + implementation project(':messaging:messaging-reliability-jdbc-postgresql') + implementation project(':messaging:messaging-admin') implementation 'org.springframework.boot:spring-boot-autoconfigure' implementation 'org.springframework.boot:spring-boot-actuator' - // The Reactor facade lives here, not in core-api: the core contract stays CompletionStage so - // that a service which does not use Reactor never inherits it. api, because - // ReactiveMessagePublisher returns Mono — a facade whose return type an adopter cannot name - // is a facade they cannot call. - api 'io.projectreactor:reactor-core' - testImplementation 'io.projectreactor:reactor-test' testImplementation 'org.springframework.boot:spring-boot-test' testImplementation 'org.springframework:spring-test' // Provider auto-configurations live in capability artifacts. Tests that exercise provider - // selection add them only to the test graph; the production starter remains broker-neutral. + // selection add it only to the test graph; the production starter remains broker-neutral. testImplementation project(':messaging:messaging-spring-boot-starter-kafka') - testImplementation project(':messaging:messaging-spring-boot-starter-rabbit') // Test-only, and specifically so the configuration reference can be executed rather than read: // MessagingConfigurationBindingTest loads the YAML blocks out of the shipped document and boots // a context on them. Spring Boot 4 treats snakeyaml as optional, so a document-driven test has @@ -71,6 +62,13 @@ dependencies { integrationTestImplementation project(':messaging:messaging-spring-boot-starter-kafka') integrationTestImplementation 'org.testcontainers:testcontainers-junit-jupiter' integrationTestImplementation 'org.testcontainers:testcontainers-kafka' + // The consume round trip's inbox lane: a real PostgreSQL behind the JDBC inbox, joined to the + // handler's own write by a real transaction manager. + integrationTestImplementation project(':messaging:messaging-reliability') + integrationTestImplementation project(':messaging:messaging-reliability-jdbc-postgresql') + integrationTestImplementation 'org.springframework:spring-jdbc' + integrationTestImplementation 'org.testcontainers:testcontainers-postgresql' + integrationTestImplementation 'org.postgresql:postgresql' } strictTestLanes { diff --git a/messaging/messaging-spring-boot-starter/gradle.lockfile b/messaging/messaging-spring-boot-starter/gradle.lockfile index 7f5981c..d178e83 100644 --- a/messaging/messaging-spring-boot-starter/gradle.lockfile +++ b/messaging/messaging-spring-boot-starter/gradle.lockfile @@ -32,7 +32,6 @@ com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,integrat com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins com.puppycrawl.tools:checkstyle:13.5.0=checkstyle -com.rabbitmq:amqp-client:5.27.1=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath commons-beanutils:commons-beanutils:1.11.0=checkstyle commons-codec:commons-codec:1.19.0=integrationTestCompileClasspath,integrationTestRuntimeClasspath commons-collections:commons-collections:3.2.2=checkstyle @@ -40,26 +39,11 @@ commons-io:commons-io:2.20.0=integrationTestCompileClasspath,integrationTestRunt commons-io:commons-io:2.21.0=spotbugs commons-logging:commons-logging:1.3.6=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath info.picocli:picocli:4.7.7=checkstyle -io.cloudevents:cloudevents-api:4.0.1=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.cloudevents:cloudevents-core:4.0.1=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor io.micrometer:micrometer-commons:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.micrometer:micrometer-core:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath io.micrometer:micrometer-observation:1.16.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-buffer:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-base:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-compression:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-marshalling:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec-protobuf:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-codec:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-common:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-handler:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-resolver:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-transport-native-unix-common:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.netty:netty-transport:4.2.17.Final=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -io.projectreactor:reactor-core:3.8.7=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -io.projectreactor:reactor-test:3.8.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath javax.inject:javax.inject:1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs net.bytebuddy:byte-buddy:1.17.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath @@ -83,6 +67,7 @@ org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle org.apache.xbean:xbean-reflect:3.7=checkstyle org.apiguardian:apiguardian-api:1.1.2=integrationTestCompileClasspath,testCompileClasspath org.assertj:assertj-core:3.27.7=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath +org.checkerframework:checker-qual:3.55.1=integrationTestRuntimeClasspath org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle @@ -110,14 +95,12 @@ org.ow2.asm:asm-tree:9.10.1=spotbugs org.ow2.asm:asm-util:9.10.1=spotbugs org.ow2.asm:asm:9.10.1=spotbugs org.pcollections:pcollections:4.0.1=annotationProcessor,integrationTestAnnotationProcessor,testAnnotationProcessor -org.reactivestreams:reactive-streams:1.0.4=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath +org.postgresql:postgresql:42.7.13=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.reflections:reflections:0.10.2=checkstyle org.rnorth.duct-tape:duct-tape:1.0.8=integrationTestCompileClasspath,integrationTestRuntimeClasspath -org.slf4j:slf4j-api:2.0.18=integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs,spotbugsSlf4j,testCompileClasspath,testRuntimeClasspath +org.slf4j:slf4j-api:2.0.18=integrationTestCompileClasspath,integrationTestRuntimeClasspath,spotbugs,spotbugsSlf4j,testRuntimeClasspath org.slf4j:slf4j-simple:2.0.17=spotbugsSlf4j org.slf4j:slf4j-simple:2.0.18=checkstyle -org.springframework.amqp:spring-amqp:4.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework.amqp:spring-rabbit:4.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-actuator:4.0.8=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-autoconfigure:4.0.8=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework.boot:spring-boot-configuration-processor:4.0.8=annotationProcessor @@ -128,12 +111,14 @@ org.springframework:spring-beans:7.0.9=compileClasspath,integrationTestCompileCl org.springframework:spring-context:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework:spring-core:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.springframework:spring-expression:7.0.9=compileClasspath,integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-jdbc:7.0.9=integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath -org.springframework:spring-messaging:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath +org.springframework:spring-jdbc:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath org.springframework:spring-test:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,testCompileClasspath,testRuntimeClasspath -org.springframework:spring-tx:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath +org.springframework:spring-tx:7.0.9=integrationTestCompileClasspath,integrationTestRuntimeClasspath,runtimeClasspath,testRuntimeClasspath +org.testcontainers:testcontainers-database-commons:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath +org.testcontainers:testcontainers-jdbc:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.testcontainers:testcontainers-junit-jupiter:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.testcontainers:testcontainers-kafka:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath +org.testcontainers:testcontainers-postgresql:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.testcontainers:testcontainers:2.0.5=integrationTestCompileClasspath,integrationTestRuntimeClasspath org.xerial.snappy:snappy-java:1.1.10.7=integrationTestRuntimeClasspath,testRuntimeClasspath org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs diff --git a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingKafkaConsumeQualificationTest.java b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingKafkaConsumeQualificationTest.java new file mode 100644 index 0000000..7083bc6 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingKafkaConsumeQualificationTest.java @@ -0,0 +1,460 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.ContentType; +import dev.caskeleton.messaging.api.MessageEnvelope; +import dev.caskeleton.messaging.api.MessageId; +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.ProducerId; +import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; +import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; +import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.delivery.OrderingScope; +import dev.caskeleton.messaging.api.destination.DestinationKind; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.error.FailureCategory; +import dev.caskeleton.messaging.api.error.FailureDescriptor; +import dev.caskeleton.messaging.api.header.MessageHeaders; +import dev.caskeleton.messaging.api.header.ReservedHeaders; +import dev.caskeleton.messaging.api.publish.MessagePublisher; +import dev.caskeleton.messaging.api.publish.PublishCompletion; +import dev.caskeleton.messaging.api.publish.PublishOptions; +import dev.caskeleton.messaging.inbox.IdempotentConsumer; +import dev.caskeleton.messaging.inbox.JdbcInboxRepository; +import dev.caskeleton.messaging.policy.ApiTier; +import dev.caskeleton.messaging.policy.ConsumerPolicy; +import dev.caskeleton.messaging.policy.DeadLetterPolicy; +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.policy.OrderingImpact; +import dev.caskeleton.messaging.policy.PayloadPolicy; +import dev.caskeleton.messaging.policy.PhysicalDestination; +import dev.caskeleton.messaging.policy.ProducerPolicy; +import dev.caskeleton.messaging.policy.RetryMode; +import dev.caskeleton.messaging.policy.RetryPolicy; +import dev.caskeleton.messaging.policy.SchemaPolicy; +import dev.caskeleton.messaging.schema.MessageContractKey; +import dev.caskeleton.messaging.schema.SchemaCompatibility; +import java.io.IOException; +import java.io.InputStream; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.time.Instant; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Properties; +import java.util.Set; +import java.util.concurrent.BlockingQueue; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionStage; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.Function; +import javax.sql.DataSource; +import org.apache.kafka.clients.admin.Admin; +import org.apache.kafka.clients.admin.AdminClientConfig; +import org.apache.kafka.clients.consumer.ConsumerConfig; +import org.apache.kafka.clients.consumer.ConsumerRecord; +import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.clients.consumer.OffsetAndMetadata; +import org.apache.kafka.common.TopicPartition; +import org.apache.kafka.common.serialization.ByteArrayDeserializer; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.condition.EnabledIf; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.jdbc.datasource.DataSourceTransactionManager; +import org.springframework.jdbc.datasource.DriverManagerDataSource; +import org.springframework.transaction.support.TransactionTemplate; +import org.testcontainers.DockerClientFactory; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.kafka.KafkaContainer; +import org.testcontainers.postgresql.PostgreSQLContainer; + +/** + * A message published through the assembled platform is consumed by a {@code MessageHandler} bean + * on a real broker (owner decision 10, S7-#2). + * + *

The consume side had three contracts and no assembly: the typed handler had no caller, the + * Kafka consumer runtime was never constructed, and the transport the starter built refused every + * subscription with {@code KAFKA_CONSUMER_NOT_CONFIGURED}. These lanes run the whole path — + * publish, poll, decode, inbox, handler, settlement, offset commit — against Kafka, and against + * PostgreSQL for the inbox. + */ +@Testcontainers +@EnabledIf(value = "dockerAvailable", disabledReason = "Docker not available for the live broker") +class MessagingKafkaConsumeQualificationTest { + + private static final MessageType CREATED = new MessageType("order.created"); + + @Container static final KafkaContainer KAFKA = new KafkaContainer("apache/kafka:4.1.0"); + + // @Container: the Testcontainers JUnit extension owns this lifecycle. + @SuppressWarnings("resource") + @Container + static final PostgreSQLContainer POSTGRES = + new PostgreSQLContainer("postgres:16-alpine") + .withDatabaseName("messaging") + .withUsername("messaging") + .withPassword("messaging"); + + static boolean dockerAvailable() { + return DockerClientFactory.instance().isDockerAvailable(); + } + + /** The payload the handlers are typed on. */ + record OrderCreated(String orderId) {} + + @Test + @DisplayName("플랫폼으로 발행한 메시지를 handler bean이 typed payload로 받고 offset이 commit된다") + void aPublishedMessageIsConsumedByTheHandlerBeanAndItsOffsetIsCommitted() { + // given + DestinationName orders = new DestinationName("consume-round-trip"); + BlockingQueue received = new LinkedBlockingQueue<>(); + MessageHandler handler = + handler( + orders, + delivery -> { + received.add(delivery.message().payload()); + return HandleResult.success(); + }); + + runner(orders, handler, RetryPolicy.none(), ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED) + .run( + context -> { + assertThat(context).hasNotFailed(); + + // when + publish(context.getBean(MessagePublisher.class), orders, envelope(MessageId.newId())); + + // then + assertThat(received.poll(60, TimeUnit.SECONDS)) + .as("the handler bean is called with the decoded payload") + .isEqualTo(new OrderCreated("o-1")); + assertThat(committedOffset(group(orders), orders.value(), 1)) + .as("the offset is committed once the delivery was settled") + .isEqualTo(1L); + }); + } + + @Test + @DisplayName("같은 메시지가 다시 배달돼도 inbox가 handler의 효과를 한 번만 남긴다") + void aRedeliveredMessageIsAppliedOnceThroughThePostgresInbox() throws Exception { + // given + DestinationName orders = new DestinationName("consume-inbox"); + DataSource dataSource = dataSource(); + JdbcTemplate jdbc = new JdbcTemplate(dataSource); + jdbc.execute("DROP TABLE IF EXISTS messaging_inbox"); + jdbc.execute("DROP TABLE IF EXISTS order_projection"); + jdbc.execute(inboxMigration()); + jdbc.execute("CREATE TABLE order_projection (order_id TEXT NOT NULL)"); + TransactionTemplate transactions = + new TransactionTemplate(new DataSourceTransactionManager(dataSource)); + IdempotentConsumer inbox = + new IdempotentConsumer( + new JdbcInboxRepository(dataSource), + new IdempotentConsumer.TransactionRunner() { + @Override + public T inTransaction(java.util.function.Supplier work) { + return transactions.execute(status -> work.get()); + } + }); + AtomicInteger calls = new AtomicInteger(); + MessageHandler handler = + handler( + orders, + delivery -> { + calls.incrementAndGet(); + // The business write joins the transaction the inbox reservation runs in. + jdbc.update( + "INSERT INTO order_projection (order_id) VALUES (?)", + delivery.message().payload().orderId()); + return HandleResult.success(); + }); + MessageEnvelope message = envelope(MessageId.newId()); + + runner(orders, handler, RetryPolicy.none(), ExternalSideEffectGuarantee.INBOX_TRANSACTIONAL) + .withBean(IdempotentConsumer.class, () -> inbox) + .run( + context -> { + assertThat(context).hasNotFailed(); + MessagePublisher publisher = context.getBean(MessagePublisher.class); + + // when + publish(publisher, orders, message); + publish(publisher, orders, message); + + // then + assertThat(committedOffset(group(orders), orders.value(), 2)) + .as("both copies were settled") + .isEqualTo(2L); + assertThat(jdbc.queryForObject("SELECT count(*) FROM order_projection", Long.class)) + .as("the second copy found the reservation and did not apply the effect again") + .isEqualTo(1L); + assertThat(calls.get()).isEqualTo(1); + }); + } + + @Test + @DisplayName("계속 Retry로 답하는 메시지는 시도 횟수를 다 쓰면 DLQ 토픽에 남고 offset은 넘어간다") + void aMessageItsHandlerKeepsRetryingIsParkedOnTheDeadLetterTopicOnceItsAttemptsAreSpent() { + // given + DestinationName orders = new DestinationName("consume-retry"); + AtomicInteger attempts = new AtomicInteger(); + MessageHandler handler = + handler( + orders, + delivery -> { + attempts.incrementAndGet(); + return new HandleResult.Retry( + FailureDescriptor.of( + FailureCategory.PROCESSING_TRANSIENT, "DOWNSTREAM_BUSY", "try later")); + }); + RetryPolicy twoAttempts = + new RetryPolicy( + RetryMode.PAUSE_PARTITION, + 2, + Duration.ZERO, + Duration.ZERO, + 1.0, + false, + OrderingImpact.PRESERVE, + Set.of(), + Set.of()); + MessageId id = MessageId.newId(); + + runner(orders, handler, twoAttempts, ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED) + .run( + context -> { + assertThat(context).hasNotFailed(); + + // when + publish(context.getBean(MessagePublisher.class), orders, envelope(id)); + + // then + assertThat(readMessageIds(dlq(orders).value())) + .as("the message is parked where someone can redrive it, identity intact") + .contains(id.value().toString()); + assertThat(committedOffset(group(orders), orders.value(), 1)).isEqualTo(1L); + assertThat(attempts.get()).isEqualTo(2); + }); + } + + private static ApplicationContextRunner runner( + DestinationName orders, + MessageHandler handler, + RetryPolicy retry, + ExternalSideEffectGuarantee sideEffect) { + // Created up front. A consumer that subscribes before its topic exists waits for the next + // metadata refresh — five minutes by default — before it is assigned anything. + createTopics(orders.value(), dlq(orders).value()); + return new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=" + KAFKA.getBootstrapServers(), + "app.messaging.shutdown.drain-deadline=5s") + .withBean( + MessageContracts.class, + () -> + new MessageContracts( + Map.of( + new MessageContractKey(CREATED, new SchemaVersion(1)), OrderCreated.class))) + .withBean( + orders.value() + "Profile", + DestinationProfile.class, + () -> profile(orders, dlq(orders), retry, sideEffect)) + .withBean( + orders.value() + "DlqProfile", + DestinationProfile.class, + () -> + profile( + dlq(orders), + null, + RetryPolicy.none(), + ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED)) + .withBean(orders.value() + "Handler", MessageHandler.class, () -> handler); + } + + private static MessageHandler handler( + DestinationName orders, Function, HandleResult> body) { + return new MessageHandler<>() { + @Override + public MessageDestination destination() { + return new MessageDestination<>(orders, CREATED, OrderCreated.class); + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + return CompletableFuture.completedFuture(body.apply(delivery)); + } + }; + } + + private static void publish( + MessagePublisher publisher, DestinationName orders, MessageEnvelope message) { + PublishCompletion completion = + publisher + .publish( + new MessageDestination<>(orders, CREATED, OrderCreated.class), + message, + PublishOptions.defaults()) + .toCompletableFuture() + .join() + .completion(); + assertThat(completion).isEqualTo(PublishCompletion.CONFIRMED); + } + + private static DestinationName dlq(DestinationName source) { + return new DestinationName(source.value() + "-dlq"); + } + + private static String group(DestinationName destination) { + return destination.value() + "-projection"; + } + + private static DestinationProfile profile( + DestinationName name, + DestinationName deadLetter, + RetryPolicy retry, + ExternalSideEffectGuarantee sideEffect) { + ConsumerPolicy defaults = ConsumerPolicy.defaults(group(name)); + return new DestinationProfile( + name, + "kafka", + DestinationKind.EVENT_STREAM, + PhysicalDestination.kafkaTopic(name.value()), + new SchemaPolicy( + ContentType.JSON, SchemaCompatibility.BACKWARD_TRANSITIVE, Set.of(CREATED)), + deadLetter == null ? DeliveryGuarantee.AT_MOST_ONCE : DeliveryGuarantee.AT_LEAST_ONCE, + OrderingScope.NONE, + sideEffect, + ProducerPolicy.defaults(), + new ConsumerPolicy( + defaults.group(), + defaults.concurrency(), + defaults.maxInFlightPerOrderingUnit(), + defaults.prefetch(), + Duration.ofSeconds(10), + false), + retry, + deadLetter == null ? DeadLetterPolicy.disabled() : DeadLetterPolicy.to(deadLetter), + PayloadPolicy.defaults(), + ApiTier.M1, + false, + false, + true); + } + + private static MessageEnvelope envelope(MessageId id) { + return new MessageEnvelope<>( + id, + CREATED, + new SchemaVersion(1), + Instant.parse("2026-09-29T00:00:00Z"), + Optional.empty(), + new ProducerId("orders"), + Optional.empty(), + Optional.empty(), + ContentType.JSON, + Optional.empty(), + Optional.empty(), + Optional.empty(), + TraceContext.none(), + MessageHeaders.empty(), + new OrderCreated("o-1")); + } + + private static void createTopics(String... topics) { + try (Admin admin = + Admin.create( + Map.of(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA.getBootstrapServers()))) { + admin + .createTopics( + java.util.Arrays.stream(topics) + .map(topic -> new org.apache.kafka.clients.admin.NewTopic(topic, 1, (short) 1)) + .toList()) + .all() + .get(30, TimeUnit.SECONDS); + } catch (Exception failed) { + throw new IllegalStateException("could not create " + List.of(topics), failed); + } + } + + /** Waits, bounded, until the group has committed at least {@code expected} on partition 0. */ + private static long committedOffset(String group, String topic, long expected) throws Exception { + try (Admin admin = + Admin.create( + Map.of(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA.getBootstrapServers()))) { + Instant deadline = Instant.now().plusSeconds(60); + long seen = -1; + while (Instant.now().isBefore(deadline)) { + Map offsets = + admin.listConsumerGroupOffsets(group).partitionsToOffsetAndMetadata().get(); + OffsetAndMetadata committed = offsets.get(new TopicPartition(topic, 0)); + seen = committed == null ? -1 : committed.offset(); + if (seen >= expected) { + return seen; + } + TimeUnit.MILLISECONDS.sleep(200); + } + return seen; + } + } + + /** Reads the message ids that reached a topic, with a bound rather than an open-ended loop. */ + private static List readMessageIds(String topic) { + Properties config = new Properties(); + config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, KAFKA.getBootstrapServers()); + config.put(ConsumerConfig.GROUP_ID_CONFIG, topic + "-reader"); + config.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); + config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class); + config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class); + try (KafkaConsumer consumer = new KafkaConsumer<>(config)) { + consumer.subscribe(List.of(topic)); + Instant deadline = Instant.now().plusSeconds(60); + while (Instant.now().isBefore(deadline)) { + List ids = + java.util.stream.StreamSupport.stream( + consumer.poll(Duration.ofSeconds(1)).records(topic).spliterator(), false) + .map(MessagingKafkaConsumeQualificationTest::messageId) + .toList(); + if (!ids.isEmpty()) { + return ids; + } + } + return List.of(); + } + } + + private static String messageId(ConsumerRecord record) { + return new String( + record.headers().lastHeader(ReservedHeaders.MESSAGE_ID).value(), StandardCharsets.UTF_8); + } + + private static DataSource dataSource() { + return new DriverManagerDataSource( + POSTGRES.getJdbcUrl(), POSTGRES.getUsername(), POSTGRES.getPassword()); + } + + private static String inboxMigration() throws IOException { + try (InputStream sql = + MessagingKafkaConsumeQualificationTest.class.getResourceAsStream( + "/db/migration/messaging/V1101__messaging_inbox.sql")) { + assertThat(sql).as("the inbox migration ships with the JDBC inbox").isNotNull(); + return new String(sql.readAllBytes(), StandardCharsets.UTF_8); + } + } +} diff --git a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java index 8fdc0a2..42c1d15 100644 --- a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java +++ b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingLiveRoundTripQualificationTest.java @@ -77,7 +77,8 @@ class MessagingLiveRoundTripQualificationTest { .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=" + KAFKA.getBootstrapServers()) + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=" + KAFKA.getBootstrapServers()) // Both, because ValidatedDestinationRegistry refuses a profile whose dead-letter target is // not itself registered — a good invariant, and one a fixture declaring only the happy path // walks straight into. @@ -169,7 +170,7 @@ class MessagingLiveRoundTripQualificationTest { * same constant: a round trip that read a different topic from the one the transport wrote would * pass or fail for reasons unrelated to the platform. */ - private static dev.caskeleton.messaging.policy.DestinationProfile profile( + static dev.caskeleton.messaging.policy.DestinationProfile profile( DestinationName name, DestinationName deadLetter) { return new dev.caskeleton.messaging.policy.DestinationProfile( name, @@ -192,13 +193,13 @@ class MessagingLiveRoundTripQualificationTest { ? dev.caskeleton.messaging.policy.DeadLetterPolicy.disabled() : dev.caskeleton.messaging.policy.DeadLetterPolicy.to(deadLetter), dev.caskeleton.messaging.policy.PayloadPolicy.defaults(), - dev.caskeleton.messaging.policy.CapabilityTier.M1, + dev.caskeleton.messaging.policy.ApiTier.M1, false, false, false); } - private static MessageEnvelope envelope() { + static MessageEnvelope envelope() { return new MessageEnvelope<>( MessageId.newId(), new MessageType("order.created"), diff --git a/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java new file mode 100644 index 0000000..5e823a3 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/integrationTest/java/dev/caskeleton/messaging/autoconfigure/MessagingSecuredBrokerQualificationTest.java @@ -0,0 +1,395 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.publish.MessagePublisher; +import dev.caskeleton.messaging.api.publish.PublishCompletion; +import dev.caskeleton.messaging.api.publish.PublishOptions; +import dev.caskeleton.messaging.api.publish.PublishResult; +import dev.caskeleton.messaging.security.CredentialProvider; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Path; +import java.time.Instant; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.UUID; +import java.util.concurrent.TimeUnit; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.condition.EnabledIf; +import org.junit.jupiter.api.io.TempDir; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; +import org.testcontainers.DockerClientFactory; +import org.testcontainers.kafka.KafkaContainer; +import org.testcontainers.utility.MountableFile; + +/** + * A publish reaches a broker that trusts nobody by default (Review Focus 5, M-2e/NEW-4b, M-2d). + * + *

Both brokers present a certificate signed by a CA generated for this test, which no JVM trusts + * — so a publish is confirmed only if the configured trust store reached the client. The first + * broker then requires SASL PLAIN, and accepts only the user name the credential provider returns; + * the second requires a client certificate, which only the configured key store can present. + * + *

No secret is written in this file: the store and broker passwords are generated per run. + */ +@EnabledIf(value = "dockerAvailable", disabledReason = "Docker not available for the live broker") +class MessagingSecuredBrokerQualificationTest { + + private static final String IMAGE = "apache/kafka:4.1.0"; + + private static final String BROKER_USER = "svc-orders"; + + private static final String BROKER_STORE = "/etc/kafka/secrets/broker.p12"; + + private static final String BROKER_TRUST = "/etc/kafka/secrets/truststore.p12"; + + static boolean dockerAvailable() { + return DockerClientFactory.instance().isDockerAvailable(); + } + + @Test + void aPublishIsConfirmedByAPrivateCaBrokerThatRequiresSaslAsTheProviderUser( + @TempDir Path directory) { + // given + PrivateCa ca = new PrivateCa(directory); + String userPassword = UUID.randomUUID().toString(); + try (KafkaContainer kafka = + broker(ca, "SASL_SSL") + .withEnv("KAFKA_SASL_ENABLED_MECHANISMS", "PLAIN") + .withEnv( + "KAFKA_LISTENER_NAME_PLAINTEXT_PLAIN_SASL_JAAS_CONFIG", + "org.apache.kafka.common.security.plain.PlainLoginModule required user_" + + BROKER_USER + + "=\"" + + userPassword + + "\";")) { + kafka.start(); + ApplicationContextRunner runner = + platform(kafka, ca, Map.of("orders-producer", userPassword, "orders-consumer", "unused")) + .withPropertyValues( + "app.messaging.security.kafka.producer.type=USERNAME_PASSWORD", + "app.messaging.security.kafka.producer.credential-id=orders-producer", + "app.messaging.security.kafka.consumer.type=USERNAME_PASSWORD", + "app.messaging.security.kafka.consumer.credential-id=orders-consumer"); + + // when + List results = new ArrayList<>(); + runner.run(context -> results.add(publish(context.getBean(MessagePublisher.class)))); + + // then + assertThat(results) + .extracting(PublishResult::completion) + .containsExactly(PublishCompletion.CONFIRMED); + } + } + + @Test + void aPublishIsConfirmedByAPrivateCaBrokerThatRequiresAClientCertificate( + @TempDir Path directory) { + // given + PrivateCa ca = new PrivateCa(directory); + Path clientStore = ca.issue("orders-producer", "dns:orders-producer"); + try (KafkaContainer kafka = + broker(ca, "SSL") + .withEnv("KAFKA_SSL_CLIENT_AUTH", "required") + .withEnv("KAFKA_SSL_TRUSTSTORE_LOCATION", BROKER_TRUST) + .withEnv("KAFKA_SSL_TRUSTSTORE_TYPE", "PKCS12") + .withEnv("KAFKA_SSL_TRUSTSTORE_PASSWORD", ca.password) + .withCopyToContainer(MountableFile.forHostPath(ca.trustStore, 0644), BROKER_TRUST)) { + kafka.start(); + ApplicationContextRunner runner = + platform( + kafka, + ca, + Map.of("orders-producer-tls", ca.password, "orders-consumer-tls", ca.password)) + .withPropertyValues( + "app.messaging.security.kafka.producer.type=MUTUAL_TLS", + "app.messaging.security.kafka.producer.credential-id=orders-producer-tls", + "app.messaging.security.kafka.producer.key-store.location=" + clientStore, + "app.messaging.security.kafka.consumer.type=MUTUAL_TLS", + "app.messaging.security.kafka.consumer.credential-id=orders-consumer-tls", + "app.messaging.security.kafka.consumer.key-store.location=" + clientStore); + + // when + List results = new ArrayList<>(); + runner.run(context -> results.add(publish(context.getBean(MessagePublisher.class)))); + + // then + assertThat(results) + .extracting(PublishResult::completion) + .containsExactly(PublishCompletion.CONFIRMED); + } + } + + /** + * The Testcontainers broker with its external listener switched to a secured protocol. + * + *

The container names its host-facing listener {@code PLAINTEXT} and advertises it by that + * name; only the protocol that name maps to changes here. + */ + @SuppressWarnings("resource") + private static KafkaContainer broker(PrivateCa ca, String protocol) { + Path brokerStore = ca.issue("broker", ca.brokerSubjectAlternativeNames()); + return new KafkaContainer(IMAGE) + .withEnv( + "KAFKA_LISTENER_SECURITY_PROTOCOL_MAP", + "BROKER:PLAINTEXT,PLAINTEXT:" + protocol + ",CONTROLLER:PLAINTEXT") + .withEnv("KAFKA_SSL_KEYSTORE_LOCATION", BROKER_STORE) + .withEnv("KAFKA_SSL_KEYSTORE_TYPE", "PKCS12") + .withEnv("KAFKA_SSL_KEYSTORE_PASSWORD", ca.password) + .withEnv("KAFKA_SSL_KEY_PASSWORD", ca.password) + .withCopyToContainer(MountableFile.forHostPath(brokerStore, 0644), BROKER_STORE); + } + + /** The assembled platform, pointed at the broker through the private CA's trust store. */ + private static ApplicationContextRunner platform( + KafkaContainer kafka, PrivateCa ca, Map secrets) { + Map all = new java.util.HashMap<>(secrets); + all.put("kafka-trust", ca.password); + return new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=" + kafka.getBootstrapServers(), + "app.messaging.brokers.kafka.tls-enabled=true", + "app.messaging.brokers.kafka.authentication-enabled=true", + "app.messaging.security.kafka.trust-store.location=" + ca.trustStore, + "app.messaging.security.kafka.trust-store.password-credential-id=kafka-trust", + "app.messaging.security.kafka.access.publishable[0]=order-events") + .withBean(CredentialProvider.class, () -> provider(all)) + .withBean( + MessageContracts.class, + () -> + new MessageContracts( + Map.of( + new dev.caskeleton.messaging.schema.MessageContractKey( + new MessageType("order.created"), new SchemaVersion(1)), + String.class))) + .withBean( + "ordersProfile", + dev.caskeleton.messaging.policy.DestinationProfile.class, + () -> + MessagingLiveRoundTripQualificationTest.profile( + new DestinationName("order-events"), new DestinationName("order-events-dlq"))) + .withBean( + "ordersDlqProfile", + dev.caskeleton.messaging.policy.DestinationProfile.class, + () -> + MessagingLiveRoundTripQualificationTest.profile( + new DestinationName("order-events-dlq"), null)); + } + + private static PublishResult publish(MessagePublisher publisher) { + return publisher + .publish( + new MessageDestination<>( + new DestinationName("order-events"), + new MessageType("order.created"), + String.class), + MessagingLiveRoundTripQualificationTest.envelope(), + PublishOptions.defaults()) + .toCompletableFuture() + .join(); + } + + private static CredentialProvider provider(Map secrets) { + return new CredentialProvider() { + @Override + public char[] resolve(String credentialId) { + return secrets.get(credentialId).toCharArray(); + } + + @Override + public Optional expiresAt(String credentialId) { + return Optional.empty(); + } + + @Override + public Optional username(String credentialId) { + return Optional.of(BROKER_USER); + } + }; + } + + /** + * A certificate authority made for one test run with the JDK's {@code keytool}. + * + *

Every store shares one generated password, which is also what the credential provider hands + * the client for the trust store and the client key store. + */ + private static final class PrivateCa { + + private final Path directory; + + private final String password = UUID.randomUUID().toString(); + + private final Path authority; + + private final Path certificate; + + private final Path trustStore; + + private PrivateCa(Path directory) { + this.directory = directory; + this.authority = directory.resolve("ca.p12"); + this.certificate = directory.resolve("ca.pem"); + this.trustStore = directory.resolve("truststore.p12"); + keytool( + "-genkeypair", + "-alias", + "ca", + "-keyalg", + "RSA", + "-keysize", + "2048", + "-validity", + "2", + "-dname", + "CN=messaging-test-ca", + "-ext", + "bc:c", + "-keystore", + authority.toString(), + "-storetype", + "PKCS12", + "-storepass", + password); + keytool( + "-exportcert", + "-rfc", + "-alias", + "ca", + "-keystore", + authority.toString(), + "-storepass", + password, + "-file", + certificate.toString()); + keytool( + "-importcert", + "-noprompt", + "-alias", + "ca", + "-file", + certificate.toString(), + "-keystore", + trustStore.toString(), + "-storetype", + "PKCS12", + "-storepass", + password); + } + + /** The names the client may reach the broker by, so hostname verification can pass. */ + private String brokerSubjectAlternativeNames() { + String host = DockerClientFactory.instance().dockerHostIpAddress(); + String dockerHost = + host.chars().allMatch(c -> Character.isDigit(c) || c == '.') + ? "ip:" + host + : "dns:" + host; + return "dns:localhost,ip:127.0.0.1," + dockerHost; + } + + /** Issues a key store holding a CA-signed certificate for {@code name}. */ + private Path issue(String name, String subjectAlternativeNames) { + Path store = directory.resolve(name + ".p12"); + Path request = directory.resolve(name + ".csr"); + Path signed = directory.resolve(name + ".pem"); + keytool( + "-genkeypair", + "-alias", + name, + "-keyalg", + "RSA", + "-keysize", + "2048", + "-validity", + "2", + "-dname", + "CN=" + name, + "-keystore", + store.toString(), + "-storetype", + "PKCS12", + "-storepass", + password); + keytool( + "-certreq", + "-alias", + name, + "-keystore", + store.toString(), + "-storepass", + password, + "-file", + request.toString()); + keytool( + "-gencert", + "-rfc", + "-alias", + "ca", + "-keystore", + authority.toString(), + "-storepass", + password, + "-infile", + request.toString(), + "-outfile", + signed.toString(), + "-validity", + "2", + "-ext", + "SAN=" + subjectAlternativeNames); + keytool( + "-importcert", + "-noprompt", + "-alias", + "ca", + "-file", + certificate.toString(), + "-keystore", + store.toString(), + "-storepass", + password); + keytool( + "-importcert", + "-alias", + name, + "-file", + signed.toString(), + "-keystore", + store.toString(), + "-storepass", + password); + return store; + } + + private static void keytool(String... arguments) { + List command = new ArrayList<>(); + command.add(Path.of(System.getProperty("java.home"), "bin", "keytool").toString()); + command.addAll(List.of(arguments)); + try { + Process process = new ProcessBuilder(command).redirectErrorStream(true).start(); + String output = new String(process.getInputStream().readAllBytes(), StandardCharsets.UTF_8); + if (!process.waitFor(60, TimeUnit.SECONDS) || process.exitValue() != 0) { + throw new IllegalStateException("keytool " + arguments[0] + " failed: " + output); + } + } catch (IOException failure) { + throw new IllegalStateException("keytool could not run", failure); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + throw new IllegalStateException("keytool was interrupted", interrupted); + } + } + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSecuritySettings.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSecuritySettings.java index 0b629b1..ddc56f8 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSecuritySettings.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSecuritySettings.java @@ -8,7 +8,9 @@ import org.springframework.boot.context.properties.bind.DefaultValue; * *

No field here holds a secret, and that is a structural rule rather than a convention: the * platform stores a credential id and resolves the material through a {@code CredentialProvider} at - * connect time, so a configuration dump or a heap dump yields a name and nothing usable. + * connect time, so a configuration dump or a heap dump yields a name and nothing usable. Key and + * trust store passwords follow the same rule — the stores are named by path, their passwords by + * credential id. * *

The admin credential is bindable and its presence on a production broker is a startup failure. * Leaving it unbindable would have been the safer-looking choice and the wrong one: an operator who @@ -19,6 +21,7 @@ import org.springframework.boot.context.properties.bind.DefaultValue; * @param consumer the consumer credential * @param admin the admin credential, which an application runtime must not hold in production * @param hostnameVerification whether the server certificate hostname is verified + * @param trustStore the certificates the connection trusts; absent means the JVM default store * @param access which destinations each role may touch */ public record BrokerSecuritySettings( @@ -26,6 +29,7 @@ public record BrokerSecuritySettings( Credential consumer, Credential admin, @DefaultValue("true") boolean hostnameVerification, + TrustStore trustStore, @DefaultValue Access access) { /** @@ -33,12 +37,35 @@ public record BrokerSecuritySettings( * * @param type the authentication mechanism * @param credentialId the identifier the credential provider resolves + * @param keyStore the client certificate, for {@code MUTUAL_TLS} and nothing else */ - public record Credential(String type, String credentialId) {} + public record Credential(String type, String credentialId, KeyStore keyStore) {} + + /** + * A client certificate store, unlocked by the credential it belongs to. + * + * @param location the key store file + * @param type the key store format + */ + public record KeyStore(String location, @DefaultValue("PKCS12") String type) {} + + /** + * The certificates the connection trusts. + * + * @param location the trust store file + * @param type the trust store format + * @param passwordCredentialId the credential that unlocks the store, when it is locked + */ + public record TrustStore( + String location, @DefaultValue("PKCS12") String type, String passwordCredentialId) {} /** * Which destinations each role may touch. * + *

{@code publishable} is enforced: once a broker has a security entry, the platform publishes + * only to the declared destinations of that broker that this list names. An empty list therefore + * publishes nothing. + * * @param publishable destinations the producer credential may publish to * @param consumable destinations the consumer credential may consume * @param administrable destinations the admin credential may operate on diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java index 5274dd5..1b491be 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/BrokerSettings.java @@ -1,69 +1,36 @@ package dev.caskeleton.messaging.autoconfigure; -import java.time.Duration; -import java.util.List; import org.springframework.boot.context.properties.bind.DefaultValue; /** - * One entry of {@code app.messaging.brokers}. + * The broker-neutral part of one {@code app.messaging.brokers} entry. * - *

Both broker families are described by one record because a deployment names its brokers in one - * map, and a reader of that map should not have to know which half of the fields apply. Which half - * does apply is decided by {@link #type()}, and the compiler refuses a field belonging to the other - * family rather than ignoring it — a Kafka broker carrying {@code prefetch} is a configuration the - * author believed was doing something. + *

This starter is broker-neutral, and its settings are too. The client settings of a family — + * Kafka's bootstrap servers, acks, idempotence — are bound by that family's starter from the same + * entry ({@code KafkaBrokerSettings} in {@code messaging-spring-boot-starter-kafka}). They used to + * be fields here, so the neutral starter carried the union of every broker's client knobs and a + * deployment that shipped no Kafka transport still bound Kafka settings. {@link + * MessagingConfigurationKeyValidator} accepts a family's keys only when that family's settings + * record is on the classpath ({@link MessagingProviderSelection#PROVIDER_BROKER_SETTINGS}). * *

{@code tlsEnabled} and {@code production} live here rather than beside the credentials, so a * broker is described once. They were previously implied by whichever object happened to be * constructed, which is how a profile can be production in one validator's view and not in * another's. * - * @param type the broker family, {@code kafka} or {@code rabbitmq} + * @param type the broker family; {@code kafka} is the only one * @param stable whether this profile claims the Stable guarantees * @param production whether this profile is used in production * @param tlsEnabled whether transport encryption is on * @param authenticationEnabled whether broker authentication is on - * @param bootstrapServers Kafka broker addresses - * @param enableIdempotence Kafka producer duplicate suppression on internal retry - * @param acks the Kafka acknowledgement level - * @param maxInFlightRequestsPerConnection the Kafka in-flight request ceiling - * @param deliveryTimeout the Kafka producer delivery deadline - * @param enableAutoCommit whether the Kafka consumer commits offsets on a timer - * @param consumerGroup the Kafka consumer group - * @param addresses RabbitMQ broker addresses - * @param publisherConfirms whether correlated publisher confirms are on - * @param publisherReturns whether unroutable returns are delivered - * @param mandatory whether publishes carry the mandatory flag - * @param confirmTimeout how long to wait for a RabbitMQ confirm - * @param autoAck whether the RabbitMQ consumer acknowledges on delivery - * @param prefetch the RabbitMQ consumer prefetch window - * @param quorumQueues whether durable work queues are declared as quorum queues */ public record BrokerSettings( String type, @DefaultValue("false") boolean stable, @DefaultValue("false") boolean production, @DefaultValue("false") boolean tlsEnabled, - @DefaultValue("false") boolean authenticationEnabled, - @DefaultValue List bootstrapServers, - @DefaultValue("true") boolean enableIdempotence, - @DefaultValue("all") String acks, - @DefaultValue("5") int maxInFlightRequestsPerConnection, - @DefaultValue("30s") Duration deliveryTimeout, - @DefaultValue("false") boolean enableAutoCommit, - @DefaultValue("") String consumerGroup, - @DefaultValue List addresses, - @DefaultValue("true") boolean publisherConfirms, - @DefaultValue("true") boolean publisherReturns, - @DefaultValue("true") boolean mandatory, - @DefaultValue("5s") Duration confirmTimeout, - @DefaultValue("false") boolean autoAck, - @DefaultValue("16") int prefetch, - @DefaultValue("true") boolean quorumQueues) { + @DefaultValue("false") boolean authenticationEnabled) { /** The broker family a Kafka entry declares. */ public static final String KAFKA = "kafka"; - - /** The broker family a RabbitMQ entry declares. */ - public static final String RABBITMQ = "rabbitmq"; } diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBatchMessagePublisher.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBatchMessagePublisher.java deleted file mode 100644 index c45124d..0000000 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBatchMessagePublisher.java +++ /dev/null @@ -1,262 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.error.MessageTooLargeException; -import dev.caskeleton.messaging.api.publish.BatchMessagePublisher; -import dev.caskeleton.messaging.api.publish.BatchPublishItemResult; -import dev.caskeleton.messaging.api.publish.BatchPublishOptions; -import dev.caskeleton.messaging.api.publish.BatchPublishResult; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishRequest; -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Objects; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.function.Supplier; - -/** - * Fans a batch out over the single-message publisher and keeps every entry's outcome. - * - *

Entries are submitted concurrently and awaited together, because a batch's value is the - * overlap: submitting serially would make a 500-entry batch take 500 round trips, which is slower - * than the loop the caller was trying to avoid. - * - *

A failed entry never fails the batch. Each entry's exception is converted into that entry's - * result, so an oversized message at index 7 does not hide the outcomes of the other 499 — the - * caller needs to know exactly which entries to resubmit, and a thrown exception tells it nothing. - * - *

{@code stopOnFirstRejection} short-circuits the submission, not the awaiting. Entries - * already in flight when the first rejection arrives cannot be recalled, so they are still awaited - * and reported; pretending otherwise would leave the caller unsure whether they landed. - */ -public final class DefaultBatchMessagePublisher implements BatchMessagePublisher, AutoCloseable { - - private final MessagePublisher publisher; - private final Supplier clock; - private final java.util.concurrent.ScheduledExecutorService deadlines; - - /** - * Creates a batch publisher over the single-message publisher. - * - * @param publisher the platform publisher - * @param clock supplies the current instant - */ - public DefaultBatchMessagePublisher(MessagePublisher publisher, Supplier clock) { - this( - publisher, - clock, - java.util.concurrent.Executors.newSingleThreadScheduledExecutor( - runnable -> { - Thread thread = new Thread(runnable, "messaging-batch-deadline"); - thread.setDaemon(true); - return thread; - })); - } - - DefaultBatchMessagePublisher( - MessagePublisher publisher, - Supplier clock, - java.util.concurrent.ScheduledExecutorService deadlines) { - this.publisher = Objects.requireNonNull(publisher, "publisher must not be null"); - this.clock = Objects.requireNonNull(clock, "clock must not be null"); - this.deadlines = Objects.requireNonNull(deadlines, "deadlines must not be null"); - } - - @Override - public CompletionStage publish( - List> requests, BatchPublishOptions options) { - Objects.requireNonNull(requests, "requests must not be null"); - Objects.requireNonNull(options, "options must not be null"); - - if (requests.size() > options.maxBatchSize()) { - throw new MessageTooLargeException( - "BATCH_COUNT_EXCEEDED", - "batch of %d entries exceeds the %d entry limit" - .formatted(requests.size(), options.maxBatchSize())); - } - - Instant startedAt = clock.get(); - List> pending = new ArrayList<>(requests.size()); - // Index and message id per slot, so an entry still outstanding at the deadline can still be - // reported against the input position it came from. - List placeholders = new ArrayList<>(requests.size()); - - for (int index = 0; index < requests.size(); index++) { - placeholders.add(outstanding(index, requests.get(index).message().messageId())); - if (options.stopOnFirstRejection() && anyRejected(pending)) { - // Every remaining index still gets a result. Breaking out shortened the result list, so a - // caller could not tell "not submitted" from "result lost" — and those need opposite - // actions. - pending.add( - CompletableFuture.completedFuture( - notSubmitted(index, requests.get(index).message().messageId(), startedAt))); - continue; - } - CompletableFuture submitted = submit(requests.get(index), index); - pending.add(submitted); - if (options.stopOnFirstRejection()) { - // Sequential under stop-on-first-rejection: waiting for this entry's outcome is the only - // way the flag can mean anything. The previous loop only saw a rejection that had already - // resolved, which for an asynchronous broker is never — so every remaining entry was - // submitted anyway and the option did nothing. - awaitQuietly(submitted, remaining(startedAt, options.timeout())); - } - } - - CompletableFuture completed = new CompletableFuture<>(); - CompletableFuture all = - CompletableFuture.allOf(pending.toArray(CompletableFuture[]::new)); - // The returned stage is deliberately dropped: `completed` is what the caller holds, and this - // callback's only job is to fill it in. Nothing downstream observes the intermediate stage. - CompletableFuture ignoredCompletion = - all.whenComplete( - (ignored, failure) -> - completed.complete( - new BatchPublishResult( - resolveAll(pending, placeholders), - Duration.between(startedAt, clock.get())))); - assert ignoredCompletion != null; - - // The whole-batch deadline the options document. It was never read, so a publisher that never - // completed left the caller's stage pending forever — a timeout that exists only in the - // javadoc. - // Same for the scheduled future: cancelling it early would only save a no-op, and the task - // checks `isDone` before completing anything. - java.util.concurrent.ScheduledFuture ignoredDeadline = - deadlines.schedule( - () -> { - if (!completed.isDone()) { - completed.complete( - new BatchPublishResult( - resolveAll(pending, placeholders), - Duration.between(startedAt, clock.get()))); - } - }, - options.timeout().toMillis(), - java.util.concurrent.TimeUnit.MILLISECONDS); - assert ignoredDeadline != null; - return completed; - } - - @Override - public void close() { - deadlines.shutdownNow(); - } - - private boolean anyRejected(List> pending) { - return pending.stream() - .anyMatch( - future -> - future.isDone() - && !future.isCompletedExceptionally() - && future.join().result().completion() == PublishCompletion.REJECTED); - } - - private void awaitQuietly(CompletableFuture future, Duration budget) { - if (budget.isNegative() || budget.isZero()) { - return; - } - try { - future.get(budget.toMillis(), java.util.concurrent.TimeUnit.MILLISECONDS); - } catch (InterruptedException interrupted) { - Thread.currentThread().interrupt(); - } catch (java.util.concurrent.ExecutionException - | java.util.concurrent.TimeoutException other) { - // Both are outcomes the result already carries; nothing to add here. - } - } - - private Duration remaining(Instant startedAt, Duration timeout) { - return timeout.minus(Duration.between(startedAt, clock.get())); - } - - /** - * An entry the batch never submitted because an earlier one was rejected. - * - *

Distinct from a failure: nothing was sent, so the caller may submit it unchanged. - */ - private BatchPublishItemResult notSubmitted( - int index, dev.caskeleton.messaging.api.MessageId messageId, Instant startedAt) { - return new BatchPublishItemResult( - index, - messageId, - new dev.caskeleton.messaging.api.publish.PublishResult( - PublishCompletion.REJECTED, - dev.caskeleton.messaging.api.publish.PublishEvidence.notTransmitted(), - dev.caskeleton.messaging.api.publish.RoutingOutcome.UNKNOWN, - java.util.Optional.empty(), - 1, - Duration.between(startedAt, clock.get()), - java.util.Optional.of( - dev.caskeleton.messaging.api.error.FailureDescriptor.of( - dev.caskeleton.messaging.api.error.FailureCategory.CONFIGURATION, - "NOT_SUBMITTED_AFTER_REJECTION", - "an earlier entry was rejected and this batch stops on the first rejection")))); - } - - private List resolveAll( - List> pending, - List placeholders) { - List resolved = new ArrayList<>(pending.size()); - for (int index = 0; index < pending.size(); index++) { - CompletableFuture future = pending.get(index); - resolved.add( - future.isDone() && !future.isCompletedExceptionally() - ? future.join() - : placeholders.get(index)); - } - return List.copyOf(resolved); - } - - /** An entry still outstanding when the deadline arrived is ambiguous, never a success. */ - private BatchPublishItemResult outstanding( - int index, dev.caskeleton.messaging.api.MessageId messageId) { - return new BatchPublishItemResult( - index, - messageId, - new dev.caskeleton.messaging.api.publish.PublishResult( - PublishCompletion.AMBIGUOUS, - dev.caskeleton.messaging.api.publish.PublishEvidence.ambiguous(), - dev.caskeleton.messaging.api.publish.RoutingOutcome.UNKNOWN, - java.util.Optional.empty(), - 1, - Duration.ZERO, - java.util.Optional.of( - dev.caskeleton.messaging.api.error.FailureDescriptor.of( - dev.caskeleton.messaging.api.error.FailureCategory.AMBIGUOUS, - "BATCH_DEADLINE_EXCEEDED", - "the batch deadline elapsed before this entry produced evidence")))); - } - - private CompletableFuture submit( - PublishRequest request, int index) { - @SuppressWarnings("unchecked") - PublishRequest typed = (PublishRequest) request; - MessageDestination destination = typed.destination(); - MessageEnvelope message = typed.message(); - - try { - return publisher - .publish(destination, message, typed.options()) - .toCompletableFuture() - .handle( - (result, failure) -> - new BatchPublishItemResult( - index, - message.messageId(), - failure == null ? result : PublishResults.from(failure))) - .toCompletableFuture(); - } catch (RuntimeException synchronousFailure) { - // A publisher that validates eagerly throws instead of returning a failed stage. Converting - // it here keeps the "one result per index" contract that the caller resubmits from. - return CompletableFuture.completedFuture( - new BatchPublishItemResult( - index, message.messageId(), PublishResults.from(synchronousFailure))); - } - } -} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBlockingMessagePublisher.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBlockingMessagePublisher.java deleted file mode 100644 index 9931b91..0000000 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultBlockingMessagePublisher.java +++ /dev/null @@ -1,65 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.error.MessagePublishTimeoutException; -import dev.caskeleton.messaging.api.publish.BlockingMessagePublisher; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import java.util.Objects; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.TimeoutException; - -/** - * Converts the asynchronous publish lifecycle into a blocking call. - * - *

Three details make this safe rather than a thread-pool trap. The wait is bounded by the call's - * own deadline. A timeout is reported as {@link MessagePublishTimeoutException}, which the error - * model classifies as ambiguous — the broker may still hold the message, and the caller must not - * treat it as a failed send. And an interrupt restores the thread's interrupt flag instead of - * swallowing it, so a shutdown that interrupts a blocked publisher still unwinds. - */ -public final class DefaultBlockingMessagePublisher implements BlockingMessagePublisher { - - private final MessagePublisher delegate; - - /** - * Creates a blocking facade. - * - * @param delegate the asynchronous publisher - */ - public DefaultBlockingMessagePublisher(MessagePublisher delegate) { - this.delegate = Objects.requireNonNull(delegate, "delegate must not be null"); - } - - @Override - public PublishResult publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options) { - Objects.requireNonNull(options, "options must not be null"); - CompletableFuture future = - delegate.publish(destination, message, options).toCompletableFuture(); - try { - return future.get(options.timeout().toMillis(), TimeUnit.MILLISECONDS); - } catch (TimeoutException exception) { - future.cancel(true); - throw new MessagePublishTimeoutException( - "PUBLISH_TIMEOUT", - "no publish outcome within " + options.timeout() + "; the broker may hold the message", - exception); - } catch (InterruptedException exception) { - Thread.currentThread().interrupt(); - future.cancel(true); - throw new MessagePublishTimeoutException( - "PUBLISH_INTERRUPTED", "the publishing thread was interrupted", exception); - } catch (ExecutionException exception) { - Throwable cause = exception.getCause(); - if (cause instanceof RuntimeException runtime) { - throw runtime; - } - throw new IllegalStateException("publish failed", cause); - } - } -} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java deleted file mode 100644 index a17c0e3..0000000 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DefaultReactiveMessagePublisher.java +++ /dev/null @@ -1,39 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import java.util.Objects; -import reactor.core.publisher.Mono; - -/** - * Converts the asynchronous publish lifecycle into a cold, cancellable {@code Mono}. - * - *

{@code Mono.fromCompletionStage} with a supplier is used deliberately: the eager overload - * would start the publish when the {@code Mono} is assembled rather than when it is subscribed, - * which makes a request never sent by an unsubscribed pipeline indistinguishable from one that was. - */ -public final class DefaultReactiveMessagePublisher implements ReactiveMessagePublisher { - - private final MessagePublisher delegate; - - /** - * Creates a Reactor facade. - * - * @param delegate the asynchronous publisher - */ - public DefaultReactiveMessagePublisher(MessagePublisher delegate) { - this.delegate = Objects.requireNonNull(delegate, "delegate must not be null"); - } - - @Override - public Mono publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(message, "message must not be null"); - Objects.requireNonNull(options, "options must not be null"); - return Mono.fromCompletionStage(() -> delegate.publish(destination, message, options)); - } -} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DestinationSettings.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DestinationSettings.java index 35304c7..8184d6a 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DestinationSettings.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/DestinationSettings.java @@ -5,7 +5,7 @@ import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; import dev.caskeleton.messaging.api.delivery.OrderingScope; import dev.caskeleton.messaging.api.destination.ConfirmationRequirement; import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.OrderingImpact; import dev.caskeleton.messaging.policy.RetryMode; import dev.caskeleton.messaging.schema.SchemaCompatibility; @@ -48,7 +48,7 @@ import org.springframework.boot.context.properties.bind.DefaultValue; public record DestinationSettings( String broker, DestinationKind kind, - @DefaultValue("M1") CapabilityTier tier, + @DefaultValue("M1") ApiTier tier, @DefaultValue Physical physical, @DefaultValue Schema schema, @DefaultValue Guarantees guarantees, diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessageConsumers.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessageConsumers.java new file mode 100644 index 0000000..332baa5 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessageConsumers.java @@ -0,0 +1,46 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.transport.TransportDelivery; +import java.util.List; +import java.util.Objects; +import java.util.concurrent.CompletionStage; +import java.util.function.Function; + +/** + * The destinations this application consumes, each with the broker-neutral pipeline its deliveries + * go through. + * + *

Built by {@link MessagingConsumeConfiguration} from the {@code MessageHandler} beans; a broker + * starter opens one consumer per entry and hands every delivery to its {@link Consumer#sink()}. The + * split keeps decoding, inbox, dead-lettering and settlement rules out of each broker starter. + * + * @param consumers one entry per consumed destination + */ +public record MessageConsumers(List consumers) { + + public MessageConsumers { + consumers = List.copyOf(Objects.requireNonNull(consumers, "consumers must not be null")); + } + + /** + * One consumed destination. + * + * @param profile the destination profile + * @param group the consumer group, which is also the inbox consumer identity + * @param sink the pipeline each still-encoded delivery goes through + */ + public record Consumer( + DestinationProfile profile, + String group, + Function> sink) { + + public Consumer { + Objects.requireNonNull(profile, "profile must not be null"); + Objects.requireNonNull(sink, "sink must not be null"); + if (group == null || group.isBlank()) { + throw new IllegalArgumentException("group must not be blank"); + } + } + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java index dca2d39..2a24d32 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationCompiler.java @@ -14,6 +14,7 @@ import dev.caskeleton.messaging.policy.RetryPolicy; import dev.caskeleton.messaging.policy.SchemaPolicy; import dev.caskeleton.messaging.security.BrokerCredentialProfile; import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import dev.caskeleton.messaging.security.BrokerTrustStore; import dev.caskeleton.messaging.security.DestinationAccessPolicy; import java.util.ArrayList; import java.util.LinkedHashSet; @@ -92,28 +93,13 @@ public final class MessagingConfigurationCompiler { } private static void validateBrokerSettings(String name, BrokerSettings broker) { - switch (familyOf(name, broker)) { - case BrokerSettings.KAFKA -> { - if (broker.bootstrapServers().isEmpty()) { - throw configurationError( - key("brokers", name, "bootstrap-servers"), - "a Kafka broker needs at least one address; a client with none defaults to " - + "localhost:9092 and fails at the first publish rather than at startup"); - } - } - case BrokerSettings.RABBITMQ -> { - if (broker.addresses().isEmpty()) { - throw configurationError( - key("brokers", name, "addresses"), - "a RabbitMQ broker needs at least one address; a client with none defaults to " - + "localhost and connects as guest"); - } - } - default -> - throw configurationError( - key("brokers", name, "type"), - "'%s' is not a broker family this platform binds; use '%s' or '%s'" - .formatted(broker.type(), BrokerSettings.KAFKA, BrokerSettings.RABBITMQ)); + // Only the family is checked here. Its client settings — addresses, acks — belong to the + // family's starter, which binds and validates them when it builds the client. + if (!MessagingProviderSelection.registeredBrokerIds().contains(familyOf(name, broker))) { + throw configurationError( + key("brokers", name, "type"), + "'%s' is not a broker family this platform has a transport for; use one of %s" + .formatted(broker.type(), MessagingProviderSelection.registeredBrokerIds())); } } @@ -243,7 +229,22 @@ public final class MessagingConfigurationCompiler { new DestinationAccessPolicy( destinationNames(security.access().publishable()), destinationNames(security.access().consumable()), - destinationNames(security.access().administrable()))); + destinationNames(security.access().administrable())), + Optional.ofNullable(security.trustStore()).map(store -> trustStore(name, store))); + } + + private static BrokerTrustStore trustStore( + String broker, BrokerSecuritySettings.TrustStore store) { + if (blankToNull(store.location()) == null) { + throw configurationError( + key("security", broker, "trust-store.location"), + "a trust store entry needs the file it names; without one the JVM default store is used," + + " which is what leaving the entry out already says"); + } + return new BrokerTrustStore( + store.location(), + store.type(), + Optional.ofNullable(blankToNull(store.passwordCredentialId()))); } private static BrokerCredentialProfile credential( @@ -260,10 +261,25 @@ public final class MessagingConfigurationCompiler { } String type = credential.type() == null ? "" : credential.type().toUpperCase(Locale.ROOT); String id = credential.credentialId(); + BrokerSecuritySettings.KeyStore keyStore = credential.keyStore(); + boolean hasKeyStore = keyStore != null && blankToNull(keyStore.location()) != null; + if ("MUTUAL_TLS".equals(type) && !hasKeyStore) { + throw configurationError( + key("security", broker, role + ".key-store"), + "MUTUAL_TLS presents a client certificate, and without a key store there is none to" + + " present; the broker refuses the handshake"); + } + if (!"MUTUAL_TLS".equals(type) && keyStore != null) { + throw configurationError( + key("security", broker, role + ".key-store"), + "only a MUTUAL_TLS credential presents a key store; a %s credential would ignore it" + .formatted(credential.type())); + } return switch (type) { case "SASL_SCRAM" -> new BrokerCredentialProfile.SaslScram(id); case "OAUTH2" -> new BrokerCredentialProfile.OAuth2(id); - case "MUTUAL_TLS" -> new BrokerCredentialProfile.MutualTls(id); + case "MUTUAL_TLS" -> + new BrokerCredentialProfile.MutualTls(id, keyStore.location(), keyStore.type()); case "USERNAME_PASSWORD" -> new BrokerCredentialProfile.UsernamePassword(id); case "NKEY" -> new BrokerCredentialProfile.Nkey(id); default -> diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java index 84e9baa..56c5b67 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationKeyValidator.java @@ -13,6 +13,7 @@ import org.springframework.beans.factory.InitializingBean; import org.springframework.core.env.ConfigurableEnvironment; import org.springframework.core.env.EnumerablePropertySource; import org.springframework.core.env.PropertySource; +import org.springframework.util.ClassUtils; /** * Refuses a key inside the platform's own configuration sections that nothing binds. @@ -38,29 +39,6 @@ import org.springframework.core.env.PropertySource; */ public final class MessagingConfigurationKeyValidator implements InitializingBean { - /** Broker keys that only a Kafka entry may carry. */ - private static final Set KAFKA_ONLY_KEYS = - Set.of( - "bootstrap-servers", - "enable-idempotence", - "acks", - "max-in-flight-requests-per-connection", - "delivery-timeout", - "enable-auto-commit", - "consumer-group"); - - /** Broker keys that only a RabbitMQ entry may carry. */ - private static final Set RABBIT_ONLY_KEYS = - Set.of( - "addresses", - "publisher-confirms", - "publisher-returns", - "mandatory", - "confirm-timeout", - "auto-ack", - "prefetch", - "quorum-queues"); - private final ConfigurableEnvironment environment; /** @@ -92,18 +70,18 @@ public final class MessagingConfigurationKeyValidator implements InitializingBea List rejected = new ArrayList<>(); sections.forEach( (section, shape) -> { - Set legal = leafPaths(shape); + Set common = leafPaths(shape); for (String key : keysUnder(MessagingSettings.PREFIX + '.' + section + '.')) { Entry entry = split(key, MessagingSettings.PREFIX + '.' + section + '.'); if (entry.leaf().isEmpty()) { continue; } + Set legal = + "brokers".equals(section) ? brokerLeafPaths(entry.name(), common) : common; if (!legal.contains(entry.leaf())) { rejected.add( "%s binds nothing; %s entries accept %s" .formatted(key, section, legal.stream().sorted().toList())); - } else if ("brokers".equals(section)) { - wrongFamily(entry).ifPresent(rejected::add); } } }); @@ -126,29 +104,30 @@ public final class MessagingConfigurationKeyValidator implements InitializingBea } /** - * Reports a broker key that belongs to the other broker family. + * The keys a broker entry accepts: the neutral ones, plus its family's own when the family's + * starter is on the classpath. * - *

Both families are described by one record, so a Kafka entry carrying {@code prefetch} binds - * without complaint and does nothing. The author of that line believed it was doing something, - * which is the same silence this class exists to end. + *

A family whose settings record is absent accepts only the neutral keys. Its client keys bind + * nothing in that deployment — there is no transport to hand them to — and {@link + * MessagingProviderSelection} refuses a selected family whose starter is missing anyway. */ - private java.util.Optional wrongFamily(Entry entry) { - String declared = - environment.getProperty( - MessagingSettings.PREFIX + ".brokers." + entry.name() + ".type", ""); - String family = declared.toLowerCase(Locale.ROOT); - String leaf = entry.leaf(); - if (BrokerSettings.KAFKA.equals(family) && RABBIT_ONLY_KEYS.contains(leaf)) { - return java.util.Optional.of( - "%s.brokers.%s.%s is a RabbitMQ setting on a Kafka broker" - .formatted(MessagingSettings.PREFIX, entry.name(), leaf)); + private Set brokerLeafPaths(String brokerName, Set common) { + String type = + environment.getProperty(MessagingSettings.PREFIX + ".brokers." + brokerName + ".type", ""); + String shapeName = + MessagingProviderSelection.PROVIDER_BROKER_SETTINGS.get( + type.trim().toLowerCase(Locale.ROOT)); + if (shapeName == null + || !ClassUtils.isPresent( + shapeName, MessagingConfigurationKeyValidator.class.getClassLoader())) { + return common; } - if (BrokerSettings.RABBITMQ.equals(family) && KAFKA_ONLY_KEYS.contains(leaf)) { - return java.util.Optional.of( - "%s.brokers.%s.%s is a Kafka setting on a RabbitMQ broker" - .formatted(MessagingSettings.PREFIX, entry.name(), leaf)); - } - return java.util.Optional.empty(); + Set legal = new LinkedHashSet<>(common); + legal.addAll( + leafPaths( + ClassUtils.resolveClassName( + shapeName, MessagingConfigurationKeyValidator.class.getClassLoader()))); + return legal; } /** One configuration key split into the map entry it configures and the leaf inside it. */ diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfiguration.java new file mode 100644 index 0000000..0dc8879 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfiguration.java @@ -0,0 +1,151 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.error.MessagingConfigurationException; +import dev.caskeleton.messaging.inbox.IdempotentConsumer; +import dev.caskeleton.messaging.inbox.InboxMessageHandler; +import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.runtime.DefaultDeliveryProcessor; +import dev.caskeleton.messaging.schema.MessageCodecRegistry; +import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import dev.caskeleton.messaging.security.DestinationAccessPolicy; +import java.time.Instant; +import java.time.InstantSource; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.CompletableFuture; +import java.util.stream.Stream; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * The broker-neutral consume pipeline, assembled from the application's {@link MessageHandler} + * beans (owner decision 10, S7-#2). + * + *

On when present: no handler bean, no consumer. {@code app.messaging.enabled=false} turns the + * whole platform, consumers included, off. Every contradiction a handler can declare fails startup + * here, before any broker connection opens: + * + *

    + *
  • a handler for a destination nobody declared; + *
  • a consumed destination without {@code consumer.group} — the group is also the inbox + * identity, and inventing one would split or merge consumers silently; + *
  • a destination the broker's security profile does not list as consumable; + *
  • {@code INBOX_TRANSACTIONAL} without an {@link IdempotentConsumer} bean to run it. + *
+ */ +@Configuration(proxyBeanMethods = false) +public class MessagingConsumeConfiguration { + + /** + * One consume pipeline per destination at least one handler names. + * + * @param handlers the application's handlers + * @param destinations the validated destination registry + * @param codecs the codec registry + * @param deadLetters publishes to a destination's dead-letter destination + * @param inbox runs a handler inside the inbox reservation's transaction, when supplied + * @param compiled the security profiles compiled from configuration + * @param declaredSecurity the security profiles the application declared as beans + * @return the consumers + */ + @Bean + @ConditionalOnBean(MessageHandler.class) + @ConditionalOnMissingBean + public MessageConsumers messageConsumers( + ObjectProvider> handlers, + ValidatedDestinationRegistry destinations, + MessageCodecRegistry codecs, + DeadLetterOrchestrator deadLetters, + ObjectProvider inbox, + CompiledMessagingConfiguration compiled, + ObjectProvider declaredSecurity) { + Map>> byDestination = new LinkedHashMap<>(); + handlers + .orderedStream() + .forEach( + handler -> + byDestination + .computeIfAbsent(handler.destination().name(), name -> new ArrayList<>()) + .add(handler)); + + Map access = new LinkedHashMap<>(); + Stream.concat(compiled.security().stream(), declaredSecurity.orderedStream()) + .forEach(profile -> access.putIfAbsent(profile.broker(), profile.access())); + + List consumers = new ArrayList<>(); + byDestination.forEach( + (name, forDestination) -> { + DestinationProfile profile = destinations.require(name); + String group = + profile + .consumer() + .group() + .filter(value -> !value.isBlank()) + .orElseThrow( + () -> + refused( + name, + "is consumed by a handler but declares no consumer.group; the group" + + " is also the inbox identity, so the platform does not invent" + + " one")); + DestinationAccessPolicy acl = access.get(profile.broker()); + if (acl != null && !acl.mayConsume(name)) { + throw refused( + name, + "is not in app.messaging.security." + profile.broker() + ".access.consumable"); + } + List> pipeline = + profile.externalSideEffectGuarantee() + == ExternalSideEffectGuarantee.INBOX_TRANSACTIONAL + ? forDestination.stream() + .>map( + handler -> inboxed(handler, inbox.getIfAvailable(), group, name)) + .toList() + : forDestination; + DefaultDeliveryProcessor processor = + new DefaultDeliveryProcessor( + profile, + pipeline, + codecs, + (delivery, failure) -> + // The processor acknowledges the source itself once the publish confirms, + // so the orchestrator's settlement callback has nothing left to do. + deadLetters + .deadLetter( + profile, + delivery, + failure, + () -> CompletableFuture.completedFuture(null)) + .thenApply(result -> result.publishResult().completion()), + group, + InstantSource.system()); + consumers.add(new MessageConsumers.Consumer(profile, group, processor::process)); + }); + return new MessageConsumers(consumers); + } + + private static MessageHandler inboxed( + MessageHandler handler, IdempotentConsumer inbox, String group, DestinationName name) { + if (inbox == null) { + throw refused( + name, + "declares INBOX_TRANSACTIONAL but no IdempotentConsumer bean exists; without one the" + + " handler's side effect is not protected against redelivery"); + } + return new InboxMessageHandler<>(handler, inbox, group, Instant::now); + } + + private static MessagingConfigurationException refused(DestinationName name, String reason) { + return new MessagingConfigurationException( + "MESSAGING_CONSUMER_INVALID", "destination '" + name.value() + "' " + reason); + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java index 81b0602..c7fa030 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingCoreAutoConfiguration.java @@ -1,550 +1,26 @@ package dev.caskeleton.messaging.autoconfigure; -import dev.caskeleton.messaging.api.publish.BatchMessagePublisher; -import dev.caskeleton.messaging.api.publish.BlockingMessagePublisher; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.observation.CardinalityGuard; -import dev.caskeleton.messaging.observation.MessagingRedactor; -import dev.caskeleton.messaging.policy.BackoffCalculator; -import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; -import dev.caskeleton.messaging.policy.DefaultRetryDecisionEngine; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.DestinationProfileValidator; -import dev.caskeleton.messaging.policy.RetryDecisionEngine; -import dev.caskeleton.messaging.security.BrokerTlsPolicy; -import dev.caskeleton.messaging.security.CredentialProvider; -import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; -import dev.caskeleton.messaging.security.MessageSecurityValidator; -import dev.caskeleton.messaging.transport.DefaultMessagingRuntimeRegistry; -import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; -import dev.caskeleton.messaging.transport.MessagingRuntimeRegistry; -import java.time.Instant; -import java.util.List; -import org.springframework.beans.factory.ObjectProvider; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; /** - * Wires the platform's policy, transport, and observability primitives. + * Wires the platform's configuration, publish pipeline and observability. * - *

Destination profiles are validated at context refresh through a bean that fails to construct - * when the registry is contradictory. Startup failure is the point: a profile that promises ordered - * delivery while configuring a reordering retry does not fail on the happy path, it fails months - * later on the first retry, looking like a data bug rather than a configuration one. + *

It was one 550-line class with 28 beans (S7-#9a). It is now the entry point that imports the + * three parts, plus the consume pipeline (S7-#2), so an adopter or a test that names this class + * still gets the whole core. * - *

The M2 and M3 surfaces are deliberately not auto-configured. Manual settlement, native - * capabilities, and the Admin Plane are opt-in beans with their own credentials, so a service - * cannot acquire them by adding the starter to its classpath. + * @see MessagingValidationConfiguration + * @see MessagingPublishConfiguration + * @see MessagingObservabilityConfiguration + * @see MessagingConsumeConfiguration */ @Configuration(proxyBeanMethods = false) @EnableConfigurationProperties(MessagingSettings.class) -public class MessagingCoreAutoConfiguration { - - /** - * Fails the context when a production broker profile has no credential source. - * - *

One actionable error instead of a missing-bean message naming an internal registry type. - */ - @Bean - public MessagingCredentialRequirementValidator messagingCredentialRequirementValidator( - org.springframework.beans.factory.ObjectProvider< - dev.caskeleton.messaging.security.BrokerSecurityProfile> - profiles, - CompiledMessagingConfiguration compiled, - org.springframework.beans.factory.ObjectProvider< - dev.caskeleton.messaging.security.CredentialProvider> - credentials, - org.springframework.core.env.Environment environment) { - return new MessagingCredentialRequirementValidator( - profiles, compiled.security(), credentials, environment); - } - - /** - * Fails the context when configuration uses a prefix nothing binds. - * - *

Registered here rather than left to review because the failure it catches is silent: a key - * under {@code backend.messaging} or a bare {@code messaging} looks configured and does nothing. - */ - @Bean - public MessagingPrefixMigrationValidator messagingPrefixMigrationValidator( - org.springframework.core.env.ConfigurableEnvironment environment) { - return new MessagingPrefixMigrationValidator(environment); - } - - /** - * Compiles {@code app.messaging.destinations}, {@code .brokers} and {@code .security} into the - * platform's own profile objects. - * - *

Separate from the registry bean so that a configuration error fails while compiling, with - * the offending key in the message, rather than later as a validation failure over an object - * whose origin the operator then has to work out. - * - * @param settings the bound configuration - * @return the compiled profiles - */ - @Bean - @ConditionalOnMissingBean - public CompiledMessagingConfiguration compiledMessagingConfiguration(MessagingSettings settings) { - return new MessagingConfigurationCompiler(settings).compile(); - } - - /** - * Refuses a key under the platform's own sections that nothing binds. - * - *

Registered here rather than left to review because a misspelt key inside a bound section is - * silent in a way a misspelt prefix is not: the section binds, the entry appears, and the one - * setting the operator came to change is the one that did not take. - * - * @param environment the resolved environment - * @return the key validator - */ - @Bean - @ConditionalOnMissingBean - public MessagingConfigurationKeyValidator messagingConfigurationKeyValidator( - org.springframework.core.env.ConfigurableEnvironment environment) { - return new MessagingConfigurationKeyValidator(environment); - } - - /** - * Validates every registered destination profile at startup. - * - *

Returns a marker rather than a service because its only job is to fail the context when the - * registry is inconsistent. - * - *

Configured profiles and bean-declared ones are merged before validation, not after, because - * the rules that matter most across a registry — a dead letter destination that exists, a retry - * graph with no cycle — are questions about the whole set. Validating the two halves separately - * would pass a deployment whose configured destination dead-letters into a bean-declared one that - * points back. - * - * @param profiles the profiles the application declared as beans - * @param compiled the profiles compiled from configuration - * @return the validated registry - */ - @Bean - @ConditionalOnMissingBean - public ValidatedDestinationRegistry validatedDestinationRegistry( - ObjectProvider profiles, CompiledMessagingConfiguration compiled) { - List registered = - java.util.stream.Stream.concat(compiled.destinations().stream(), profiles.orderedStream()) - .toList(); - new DestinationProfileValidator().validateAll(registered); - return new ValidatedDestinationRegistry(registered); - } - - /** - * Returns the destination profile validator. - * - * @return the validator - */ - @Bean - @ConditionalOnMissingBean - public DestinationProfileValidator destinationProfileValidator() { - return new DestinationProfileValidator(); - } - - /** - * Returns the broker security validator. - * - * @return the validator - */ - @Bean - @ConditionalOnMissingBean - public MessageSecurityValidator messageSecurityValidator() { - return new MessageSecurityValidator(); - } - - /** - * Returns the retry decision engine. - * - * @return the engine - */ - @Bean - @ConditionalOnMissingBean - public RetryDecisionEngine retryDecisionEngine() { - return new DefaultRetryDecisionEngine(new BackoffCalculator()); - } - - /** - * Returns the dead letter orchestrator. - * - * @param publisher the platform publisher - * @return the orchestrator - */ - @Bean - @ConditionalOnMissingBean - public DeadLetterOrchestrator deadLetterOrchestrator(MessagePublisher publisher) { - return new DeadLetterOrchestrator(publisher); - } - - /** - * Returns the runtime registry used for credential and topology rotation. - * - * @param properties the bound configuration - * @return the registry - */ - @Bean - @ConditionalOnMissingBean - public MessagingRuntimeRegistry messagingRuntimeRegistry(MessagingSettings properties) { - return new DefaultMessagingRuntimeRegistry(properties.getShutdown().getDrainDeadline()); - } - - /** - * Returns the graceful shutdown coordinator. - * - * @param properties the bound configuration - * @return the coordinator - */ - @Bean - @ConditionalOnMissingBean - public GracefulShutdownCoordinator gracefulShutdownCoordinator(MessagingSettings properties) { - return new GracefulShutdownCoordinator(properties.getShutdown().getDrainDeadline()); - } - - /** - * Runs the security validator over every broker security profile, declared or configured. - * - * @param profiles the security profiles the application declared as beans - * @param compiled the security profiles compiled from configuration - * @param validator the security validator - * @return the startup validation - */ - @Bean - public StartupProfileValidation - brokerSecurityStartupValidation( - ObjectProvider profiles, - CompiledMessagingConfiguration compiled, - MessageSecurityValidator validator) { - return new StartupProfileValidation<>( - () -> - java.util.stream.Stream.concat(compiled.security().stream(), profiles.orderedStream()) - .toList(), - validator::validate); - } - - /** - * Drives the shutdown phases the admission gate and the drain coordinator only implemented. - * - * @param admission the publish admission gate - * @param drain the in-flight work coordinator - * @param properties the bound configuration supplying the drain deadline - * @return the lifecycle - */ - @Bean - @ConditionalOnMissingBean - public MessagingShutdownLifecycle messagingShutdownLifecycle( - dev.caskeleton.messaging.policy.MessagingAdmissionController admission, - GracefulShutdownCoordinator drain, - MessagingSettings properties) { - return new MessagingShutdownLifecycle( - admission, drain, properties.getShutdown().getDrainDeadline()); - } - - /** - * Returns the diagnostic redactor. - * - * @return the redactor - */ - @Bean - @ConditionalOnMissingBean - public MessagingRedactor messagingRedactor() { - return new MessagingRedactor(); - } - - /** - * Returns the metric cardinality guard. - * - * @return the guard - */ - @Bean - @ConditionalOnMissingBean - public CardinalityGuard cardinalityGuard() { - return new CardinalityGuard(); - } - - /** - * Binds the platform observation seam to Micrometer when a registry is available. - * - *

Narrow tests and non-observed adopters may omit Micrometer entirely. A normal Boot runtime - * with Actuator contributes a registry, in which case publish outcomes must not fall back to the - * no-op observation constructor. - * - * @param registry the Micrometer registry - * @param guard the bounded-cardinality guard - * @param redactor the diagnostics redactor - * @return the messaging metrics binding - */ - @Bean - @ConditionalOnBean(io.micrometer.core.instrument.MeterRegistry.class) - @ConditionalOnMissingBean(dev.caskeleton.messaging.observation.MessagingObservation.class) - public dev.caskeleton.messaging.observation.MessagingObservation messagingObservation( - io.micrometer.core.instrument.MeterRegistry registry, - CardinalityGuard guard, - MessagingRedactor redactor) { - return new dev.caskeleton.messaging.observation.MessagingMetrics(registry, guard, redactor); - } - - /** - * Returns the blocking publish facade. - * - * @param publisher the asynchronous publisher - * @return the blocking facade - */ - @Bean - @ConditionalOnMissingBean - public BlockingMessagePublisher blockingMessagePublisher(MessagePublisher publisher) { - return new DefaultBlockingMessagePublisher(publisher); - } - - /** - * Returns the Reactor publish facade. - * - * @param publisher the asynchronous publisher - * @return the Reactor facade - */ - @Bean - @ConditionalOnMissingBean - public ReactiveMessagePublisher reactiveMessagePublisher(MessagePublisher publisher) { - return new DefaultReactiveMessagePublisher(publisher); - } - - /** - * Returns the M2 batch publish facade. - * - * @param publisher the asynchronous publisher - * @return the batch publisher - */ - @Bean - @ConditionalOnMissingBean - public BatchMessagePublisher batchMessagePublisher(MessagePublisher publisher) { - return new DefaultBatchMessagePublisher(publisher, Instant::now); - } - - /** - * Returns the credential runtime registry, when the application supplied a credential provider. - * - *

Conditional on the provider because there is no safe default: a registry with no source - * would resolve nothing, and the adapters would connect with whatever their client's defaults - * are. - * - * @param provider the application's credential source - * @return the credential runtime registry - */ - @Bean - @ConditionalOnBean(CredentialProvider.class) - @ConditionalOnMissingBean - public CredentialRuntimeRegistry credentialRuntimeRegistry(CredentialProvider provider) { - return new CredentialRuntimeRegistry(provider); - } - - /** - * Returns the transport security policy. - * - * @return the TLS policy - */ - @Bean - @ConditionalOnMissingBean - public BrokerTlsPolicy brokerTlsPolicy() { - return new BrokerTlsPolicy(); - } - - /** - * Returns the read-only actuator endpoint. - * - * @param registry the validated destination registry - * @param admission the gate every publish passes through - * @return the endpoint - */ - @Bean - @ConditionalOnMissingBean - public MessagingEndpoint messagingEndpoint( - ValidatedDestinationRegistry registry, - dev.caskeleton.messaging.policy.MessagingAdmissionController admission) { - return new MessagingEndpoint(registry, admission); - } - - /** - * The codec registry, which had no implementation anywhere (MSG-INT-003). - * - *

The message contracts come from the deployment — a {@code Map} of message type and schema - * version to payload class — and default to empty, which means no message type can be - * encoded and every publish is refused at preparation. That is fail-closed on purpose: a codec - * that accepted an unregistered type would serialise whatever object it was handed and put bytes - * on a topic that consumers compiled against a different shape will read. - * - * @param contracts the declared message contracts, if the deployment registered any - * @return the registry, defaulting to JSON - */ - @Bean - @ConditionalOnMissingBean(dev.caskeleton.messaging.schema.MessageCodecRegistry.class) - public dev.caskeleton.messaging.runtime.RegisteredMessageCodecs messagingCodecs( - ObjectProvider contracts) { - return dev.caskeleton.messaging.runtime.RegisteredMessageCodecs.of( - dev.caskeleton.messaging.schema.json.JacksonMessageCodec.of( - contracts.getIfAvailable(MessageContracts::none).byKey())); - } - - /** - * The destination profiles the publisher resolves against, from the validated registry. - * - * @param destinations the validated registry - * @return the profile registry - */ - @Bean - @ConditionalOnMissingBean - public dev.caskeleton.messaging.runtime.DestinationProfileRegistry messagingDestinationProfiles( - ValidatedDestinationRegistry destinations) { - return dev.caskeleton.messaging.runtime.DestinationProfileRegistry.of(destinations.all()); - } - - /** - * What this deployment may publish to. - * - *

The declared destinations, and nothing else. Deny-all assembles a platform that refuses - * every publish a correctly configured deployment makes; allow-all makes the check decoration. - * Declaring a destination profile is already an act of configuration, so a message to one nobody - * declared is a typo or a module reaching past its contract — which is what this check is for. - * - * @param destinations the validated registry - * @return the access policy - */ - @Bean - @ConditionalOnMissingBean - public dev.caskeleton.messaging.security.DestinationAccessPolicy messagingDestinationAccess( - ValidatedDestinationRegistry destinations) { - return dev.caskeleton.messaging.runtime.DeclaredDestinationAccess.of(destinations.all()); - } - - /** - * Payload and in-flight admission. - * - * @param properties the bound settings - * @return the admission controller - */ - @Bean - @ConditionalOnMissingBean - public dev.caskeleton.messaging.policy.MessagingAdmissionController messagingAdmission( - MessagingSettings properties) { - return new dev.caskeleton.messaging.policy.MessagingAdmissionController( - new dev.caskeleton.messaging.policy.PayloadLimitGuard( - new dev.caskeleton.messaging.policy.PayloadPolicy( - dev.caskeleton.messaging.schema.json.JacksonMessageCodec.DEFAULT_MAX_BYTES, - dev.caskeleton.messaging.schema.json.JacksonMessageCodec.DEFAULT_MAX_BYTES / 2)), - new dev.caskeleton.messaging.policy.InFlightLimiter( - properties.getBackpressure().getGlobalLimit(), java.time.Duration.ofSeconds(5)), - properties.getBackpressure().getPerDestinationLimit()); - } - - /** - * The publisher this configuration has been consuming since it was written (MSG-INT-003). - * - *

{@code deadLetterOrchestrator}, {@code blockingMessagePublisher}, {@code - * reactiveMessagePublisher} and {@code batchMessagePublisher} all take a {@link - * MessagePublisher}, and no configuration produced one. A selected transport therefore could not - * assemble: the platform failed on a missing bean rather than publishing anywhere. {@code - * DefaultMessagePublisher} existed in {@code messaging-runtime-core} the whole time; the starter - * did not depend on that leaf. - * - * @param destinations the destination profiles - * @param access what may be published to - * @param codecs the codec registry - * @param admission payload and in-flight admission - * @param runtimes the runtime registry - * @param transport the selected transport - * @return the publisher - */ - @Bean - @ConditionalOnMissingBean(MessagePublisher.class) - public dev.caskeleton.messaging.runtime.DefaultMessagePublisher messagingPublisher( - dev.caskeleton.messaging.runtime.DestinationProfileRegistry destinations, - dev.caskeleton.messaging.security.DestinationAccessPolicy access, - dev.caskeleton.messaging.schema.MessageCodecRegistry codecs, - dev.caskeleton.messaging.policy.MessagingAdmissionController admission, - MessagingRuntimeRegistry runtimes, - dev.caskeleton.messaging.transport.MessagingTransport transport, - ObjectProvider observation) { - dev.caskeleton.messaging.observation.MessagingObservation configured = - observation.getIfAvailable(); - if (configured == null) { - return new dev.caskeleton.messaging.runtime.DefaultMessagePublisher( - destinations, access, codecs, admission, runtimes, transport); - } - return new dev.caskeleton.messaging.runtime.DefaultMessagePublisher( - destinations, access, codecs, admission, runtimes, transport, System::nanoTime, configured); - } - - /** - * Installs the selected transport into the runtime registry (MSG-INT-003). - * - *

The registry was constructed empty and nothing ever installed anything, so every publish - * failed at {@code PUBLISH_RUNTIME_UNAVAILABLE} — after resolving the destination, checking - * access and encoding the payload, which is as far as a message could get. The same "registry - * built empty" shape the notification provider registry had. - * - * @param runtimes the registry - * @param transport the selected transport - * @return the startup installation - */ - @Bean - public org.springframework.beans.factory.InitializingBean messagingRuntimeInstallation( - MessagingRuntimeRegistry runtimes, - ObjectProvider transport, - CompiledMessagingConfiguration compiled) { - return () -> { - dev.caskeleton.messaging.transport.MessagingTransport selected = transport.getIfAvailable(); - if (selected == null) { - // Not a silent skip of a check: MessagingProviderSelection is what guarantees a transport - // when a broker is selected, and it refuses startup by name when one is not. This - // configuration is also loadable on its own — an adopter composing the policy primitives - // without a transport — and demanding one here would refuse that. - return; - } - - String runtimeName = logicalRuntimeName(selected, compiled); - runtimes.install( - new dev.caskeleton.messaging.runtime.TransportMessagingRuntime( - runtimeName, selected.generation(), selected)); - }; - } - - private static String logicalRuntimeName( - dev.caskeleton.messaging.transport.MessagingTransport selected, - CompiledMessagingConfiguration compiled) { - String configuredFamily = - switch (selected.brokerName()) { - case "kafka" -> BrokerSettings.KAFKA; - case "rabbit" -> BrokerSettings.RABBITMQ; - default -> selected.brokerName(); - }; - java.util.List configuredNames = - compiled.brokers().entrySet().stream() - .filter( - entry -> - entry.getValue().type() != null - && entry.getValue().type().equalsIgnoreCase(configuredFamily)) - .map(java.util.Map.Entry::getKey) - .sorted() - .toList(); - - if (configuredNames.isEmpty()) { - // Backward compatibility for applications that declare all profiles as beans and therefore - // have no app.messaging.brokers entries. Their destination profiles historically named the - // transport family directly. - return selected.brokerName(); - } - if (configuredNames.size() != 1) { - throw new IllegalStateException( - "the selected " - + selected.brokerName() - + " transport has one client/runtime but app.messaging.brokers declares " - + configuredNames.size() - + " logical brokers " - + configuredNames - + "; configure exactly one logical broker for the selected transport"); - } - return configuredNames.getFirst(); - } -} +@org.springframework.context.annotation.Import({ + MessagingValidationConfiguration.class, + MessagingPublishConfiguration.class, + MessagingObservabilityConfiguration.class, + MessagingConsumeConfiguration.class +}) +public class MessagingCoreAutoConfiguration {} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingObservabilityConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingObservabilityConfiguration.java new file mode 100644 index 0000000..bd8ca53 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingObservabilityConfiguration.java @@ -0,0 +1,77 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.observation.CardinalityGuard; +import dev.caskeleton.messaging.observation.MessagingRedactor; +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * Metrics, diagnostic redaction and the read-only actuator endpoint. + * + *

The Micrometer binding is conditional on a registry, so a narrow test or a non-observed + * adopter assembles the platform without Micrometer. + */ +@Configuration(proxyBeanMethods = false) +public class MessagingObservabilityConfiguration { + + /** + * Returns the diagnostic redactor. + * + * @return the redactor + */ + @Bean + @ConditionalOnMissingBean + public MessagingRedactor messagingRedactor() { + return new MessagingRedactor(); + } + + /** + * Returns the metric cardinality guard. + * + * @return the guard + */ + @Bean + @ConditionalOnMissingBean + public CardinalityGuard cardinalityGuard() { + return new CardinalityGuard(); + } + + /** + * Binds the platform observation seam to Micrometer when a registry is available. + * + *

Narrow tests and non-observed adopters may omit Micrometer entirely. A normal Boot runtime + * with Actuator contributes a registry, in which case publish outcomes must not fall back to the + * no-op observation constructor. + * + * @param registry the Micrometer registry + * @param guard the bounded-cardinality guard + * @param redactor the diagnostics redactor + * @return the messaging metrics binding + */ + @Bean + @ConditionalOnBean(io.micrometer.core.instrument.MeterRegistry.class) + @ConditionalOnMissingBean(dev.caskeleton.messaging.observation.MessagingObservation.class) + public dev.caskeleton.messaging.observation.MessagingObservation messagingObservation( + io.micrometer.core.instrument.MeterRegistry registry, + CardinalityGuard guard, + MessagingRedactor redactor) { + return new dev.caskeleton.messaging.observation.MessagingMetrics(registry, guard, redactor); + } + + /** + * Returns the read-only actuator endpoint. + * + * @param registry the validated destination registry + * @param admission the gate every publish passes through + * @return the endpoint + */ + @Bean + @ConditionalOnMissingBean + public MessagingEndpoint messagingEndpoint( + ValidatedDestinationRegistry registry, + dev.caskeleton.messaging.policy.MessagingAdmissionController admission) { + return new MessagingEndpoint(registry, admission); + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPlatformRootAutoConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPlatformRootAutoConfiguration.java index 8e3b816..3534ca2 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPlatformRootAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPlatformRootAutoConfiguration.java @@ -20,13 +20,18 @@ import org.springframework.context.annotation.Import; * whichever bean won. Selection now reads {@code app.messaging.broker} against a closed registry, * and a value outside it is a startup error rather than a context with no provider at all. * - *

{@code app.messaging.enabled=false} means no beans, no clients, no threads, and no bound - * detail namespace. {@code MessagingStarterOffContractTest} holds that against a full context - * including this starter's owned imports. Vendor auto-configuration is verified in its own runtime - * lanes. + *

On when present: a missing {@code app.messaging.enabled} means on, because an application that + * put this starter on its classpath chose messaging. {@code app.messaging.enabled=false} means no + * beans, no clients, no threads, and no bound detail namespace. {@code + * MessagingStarterOffContractTest} holds that against a full context including this starter's owned + * imports. Vendor auto-configuration is verified in its own runtime lanes. */ @AutoConfiguration -@ConditionalOnProperty(prefix = MessagingSettings.PREFIX, name = "enabled", havingValue = "true") +@ConditionalOnProperty( + prefix = MessagingSettings.PREFIX, + name = "enabled", + havingValue = "true", + matchIfMissing = true) @EnableConfigurationProperties(MessagingSettings.class) @Import({ MessagingCoreAutoConfiguration.class, diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java index 227965c..b7a248f 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingProviderSelection.java @@ -13,9 +13,9 @@ import org.springframework.util.ClassUtils; /** * Selects exactly one transport, from a closed registry, by name (MSG-INT-004). * - *

Selection used to be {@code @ConditionalOnClass}: Kafka assembled because a Kafka class was on - * the classpath and Rabbit because a Rabbit class was, so an application with both libraries — one - * transitive dependency is enough — assembled both and published through whichever bean won a + *

Selection used to be {@code @ConditionalOnClass}: a transport assembled because its client + * class was on the classpath, so an application with two client libraries — one transitive + * dependency is enough — assembled both and published through whichever bean won a * {@code @ConditionalOnMissingBean} race. Nothing failed; the message simply went somewhere nobody * chose. * @@ -25,7 +25,8 @@ import org.springframework.util.ClassUtils; *

  • a broker id outside the registry — the operator set something the platform cannot honour; *
  • a registered broker whose client library is absent — a deployment that asked for Kafka and * shipped without it is told so, instead of starting with no publisher at all; - *
  • a blank broker while messaging is on — the master switch says yes and nothing says what to. + *
  • a blank broker while messaging is on — the capability switch says yes and nothing says what + * to. * * *

    The provider configurations carry no condition of their own any more. Each is reached only @@ -38,58 +39,31 @@ public class MessagingProviderSelection { /** Broker id to the client class that must be present for it to be honoured. */ static final Map REGISTERED_BROKERS = - Map.of( - "kafka", "org.apache.kafka.clients.producer.Producer", - "rabbit", "com.rabbitmq.client.Channel"); + Map.of("kafka", "org.apache.kafka.clients.producer.Producer"); /** Broker id to the configuration that assembles it. */ static final Map PROVIDER_CONFIGURATIONS = - Map.of( - "kafka", "dev.caskeleton.messaging.autoconfigure.KafkaMessagingAutoConfiguration", - "rabbit", "dev.caskeleton.messaging.autoconfigure.RabbitMessagingAutoConfiguration"); + Map.of("kafka", "dev.caskeleton.messaging.autoconfigure.KafkaMessagingAutoConfiguration"); + + /** + * Broker id to the record its starter binds the family's own keys of an {@code + * app.messaging.brokers.} entry into. + * + *

    Named rather than referenced for the same reason as {@link #PROVIDER_CONFIGURATIONS}: this + * starter is broker-neutral and cannot see the class. The key validator loads it when present. + */ + static final Map PROVIDER_BROKER_SETTINGS = + Map.of("kafka", "dev.caskeleton.messaging.autoconfigure.KafkaBrokerSettings"); /** Broker id to the starter a deployment adds to its build to obtain that transport. */ static final Map PROVIDER_STARTERS = - Map.of( - "kafka", "messaging-spring-boot-starter-kafka", - "rabbit", "messaging-spring-boot-starter-rabbit"); - - /** - * Broker ids that ship without a {@code MessagingTransport}, and why. - * - *

    Registered is not the same as assemblable. The Rabbit leaf ships its profile validator, - * failure classifier and security configurer, and no transport — the native channel bridge those - * would sit on has only test implementations. Selecting it therefore produced an unsatisfied - * dependency on {@code MessagingTransport} deep inside the core configuration, which tells an - * operator that a bean is missing rather than that the transport they chose is not finished. - * - *

    Refused here instead, next to the other selection errors, so the message names the state the - * deployment is actually in. An entry leaves this map on the day its transport does exist. - */ - static final Map BROKERS_WITHOUT_A_TRANSPORT = - Map.of( - "rabbit", - "the Rabbit adapter ships its validators and security configuration but no " - + "MessagingTransport: its native channel publisher is not implemented, so a publish " - + "has nothing to travel on"); + Map.of("kafka", "messaging-spring-boot-starter-kafka"); /** The registered broker ids, so a message can tell an operator what is available. */ public static List registeredBrokerIds() { return REGISTERED_BROKERS.keySet().stream().sorted().toList(); } - /** - * The registered broker ids that actually assemble a transport. - * - * @return the ids a deployment can select today - */ - public static List assemblableBrokerIds() { - return REGISTERED_BROKERS.keySet().stream() - .filter(id -> !BROKERS_WITHOUT_A_TRANSPORT.containsKey(id)) - .sorted() - .toList(); - } - /** * Resolves the selection, or throws naming the property and the values it accepts. * @@ -141,17 +115,6 @@ public class MessagingProviderSelection { + " module on the classpath; this core starter is broker-neutral and the deployment" + " did not compose the transport it selected"); } - String unfinished = BROKERS_WITHOUT_A_TRANSPORT.get(broker); - if (unfinished != null) { - throw new IllegalStateException( - MessagingSettings.PREFIX - + ".broker=" - + broker - + " cannot be selected: " - + unfinished - + ". Transports that can be assembled today: " - + assemblableBrokerIds()); - } return broker; } diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPublishConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPublishConfiguration.java new file mode 100644 index 0000000..79995f1 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingPublishConfiguration.java @@ -0,0 +1,304 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.api.publish.MessagePublisher; +import dev.caskeleton.messaging.policy.BackoffCalculator; +import dev.caskeleton.messaging.policy.DeadLetterOrchestrator; +import dev.caskeleton.messaging.policy.DefaultRetryDecisionEngine; +import dev.caskeleton.messaging.policy.RetryDecisionEngine; +import dev.caskeleton.messaging.transport.DefaultMessagingRuntimeRegistry; +import dev.caskeleton.messaging.transport.GracefulShutdownCoordinator; +import dev.caskeleton.messaging.transport.MessagingRuntimeRegistry; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * The publish pipeline: codecs, destination access, admission, the publisher, the runtime it + * publishes through, and the shutdown order that drains it. + * + *

    The M2 and M3 surfaces are deliberately not auto-configured. Manual settlement, native + * capabilities, and the Admin Plane are opt-in beans with their own credentials, so a service + * cannot acquire them by adding the starter to its classpath. + */ +@Configuration(proxyBeanMethods = false) +public class MessagingPublishConfiguration { + + /** + * Returns the retry decision engine. + * + * @return the engine + */ + @Bean + @ConditionalOnMissingBean + public RetryDecisionEngine retryDecisionEngine() { + return new DefaultRetryDecisionEngine(new BackoffCalculator()); + } + + /** + * Returns the dead letter orchestrator. + * + * @param publisher the platform publisher + * @return the orchestrator + */ + @Bean + @ConditionalOnMissingBean + public DeadLetterOrchestrator deadLetterOrchestrator(MessagePublisher publisher) { + return new DeadLetterOrchestrator(publisher); + } + + /** + * Returns the runtime registry used for credential and topology rotation. + * + * @param properties the bound configuration + * @return the registry + */ + @Bean + @ConditionalOnMissingBean + public MessagingRuntimeRegistry messagingRuntimeRegistry(MessagingSettings properties) { + return new DefaultMessagingRuntimeRegistry(properties.getShutdown().getDrainDeadline()); + } + + /** + * Returns the graceful shutdown coordinator. + * + * @param properties the bound configuration + * @return the coordinator + */ + @Bean + @ConditionalOnMissingBean + public GracefulShutdownCoordinator gracefulShutdownCoordinator(MessagingSettings properties) { + return new GracefulShutdownCoordinator(properties.getShutdown().getDrainDeadline()); + } + + /** + * Drives the shutdown phases the admission gate and the drain coordinator only implemented. + * + * @param admission the publish admission gate + * @param drain the in-flight work coordinator + * @param properties the bound configuration supplying the drain deadline + * @return the lifecycle + */ + @Bean + @ConditionalOnMissingBean + public MessagingShutdownLifecycle messagingShutdownLifecycle( + dev.caskeleton.messaging.policy.MessagingAdmissionController admission, + GracefulShutdownCoordinator drain, + MessagingSettings properties) { + return new MessagingShutdownLifecycle( + admission, drain, properties.getShutdown().getDrainDeadline()); + } + + /** + * The codec registry, which had no implementation anywhere (MSG-INT-003). + * + *

    The message contracts come from the deployment — a {@code Map} of message type and schema + * version to payload class — and default to empty, which means no message type can be + * encoded and every publish is refused at preparation. That is fail-closed on purpose: a codec + * that accepted an unregistered type would serialise whatever object it was handed and put bytes + * on a topic that consumers compiled against a different shape will read. + * + * @param contracts the declared message contracts, if the deployment registered any + * @return the registry, defaulting to JSON + */ + @Bean + @ConditionalOnMissingBean(dev.caskeleton.messaging.schema.MessageCodecRegistry.class) + public dev.caskeleton.messaging.runtime.RegisteredMessageCodecs messagingCodecs( + ObjectProvider contracts) { + return dev.caskeleton.messaging.runtime.RegisteredMessageCodecs.of( + dev.caskeleton.messaging.schema.json.JacksonMessageCodec.of( + contracts.getIfAvailable(MessageContracts::none).byKey())); + } + + /** + * The destination profiles the publisher resolves against, from the validated registry. + * + * @param destinations the validated registry + * @return the profile registry + */ + @Bean + @ConditionalOnMissingBean + public dev.caskeleton.messaging.runtime.DestinationProfileRegistry messagingDestinationProfiles( + ValidatedDestinationRegistry destinations) { + return dev.caskeleton.messaging.runtime.DestinationProfileRegistry.of(destinations.all()); + } + + /** + * What this deployment may publish to. + * + *

    The declared destinations, and nothing else. Deny-all assembles a platform that refuses + * every publish a correctly configured deployment makes; allow-all makes the check decoration. + * Declaring a destination profile is already an act of configuration, so a message to one nobody + * declared is a typo or a module reaching past its contract — which is what this check is for. + * + *

    A broker with a security entry narrows that further to its {@code access.publishable} list. + * The list was compiled and then read by nothing, so a destination the security entry left out + * was published to anyway. + * + * @param destinations the validated registry + * @param compiled the security profiles compiled from configuration + * @param declared the security profiles the application declared as beans + * @return the access policy + */ + @Bean + @ConditionalOnMissingBean + public dev.caskeleton.messaging.security.DestinationAccessPolicy messagingDestinationAccess( + ValidatedDestinationRegistry destinations, + CompiledMessagingConfiguration compiled, + ObjectProvider declared) { + java.util.Map byBroker = + new java.util.HashMap<>(); + java.util.stream.Stream.concat(compiled.security().stream(), declared.orderedStream()) + .forEach( + profile -> { + if (byBroker.putIfAbsent(profile.broker(), profile.access()) != null) { + throw new IllegalStateException( + "broker '" + + profile.broker() + + "' has more than one security profile; its access list is ambiguous"); + } + }); + java.util.Map< + dev.caskeleton.messaging.api.destination.DestinationName, + dev.caskeleton.messaging.policy.DestinationProfile> + permitted = new java.util.HashMap<>(); + destinations + .all() + .forEach( + (name, profile) -> { + dev.caskeleton.messaging.security.DestinationAccessPolicy acl = + byBroker.get(profile.broker()); + if (acl == null || acl.mayPublish(name)) { + permitted.put(name, profile); + } + }); + return dev.caskeleton.messaging.runtime.DeclaredDestinationAccess.of(permitted); + } + + /** + * Payload and in-flight admission. + * + * @param properties the bound settings + * @return the admission controller + */ + @Bean + @ConditionalOnMissingBean + public dev.caskeleton.messaging.policy.MessagingAdmissionController messagingAdmission( + MessagingSettings properties) { + return new dev.caskeleton.messaging.policy.MessagingAdmissionController( + new dev.caskeleton.messaging.policy.PayloadLimitGuard( + new dev.caskeleton.messaging.policy.PayloadPolicy( + dev.caskeleton.messaging.schema.json.JacksonMessageCodec.DEFAULT_MAX_BYTES, + dev.caskeleton.messaging.schema.json.JacksonMessageCodec.DEFAULT_MAX_BYTES / 2)), + new dev.caskeleton.messaging.policy.InFlightLimiter( + properties.getBackpressure().getGlobalLimit(), java.time.Duration.ofSeconds(5)), + properties.getBackpressure().getPerDestinationLimit()); + } + + /** + * The publisher this configuration has been consuming since it was written (MSG-INT-003). + * + *

    {@code deadLetterOrchestrator} takes a {@link MessagePublisher}, and no configuration + * produced one. A selected transport therefore could not assemble: the platform failed on a + * missing bean rather than publishing anywhere. {@code DefaultMessagePublisher} existed in {@code + * messaging-runtime-core} the whole time; the starter did not depend on that leaf. + * + * @param destinations the destination profiles + * @param access what may be published to + * @param codecs the codec registry + * @param admission payload and in-flight admission + * @param runtimes the runtime registry + * @param transport the selected transport + * @return the publisher + */ + @Bean + @ConditionalOnMissingBean(MessagePublisher.class) + public dev.caskeleton.messaging.runtime.DefaultMessagePublisher messagingPublisher( + dev.caskeleton.messaging.runtime.DestinationProfileRegistry destinations, + dev.caskeleton.messaging.security.DestinationAccessPolicy access, + dev.caskeleton.messaging.schema.MessageCodecRegistry codecs, + dev.caskeleton.messaging.policy.MessagingAdmissionController admission, + MessagingRuntimeRegistry runtimes, + dev.caskeleton.messaging.transport.MessagingTransport transport, + ObjectProvider observation) { + dev.caskeleton.messaging.observation.MessagingObservation configured = + observation.getIfAvailable(); + if (configured == null) { + return new dev.caskeleton.messaging.runtime.DefaultMessagePublisher( + destinations, access, codecs, admission, runtimes, transport); + } + return new dev.caskeleton.messaging.runtime.DefaultMessagePublisher( + destinations, access, codecs, admission, runtimes, transport, System::nanoTime, configured); + } + + /** + * Installs the selected transport into the runtime registry (MSG-INT-003). + * + *

    The registry was constructed empty and nothing ever installed anything, so every publish + * failed at {@code PUBLISH_RUNTIME_UNAVAILABLE} — after resolving the destination, checking + * access and encoding the payload, which is as far as a message could get. The same "registry + * built empty" shape the notification provider registry had. + * + * @param runtimes the registry + * @param transport the selected transport + * @return the startup installation + */ + @Bean + public org.springframework.beans.factory.InitializingBean messagingRuntimeInstallation( + MessagingRuntimeRegistry runtimes, + ObjectProvider transport, + CompiledMessagingConfiguration compiled) { + return () -> { + dev.caskeleton.messaging.transport.MessagingTransport selected = transport.getIfAvailable(); + if (selected == null) { + // Not a silent skip of a check: MessagingProviderSelection is what guarantees a transport + // when a broker is selected, and it refuses startup by name when one is not. This + // configuration is also loadable on its own — an adopter composing the policy primitives + // without a transport — and demanding one here would refuse that. + return; + } + + String runtimeName = logicalRuntimeName(selected, compiled); + runtimes.install( + new dev.caskeleton.messaging.runtime.TransportMessagingRuntime( + runtimeName, selected.generation(), selected)); + }; + } + + private static String logicalRuntimeName( + dev.caskeleton.messaging.transport.MessagingTransport selected, + CompiledMessagingConfiguration compiled) { + String configuredFamily = + switch (selected.brokerName()) { + case "kafka" -> BrokerSettings.KAFKA; + default -> selected.brokerName(); + }; + java.util.List configuredNames = + compiled.brokers().entrySet().stream() + .filter( + entry -> + entry.getValue().type() != null + && entry.getValue().type().equalsIgnoreCase(configuredFamily)) + .map(java.util.Map.Entry::getKey) + .sorted() + .toList(); + + if (configuredNames.isEmpty()) { + // Backward compatibility for applications that declare all profiles as beans and therefore + // have no app.messaging.brokers entries. Their destination profiles historically named the + // transport family directly. + return selected.brokerName(); + } + if (configuredNames.size() != 1) { + throw new IllegalStateException( + "the selected " + + selected.brokerName() + + " transport has one client/runtime but app.messaging.brokers declares " + + configuredNames.size() + + " logical brokers " + + configuredNames + + "; configure exactly one logical broker for the selected transport"); + } + return configuredNames.getFirst(); + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java index 72350cc..f6e096a 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingReliabilityAutoConfiguration.java @@ -1,10 +1,8 @@ package dev.caskeleton.messaging.autoconfigure; import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.inbox.IdempotentConsumer; import dev.caskeleton.messaging.inbox.InboxCleanupJob; import dev.caskeleton.messaging.inbox.InboxRetentionPolicy; -import dev.caskeleton.messaging.inbox.TransactionalInboxHandler; import dev.caskeleton.messaging.outbox.OutboxCleanupJob; import dev.caskeleton.messaging.outbox.OutboxEnvelopeFactory; import dev.caskeleton.messaging.outbox.OutboxProperties; @@ -14,7 +12,6 @@ import dev.caskeleton.messaging.outbox.OutboxRetryScheduler; import dev.caskeleton.messaging.reliability.InboxRepository; import dev.caskeleton.messaging.reliability.OutboxRepository; import java.time.Duration; -import java.time.Instant; import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; import org.springframework.context.annotation.Bean; @@ -109,6 +106,33 @@ public class MessagingReliabilityAutoConfiguration { return new OutboxRelayWorker(relay, scheduler); } + /** + * Publishes the relay's failed-pass count when a meter registry exists. + * + *

    {@code failedPasses()} was counted and read by nothing, so a relay failing every pass looked + * on every dashboard exactly like one with nothing to do. + * + * @param worker the relay worker + * @param registries the meter registry, when the application has one + * @return the registration, run once the context has the registry + */ + @Bean + @ConditionalOnBean(OutboxRelayWorker.class) + public org.springframework.beans.factory.InitializingBean outboxRelayMeters( + OutboxRelayWorker worker, + org.springframework.beans.factory.ObjectProvider + registries) { + return () -> + registries.ifAvailable( + registry -> + io.micrometer.core.instrument.FunctionCounter.builder( + "messaging.outbox.relay.failed.passes", + worker, + OutboxRelayWorker::failedPasses) + .description("Outbox relay passes that ended in an exception") + .register(registry)); + } + /** * Binds the relay worker to the context lifecycle. * @@ -169,17 +193,4 @@ public class MessagingReliabilityAutoConfiguration { public InboxCleanupJob inboxCleanupJob(InboxRepository inbox, InboxRetentionPolicy policy) { return new InboxCleanupJob(inbox, policy, 20); } - - /** - * Returns the transactional inbox handler. - * - * @param consumer the reservation-and-effect runner the application supplied - * @return the handler - */ - @Bean - @ConditionalOnBean(IdempotentConsumer.class) - @ConditionalOnMissingBean - public TransactionalInboxHandler transactionalInboxHandler(IdempotentConsumer consumer) { - return new TransactionalInboxHandler<>(consumer, Instant::now); - } } diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingSettings.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingSettings.java index 92fef98..9985b6e 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingSettings.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingSettings.java @@ -20,11 +20,6 @@ import org.springframework.boot.context.properties.ConfigurationProperties; * the failure it prevents (a deployment split across two prefixes, each half-configuring the * platform) is worse than an actionable startup error. * - *

    Every Experimental and bridge flag defaults to {@code false}. An adapter whose contract suite - * is still being proven must not become load-bearing because a default turned it on, and the same - * applies to the Spring Cloud Stream bridge, which bypasses the platform's retry and dead letter - * policy by design. - * *

    The destination, broker and security maps are what the configuration reference has always * described and nothing bound. Until they existed, a deployment that configured them exactly as * documented changed no behaviour and produced no error: the profiles the platform actually used @@ -48,8 +43,6 @@ public class MessagingSettings { /** The prefix the configuration reference documented, which nothing ever bound. */ public static final String DOCUMENTED_PREFIX = "messaging"; - private final Experimental experimental = new Experimental(); - private final Bridge bridge = new Bridge(); private final Backpressure backpressure = new Backpressure(); private final Shutdown shutdown = new Shutdown(); private Map destinations = new LinkedHashMap<>(); @@ -110,24 +103,6 @@ public class MessagingSettings { this.security = new LinkedHashMap<>(security); } - /** - * Returns the experimental adapter switches. - * - * @return the experimental settings - */ - public Experimental getExperimental() { - return experimental; - } - - /** - * Returns the optional bridge switches. - * - * @return the bridge settings - */ - public Bridge getBridge() { - return bridge; - } - /** * Returns the backpressure limits. * @@ -146,92 +121,6 @@ public class MessagingSettings { return shutdown; } - /** Experimental adapter switches, all off by default. */ - public static class Experimental { - - private boolean kafkaShare; - private boolean pulsar; - private boolean nats; - - /** - * Reports whether the Kafka Share Group adapter is enabled. - * - * @return true when enabled - */ - public boolean isKafkaShare() { - return kafkaShare; - } - - /** - * Enables or disables the Kafka Share Group adapter. - * - * @param kafkaShare whether to enable it - */ - public void setKafkaShare(boolean kafkaShare) { - this.kafkaShare = kafkaShare; - } - - /** - * Reports whether the Pulsar adapter is enabled. - * - * @return true when enabled - */ - public boolean isPulsar() { - return pulsar; - } - - /** - * Enables or disables the Pulsar adapter. - * - * @param pulsar whether to enable it - */ - public void setPulsar(boolean pulsar) { - this.pulsar = pulsar; - } - - /** - * Reports whether the NATS JetStream adapter is enabled. - * - * @return true when enabled - */ - public boolean isNats() { - return nats; - } - - /** - * Enables or disables the NATS JetStream adapter. - * - * @param nats whether to enable it - */ - public void setNats(boolean nats) { - this.nats = nats; - } - } - - /** Optional integration bridges, off by default. */ - public static class Bridge { - - private boolean springCloudStream; - - /** - * Reports whether the Spring Cloud Stream bridge is enabled. - * - * @return true when enabled - */ - public boolean isSpringCloudStream() { - return springCloudStream; - } - - /** - * Enables or disables the Spring Cloud Stream bridge. - * - * @param springCloudStream whether to enable it - */ - public void setSpringCloudStream(boolean springCloudStream) { - this.springCloudStream = springCloudStream; - } - } - /** In-flight limits. */ public static class Backpressure { diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingValidationConfiguration.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingValidationConfiguration.java new file mode 100644 index 0000000..0d6a60a --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingValidationConfiguration.java @@ -0,0 +1,194 @@ +package dev.caskeleton.messaging.autoconfigure; + +import dev.caskeleton.messaging.policy.DestinationProfile; +import dev.caskeleton.messaging.policy.DestinationProfileValidator; +import dev.caskeleton.messaging.security.BrokerTlsPolicy; +import dev.caskeleton.messaging.security.CredentialProvider; +import dev.caskeleton.messaging.security.CredentialRuntimeRegistry; +import dev.caskeleton.messaging.security.MessageSecurityValidator; +import java.util.List; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; +import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * Compiles and validates the messaging configuration, and fails the context when it is + * contradictory. + * + *

    Destination profiles are validated at context refresh through a bean that fails to construct + * when the registry is contradictory. Startup failure is the point: a profile that promises ordered + * delivery while configuring a reordering retry does not fail on the happy path, it fails months + * later on the first retry, looking like a data bug rather than a configuration one. + * + *

    One of the three parts {@link MessagingCoreAutoConfiguration} used to be in a single class + * (S7-#9a): configuration and validation here, the publish pipeline in {@link + * MessagingPublishConfiguration}, observability in {@link MessagingObservabilityConfiguration}. + */ +@Configuration(proxyBeanMethods = false) +public class MessagingValidationConfiguration { + + /** + * Fails the context when a production broker profile has no credential source. + * + *

    One actionable error instead of a missing-bean message naming an internal registry type. + */ + @Bean + public MessagingCredentialRequirementValidator messagingCredentialRequirementValidator( + org.springframework.beans.factory.ObjectProvider< + dev.caskeleton.messaging.security.BrokerSecurityProfile> + profiles, + CompiledMessagingConfiguration compiled, + org.springframework.beans.factory.ObjectProvider< + dev.caskeleton.messaging.security.CredentialProvider> + credentials, + org.springframework.core.env.Environment environment) { + return new MessagingCredentialRequirementValidator( + profiles, compiled.security(), credentials, environment); + } + + /** + * Fails the context when configuration uses a prefix nothing binds. + * + *

    Registered here rather than left to review because the failure it catches is silent: a key + * under {@code backend.messaging} or a bare {@code messaging} looks configured and does nothing. + */ + @Bean + public MessagingPrefixMigrationValidator messagingPrefixMigrationValidator( + org.springframework.core.env.ConfigurableEnvironment environment) { + return new MessagingPrefixMigrationValidator(environment); + } + + /** + * Compiles {@code app.messaging.destinations}, {@code .brokers} and {@code .security} into the + * platform's own profile objects. + * + *

    Separate from the registry bean so that a configuration error fails while compiling, with + * the offending key in the message, rather than later as a validation failure over an object + * whose origin the operator then has to work out. + * + * @param settings the bound configuration + * @return the compiled profiles + */ + @Bean + @ConditionalOnMissingBean + public CompiledMessagingConfiguration compiledMessagingConfiguration(MessagingSettings settings) { + return new MessagingConfigurationCompiler(settings).compile(); + } + + /** + * Refuses a key under the platform's own sections that nothing binds. + * + *

    Registered here rather than left to review because a misspelt key inside a bound section is + * silent in a way a misspelt prefix is not: the section binds, the entry appears, and the one + * setting the operator came to change is the one that did not take. + * + * @param environment the resolved environment + * @return the key validator + */ + @Bean + @ConditionalOnMissingBean + public MessagingConfigurationKeyValidator messagingConfigurationKeyValidator( + org.springframework.core.env.ConfigurableEnvironment environment) { + return new MessagingConfigurationKeyValidator(environment); + } + + /** + * Validates every registered destination profile at startup. + * + *

    Returns a marker rather than a service because its only job is to fail the context when the + * registry is inconsistent. + * + *

    Configured profiles and bean-declared ones are merged before validation, not after, because + * the rules that matter most across a registry — a dead letter destination that exists, a retry + * graph with no cycle — are questions about the whole set. Validating the two halves separately + * would pass a deployment whose configured destination dead-letters into a bean-declared one that + * points back. + * + * @param profiles the profiles the application declared as beans + * @param compiled the profiles compiled from configuration + * @return the validated registry + */ + @Bean + @ConditionalOnMissingBean + public ValidatedDestinationRegistry validatedDestinationRegistry( + ObjectProvider profiles, CompiledMessagingConfiguration compiled) { + List registered = + java.util.stream.Stream.concat(compiled.destinations().stream(), profiles.orderedStream()) + .toList(); + new DestinationProfileValidator().validateAll(registered); + return new ValidatedDestinationRegistry(registered); + } + + /** + * Returns the destination profile validator. + * + * @return the validator + */ + @Bean + @ConditionalOnMissingBean + public DestinationProfileValidator destinationProfileValidator() { + return new DestinationProfileValidator(); + } + + /** + * Returns the broker security validator. + * + * @return the validator + */ + @Bean + @ConditionalOnMissingBean + public MessageSecurityValidator messageSecurityValidator() { + return new MessageSecurityValidator(); + } + + /** + * Runs the security validator over every broker security profile, declared or configured. + * + * @param profiles the security profiles the application declared as beans + * @param compiled the security profiles compiled from configuration + * @param validator the security validator + * @return the startup validation + */ + @Bean + public StartupProfileValidation + brokerSecurityStartupValidation( + ObjectProvider profiles, + CompiledMessagingConfiguration compiled, + MessageSecurityValidator validator) { + return new StartupProfileValidation<>( + () -> + java.util.stream.Stream.concat(compiled.security().stream(), profiles.orderedStream()) + .toList(), + validator::validate); + } + + /** + * Returns the credential runtime registry, when the application supplied a credential provider. + * + *

    Conditional on the provider because there is no safe default: a registry with no source + * would resolve nothing, and the adapters would connect with whatever their client's defaults + * are. + * + * @param provider the application's credential source + * @return the credential runtime registry + */ + @Bean + @ConditionalOnBean(CredentialProvider.class) + @ConditionalOnMissingBean + public CredentialRuntimeRegistry credentialRuntimeRegistry(CredentialProvider provider) { + return new CredentialRuntimeRegistry(provider); + } + + /** + * Returns the transport security policy. + * + * @return the TLS policy + */ + @Bean + @ConditionalOnMissingBean + public BrokerTlsPolicy brokerTlsPolicy() { + return new BrokerTlsPolicy(); + } +} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/PublishResults.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/PublishResults.java deleted file mode 100644 index 4dd0178..0000000 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/PublishResults.java +++ /dev/null @@ -1,71 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessagingException; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.time.Duration; -import java.util.Optional; -import java.util.concurrent.CompletionException; - -/** - * Converts an exception that escaped a publish into a result the caller can act on. - * - *

    A {@link MessagingException} already carries a sanitized descriptor, including whether it is - * retryable, so its own classification is used. Anything else is reported {@code AMBIGUOUS}: an - * unrecognised exception proves nothing about whether the message reached the broker, and the - * conservative reading is the one that does not turn a lost confirmation into a duplicate. - */ -final class PublishResults { - - private PublishResults() {} - - /** - * Converts a failure into a publish result. - * - * @param failure the exception that escaped the publish - * @return the equivalent result - */ - static PublishResult from(Throwable failure) { - Throwable cause = failure instanceof CompletionException ? failure.getCause() : failure; - - if (cause instanceof MessagingException messagingFailure) { - FailureDescriptor descriptor = messagingFailure.failure(); - boolean transmitted = descriptor.category() == FailureCategory.AMBIGUOUS; - return new PublishResult( - transmitted ? PublishCompletion.AMBIGUOUS : PublishCompletion.REJECTED, - transmitted - ? new PublishEvidence( - true, - TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, - false, - ConfirmationLevel.NONE) - : PublishEvidence.notTransmitted(), - transmitted ? RoutingOutcome.UNKNOWN : RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of(descriptor)); - } - - return new PublishResult( - PublishCompletion.AMBIGUOUS, - new PublishEvidence( - true, TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, - "UNCLASSIFIED_PUBLISH_FAILURE", - "the publish failed with an unrecognised error: " - + (cause == null ? "unknown" : cause.getClass().getSimpleName())))); - } -} diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/ReactiveMessagePublisher.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/ReactiveMessagePublisher.java deleted file mode 100644 index f59bdf1..0000000 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/ReactiveMessagePublisher.java +++ /dev/null @@ -1,32 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import reactor.core.publisher.Mono; - -/** - * The Reactor facade over the asynchronous publisher. - * - *

    Lives in the starter rather than the core so that {@code messaging-core-api} never depends on - * Reactor. A service that does not use Reactor should not inherit it through a messaging contract. - * - *

    The returned {@code Mono} is cold and cancellable: nothing is published until it is - * subscribed, and cancelling the subscription cancels the underlying stage rather than leaving an - * orphaned publish in flight. - */ -public interface ReactiveMessagePublisher { - - /** - * Publishes one message. - * - * @param the payload type - * @param destination the logical destination - * @param message the envelope to publish - * @param options per-call options - * @return a cold {@code Mono} emitting the outcome and its evidence - */ - Mono publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options); -} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BatchPublisherTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BatchPublisherTest.java deleted file mode 100644 index ee8855a..0000000 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BatchPublisherTest.java +++ /dev/null @@ -1,328 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.error.MessageTooLargeException; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.BatchPublishOptions; -import dev.caskeleton.messaging.api.publish.BatchPublishResult; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishRequest; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import org.junit.jupiter.api.Test; - -class BatchPublisherTest { - - private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z"); - - private static PublishResult rejected() { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of( - dev.caskeleton.messaging.api.error.FailureDescriptor.of( - dev.caskeleton.messaging.api.error.FailureCategory.PERMANENT_BUSINESS, - "BROKER_REJECTED", - "the broker refused the message"))); - } - - private static PublishResult confirmed() { - return new PublishResult( - PublishCompletion.CONFIRMED, - new PublishEvidence( - true, - TransmissionEvidence.TRANSMITTED, - true, - ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK), - RoutingOutcome.ROUTED, - Optional.empty(), - 1, - Duration.ZERO, - Optional.empty()); - } - - /** Publishes according to a per-index script, so partial failures are reproducible. */ - private static MessagePublisher scripted(List script, List calls) { - return new MessagePublisher() { - private int index; - - @Override - public CompletionStage publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options) { - int current = index++; - calls.add(current); - return switch (script.get(current)) { - case CONFIRM -> CompletableFuture.completedFuture(confirmed()); - case FAIL_ASYNC -> - CompletableFuture.failedFuture( - new MessageTooLargeException("PAYLOAD_LIMIT_EXCEEDED", "too large")); - case THROW_SYNC -> - throw new MessageTooLargeException("PAYLOAD_LIMIT_EXCEEDED", "too large"); - }; - } - }; - } - - private enum Outcome { - CONFIRM, - FAIL_ASYNC, - THROW_SYNC - } - - private static PublishRequest request() { - MessageDestination destination = - new MessageDestination<>( - new DestinationName("orders.v1"), new MessageType("order.created"), String.class); - MessageEnvelope envelope = - new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - NOW, - Optional.of(NOW), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - "{}"); - return PublishRequest.of(destination, envelope); - } - - private static List> requests(int count) { - List> requests = new ArrayList<>(); - for (int index = 0; index < count; index++) { - requests.add(request()); - } - return requests; - } - - private static BatchPublishResult publish(List script, BatchPublishOptions options) { - List calls = new ArrayList<>(); - return new DefaultBatchMessagePublisher(scripted(script, calls), () -> NOW) - .publish(requests(script.size()), options) - .toCompletableFuture() - .join(); - } - - @Test - void closingTheBatchPublisherStopsItsDeadlineScheduler() { - var deadlines = new java.util.concurrent.ScheduledThreadPoolExecutor(1); - var publisher = - new DefaultBatchMessagePublisher( - scripted(List.of(Outcome.CONFIRM), new ArrayList<>()), () -> NOW, deadlines); - - publisher.close(); - - assertThat(deadlines.isShutdown()).isTrue(); - } - - @Test - void everyEntryGetsItsOwnResultKeyedByIndex() { - BatchPublishResult result = - publish( - List.of(Outcome.CONFIRM, Outcome.CONFIRM, Outcome.CONFIRM), - BatchPublishOptions.defaults()); - - assertThat(result.items()).hasSize(3); - assertThat(result.items()).extracting(item -> item.index()).containsExactly(0, 1, 2); - assertThat(result.allConfirmed()).isTrue(); - } - - @Test - void oneFailedEntryDoesNotHideTheOthers() { - BatchPublishResult result = - publish( - List.of(Outcome.CONFIRM, Outcome.FAIL_ASYNC, Outcome.CONFIRM), - BatchPublishOptions.defaults()); - - assertThat(result.withCompletion(PublishCompletion.CONFIRMED)).hasSize(2); - assertThat(result.withCompletion(PublishCompletion.REJECTED)) - .as("the caller needs to know exactly which entries to resubmit") - .hasSize(1); - } - - @Test - void aSynchronousFailureStillProducesAnEntryResult() { - BatchPublishResult result = - publish(List.of(Outcome.THROW_SYNC, Outcome.CONFIRM), BatchPublishOptions.defaults()); - - assertThat(result.items()).hasSize(2); - assertThat(result.items().get(0).result().completion()).isEqualTo(PublishCompletion.REJECTED); - } - - @Test - void aRejectedEntryKeepsItsSanitizedFailureCode() { - BatchPublishResult result = - publish(List.of(Outcome.FAIL_ASYNC), BatchPublishOptions.defaults()); - - assertThat(result.items().get(0).result().failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("PAYLOAD_LIMIT_EXCEEDED")); - } - - @Test - void aBatchLargerThanItsLimitIsRefusedBeforeAnythingIsPublished() { - List calls = new ArrayList<>(); - DefaultBatchMessagePublisher publisher = - new DefaultBatchMessagePublisher(scripted(List.of(Outcome.CONFIRM), calls), () -> NOW); - - assertThatThrownBy( - () -> - publisher.publish( - requests(5), new BatchPublishOptions(Duration.ofSeconds(5), 2, false))) - .isInstanceOf(MessageTooLargeException.class); - assertThat(calls).isEmpty(); - } - - @Test - void stopOnFirstRejectionSkipsTheRemainingSubmissions() { - List calls = new ArrayList<>(); - BatchPublishResult result = - new DefaultBatchMessagePublisher( - scripted( - List.of(Outcome.CONFIRM, Outcome.THROW_SYNC, Outcome.CONFIRM, Outcome.CONFIRM), - calls), - () -> NOW) - .publish(requests(4), new BatchPublishOptions(Duration.ofSeconds(5), 100, true)) - .toCompletableFuture() - .join(); - - assertThat(calls).containsExactly(0, 1); - assertThat(result.items()) - .as("truncating the list left the caller unable to tell 'not submitted' from 'lost'") - .hasSize(4); - assertThat(result.items().get(3).result().failure().orElseThrow().code()) - .isEqualTo("NOT_SUBMITTED_AFTER_REJECTION"); - } - - @Test - @org.junit.jupiter.api.DisplayName( - "the batch deadline completes outstanding entries as ambiguous") - void theBatchDeadlineCompletesOutstandingEntries() { - BatchPublishResult result = - new DefaultBatchMessagePublisher( - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - return new CompletableFuture<>(); - } - }, - () -> NOW) - .publish(requests(2), new BatchPublishOptions(Duration.ofMillis(50), 100, false)) - .toCompletableFuture() - .join(); - - assertThat(result.items()) - .as("options.timeout() was never read, so a publisher that never completes hung forever") - .hasSize(2); - assertThat(result.items()) - .allSatisfy( - item -> - assertThat(item.result().completion()) - .isEqualTo(dev.caskeleton.messaging.api.publish.PublishCompletion.AMBIGUOUS)); - } - - @Test - @org.junit.jupiter.api.DisplayName("a delayed rejection still stops the entries after it") - void aDelayedRejectionStopsLaterEntries() { - List calls = new ArrayList<>(); - java.util.concurrent.CompletableFuture - delayed = new java.util.concurrent.CompletableFuture<>(); - - BatchPublishResult result = - new DefaultBatchMessagePublisher( - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - int index = calls.size(); - calls.add(index); - if (index == 0) { - // Resolves only after the publisher returns, which is what an asynchronous - // broker rejection looks like. The old loop only saw an already-completed - // failure, so every later entry was submitted anyway. - // Completed on another thread so the rejection lands after this method - // returns — the shape an asynchronous broker rejection actually has. - new Thread(() -> delayed.complete(rejected())).start(); - return delayed; - } - return CompletableFuture.completedFuture(confirmed()); - } - }, - () -> NOW) - .publish(requests(3), new BatchPublishOptions(Duration.ofSeconds(2), 100, true)) - .toCompletableFuture() - .join(); - - assertThat(calls).containsExactly(0); - assertThat(result.items()).hasSize(3); - assertThat(result.items().get(2).result().failure().orElseThrow().code()) - .isEqualTo("NOT_SUBMITTED_AFTER_REJECTION"); - } - - @Test - void withoutStopOnFirstRejectionEveryEntryIsAttempted() { - List calls = new ArrayList<>(); - new DefaultBatchMessagePublisher( - scripted(List.of(Outcome.THROW_SYNC, Outcome.CONFIRM, Outcome.CONFIRM), calls), - () -> NOW) - .publish(requests(3), BatchPublishOptions.defaults()) - .toCompletableFuture() - .join(); - - assertThat(calls).containsExactly(0, 1, 2); - } - - @Test - void aBatchIsNotATransactionSoConfirmedEntriesStayConfirmed() { - BatchPublishResult result = - publish(List.of(Outcome.CONFIRM, Outcome.FAIL_ASYNC), BatchPublishOptions.defaults()); - - assertThat(result.items().get(0).result().completion()) - .as("nothing can un-publish an entry the broker already confirmed") - .isEqualTo(PublishCompletion.CONFIRMED); - assertThat(result.allConfirmed()).isFalse(); - } - - @Test - void theDefaultOptionsAttemptEveryEntry() { - assertThat(BatchPublishOptions.defaults().stopOnFirstRejection()).isFalse(); - } -} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BlockingFacadeTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BlockingFacadeTest.java deleted file mode 100644 index 0b60b6e..0000000 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BlockingFacadeTest.java +++ /dev/null @@ -1,148 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.error.FailureCategory; -import dev.caskeleton.messaging.api.error.FailureDescriptor; -import dev.caskeleton.messaging.api.error.MessagePublishTimeoutException; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.BatchPublishItemResult; -import dev.caskeleton.messaging.api.publish.BatchPublishResult; -import dev.caskeleton.messaging.api.publish.BlockingMessagePublisher; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import java.time.Duration; -import java.time.Instant; -import java.util.List; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import org.junit.jupiter.api.Test; - -class BlockingFacadeTest { - - private static final MessageDestination ORDERS = - new MessageDestination<>( - new DestinationName("order-events"), new MessageType("order.created"), String.class); - - @Test - void aConfirmedPublishIsReturnedDirectly() { - BlockingMessagePublisher publisher = - new DefaultBlockingMessagePublisher( - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - return CompletableFuture.completedFuture(confirmed()); - } - }); - - PublishResult result = publisher.publish(ORDERS, envelope(), PublishOptions.defaults()); - - assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED); - } - - @Test - void aStalledPublishTimesOutAsAmbiguousRatherThanBlockingForever() { - MessagePublisher stalled = - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - return new CompletableFuture<>(); - } - }; - PublishOptions shortDeadline = - new PublishOptions( - Duration.ofMillis(50), - dev.caskeleton.messaging.api.destination.ConfirmationRequirement - .REPLICATION_OR_PERSISTENCE_ACK, - Optional.empty()); - - assertThatThrownBy( - () -> - new DefaultBlockingMessagePublisher(stalled) - .publish(ORDERS, envelope(), shortDeadline)) - .isInstanceOf(MessagePublishTimeoutException.class) - .satisfies( - thrown -> - assertThat(((MessagePublishTimeoutException) thrown).category()) - .isEqualTo(FailureCategory.AMBIGUOUS)); - } - - @Test - void aBatchKeepsPerItemOutcomes() { - BatchPublishResult batch = - new BatchPublishResult( - List.of( - new BatchPublishItemResult(0, MessageId.newId(), confirmed()), - new BatchPublishItemResult(1, MessageId.newId(), ambiguous())), - Duration.ofMillis(20)); - - assertThat(batch.allConfirmed()).isFalse(); - assertThat(batch.withCompletion(PublishCompletion.CONFIRMED)).hasSize(1); - assertThat(batch.withCompletion(PublishCompletion.AMBIGUOUS)).hasSize(1); - } - - private static MessageEnvelope envelope() { - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - Instant.parse("2026-08-10T09:15:00Z"), - Optional.of(Instant.parse("2026-08-10T09:15:00Z")), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - "payload"); - } - - private static PublishResult confirmed() { - return new PublishResult( - PublishCompletion.CONFIRMED, - PublishEvidence.confirmed(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ofMillis(3), - Optional.empty()); - } - - private static PublishResult ambiguous() { - return new PublishResult( - PublishCompletion.AMBIGUOUS, - PublishEvidence.ambiguous(), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - Duration.ofSeconds(5), - Optional.of( - FailureDescriptor.of( - FailureCategory.AMBIGUOUS, "CONFIRM_TIMEOUT", "confirm timed out"))); - } -} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java new file mode 100644 index 0000000..936c710 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/BrokerAccessListTest.java @@ -0,0 +1,102 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.ContentType; +import dev.caskeleton.messaging.api.MessageEnvelope; +import dev.caskeleton.messaging.api.MessageId; +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.ProducerId; +import dev.caskeleton.messaging.api.SchemaVersion; +import dev.caskeleton.messaging.api.TraceContext; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import dev.caskeleton.messaging.api.error.FailureDescriptor; +import dev.caskeleton.messaging.api.header.MessageHeaders; +import dev.caskeleton.messaging.api.publish.MessagePublisher; +import dev.caskeleton.messaging.api.publish.PublishOptions; +import dev.caskeleton.messaging.api.publish.PublishResult; +import java.time.Instant; +import java.util.Optional; +import org.junit.jupiter.api.Test; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; + +/** + * The broker's {@code access.publishable} list is enforced by the publish path, not only compiled. + * + *

    The list was bound, validated and turned into a {@code DestinationAccessPolicy} that nothing + * read: the publisher's access check was built from the declared destinations alone, so a + * destination the security entry left out was published to anyway. + */ +class BrokerAccessListTest { + + private final ApplicationContextRunner runner = + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withUserConfiguration(KafkaProducerSecurityTest.Credentials.class) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "app.messaging.brokers.kafka-primary.type=kafka", + "app.messaging.brokers.kafka-primary.bootstrap-servers[0]=localhost:9092", + "app.messaging.security.kafka-primary.producer.type=SASL_SCRAM", + "app.messaging.security.kafka-primary.producer.credential-id=kafka-producer", + "app.messaging.security.kafka-primary.consumer.type=SASL_SCRAM", + "app.messaging.security.kafka-primary.consumer.credential-id=kafka-consumer", + "app.messaging.security.kafka-primary.access.publishable[0]=order-events") + .withPropertyValues(destination("order-events")) + .withPropertyValues(destination("audit-events")); + + @Test + void aDeclaredDestinationTheSecurityEntryDoesNotListIsRejectedAsForbidden() { + runner.run( + context -> { + // given + MessagePublisher publisher = context.getBean(MessagePublisher.class); + + // when + PublishResult result = publish(publisher, "audit-events"); + + // then + assertThat(result.failure().map(FailureDescriptor::code)).contains("PUBLISH_FORBIDDEN"); + }); + } + + private static PublishResult publish(MessagePublisher publisher, String destination) { + return publisher + .publish( + new MessageDestination<>( + new DestinationName(destination), new MessageType("order.created"), String.class), + new MessageEnvelope<>( + MessageId.newId(), + new MessageType("order.created"), + new SchemaVersion(1), + Instant.parse("2026-09-25T00:00:00Z"), + Optional.empty(), + new ProducerId("orders"), + Optional.empty(), + Optional.empty(), + ContentType.JSON, + Optional.empty(), + Optional.empty(), + Optional.empty(), + TraceContext.none(), + MessageHeaders.empty(), + "{}"), + PublishOptions.defaults()) + .toCompletableFuture() + .join(); + } + + private static String[] destination(String name) { + String prefix = "app.messaging.destinations." + name + "."; + return new String[] { + prefix + "broker=kafka-primary", + prefix + "kind=EVENT_STREAM", + prefix + "physical.topic=" + name, + prefix + "schema.message-types[0]=order.created", + prefix + "guarantees.delivery=AT_MOST_ONCE", + }; + } +} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java new file mode 100644 index 0000000..0340c8a --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerConfigTest.java @@ -0,0 +1,97 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.publish.PublishOptions; +import dev.caskeleton.messaging.kafka.KafkaSecurityConfigurer; +import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import java.util.Map; +import org.junit.jupiter.api.Test; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.assertj.AssertableApplicationContext; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; + +/** + * Where the producer connects and how it delivers come from the one broker entry. + * + *

    The producer read {@code spring.kafka.bootstrap-servers} while the compiler required and + * validated {@code app.messaging.brokers..bootstrap-servers}, so a broker was described twice + * and only one description was used. The entry's delivery settings were validated and then replaced + * by hard-coded values. + */ +class KafkaProducerConfigTest { + + private final ApplicationContextRunner runner = + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "app.messaging.brokers.kafka-primary.type=kafka", + // Resolvable names: the client refuses an address it cannot look up at construction. + "app.messaging.brokers.kafka-primary.bootstrap-servers[0]=localhost:9093", + "app.messaging.brokers.kafka-primary.bootstrap-servers[1]=127.0.0.1:9094"); + + @Test + void theProducerConnectsToTheBrokerEntryAddressesNotToSpringKafka() { + runner + .withPropertyValues("spring.kafka.bootstrap-servers=elsewhere:9092") + .run( + context -> + assertThat(producerConfig(context).get("bootstrap.servers")) + .isEqualTo("localhost:9093,127.0.0.1:9094")); + } + + @Test + void theProducerAppliesTheBrokerEntryDeliverySettings() { + runner + .withPropertyValues( + "app.messaging.brokers.kafka-primary.enable-idempotence=false", + "app.messaging.brokers.kafka-primary.acks=1", + "app.messaging.brokers.kafka-primary.max-in-flight-requests-per-connection=3", + "app.messaging.brokers.kafka-primary.delivery-timeout=45s") + .run( + context -> + assertThat(producerConfig(context)) + .extractingByKeys( + "enable.idempotence", + "acks", + "max.in.flight.requests.per.connection", + "delivery.timeout.ms") + .containsExactly(false, "1", 3, 45_000)); + } + + @Test + void theProducerStopsWaitingForMetadataWithinTheDefaultPublishDeadline() { + runner.run( + context -> + assertThat(producerConfig(context).get("max.block.ms")) + .as("the caller's thread waited up to Kafka's own 60s before any deadline applied") + .isEqualTo(PublishOptions.defaults().timeout().toMillis())); + } + + @Test + void aBrokerWithoutAnEntryFailsStartupNamingTheKey() { + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "spring.kafka.bootstrap-servers=localhost:9092") + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .rootCause() + .hasMessageContaining("app.messaging.brokers. entry with type kafka")); + } + + private static Map producerConfig(AssertableApplicationContext context) { + return KafkaMessagingAutoConfiguration.producerConfig( + context.getBean(CompiledMessagingConfiguration.class), + context.getBean(KafkaBrokerCatalog.class), + context.getBeanProvider(BrokerSecurityProfile.class), + context.getBeanProvider(KafkaSecurityConfigurer.class)); + } +} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java new file mode 100644 index 0000000..cfca082 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/KafkaProducerSecurityTest.java @@ -0,0 +1,328 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.kafka.KafkaSecurityConfigurer; +import dev.caskeleton.messaging.security.BrokerSecurityProfile; +import dev.caskeleton.messaging.security.CredentialProvider; +import java.time.Instant; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.concurrent.atomic.AtomicReference; +import javax.security.auth.callback.Callback; +import javax.security.auth.callback.NameCallback; +import javax.security.auth.callback.PasswordCallback; +import org.apache.kafka.common.security.auth.AuthenticateCallbackHandler; +import org.junit.jupiter.api.Test; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.assertj.AssertableApplicationContext; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * What the Kafka producer is actually configured with once {@code app.messaging.security} says how. + * + *

    The security settings were bound, compiled and validated, and {@code KafkaSecurityConfigurer} + * was a bean nothing called: the producer was built from the bootstrap address, serializers, acks + * and idempotence alone, so a broker that passed the TLS and credential checks was reached over + * plaintext with no authentication. + */ +class KafkaProducerSecurityTest { + + // Assembled through the root, as an application is: the import order it fixes is what lets the + // configurer see the credential registry. + private final ApplicationContextRunner runner = + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)) + .withPropertyValues( + "app.messaging.enabled=true", + "app.messaging.broker=kafka", + "app.messaging.brokers.kafka-primary.type=kafka", + "app.messaging.brokers.kafka-primary.bootstrap-servers[0]=localhost:9092"); + + private static final String[] SCRAM_CREDENTIALS = { + "app.messaging.security.kafka-primary.producer.type=SASL_SCRAM", + "app.messaging.security.kafka-primary.producer.credential-id=kafka-producer", + "app.messaging.security.kafka-primary.consumer.type=SASL_SCRAM", + "app.messaging.security.kafka-primary.consumer.credential-id=kafka-consumer", + }; + + private static final String[] MUTUAL_TLS_CREDENTIALS = { + "app.messaging.brokers.kafka-primary.tls-enabled=true", + "app.messaging.security.kafka-primary.producer.type=MUTUAL_TLS", + "app.messaging.security.kafka-primary.producer.credential-id=kafka-producer-tls", + "app.messaging.security.kafka-primary.producer.key-store.location=/etc/kafka/producer.p12", + "app.messaging.security.kafka-primary.consumer.type=MUTUAL_TLS", + "app.messaging.security.kafka-primary.consumer.credential-id=kafka-consumer-tls", + "app.messaging.security.kafka-primary.consumer.key-store.location=/etc/kafka/consumer.p12", + }; + + @Test + void aTlsBrokerWithScramCredentialsConfiguresTheProducerForSaslSsl() { + runner + .withUserConfiguration(Credentials.class) + .withPropertyValues(SCRAM_CREDENTIALS) + .withPropertyValues( + "app.messaging.brokers.kafka-primary.tls-enabled=true", + "app.messaging.brokers.kafka-primary.authentication-enabled=true") + .run( + context -> + assertThat(producerConfig(context)) + .extractingByKeys( + "security.protocol", + "sasl.mechanism", + "ssl.enabled.protocols", + "ssl.endpoint.identification.algorithm") + .containsExactly("SASL_SSL", "SCRAM-SHA-512", "TLSv1.3,TLSv1.2", "https")); + } + + @Test + void aPlaintextBrokerWithPasswordCredentialsConfiguresTheProducerForSaslPlaintext() { + runner + .withUserConfiguration(Credentials.class) + .withPropertyValues( + "app.messaging.brokers.kafka-primary.authentication-enabled=true", + "app.messaging.security.kafka-primary.producer.type=USERNAME_PASSWORD", + "app.messaging.security.kafka-primary.producer.credential-id=kafka-producer", + "app.messaging.security.kafka-primary.consumer.type=USERNAME_PASSWORD", + "app.messaging.security.kafka-primary.consumer.credential-id=kafka-consumer") + .run( + context -> + assertThat(producerConfig(context)) + .extractingByKeys("security.protocol", "sasl.mechanism") + .containsExactly("SASL_PLAINTEXT", "PLAIN")); + } + + @Test + void theProducerAuthenticatesAsTheUsernameTheProviderReturnsNotAsTheCredentialId() { + runner + .withUserConfiguration(Credentials.class) + .withPropertyValues(SCRAM_CREDENTIALS) + .withPropertyValues("app.messaging.brokers.kafka-primary.tls-enabled=true") + .run( + context -> { + // given + Map config = producerConfig(context); + + // when + Answer answer = authenticate(config); + + // then + assertThat(answer).isEqualTo(new Answer(Credentials.USERNAME, Credentials.SECRET)); + }); + } + + @Test + void aRotatedPasswordReachesTheNextAuthenticationWithoutRebuildingTheProducer() { + runner + .withUserConfiguration(RotatingCredentials.class) + .withPropertyValues(SCRAM_CREDENTIALS) + .withPropertyValues("app.messaging.brokers.kafka-primary.tls-enabled=true") + .run( + context -> { + // given + Map config = producerConfig(context); + RotatingCredentials.CURRENT.set("rotated-secret"); + + // when + Answer answer = authenticate(config); + + // then + assertThat(answer.password()).isEqualTo("rotated-secret"); + }); + } + + @Test + void aPrivateCaTrustStoreReachesTheProducer() { + runner + .withUserConfiguration(Credentials.class, StoreFilesAbsent.class) + .withPropertyValues(SCRAM_CREDENTIALS) + .withPropertyValues( + "app.messaging.brokers.kafka-primary.tls-enabled=true", + "app.messaging.security.kafka-primary.trust-store.location=/etc/kafka/private-ca.p12", + "app.messaging.security.kafka-primary.trust-store.password-credential-id=kafka-trust") + .run( + context -> + assertThat(producerConfig(context)) + .extractingByKeys( + "ssl.truststore.location", "ssl.truststore.type", "ssl.truststore.password") + .containsExactly("/etc/kafka/private-ca.p12", "PKCS12", Credentials.SECRET)); + } + + @Test + void mutualTlsPresentsTheProducerKeyStoreUnlockedByItsCredential() { + runner + .withUserConfiguration(Credentials.class, StoreFilesAbsent.class) + .withPropertyValues(MUTUAL_TLS_CREDENTIALS) + .run( + context -> + assertThat(producerConfig(context)) + .extractingByKeys( + "security.protocol", + "ssl.keystore.location", + "ssl.keystore.type", + "ssl.keystore.password") + .containsExactly( + "SSL", "/etc/kafka/producer.p12", "PKCS12", Credentials.SECRET)); + } + + @Test + void mutualTlsWithoutAKeyStoreFailsStartupNamingTheKey() { + runner + .withUserConfiguration(Credentials.class) + .withPropertyValues(MUTUAL_TLS_CREDENTIALS) + .withPropertyValues("app.messaging.security.kafka-primary.producer.key-store.location=") + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .as("a client certificate nobody configured cannot be presented") + .hasMessageContaining( + "app.messaging.security.kafka-primary.producer.key-store: MUTUAL_TLS")); + } + + @Test + void mutualTlsOnABrokerWithoutTlsFailsStartup() { + runner + .withUserConfiguration(Credentials.class) + .withPropertyValues(MUTUAL_TLS_CREDENTIALS) + .withPropertyValues("app.messaging.brokers.kafka-primary.tls-enabled=false") + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .as("a certificate only authenticates inside a TLS handshake") + .hasMessageContaining("authenticates with a client certificate but does not")); + } + + @Test + void aBrokerDeclaringTlsWithoutASecurityEntryFailsStartup() { + runner + .withPropertyValues("app.messaging.brokers.kafka-primary.tls-enabled=true") + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .as("a producer that cannot apply TLS connected over plaintext instead") + .hasMessageContaining("app.messaging.security.kafka-primary")); + } + + @Test + void aSecurityEntryWithoutACredentialProviderFailsStartup() { + runner + .withPropertyValues(SCRAM_CREDENTIALS) + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .as("a producer that cannot resolve its credential connected without one") + .hasMessageContaining("CredentialProvider")); + } + + private static Map producerConfig(AssertableApplicationContext context) { + return KafkaMessagingAutoConfiguration.producerConfig( + context.getBean(CompiledMessagingConfiguration.class), + context.getBean(KafkaBrokerCatalog.class), + context.getBeanProvider(BrokerSecurityProfile.class), + context.getBeanProvider(KafkaSecurityConfigurer.class)); + } + + /** What a SASL client is told when it asks for a name and a password. */ + private record Answer(String name, String password) {} + + /** + * Authenticates the way Kafka's SASL client does: instantiate the configured handler class, + * configure it with the client's own configuration, then ask it for a name and a password. + */ + private static Answer authenticate(Map config) throws Exception { + assertThat(config).containsKey("sasl.client.callback.handler.class"); + Class type = (Class) config.get("sasl.client.callback.handler.class"); + AuthenticateCallbackHandler handler = + (AuthenticateCallbackHandler) type.getDeclaredConstructor().newInstance(); + handler.configure(config, String.valueOf(config.get("sasl.mechanism")), List.of()); + NameCallback name = new NameCallback("name"); + PasswordCallback password = new PasswordCallback("password", false); + handler.handle(new Callback[] {name, password}); + handler.close(); + return new Answer(name.getName(), new String(password.getPassword())); + } + + /** A credential source that answers every id with the same test principal and secret. */ + @Configuration(proxyBeanMethods = false) + static class Credentials { + + static final String SECRET = "test-only-secret"; + + static final String USERNAME = "svc-orders"; + + @Bean + CredentialProvider credentialProvider() { + return new CredentialProvider() { + @Override + public char[] resolve(String credentialId) { + return SECRET.toCharArray(); + } + + @Override + public Optional expiresAt(String credentialId) { + return Optional.empty(); + } + + @Override + public Optional username(String credentialId) { + return Optional.of(USERNAME); + } + }; + } + } + + /** + * Stands in for the producer when the store paths in a test name no real file. + * + *

    A real producer opens its key and trust stores while it is constructed, so a path that does + * not exist fails startup — which is the behaviour a deployment wants, and not what these tests + * are about. They read the configuration the producer would have been built from. + */ + @Configuration(proxyBeanMethods = false) + static class StoreFilesAbsent { + + @Bean + org.apache.kafka.clients.producer.Producer messagingKafkaProducer() { + return new org.apache.kafka.clients.producer.MockProducer<>(); + } + } + + /** A credential store whose secret has already expired, so every resolution reads it again. */ + @Configuration(proxyBeanMethods = false) + static class RotatingCredentials { + + static final AtomicReference CURRENT = new AtomicReference<>("first-secret"); + + @Bean + CredentialProvider credentialProvider() { + CURRENT.set("first-secret"); + return new CredentialProvider() { + @Override + public char[] resolve(String credentialId) { + return CURRENT.get().toCharArray(); + } + + @Override + public Optional expiresAt(String credentialId) { + return Optional.of(Instant.EPOCH); + } + + @Override + public Optional username(String credentialId) { + return Optional.of(Credentials.USERNAME); + } + }; + } + } +} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java index ba6b2c7..d3c02ea 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingAutoConfigurationTest.java @@ -11,11 +11,10 @@ import dev.caskeleton.messaging.api.delivery.OrderingScope; import dev.caskeleton.messaging.api.destination.DestinationKind; import dev.caskeleton.messaging.api.destination.DestinationName; import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.publish.BlockingMessagePublisher; import dev.caskeleton.messaging.api.publish.MessagePublisher; import dev.caskeleton.messaging.api.publish.PublishOptions; import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.ConsumerPolicy; import dev.caskeleton.messaging.policy.DeadLetterPolicy; import dev.caskeleton.messaging.policy.DestinationProfile; @@ -49,8 +48,6 @@ class MessagingAutoConfigurationTest { context -> assertThat(context) .hasSingleBean(ValidatedDestinationRegistry.class) - .hasSingleBean(BlockingMessagePublisher.class) - .hasSingleBean(ReactiveMessagePublisher.class) .hasSingleBean(MessagingAdmissionController.class) .hasSingleBean(GracefulShutdownCoordinator.class)); } @@ -98,30 +95,16 @@ class MessagingAutoConfigurationTest { .hasMessageContaining("exactly one logical broker")); } - @Test - void experimentalAdaptersAndTheBridgeAreOffByDefault() { - runner.run( - context -> { - MessagingSettings properties = context.getBean(MessagingSettings.class); - assertThat(properties.getExperimental().isKafkaShare()).isFalse(); - assertThat(properties.getExperimental().isPulsar()).isFalse(); - assertThat(properties.getExperimental().isNats()).isFalse(); - assertThat(properties.getBridge().isSpringCloudStream()).isFalse(); - }); - } - @Test void propertiesBindFromConfiguration() { runner .withPropertyValues( - "app.messaging.experimental.pulsar=true", "app.messaging.backpressure.global-limit=64", "app.messaging.backpressure.per-destination-limit=8", "app.messaging.shutdown.drain-deadline=15s") .run( context -> { MessagingSettings properties = context.getBean(MessagingSettings.class); - assertThat(properties.getExperimental().isPulsar()).isTrue(); assertThat(properties.getBackpressure().getGlobalLimit()).isEqualTo(64); assertThat(properties.getShutdown().getDrainDeadline()) .isEqualTo(java.time.Duration.ofSeconds(15)); @@ -314,7 +297,7 @@ class MessagingAutoConfigurationTest { retry, DeadLetterPolicy.to(new DestinationName("order-events-dlq")), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, false, false, false); @@ -328,7 +311,9 @@ class MessagingAutoConfigurationTest { org.springframework.boot.autoconfigure.AutoConfigurations.of( MessagingCoreAutoConfiguration.class, KafkaMessagingAutoConfiguration.class)) .withUserConfiguration(PublisherConfiguration.class, AutoCommittingKafkaProfile.class) - .withPropertyValues("spring.kafka.bootstrap-servers=localhost:9092") + .withPropertyValues( + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=localhost:9092") .run( context -> org.assertj.core.api.Assertions.assertThat(context) diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java index 437ca3f..f365f6e 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConfigurationBindingTest.java @@ -67,15 +67,15 @@ class MessagingConfigurationBindingTest { CompiledMessagingConfiguration compiled = context.getBean(CompiledMessagingConfiguration.class); - assertThat(compiled.brokers()).containsKeys("kafka-primary", "rabbit-primary"); - BrokerSettings kafka = compiled.brokers().get("kafka-primary"); - assertThat(kafka.type()).isEqualTo(BrokerSettings.KAFKA); + assertThat(compiled.brokers()).containsOnlyKeys("kafka-primary"); + assertThat(compiled.brokers().get("kafka-primary").type()) + .isEqualTo(BrokerSettings.KAFKA); + KafkaBrokerSettings kafka = + KafkaBrokerSettings.bindAll(context.getEnvironment(), compiled) + .get("kafka-primary"); assertThat(kafka.bootstrapServers()) .containsExactly("broker-1:9093", "broker-2:9093"); assertThat(kafka.enableAutoCommit()).isFalse(); - BrokerSettings rabbit = compiled.brokers().get("rabbit-primary"); - assertThat(rabbit.type()).isEqualTo(BrokerSettings.RABBITMQ); - assertThat(rabbit.quorumQueues()).isTrue(); assertThat(compiled.security()) .singleElement() .satisfies( @@ -175,7 +175,7 @@ class MessagingConfigurationBindingTest { } @Test - void aRabbitSettingOnAKafkaBrokerFailsNamingTheKey() { + void aRabbitMqSettingBindsNothingAndFailsNamingTheKey() { runner .withPropertyValues(minimal()) .withPropertyValues("app.messaging.brokers.kafka-primary.prefetch=32") @@ -184,8 +184,22 @@ class MessagingConfigurationBindingTest { assertThat(context) .hasFailed() .getFailure() - .hasMessageContaining("app.messaging.brokers.kafka-primary.prefetch") - .hasMessageContaining("RabbitMQ setting on a Kafka broker")); + .hasMessageContaining( + "app.messaging.brokers.kafka-primary.prefetch binds nothing")); + } + + @Test + void aRabbitMqBrokerIsRefusedBecauseNoTransportExists() { + runner + .withPropertyValues( + "app.messaging.brokers.rabbit-primary.type=rabbitmq", + "app.messaging.brokers.rabbit-primary.bootstrap-servers[0]=rabbit:5672") + .run( + context -> + assertThat(context) + .hasFailed() + .getFailure() + .hasMessageContaining("app.messaging.brokers.rabbit-primary.type")); } @Test @@ -242,14 +256,32 @@ class MessagingConfigurationBindingTest { .hasMessageContaining("TRUST_ME")); } + @Test + @org.junit.jupiter.api.DisplayName("주소가 없는 Kafka broker는 Kafka starter가 그 키를 이름으로 대며 기동을 거부한다") + void aKafkaBrokerWithoutAnAddressIsRefusedByTheKafkaStarterNamingTheKey() { + // given + ApplicationContextRunner kafka = + runner + .withConfiguration(AutoConfigurations.of(KafkaMessagingAutoConfiguration.class)) + .withPropertyValues("app.messaging.brokers.kafka-primary.type=kafka"); + + // when + kafka.run( + context -> + // then + assertThat(context) + .hasFailed() + .getFailure() + .rootCause() + .hasMessageContaining("app.messaging.brokers.kafka-primary.bootstrap-servers")); + } + @Test void aProductionKafkaBrokerWithoutTransportSecurityFailsStartup() { runner .withConfiguration(AutoConfigurations.of(KafkaMessagingAutoConfiguration.class)) .withPropertyValues(minimal()) - .withPropertyValues( - "spring.kafka.bootstrap-servers=localhost:9092", - "app.messaging.brokers.kafka-primary.production=true") + .withPropertyValues("app.messaging.brokers.kafka-primary.production=true") .run( context -> assertThat(context) diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfigurationTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfigurationTest.java new file mode 100644 index 0000000..b8471f8 --- /dev/null +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingConsumeConfigurationTest.java @@ -0,0 +1,117 @@ +package dev.caskeleton.messaging.autoconfigure; + +import static org.assertj.core.api.Assertions.assertThat; + +import dev.caskeleton.messaging.api.MessageType; +import dev.caskeleton.messaging.api.delivery.HandleResult; +import dev.caskeleton.messaging.api.delivery.MessageDelivery; +import dev.caskeleton.messaging.api.delivery.MessageHandler; +import dev.caskeleton.messaging.api.destination.DestinationName; +import dev.caskeleton.messaging.api.destination.MessageDestination; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionStage; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.springframework.boot.autoconfigure.AutoConfigurations; +import org.springframework.boot.test.context.runner.ApplicationContextRunner; + +/** The consume pipeline turns on with a handler bean and refuses what it cannot honour. */ +class MessagingConsumeConfigurationTest { + + private final ApplicationContextRunner runner = + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingCoreAutoConfiguration.class)) + .withUserConfiguration(MessagingAutoConfigurationTest.PublisherConfiguration.class) + .withPropertyValues( + "app.messaging.brokers.kafka-primary.type=kafka", + "app.messaging.brokers.kafka-primary.bootstrap-servers[0]=broker-1:9093", + "app.messaging.destinations.order-events.broker=kafka-primary", + "app.messaging.destinations.order-events.kind=EVENT_STREAM", + "app.messaging.destinations.order-events.physical.topic=order.events.v1", + "app.messaging.destinations.order-events.schema.message-types[0]=order.created", + "app.messaging.destinations.order-events.guarantees.delivery=AT_MOST_ONCE"); + + @Test + @DisplayName("handler bean이 없으면 consumer를 만들지 않는다") + void withoutAHandlerBeanNoConsumerIsAssembled() { + // when + runner.run( + context -> + // then + assertThat(context).hasNotFailed().doesNotHaveBean(MessageConsumers.class)); + } + + @Test + @DisplayName("handler bean이 있으면 그 destination의 consumer가 group과 함께 조립된다") + void aHandlerBeanAssemblesAConsumerForItsDestinationWithItsGroup() { + // when + runner + .withPropertyValues("app.messaging.destinations.order-events.consumer.group=projection") + .withBean("ordersHandler", MessageHandler.class, MessagingConsumeConfigurationTest::handler) + .run( + context -> { + // then + assertThat(context).hasNotFailed(); + assertThat(context.getBean(MessageConsumers.class).consumers()) + .singleElement() + .satisfies( + consumer -> { + assertThat(consumer.profile().name().value()).isEqualTo("order-events"); + assertThat(consumer.group()).isEqualTo("projection"); + }); + }); + } + + @Test + @DisplayName("consumer.group이 없는 destination을 소비하는 handler는 기동을 거부한다") + void aHandlerForADestinationWithoutAConsumerGroupFailsStartup() { + // when + runner + .withBean("ordersHandler", MessageHandler.class, MessagingConsumeConfigurationTest::handler) + .run( + context -> + // then + assertThat(context) + .hasFailed() + .getFailure() + .rootCause() + .hasMessageContaining("order-events") + .hasMessageContaining("consumer.group")); + } + + @Test + @DisplayName("INBOX_TRANSACTIONAL destination에 IdempotentConsumer가 없으면 기동을 거부한다") + void anInboxDestinationWithoutAnIdempotentConsumerFailsStartup() { + // when + runner + .withPropertyValues( + "app.messaging.destinations.order-events.consumer.group=projection", + "app.messaging.destinations.order-events.guarantees.external-side-effect=" + + "INBOX_TRANSACTIONAL") + .withBean("ordersHandler", MessageHandler.class, MessagingConsumeConfigurationTest::handler) + .run( + context -> + // then + assertThat(context) + .hasFailed() + .getFailure() + .rootCause() + .hasMessageContaining("INBOX_TRANSACTIONAL") + .hasMessageContaining("IdempotentConsumer")); + } + + private static MessageHandler handler() { + return new MessageHandler<>() { + @Override + public MessageDestination destination() { + return new MessageDestination<>( + new DestinationName("order-events"), new MessageType("order.created"), String.class); + } + + @Override + public CompletionStage handle(MessageDelivery delivery) { + return CompletableFuture.completedFuture(HandleResult.success()); + } + }; + } +} diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingEndpointTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingEndpointTest.java index d69b9f2..75c1bfa 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingEndpointTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingEndpointTest.java @@ -9,7 +9,7 @@ import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; import dev.caskeleton.messaging.api.delivery.OrderingScope; import dev.caskeleton.messaging.api.destination.DestinationKind; import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.policy.CapabilityTier; +import dev.caskeleton.messaging.policy.ApiTier; import dev.caskeleton.messaging.policy.ConsumerPolicy; import dev.caskeleton.messaging.policy.DeadLetterPolicy; import dev.caskeleton.messaging.policy.DestinationProfile; @@ -47,7 +47,7 @@ class MessagingEndpointTest { RetryPolicy.none(), DeadLetterPolicy.to(new DestinationName("orders.v1.dlq")), PayloadPolicy.defaults(), - CapabilityTier.M1, + ApiTier.M1, true, true, false); diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java index 6527539..db68ef1 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingOutboxRelayLifecycleTest.java @@ -73,6 +73,55 @@ class MessagingOutboxRelayLifecycleTest { .isFalse(); } + @Test + void aRelayPassThatFailsIsCountedOnTheMeterRegistry() { + new ApplicationContextRunner() + .withConfiguration(AutoConfigurations.of(MessagingReliabilityAutoConfiguration.class)) + .withBean( + io.micrometer.core.instrument.MeterRegistry.class, + io.micrometer.core.instrument.simple.SimpleMeterRegistry::new) + .withBean( + OutboxRepository.class, + () -> + (OutboxRepository) + java.lang.reflect.Proxy.newProxyInstance( + getClass().getClassLoader(), + new Class[] {OutboxRepository.class}, + (proxy, method, arguments) -> { + throw new IllegalStateException("the outbox database is unreachable"); + })) + .withBean( + OutboxEnvelopeFactory.class, + () -> new OutboxEnvelopeFactory(new ProducerId("order-api"))) + .withBean(MessagePublisher.class, () -> new RelayDependencies().messagePublisher()) + .run( + context -> { + // given + io.micrometer.core.instrument.MeterRegistry registry = + context.getBean(io.micrometer.core.instrument.MeterRegistry.class); + + // when + double failed = awaitFailedPasses(registry); + + // then + assertThat(failed) + .as("a pass that failed was swallowed with no log and a counter nothing read") + .isPositive(); + }); + } + + /** Reads the failed-pass counter until the relay's first pass has had time to fail. */ + private static double awaitFailedPasses(io.micrometer.core.instrument.MeterRegistry registry) + throws InterruptedException { + long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(5); + double failed = 0; + while (failed == 0 && System.nanoTime() < deadline) { + Thread.sleep(10); + failed = registry.get("messaging.outbox.relay.failed.passes").functionCounter().count(); + } + return failed; + } + /** The two beans only the application can supply, plus the publisher the platform needs. */ @Configuration(proxyBeanMethods = false) static class RelayDependencies { @@ -122,28 +171,11 @@ class MessagingOutboxRelayLifecycleTest { throw new UnsupportedOperationException("not used by this test"); } - @Override - public List leaseBatch(int batchSize, Duration leaseDuration, Instant now) { - throw new UnsupportedOperationException("not used by this test"); - } - - @Override - public List claimBatch( - String owner, int batchSize, Duration leaseDuration, Instant now) { - throw new UnsupportedOperationException("not used by this test"); - } - @Override public OutboxTransitionResult markPublished(OutboxLease lease, Instant now) { throw new UnsupportedOperationException("not used by this test"); } - @Override - public OutboxTransitionResult markAmbiguous( - OutboxLease lease, String failureCode, Instant now) { - throw new UnsupportedOperationException("not used by this test"); - } - @Override public OutboxTransitionResult markAmbiguous( OutboxLease lease, String failureCode, Instant now, Instant nextAttemptAt) { @@ -170,11 +202,6 @@ class MessagingOutboxRelayLifecycleTest { throw new UnsupportedOperationException("not used by this test"); } - @Override - public int purgePublishedBefore(Instant publishedBefore) { - throw new UnsupportedOperationException("not used by this test"); - } - @Override public int purgePublishedBefore(Instant publishedBefore, int limit) { throw new UnsupportedOperationException("not used by this test"); diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java index 033fb0d..e611c07 100644 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java +++ b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/MessagingStarterOffContractTest.java @@ -15,17 +15,16 @@ import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; import org.springframework.boot.autoconfigure.AutoConfigurations; import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.util.ClassUtils; /** * Off means nothing, and on means exactly one transport (MSG-INT-004). * *

    The starter registered five auto-configurations and none carried a messaging master condition, * so an application acquired the platform by having the starter on its classpath. Worse, provider - * selection was {@code @ConditionalOnClass} while this starter depends on {@code messaging-kafka} - * and {@code messaging-rabbit}, each of which exposes its client library as {@code api} — so both - * conditions are true for **every** adopter, always. Selection by classpath could not select; it - * assembled both and let a {@code @ConditionalOnMissingBean} race decide where messages went. + * selection was {@code @ConditionalOnClass} while this starter depended on two broker adapters, + * each exposing its client library as {@code api} — so both conditions were true for **every** + * adopter, always. Selection by classpath could not select; it assembled both and let a + * {@code @ConditionalOnMissingBean} race decide where messages went. * *

    The cases below are the replacement contract. The off case is deliberately checked on bean * names and live threads rather than on a handful of types: a platform that leaves a scheduler @@ -38,40 +37,43 @@ class MessagingStarterOffContractTest { .withConfiguration(AutoConfigurations.of(MessagingPlatformRootAutoConfiguration.class)); @Test - @DisplayName( - "both client libraries really are on this classpath, so the old condition could not select") - void bothClientLibrariesArePresent() { - ClassLoader loader = getClass().getClassLoader(); - - assertThat(ClassUtils.isPresent("org.apache.kafka.clients.producer.Producer", loader)) - .as("messaging-kafka exposes kafka-clients as api") - .isTrue(); - assertThat(ClassUtils.isPresent("com.rabbitmq.client.Channel", loader)) - .as("messaging-rabbit exposes spring-rabbit as api") - .isTrue(); + @DisplayName("present without the switch is on") + void presentWithoutTheSwitchIsOn() { + runner + .withPropertyValues( + "app.messaging.broker=kafka", + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=localhost:9092") + .run( + context -> { + assertThat(context).hasNotFailed(); + assertThat(context).hasSingleBean(MessagePublisher.class); + }); } @Test @DisplayName("with the switch off there are no platform beans") void offMeansNoBeans() { - runner.run( - context -> { - assertThat(context).hasNotFailed(); + runner + .withPropertyValues("app.messaging.enabled=false") + .run( + context -> { + assertThat(context).hasNotFailed(); - Set platformBeans = - java.util.Arrays.stream(context.getBeanDefinitionNames()) - .filter( - name -> { - Class type = context.getBeanFactory().getType(name, false); - return type != null - && type.getName().startsWith("dev.caskeleton.messaging."); - }) - .collect(Collectors.toSet()); + Set platformBeans = + java.util.Arrays.stream(context.getBeanDefinitionNames()) + .filter( + name -> { + Class type = context.getBeanFactory().getType(name, false); + return type != null + && type.getName().startsWith("dev.caskeleton.messaging."); + }) + .collect(Collectors.toSet()); - assertThat(platformBeans) - .as("off is a structural fact, not a set of individually-conditioned beans") - .isEmpty(); - }); + assertThat(platformBeans) + .as("off is a structural fact, not a set of individually-conditioned beans") + .isEmpty(); + }); } @Test @@ -79,7 +81,9 @@ class MessagingStarterOffContractTest { void offMeansNoThreads() { Set before = liveThreadNames(); - runner.run(context -> assertThat(context).hasNotFailed()); + runner + .withPropertyValues("app.messaging.enabled=false") + .run(context -> assertThat(context).hasNotFailed()); assertThat(newMessagingThreadsSince(before)) .as("a bean that exists but was never started is acceptable; a running thread is not") @@ -111,63 +115,25 @@ class MessagingStarterOffContractTest { .hasFailed() .getFailure() .rootCause() - .hasMessageContaining("registered: [kafka, rabbit]")); + .hasMessageContaining("registered: [kafka]")); } @Test - @DisplayName("selecting Kafka assembles Kafka and not Rabbit") + @DisplayName("selecting Kafka assembles Kafka") void selectingKafkaAssemblesOnlyKafka() { withAPublisher() .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=localhost:9092") + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=localhost:9092") .run( context -> { assertThat(context).hasNotFailed(); assertThat(context).hasSingleBean(KafkaMessagingAutoConfiguration.class); - assertThat(context) - .as("both client libraries are present, so only the property can be selecting") - .doesNotHaveBean(RabbitMessagingAutoConfiguration.class); }); } - @Test - @DisplayName("selecting Rabbit is refused, and the message says the transport is not finished") - void selectingRabbitIsRefused() { - // This test used to run under `withAPublisher()` and assert the context started. The fake - // MessagePublisher tripped @ConditionalOnMissingBean and removed the very bean whose missing - // dependency is the defect — so a configuration that cannot start in any deployment passed as - // "assembles Rabbit and not Kafka". Without the fake, the real behaviour appears. - // - // Rabbit ships validators and a security configurer and no MessagingTransport, because its - // native channel publisher has only test implementations. That is refused at selection now, - // where the message can say so, rather than surfacing as "No qualifying bean of type - // MessagingTransport" from inside the core configuration. - runner - .withPropertyValues("app.messaging.enabled=true", "app.messaging.broker=rabbit") - .run( - context -> { - assertThat(context).hasFailed(); - assertThat(context.getStartupFailure()) - .rootCause() - .hasMessageContaining("app.messaging.broker=rabbit") - .hasMessageContaining("no MessagingTransport") - .hasMessageContaining("kafka"); - }); - } - - @Test - @DisplayName("a fake publisher does not make an unassemblable transport look assemblable") - void aFakePublisherDoesNotHideAnUnassemblableTransport() { - // The guard on the test above. Supplying a MessagePublisher removes the dependency that fails, - // so if the refusal ever moves back to bean wiring this case goes green while the deployment - // stays broken. Selection-time refusal is what makes it fail either way. - withAPublisher() - .withPropertyValues("app.messaging.enabled=true", "app.messaging.broker=rabbit") - .run(context -> assertThat(context).hasFailed()); - } - @Test @DisplayName("MSG-INT-003: a selected transport now assembles a publisher") void aSelectedTransportAssemblesAPublisher() { @@ -175,7 +141,8 @@ class MessagingStarterOffContractTest { .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=localhost:9092") + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=localhost:9092") .run( context -> { assertThat(context) @@ -202,7 +169,8 @@ class MessagingStarterOffContractTest { .withPropertyValues( "app.messaging.enabled=true", "app.messaging.broker=kafka", - "spring.kafka.bootstrap-servers=localhost:9092") + "app.messaging.brokers.kafka.type=kafka", + "app.messaging.brokers.kafka.bootstrap-servers[0]=localhost:9092") .run( context -> { assertThat(context) @@ -240,7 +208,7 @@ class MessagingStarterOffContractTest { .hasFailed() .getFailure() .rootCause() - .hasMessageContaining("spring.kafka.bootstrap-servers")); + .hasMessageContaining("app.messaging.brokers.")); } /** @@ -279,8 +247,6 @@ class MessagingStarterOffContractTest { name -> { String lower = name.toLowerCase(java.util.Locale.ROOT); return lower.contains("kafka") - || lower.contains("rabbit") - || lower.contains("amqp") || lower.contains("messaging") || lower.contains("outbox"); }) diff --git a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java b/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java deleted file mode 100644 index 21931ea..0000000 --- a/messaging/messaging-spring-boot-starter/src/test/java/dev/caskeleton/messaging/autoconfigure/ReactiveFacadeTest.java +++ /dev/null @@ -1,134 +0,0 @@ -package dev.caskeleton.messaging.autoconfigure; - -import static org.assertj.core.api.Assertions.assertThat; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageEnvelope; -import dev.caskeleton.messaging.api.MessageId; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.ProducerId; -import dev.caskeleton.messaging.api.SchemaVersion; -import dev.caskeleton.messaging.api.TraceContext; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.destination.MessageDestination; -import dev.caskeleton.messaging.api.header.MessageHeaders; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.MessagePublisher; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishOptions; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import java.time.Duration; -import java.time.Instant; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicInteger; -import org.junit.jupiter.api.Test; -import reactor.core.publisher.Mono; -import reactor.test.StepVerifier; - -class ReactiveFacadeTest { - - private static final MessageDestination ORDERS = - new MessageDestination<>( - new DestinationName("order-events"), new MessageType("order.created"), String.class); - - private final AtomicInteger publishes = new AtomicInteger(); - - @Test - void nothingIsPublishedUntilTheMonoIsSubscribed() { - ReactiveMessagePublisher publisher = new DefaultReactiveMessagePublisher(counting()); - - Mono mono = publisher.publish(ORDERS, envelope(), PublishOptions.defaults()); - - assertThat(publishes.get()).as("assembly must not publish").isZero(); - - mono.block(); - - assertThat(publishes.get()).isEqualTo(1); - } - - @Test - void eachSubscriptionPublishesOnce() { - ReactiveMessagePublisher publisher = new DefaultReactiveMessagePublisher(counting()); - - Mono mono = publisher.publish(ORDERS, envelope(), PublishOptions.defaults()); - mono.block(); - mono.block(); - - assertThat(publishes.get()).isEqualTo(2); - } - - @Test - void theOutcomeIsEmittedAndTheMonoCompletes() { - ReactiveMessagePublisher publisher = new DefaultReactiveMessagePublisher(counting()); - - StepVerifier.create(publisher.publish(ORDERS, envelope(), PublishOptions.defaults())) - .assertNext( - result -> assertThat(result.completion()).isEqualTo(PublishCompletion.CONFIRMED)) - .verifyComplete(); - } - - @Test - void aFailedPublishPropagatesAsAnErrorSignal() { - MessagePublisher failing = - new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, - MessageEnvelope message, - PublishOptions options) { - return CompletableFuture.failedFuture(new IllegalStateException("broker down")); - } - }; - - StepVerifier.create( - new DefaultReactiveMessagePublisher(failing) - .publish(ORDERS, envelope(), PublishOptions.defaults())) - .expectErrorMessage("broker down") - .verify(); - } - - private MessagePublisher counting() { - return new MessagePublisher() { - @Override - public CompletionStage publish( - MessageDestination destination, MessageEnvelope message, PublishOptions options) { - publishes.incrementAndGet(); - return CompletableFuture.completedFuture(confirmed()); - } - }; - } - - private static PublishResult confirmed() { - return new PublishResult( - PublishCompletion.CONFIRMED, - PublishEvidence.confirmed(ConfirmationLevel.REPLICATION_OR_PERSISTENCE_ACK), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ofMillis(2), - Optional.empty()); - } - - private static MessageEnvelope envelope() { - return new MessageEnvelope<>( - MessageId.newId(), - new MessageType("order.created"), - new SchemaVersion(1), - Instant.parse("2026-08-10T09:15:00Z"), - Optional.of(Instant.parse("2026-08-10T09:15:00Z")), - new ProducerId("order-api"), - Optional.empty(), - Optional.empty(), - ContentType.JSON, - Optional.empty(), - Optional.empty(), - Optional.empty(), - TraceContext.none(), - MessageHeaders.empty(), - "payload"); - } -} diff --git a/messaging/messaging-spring-boot-starter/src/test/resources/application-valid.yml b/messaging/messaging-spring-boot-starter/src/test/resources/application-valid.yml index 8101b6e..b1894f0 100644 --- a/messaging/messaging-spring-boot-starter/src/test/resources/application-valid.yml +++ b/messaging/messaging-spring-boot-starter/src/test/resources/application-valid.yml @@ -1,16 +1,6 @@ # A configuration the platform accepts. -# -# Every experimental adapter is off and the bridge is off, which is the shipped default: an -# adapter whose contract suite is still being proven must never become load-bearing because a -# configuration defaulted it on. backend: messaging: - experimental: - kafka-share: false - pulsar: false - nats: false - bridge: - spring-cloud-stream: false backpressure: # The per-destination limit stays below the global one; a per-destination limit at or above # the global limit would let one destination consume every slot in the process. diff --git a/messaging/messaging-spring-cloud-stream-bridge/build.gradle b/messaging/messaging-spring-cloud-stream-bridge/build.gradle deleted file mode 100644 index d104567..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/build.gradle +++ /dev/null @@ -1,10 +0,0 @@ -plugins { - id 'ca.platform-module' -} - -dependencies { - api project(':messaging:messaging-core-api') - api project(':messaging:messaging-policy') - api project(':messaging:messaging-transport-spi') - -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingCapabilityReport.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingCapabilityReport.java deleted file mode 100644 index fb34a1a..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingCapabilityReport.java +++ /dev/null @@ -1,88 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import java.util.List; -import java.util.Objects; - -/** - * States, per binding, which platform guarantees the bridge does not provide. - * - *

    An explicit report rather than silence. The binder does provide retry and dead-lettering of - * its own, so a binding looks like it has them; what it does not have is the platform's versions — - * bounded attempts under the destination's retry policy, and a dead-letter publish confirmed before - * the source is settled. An operator comparing a bridged binding to a native one needs that - * difference written down, because nothing at runtime will show it. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - * @param publishEvidenceAvailable whether the bridge can report broker publish evidence - * @param platformRetryApplied whether the destination's retry policy is enforced - * @param confirmedDeadLetterApplied whether dead-letter publishes are confirmed before settlement - * @param orderingEnforced whether the destination's ordering scope is enforced - */ -public record BindingCapabilityReport( - DestinationName destination, - String bindingName, - boolean publishEvidenceAvailable, - boolean platformRetryApplied, - boolean confirmedDeadLetterApplied, - boolean orderingEnforced) { - - public BindingCapabilityReport { - Objects.requireNonNull(destination, "destination must not be null"); - if (bindingName == null || bindingName.isBlank()) { - throw new IllegalArgumentException("bindingName must not be blank"); - } - } - - /** - * Returns the report for a binding served through the bridge. - * - *

    Every platform guarantee is false, because the bridge hands the message to the binder and - * the binder is what decides these. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - * @return the report - */ - public static BindingCapabilityReport bridged(DestinationName destination, String bindingName) { - return new BindingCapabilityReport(destination, bindingName, false, false, false, false); - } - - /** - * Returns the guarantees this binding does not carry, as sanitized text. - * - * @return the gaps, empty when nothing is missing - */ - public List gaps() { - List gaps = new java.util.ArrayList<>(); - if (!publishEvidenceAvailable) { - gaps.add( - "publish evidence: the binder reports a send, not a broker confirmation, so an " - + "ambiguous publish is indistinguishable from a confirmed one"); - } - if (!platformRetryApplied) { - gaps.add( - "retry: the binder's own retry runs instead of the destination's retry policy, with its " - + "own attempt budget and backoff"); - } - if (!confirmedDeadLetterApplied) { - gaps.add( - "dead letter: the binder settles the source without waiting for the dead-letter publish " - + "to confirm, so a dead-letter outage loses the message"); - } - if (!orderingEnforced) { - gaps.add("ordering: the binder's concurrency settings decide ordering, not the profile"); - } - return List.copyOf(gaps); - } - - /** - * Reports whether this binding carries every platform guarantee. - * - * @return true when nothing is missing - */ - public boolean isFullyGuaranteed() { - return gaps().isEmpty(); - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingProfileValidator.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingProfileValidator.java deleted file mode 100644 index cb3d159..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/BindingProfileValidator.java +++ /dev/null @@ -1,97 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.util.Objects; -import java.util.regex.Pattern; - -/** - * Checks a binding against its destination before the bridge is wired. - * - *

    Runs on top of {@link StreamBridgePolicyGuard}: the guard decides whether a destination is - * eligible for the bridge at all, and this decides whether a particular binding is configured - * coherently for it. - * - *

    The binder's extended properties are the sharp edge. Stream lets a binding override the - * serializer, the acknowledgement mode, and the concurrency, and each of those silently replaces - * something the destination profile already decided. Rather than merging the two — which produces a - * configuration nobody can read — a conflicting extended property is rejected and the operator is - * told which side to remove. - */ -public final class BindingProfileValidator { - - private static final Pattern BINDING_NAME = Pattern.compile("[a-zA-Z][a-zA-Z0-9-]{0,63}"); - - /** Extended properties that would override a platform decision. */ - private static final java.util.Set CONFLICTING_PROPERTIES = - java.util.Set.of( - "autoBindDlq", - "republishToDlq", - "maxAttempts", - "backOffInitialInterval", - "autoCommitOffset", - "ackMode", - "useNativeEncoding", - "contentType"); - - private final StreamBridgePolicyGuard guard = new StreamBridgePolicyGuard(); - - /** - * Validates one binding. - * - * @param profile the destination profile - * @param bindingName the Stream binding name - * @param extendedProperties the binder extended properties configured for the binding - * @param enabled whether the bridge is switched on - * @return the capability report for the accepted binding - */ - public BindingCapabilityReport validate( - DestinationProfile profile, - String bindingName, - java.util.Map extendedProperties, - boolean enabled) { - Objects.requireNonNull(profile, "profile must not be null"); - Objects.requireNonNull(extendedProperties, "extendedProperties must not be null"); - - guard.validate(profile, enabled); - - if (bindingName == null || !BINDING_NAME.matcher(bindingName).matches()) { - throw new MessagingConfigurationException( - "INVALID_BINDING_NAME", - "binding name must match %s: %s".formatted(BINDING_NAME.pattern(), bindingName)); - } - - for (String property : extendedProperties.keySet()) { - if (CONFLICTING_PROPERTIES.contains(property)) { - throw new MessagingConfigurationException( - "BINDING_OVERRIDES_PLATFORM_POLICY", - "binding %s sets %s, which overrides what the %s destination profile already decides; " - .formatted(bindingName, property, profile.name().value()) - + "remove it from the binding or move the destination to the native adapter"); - } - } - - if (profile.production()) { - throw new MessagingConfigurationException( - "BRIDGE_ON_PRODUCTION_DESTINATION", - "destination %s is marked production; the bridge does not carry the platform's publish " - .formatted(profile.name().value()) - + "evidence, retry, or confirmed dead lettering"); - } - - return BindingCapabilityReport.bridged(profile.name(), bindingName); - } - - /** - * Returns the report for a destination that is not bridged at all. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - * @return a report stating every guarantee holds - */ - public static BindingCapabilityReport nativeAdapter( - DestinationName destination, String bindingName) { - return new BindingCapabilityReport(destination, bindingName, true, true, true, true); - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/MessagingBindingBridge.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/MessagingBindingBridge.java deleted file mode 100644 index 49be29e..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/MessagingBindingBridge.java +++ /dev/null @@ -1,34 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.destination.DestinationName; - -/** - * Connects a logical destination to a Spring Cloud Stream binding. - * - *

    The bridge is an interoperability seam, not a second messaging API. Its whole reason to exist - * is that a service already has Stream bindings and needs to reach the same destinations without a - * rewrite. - * - *

    Binder semantics are never promoted to platform guarantees. Stream's binder has its own retry, - * its own dead-letter, and its own acknowledgement mode, and they look enough like the platform's - * to be mistaken for them — so a destination that actually relies on the platform's versions is - * refused by {@link StreamBridgePolicyGuard} rather than served with the binder's. - */ -public interface MessagingBindingBridge { - - /** - * Binds a Stream output to a logical destination. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - */ - void bindPublisher(DestinationName destination, String bindingName); - - /** - * Binds a Stream input to a logical destination. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - */ - void bindConsumer(DestinationName destination, String bindingName); -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamConsumerBridge.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamConsumerBridge.java deleted file mode 100644 index e0b304f..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamConsumerBridge.java +++ /dev/null @@ -1,87 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import java.util.Map; -import java.util.Objects; -import java.util.concurrent.ConcurrentHashMap; - -/** - * Receives from a Spring Cloud Stream input binding and hands the message to a platform handler. - * - *

    Settlement stays with the binder. The bridge cannot acknowledge, retry, or dead-letter a - * message itself, because Stream's binder already owns the acknowledgement for that binding and two - * things settling one message is worse than either doing it alone. - * - *

    What the bridge does own is the translation and the honesty about it: a handler failure is - * rethrown so the binder's error channel sees it, rather than being converted into a platform - * {@code HandleResult} that nothing downstream would act on. - */ -public final class SpringCloudStreamConsumerBridge { - - private final Map handlers = new ConcurrentHashMap<>(); - private final Map destinations = new ConcurrentHashMap<>(); - - /** - * Registers a handler for a binding. - * - * @param destination the logical destination - * @param bindingName the Stream binding name - * @param handler the handler to invoke - */ - public void register(DestinationName destination, String bindingName, BridgedHandler handler) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(handler, "handler must not be null"); - if (bindingName == null || bindingName.isBlank()) { - throw new IllegalArgumentException("bindingName must not be blank"); - } - handlers.put(bindingName, handler); - destinations.put(bindingName, destination); - } - - /** - * Dispatches one message received on a binding. - * - * @param bindingName the Stream binding name - * @param payload the encoded payload - * @param headers the received headers - * @throws MessagingConfigurationException when nothing is registered for the binding - */ - public void dispatch(String bindingName, byte[] payload, Map headers) { - Objects.requireNonNull(payload, "payload must not be null"); - Objects.requireNonNull(headers, "headers must not be null"); - - BridgedHandler handler = handlers.get(bindingName); - DestinationName destination = destinations.get(bindingName); - if (handler == null || destination == null) { - throw new MessagingConfigurationException( - "NO_BRIDGED_HANDLER", "no handler is registered for Stream binding " + bindingName); - } - // Not caught. The binder's error channel is what retries and dead-letters this binding, and - // swallowing the failure here would acknowledge a message nothing handled. - handler.handle(destination, payload, headers); - } - - /** - * Returns how many bindings have a handler. - * - * @return the registered binding count - */ - public int registeredBindings() { - return handlers.size(); - } - - /** A platform handler reached through a Stream binding. */ - @FunctionalInterface - public interface BridgedHandler { - - /** - * Handles one bridged message. - * - * @param destination the logical destination - * @param payload the encoded payload - * @param headers the received headers - */ - void handle(DestinationName destination, byte[] payload, Map headers); - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamPublisherBridge.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamPublisherBridge.java deleted file mode 100644 index 059144f..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/SpringCloudStreamPublisherBridge.java +++ /dev/null @@ -1,156 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishEvidence; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.RoutingOutcome; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.time.Duration; -import java.util.Map; -import java.util.Objects; -import java.util.Optional; -import java.util.concurrent.ConcurrentHashMap; - -/** - * Framework-neutral publish evidence bridge for a future Spring Cloud Stream adapter. - * - *

    Despite the historical module/class name, this type does not itself depend on Spring Cloud - * Stream and does not auto-configure {@code StreamBridge}. A product integrating Spring Cloud - * Stream supplies the {@link ChannelSend} seam explicitly. Keeping that limitation in the type - * contract prevents this module from being mistaken for a shipped Spring Cloud Stream integration. - * - *

    Sends through a Spring Cloud Stream output binding and reports what that actually proves. - * - *

    The result is deliberately {@code AMBIGUOUS} rather than {@code CONFIRMED}. A Stream {@code - * send} returns a boolean from the message channel — it says the binder accepted the message, not - * that a broker did. Reporting that as confirmed would put the platform's strongest word on the - * binder's weakest evidence, and a caller reading {@code CONFIRMED} would stop worrying about a - * message that may never have left the process. - * - *

    A caller that needs real publish evidence has to use the native adapter. That is the honest - * trade the bridge exists to make visible. - */ -public final class SpringCloudStreamPublisherBridge implements MessagingBindingBridge { - - private final Map outputBindings = new ConcurrentHashMap<>(); - private final Map inputBindings = new ConcurrentHashMap<>(); - private final ChannelSend send; - - /** - * Creates a publisher bridge. - * - * @param send the Stream channel send operation - */ - public SpringCloudStreamPublisherBridge(ChannelSend send) { - this.send = Objects.requireNonNull(send, "send must not be null"); - } - - @Override - public void bindPublisher(DestinationName destination, String bindingName) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(bindingName, "bindingName must not be null"); - outputBindings.put(destination, bindingName); - } - - @Override - public void bindConsumer(DestinationName destination, String bindingName) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(bindingName, "bindingName must not be null"); - inputBindings.put(destination, bindingName); - } - - /** - * Publishes through the bound output binding. - * - * @param destination the logical destination - * @param payload the encoded payload - * @param headers the headers to carry - * @return the outcome, never stronger than the binder can prove - */ - public PublishResult publish( - DestinationName destination, byte[] payload, Map headers) { - Objects.requireNonNull(destination, "destination must not be null"); - Objects.requireNonNull(payload, "payload must not be null"); - Objects.requireNonNull(headers, "headers must not be null"); - - String binding = outputBindings.get(destination); - if (binding == null) { - throw new MessagingConfigurationException( - "NO_OUTPUT_BINDING", "no Stream output binding is bound to " + destination.value()); - } - - boolean accepted = send.send(binding, payload, headers); - return accepted ? acceptedByBinder() : rejectedByBinder(); - } - - /** - * Returns the binding bound to a destination for consuming. - * - * @param destination the logical destination - * @return the binding name, when one is bound - */ - public Optional consumerBinding(DestinationName destination) { - return Optional.ofNullable(inputBindings.get(destination)); - } - - /** - * Returns the binding bound to a destination for publishing. - * - * @param destination the logical destination - * @return the binding name, when one is bound - */ - public Optional publisherBinding(DestinationName destination) { - return Optional.ofNullable(outputBindings.get(destination)); - } - - private static PublishResult acceptedByBinder() { - return new PublishResult( - // The binder accepted it; no broker has confirmed anything. AMBIGUOUS is the only honest - // answer, and it is what makes the bridge's weaker guarantee visible to the caller. - PublishCompletion.AMBIGUOUS, - new PublishEvidence( - true, TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED, false, ConfirmationLevel.NONE), - RoutingOutcome.UNKNOWN, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of( - dev.caskeleton.messaging.api.error.FailureDescriptor.of( - dev.caskeleton.messaging.api.error.FailureCategory.AMBIGUOUS, - "STREAM_BRIDGE_NO_BROKER_EVIDENCE", - "the Stream binder accepted the message; no broker confirmation is available"))); - } - - private static PublishResult rejectedByBinder() { - return new PublishResult( - PublishCompletion.REJECTED, - PublishEvidence.notTransmitted(), - RoutingOutcome.NOT_APPLICABLE, - Optional.empty(), - 1, - Duration.ZERO, - Optional.of( - dev.caskeleton.messaging.api.error.FailureDescriptor.of( - dev.caskeleton.messaging.api.error.FailureCategory.TRANSIENT_INFRASTRUCTURE, - "STREAM_BRIDGE_SEND_REFUSED", - "the Stream channel refused the message"))); - } - - /** The Stream channel send, isolated so the bridge is testable without a binder. */ - @FunctionalInterface - public interface ChannelSend { - - /** - * Sends one message to a binding. - * - * @param bindingName the Stream binding name - * @param payload the encoded payload - * @param headers the headers to carry - * @return whether the channel accepted the message - */ - boolean send(String bindingName, byte[] payload, Map headers); - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/StreamBridgePolicyGuard.java b/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/StreamBridgePolicyGuard.java deleted file mode 100644 index e676f35..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/main/java/dev/caskeleton/messaging/streambridge/StreamBridgePolicyGuard.java +++ /dev/null @@ -1,52 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.DestinationProfile; -import java.util.Objects; - -/** - * Keeps the optional Spring Cloud Stream bridge inside platform policy. - * - *

    The bridge exists for interoperability with existing Spring Cloud Stream bindings, and its - * risk is specific: Stream owns its own binder configuration, so a binding can quietly acquire its - * own serializer, its own error handling, and its own acknowledgement mode — none of which the - * destination profile knows about. - * - *

    So the bridge is only permitted where the platform's guarantees are not the thing being relied - * on: a destination that declares an ordering scope, a retry policy, or a dead letter destination - * must go through the native adapter, where those are actually enforced. - */ -public final class StreamBridgePolicyGuard { - - /** - * Validates that a destination may be served through the bridge. - * - * @param profile the destination profile - * @param enabled whether the bridge is switched on - */ - public void validate(DestinationProfile profile, boolean enabled) { - Objects.requireNonNull(profile, "profile must not be null"); - - if (!enabled) { - throw new MessagingConfigurationException( - "STREAM_BRIDGE_DISABLED", - "the Spring Cloud Stream bridge is optional and disabled unless " - + "backend.messaging.bridge.spring-cloud-stream=true"); - } - if (profile.isOrdered()) { - throw new MessagingConfigurationException( - "STREAM_BRIDGE_ORDERING_UNSUPPORTED", - "an ordered destination must use the native adapter: " + profile.name().value()); - } - if (profile.retry().mode() != dev.caskeleton.messaging.policy.RetryMode.NONE) { - throw new MessagingConfigurationException( - "STREAM_BRIDGE_RETRY_UNSUPPORTED", - "the bridge does not apply the platform retry policy: " + profile.name().value()); - } - if (profile.deadLetter().enabled()) { - throw new MessagingConfigurationException( - "STREAM_BRIDGE_DLQ_UNSUPPORTED", - "the bridge does not apply confirmed dead lettering: " + profile.name().value()); - } - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BindingProfileValidatorTest.java b/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BindingProfileValidatorTest.java deleted file mode 100644 index 1f5f403..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BindingProfileValidatorTest.java +++ /dev/null @@ -1,164 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatCode; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.ContentType; -import dev.caskeleton.messaging.api.MessageType; -import dev.caskeleton.messaging.api.delivery.DeliveryGuarantee; -import dev.caskeleton.messaging.api.delivery.ExternalSideEffectGuarantee; -import dev.caskeleton.messaging.api.delivery.OrderingScope; -import dev.caskeleton.messaging.api.destination.DestinationKind; -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.policy.CapabilityTier; -import dev.caskeleton.messaging.policy.ConsumerPolicy; -import dev.caskeleton.messaging.policy.DeadLetterPolicy; -import dev.caskeleton.messaging.policy.DestinationProfile; -import dev.caskeleton.messaging.policy.PayloadPolicy; -import dev.caskeleton.messaging.policy.PhysicalDestination; -import dev.caskeleton.messaging.policy.ProducerPolicy; -import dev.caskeleton.messaging.policy.RetryPolicy; -import dev.caskeleton.messaging.policy.SchemaPolicy; -import dev.caskeleton.messaging.schema.SchemaCompatibility; -import java.util.Map; -import java.util.Set; -import org.junit.jupiter.api.Test; - -class BindingProfileValidatorTest { - - private final BindingProfileValidator validator = new BindingProfileValidator(); - - private static DestinationProfile profile( - OrderingScope ordering, RetryPolicy retry, DeadLetterPolicy deadLetter, boolean production) { - return new DestinationProfile( - new DestinationName("orders.v1"), - "kafka-primary", - DestinationKind.EVENT_STREAM, - PhysicalDestination.kafkaTopic("orders.v1"), - new SchemaPolicy( - ContentType.JSON, - SchemaCompatibility.BACKWARD, - Set.of(new MessageType("order.created"))), - DeliveryGuarantee.AT_LEAST_ONCE, - ordering, - ExternalSideEffectGuarantee.IDEMPOTENCY_REQUIRED, - ProducerPolicy.defaults(), - ConsumerPolicy.defaults("orders"), - retry, - deadLetter, - PayloadPolicy.defaults(), - CapabilityTier.M1, - production, - false, - false); - } - - private static DestinationProfile bridgeable() { - return profile(OrderingScope.NONE, RetryPolicy.none(), DeadLetterPolicy.disabled(), false); - } - - @Test - void aBridgeableDestinationIsAccepted() { - BindingCapabilityReport report = validator.validate(bridgeable(), "orders-out", Map.of(), true); - - assertThat(report.bindingName()).isEqualTo("orders-out"); - } - - @Test - void aDisabledBridgeRefusesEverything() { - assertThatThrownBy(() -> validator.validate(bridgeable(), "orders-out", Map.of(), false)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("disabled"); - } - - @Test - void anOrderedDestinationMustUseTheNativeAdapter() { - assertThatThrownBy( - () -> - validator.validate( - profile( - OrderingScope.KEY, RetryPolicy.none(), DeadLetterPolicy.disabled(), false), - "orders-out", - Map.of(), - true)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("ordered"); - } - - @Test - void aDestinationWithADeadLetterMustUseTheNativeAdapter() { - assertThatThrownBy( - () -> - validator.validate( - profile( - OrderingScope.NONE, - RetryPolicy.none(), - DeadLetterPolicy.to(new DestinationName("orders.v1.dlq")), - false), - "orders-out", - Map.of(), - true)) - .as("the binder settles the source without confirming the dead-letter publish") - .isInstanceOf(MessagingConfigurationException.class); - } - - @Test - void aProductionDestinationIsRefused() { - assertThatThrownBy( - () -> - validator.validate( - profile( - OrderingScope.NONE, RetryPolicy.none(), DeadLetterPolicy.disabled(), true), - "orders-out", - Map.of(), - true)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("production"); - } - - @Test - void aBinderPropertyThatOverridesPlatformPolicyIsRejectedRatherThanMerged() { - assertThatThrownBy( - () -> validator.validate(bridgeable(), "orders-out", Map.of("maxAttempts", "10"), true)) - .as("merging the two produces a configuration nobody can read") - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("maxAttempts"); - } - - @Test - void aHarmlessBinderPropertyIsAllowedThrough() { - assertThatCode( - () -> validator.validate(bridgeable(), "orders-out", Map.of("concurrency", "4"), true)) - .doesNotThrowAnyException(); - } - - @Test - void anInvalidBindingNameIsRejected() { - assertThatThrownBy(() -> validator.validate(bridgeable(), "9-bad name!", Map.of(), true)) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("binding name"); - } - - @Test - void aBridgedBindingReportsEveryGuaranteeItDoesNotCarry() { - BindingCapabilityReport report = validator.validate(bridgeable(), "orders-out", Map.of(), true); - - assertThat(report.isFullyGuaranteed()).isFalse(); - assertThat(report.gaps()).hasSize(4); - assertThat(String.join(" ", report.gaps())) - .contains("publish evidence") - .contains("retry") - .contains("dead letter") - .contains("ordering"); - } - - @Test - void aNativeBindingReportsNoGaps() { - assertThat( - BindingProfileValidator.nativeAdapter(new DestinationName("orders.v1"), "orders-out") - .isFullyGuaranteed()) - .isTrue(); - } -} diff --git a/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BridgePublishEvidenceTest.java b/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BridgePublishEvidenceTest.java deleted file mode 100644 index 411f04f..0000000 --- a/messaging/messaging-spring-cloud-stream-bridge/src/test/java/dev/caskeleton/messaging/streambridge/BridgePublishEvidenceTest.java +++ /dev/null @@ -1,131 +0,0 @@ -package dev.caskeleton.messaging.streambridge; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -import dev.caskeleton.messaging.api.destination.DestinationName; -import dev.caskeleton.messaging.api.error.MessagingConfigurationException; -import dev.caskeleton.messaging.api.publish.ConfirmationLevel; -import dev.caskeleton.messaging.api.publish.PublishCompletion; -import dev.caskeleton.messaging.api.publish.PublishResult; -import dev.caskeleton.messaging.api.publish.TransmissionEvidence; -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; -import org.junit.jupiter.api.Test; - -class BridgePublishEvidenceTest { - - private static final DestinationName ORDERS = new DestinationName("orders.v1"); - private static final byte[] PAYLOAD = "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8); - - private static SpringCloudStreamPublisherBridge bridge(boolean channelAccepts) { - SpringCloudStreamPublisherBridge bridge = - new SpringCloudStreamPublisherBridge((binding, payload, headers) -> channelAccepts); - bridge.bindPublisher(ORDERS, "orders-out"); - return bridge; - } - - @Test - void anAcceptedSendIsAmbiguousBecauseNoBrokerConfirmedIt() { - PublishResult result = bridge(true).publish(ORDERS, PAYLOAD, Map.of()); - - assertThat(result.completion()) - .as("a channel send says the binder took it, not that a broker did") - .isEqualTo(PublishCompletion.AMBIGUOUS); - assertThat(result.evidence().brokerAccepted()).isFalse(); - assertThat(result.evidence().confirmationLevel()).isEqualTo(ConfirmationLevel.NONE); - } - - @Test - void anAcceptedSendReportsThatTransmissionIsUnknown() { - assertThat(bridge(true).publish(ORDERS, PAYLOAD, Map.of()).evidence().transmission()) - .isEqualTo(TransmissionEvidence.MAY_HAVE_BEEN_TRANSMITTED); - } - - @Test - void theFailureDescriptorNamesTheMissingEvidence() { - assertThat(bridge(true).publish(ORDERS, PAYLOAD, Map.of()).failure()) - .hasValueSatisfying( - failure -> assertThat(failure.code()).isEqualTo("STREAM_BRIDGE_NO_BROKER_EVIDENCE")); - } - - @Test - void aRefusedChannelSendIsRejectedNotAmbiguous() { - PublishResult result = bridge(false).publish(ORDERS, PAYLOAD, Map.of()); - - assertThat(result.completion()).isEqualTo(PublishCompletion.REJECTED); - assertThat(result.evidence().transmission()).isEqualTo(TransmissionEvidence.NOT_TRANSMITTED); - } - - @Test - void publishingToAnUnboundDestinationFailsLoudly() { - SpringCloudStreamPublisherBridge bridge = - new SpringCloudStreamPublisherBridge((binding, payload, headers) -> true); - - assertThatThrownBy(() -> bridge.publish(ORDERS, PAYLOAD, Map.of())) - .isInstanceOf(MessagingConfigurationException.class) - .hasMessageContaining("orders.v1"); - } - - @Test - void thePayloadReachesTheBoundBinding() { - List bindings = new ArrayList<>(); - SpringCloudStreamPublisherBridge bridge = - new SpringCloudStreamPublisherBridge( - (binding, payload, headers) -> { - bindings.add(binding); - return true; - }); - bridge.bindPublisher(ORDERS, "orders-out"); - - bridge.publish(ORDERS, PAYLOAD, Map.of()); - - assertThat(bindings).containsExactly("orders-out"); - } - - @Test - void bindingsAreQueryableInBothDirections() { - SpringCloudStreamPublisherBridge bridge = bridge(true); - bridge.bindConsumer(ORDERS, "orders-in"); - - assertThat(bridge.publisherBinding(ORDERS)).hasValue("orders-out"); - assertThat(bridge.consumerBinding(ORDERS)).hasValue("orders-in"); - } - - @Test - void aHandlerFailureIsRethrownSoTheBindersErrorChannelSeesIt() { - SpringCloudStreamConsumerBridge consumer = new SpringCloudStreamConsumerBridge(); - consumer.register( - ORDERS, - "orders-in", - (destination, payload, headers) -> { - throw new IllegalStateException("handler failed"); - }); - - assertThatThrownBy(() -> consumer.dispatch("orders-in", PAYLOAD, Map.of())) - .as("swallowing it would acknowledge a message nothing handled") - .isInstanceOf(IllegalStateException.class); - } - - @Test - void dispatchingToAnUnregisteredBindingFailsLoudly() { - assertThatThrownBy( - () -> new SpringCloudStreamConsumerBridge().dispatch("orders-in", PAYLOAD, Map.of())) - .isInstanceOf(MessagingConfigurationException.class); - } - - @Test - void aRegisteredHandlerReceivesTheDestinationItWasBoundTo() { - List seen = new ArrayList<>(); - SpringCloudStreamConsumerBridge consumer = new SpringCloudStreamConsumerBridge(); - consumer.register( - ORDERS, "orders-in", (destination, payload, headers) -> seen.add(destination)); - - consumer.dispatch("orders-in", PAYLOAD, Map.of()); - - assertThat(seen).containsExactly(ORDERS); - assertThat(consumer.registeredBindings()).isEqualTo(1); - } -} diff --git a/messaging/messaging-testkit/build.gradle b/messaging/messaging-testkit/build.gradle index 559464f..0b4b657 100644 --- a/messaging/messaging-testkit/build.gradle +++ b/messaging/messaging-testkit/build.gradle @@ -1,10 +1,15 @@ plugins { - id 'ca.jmh-benchmarks' + id 'ca.platform-module' + id 'me.champeau.jmh' +} + +jmh { + jmhVersion = libs.versions.jmh.get() + includeTests = true } dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') api project(':messaging:messaging-policy') api project(':messaging:messaging-transport-spi') diff --git a/messaging/messaging-testkit/gradle.lockfile b/messaging/messaging-testkit/gradle.lockfile index 98b1f9d..4334c11 100644 --- a/messaging/messaging-testkit/gradle.lockfile +++ b/messaging/messaging-testkit/gradle.lockfile @@ -35,12 +35,12 @@ io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,jmhAnnotation javax.inject:javax.inject:1=annotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor jaxen:jaxen:2.0.6=spotbugs net.bytebuddy:byte-buddy:1.17.8=compileClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath -net.sf.jopt-simple:jopt-simple:5.0.4=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath +net.sf.jopt-simple:jopt-simple:5.0.4=jmh,jmhCompileClasspath,jmhRuntimeClasspath net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs org.antlr:antlr4-runtime:4.13.2=checkstyle org.apache.bcel:bcel:6.12.0=spotbugs org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs -org.apache.commons:commons-math3:3.6.1=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath +org.apache.commons:commons-math3:3.6.1=jmh,jmhCompileClasspath,jmhRuntimeClasspath org.apache.commons:commons-text:1.15.0=spotbugs org.apache.commons:commons-text:1.3=checkstyle org.apache.httpcomponents:httpclient:4.5.13=checkstyle @@ -71,13 +71,16 @@ org.junit.platform:junit-platform-launcher:6.0.3=jmhRuntimeClasspath,testRuntime org.junit:junit-bom:6.0.3=compileClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.junit:junit-bom:6.1.0=spotbugs org.mockito:mockito-core:5.20.0=mockitoAgent -org.openjdk.jmh:jmh-core:1.37=jmhAnnotationProcessor,jmhCompileClasspath,jmhRuntimeClasspath -org.openjdk.jmh:jmh-generator-annprocess:1.37=jmhAnnotationProcessor +org.openjdk.jmh:jmh-core:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-asm:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-bytecode:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath +org.openjdk.jmh:jmh-generator-reflection:1.37=jmh,jmhCompileClasspath,jmhRuntimeClasspath org.opentest4j:opentest4j:1.3.0=compileClasspath,jmhCompileClasspath,jmhRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath org.ow2.asm:asm-analysis:9.10.1=spotbugs org.ow2.asm:asm-commons:9.10.1=spotbugs org.ow2.asm:asm-tree:9.10.1=spotbugs org.ow2.asm:asm-util:9.10.1=spotbugs +org.ow2.asm:asm:9.0=jmh,jmhCompileClasspath,jmhRuntimeClasspath org.ow2.asm:asm:9.10.1=spotbugs org.pcollections:pcollections:4.0.1=annotationProcessor,jmhAnnotationProcessor,testAnnotationProcessor org.reflections:reflections:0.10.2=checkstyle diff --git a/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java b/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java index 13d7486..482d9b4 100644 --- a/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java +++ b/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/CompatibilityMatrix.java @@ -5,22 +5,22 @@ import java.util.Map; import java.util.Objects; /** - * The certified broker versions and support tier for every adapter. + * The certified broker versions and support grade for every adapter. * *

    Held as code with a test over it rather than as a document, because a support matrix that * lives only in prose drifts the first time an adapter is added. The test asserts the rules the * design fixes: a Stable adapter must run the shared contract and be certified against a live * broker, and an Experimental adapter must be disabled by default. * - *

    The certification claim is derived from lane evidence rather than declared beside the tier, so - * an adapter cannot be promoted by editing a boolean. + *

    The certification claim is derived from lane evidence rather than declared beside the grade, + * so an adapter cannot be promoted by editing a boolean. */ public final class CompatibilityMatrix { private CompatibilityMatrix() {} - /** The support tier an adapter has earned. */ - public enum Tier { + /** The support grade an adapter has earned. */ + public enum SupportGrade { /** Passes the shared adapter contract unchanged. */ STABLE, @@ -36,14 +36,14 @@ public final class CompatibilityMatrix { * * @param adapter the module name * @param brokerVersions the broker versions the adapter is certified against - * @param tier the support tier + * @param grade the support grade * @param runsSharedContract whether the module runs {@link MessagingAdapterContract} * @param enabledByDefault whether the adapter is active without an explicit opt-in */ public record Entry( String adapter, List brokerVersions, - Tier tier, + SupportGrade grade, boolean runsSharedContract, boolean enabledByDefault) { @@ -51,9 +51,9 @@ public final class CompatibilityMatrix { * Reports whether a lane has run this adapter against a real broker. * *

    Read from the evidence rather than declared. As a field it was a boolean an author set - * next to the tier, and RabbitMQ carried {@code true} while no fault scenario had ever been - * executed against it — the same shape as the coverage matrix that used to assert its own - * constants. + * next to the grade, and an Experimental adapter carried {@code true} while no fault scenario + * had ever been executed against it — the same shape as the coverage matrix that used to assert + * its own constants. * * @return true when {@link CertifiedEvidence} has live-broker evidence for this adapter */ @@ -63,7 +63,7 @@ public final class CompatibilityMatrix { } public Entry { - Objects.requireNonNull(tier, "tier must not be null"); + Objects.requireNonNull(grade, "grade must not be null"); Objects.requireNonNull(brokerVersions, "brokerVersions must not be null"); if (adapter == null || adapter.isBlank()) { throw new IllegalArgumentException("adapter must not be blank"); @@ -78,36 +78,11 @@ public final class CompatibilityMatrix { private static final Map ENTRIES = Map.of( "messaging-kafka", - // 4.1 only. The declared 4.2/4.3 were never run: the fixtures pin apache/kafka:4.1.0 - // and - // the locked client is 4.1.1, so the matrix and the support document agreed with each - // other and with nothing that had executed. A version returns here when a lane runs - // against its image. - new Entry("messaging-kafka", List.of("4.1"), Tier.STABLE, true, true), - "messaging-rabbit", - // Experimental until a lane runs a fault scenario against a RabbitMQ container. The - // adapter passes the shared contract and its broker IT exercises the routable and - // unroutable happy paths, but Stable in this repository means fault evidence, and - // there is none: connection-refused, cut-after-write, confirm-timeout, - // settlement-lost and high-latency have never been executed here. - new Entry("messaging-rabbit", List.of("4.3"), Tier.EXPERIMENTAL, true, false), - "messaging-kafka-share-experimental", - new Entry( - "messaging-kafka-share-experimental", - List.of("4.2", "4.3"), - Tier.EXPERIMENTAL, - false, - false), - "messaging-pulsar-experimental", - new Entry( - "messaging-pulsar-experimental", - List.of("4.0", "4.2"), - Tier.EXPERIMENTAL, - false, - false), - "messaging-nats-experimental", - new Entry( - "messaging-nats-experimental", List.of("2.14"), Tier.EXPERIMENTAL, false, false)); + // 4.1 only. The declared 4.2/4.3 were never run: the fixtures pin apache/kafka:4.1.0 and + // the locked client is 4.1.1, so the matrix and the support document agreed with each + // other and with nothing that had executed. A version returns here when a lane runs + // against its image. + new Entry("messaging-kafka", List.of("4.1"), SupportGrade.STABLE, true, true)); /** * Returns every certified adapter. diff --git a/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/MessagingAdapterHarness.java b/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/MessagingAdapterHarness.java index 4df6fa8..a17f10e 100644 --- a/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/MessagingAdapterHarness.java +++ b/messaging/messaging-testkit/src/main/java/dev/caskeleton/messaging/testkit/MessagingAdapterHarness.java @@ -9,8 +9,8 @@ import java.util.concurrent.CompletionStage; * The seam each broker adapter implements to run the common contract. * *

    The harness exposes outcomes, not internals: what the platform decided, what reached the dead - * letter destination, and what is still unsettled. Kafka and RabbitMQ answer those three questions - * with completely different machinery, and the contract only passes if the answers match. + * letter destination, and what is still unsettled. Each broker answers those three questions with + * its own machinery, and the contract only passes if the answers match. */ public interface MessagingAdapterHarness extends AutoCloseable { diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CertifiedEvidenceTest.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CertifiedEvidenceTest.java index 0ed64f1..a8eacb4 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CertifiedEvidenceTest.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CertifiedEvidenceTest.java @@ -53,8 +53,8 @@ class CertifiedEvidenceTest { assertThat(CertifiedEvidence.knownGaps("messaging-kafka")) .as("a Kafka producer buffers before it learns a connection exists, so this stays unproven") .contains(NetworkFaultScenario.CONNECTION_REFUSED); - assertThat(CertifiedEvidence.knownGaps("messaging-rabbit")) - .as("no lane runs a fault scenario against RabbitMQ, so every scenario is a gap") + assertThat(CertifiedEvidence.knownGaps("messaging-artemis")) + .as("an adapter no lane has run has every scenario as a gap") .containsExactlyElementsOf(NetworkFaultScenario.all()); } diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java index f126796..2bff244 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CompatibilityMatrixTest.java @@ -23,7 +23,7 @@ class CompatibilityMatrixTest { @Test void everyStableAdapterRunsTheSharedContract() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .isNotEmpty() .allSatisfy(entry -> assertThat(entry.runsSharedContract()).isTrue()); } @@ -31,17 +31,15 @@ class CompatibilityMatrixTest { @Test void noExperimentalAdapterIsEnabledByDefault() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.EXPERIMENTAL) - .isNotEmpty() + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.EXPERIMENTAL) .allSatisfy(entry -> assertThat(entry.enabledByDefault()).isFalse()); } @Test void theStableSetIsExactlyWhatALaneHasCertified() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .extracting(CompatibilityMatrix.Entry::adapter) - .as("RabbitMQ passes the shared contract, but no fault scenario has been run against it") .containsExactly("messaging-kafka"); } @@ -61,7 +59,7 @@ class CompatibilityMatrixTest { @Test void everyStableAdapterHasOnlyExplicitlyApprovedEvidenceGaps() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .allSatisfy( entry -> assertThat(CertifiedEvidence.approvedGaps(entry.adapter())) @@ -100,14 +98,14 @@ class CompatibilityMatrixTest { assertThatThrownBy( () -> new CompatibilityMatrix.Entry( - "messaging-x", List.of(), CompatibilityMatrix.Tier.STABLE, true, true)) + "messaging-x", List.of(), CompatibilityMatrix.SupportGrade.STABLE, true, true)) .isInstanceOf(IllegalArgumentException.class); } @Test void everyStableAdapterIsCertifiedAgainstALiveBroker() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .isNotEmpty() .allSatisfy( entry -> @@ -121,7 +119,7 @@ class CompatibilityMatrixTest { @Test void noExperimentalAdapterClaimsLiveBrokerCertification() { assertThat(CompatibilityMatrix.entries().values()) - .filteredOn(entry -> entry.tier() == CompatibilityMatrix.Tier.EXPERIMENTAL) + .filteredOn(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.EXPERIMENTAL) .allSatisfy(entry -> assertThat(entry.hasLiveBrokerCertification()).isFalse()); } @@ -129,7 +127,11 @@ class CompatibilityMatrixTest { void anAdapterWithoutEvidenceCannotBeCalledStable() { CompatibilityMatrix.Entry undeclared = new CompatibilityMatrix.Entry( - "messaging-artemis", List.of("2.42"), CompatibilityMatrix.Tier.STABLE, true, false); + "messaging-artemis", + List.of("2.42"), + CompatibilityMatrix.SupportGrade.STABLE, + true, + false); assertThat(undeclared.hasLiveBrokerCertification()) .as("promoting an adapter used to be a boolean an author could set beside the tier") diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java index adb286a..be044b0 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/CrossBrokerContractSuite.java @@ -35,7 +35,7 @@ class CrossBrokerContractSuite { private static List stableAdapters() { return CompatibilityMatrix.entries().values().stream() - .filter(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filter(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .map(CompatibilityMatrix.Entry::adapter) .toList(); } @@ -48,19 +48,16 @@ class CrossBrokerContractSuite { assertThat(matrix.gapsFor("messaging-kafka")) .as("what the Kafka lanes have not exercised against a live broker") .isEqualTo(CertifiedEvidence.knownGaps("messaging-kafka")); - assertThat(matrix.gapsFor("messaging-rabbit")) - .as("what the Rabbit lanes have not exercised against a live broker") - .isEqualTo(CertifiedEvidence.knownGaps("messaging-rabbit")); } @Test void evidenceThatDisagreesWithItsScenarioIsNotCoverage() { BrokerCertificationEvidence contradicting = new BrokerCertificationEvidence( - "messaging-rabbit", - "rabbitmq:4.3-management", + "messaging-kafka", + "apache/kafka:4.1.0", NetworkFaultScenario.CONNECTION_CUT_AFTER_WRITE, - "RabbitBrokerIT#anUnroutablePublishIsRejectedEvenThoughTheExchangeConfirmedIt", + "SomeIT#someScenario", "REJECTED", "repository-local", java.time.Instant.parse("2026-08-15T00:00:00Z")); @@ -91,7 +88,7 @@ class CrossBrokerContractSuite { void noExperimentalAdapterClaimsLiveBrokerFaultCoverage() { List experimental = CompatibilityMatrix.entries().values().stream() - .filter(entry -> entry.tier() == CompatibilityMatrix.Tier.EXPERIMENTAL) + .filter(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.EXPERIMENTAL) .map(CompatibilityMatrix.Entry::adapter) .toList(); @@ -105,14 +102,6 @@ class CrossBrokerContractSuite { .isNotEqualTo(Coverage.LIVE_BROKER))); } - @Test - void anExperimentalAdapterIsAllowedGapsButTheyAreRecorded() { - assertThat(matrix.isComplete("messaging-pulsar-experimental")) - .as("the Experimental tier is a statement of fact, not a disclaimer") - .isFalse(); - assertThat(matrix.gapsFor("messaging-pulsar-experimental")).isNotEmpty(); - } - @Test void aFaultThatWasNeverRecordedReadsAsUncoveredRatherThanPassing() { assertThat(matrix.coverageOf("messaging-artemis", NetworkFaultScenario.CONFIRM_TIMEOUT)) diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java index 0b7471a..cc5755b 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/MessagingDocumentationContractTest.java @@ -62,7 +62,7 @@ class MessagingDocumentationContractTest { String matrix = read("support-matrix.md"); CompatibilityMatrix.entries().values().stream() - .filter(entry -> entry.tier() == CompatibilityMatrix.Tier.STABLE) + .filter(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.STABLE) .forEach(entry -> assertThat(matrix).contains(brokerLabel(entry.adapter()))); } @@ -71,7 +71,7 @@ class MessagingDocumentationContractTest { String matrix = read("support-matrix.md"); CompatibilityMatrix.entries().values().stream() - .filter(entry -> entry.tier() == CompatibilityMatrix.Tier.EXPERIMENTAL) + .filter(entry -> entry.grade() == CompatibilityMatrix.SupportGrade.EXPERIMENTAL) .forEach( entry -> assertThat(matrix) @@ -135,10 +135,6 @@ class MessagingDocumentationContractTest { private static String brokerLabel(String adapter) { return switch (adapter) { case "messaging-kafka" -> "Kafka"; - case "messaging-rabbit" -> "RabbitMQ"; - case "messaging-pulsar-experimental" -> "Pulsar"; - case "messaging-nats-experimental" -> "NATS"; - case "messaging-kafka-share-experimental" -> "Kafka Share"; default -> adapter; }; } diff --git a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/contract/MessagingPublicSurfaceContractTest.java b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/contract/MessagingPublicSurfaceContractTest.java index 8cd6a3c..00756f7 100644 --- a/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/contract/MessagingPublicSurfaceContractTest.java +++ b/messaging/messaging-testkit/src/test/java/dev/caskeleton/messaging/testkit/contract/MessagingPublicSurfaceContractTest.java @@ -96,7 +96,7 @@ class MessagingPublicSurfaceContractTest { assertThat(leaves) .as("a scan that found no leaf would report every violation as absent") - .hasSizeGreaterThan(20); + .hasSizeGreaterThan(15); assertThat( exposedLibraries( leaves.stream().filter(isNamed("messaging-kafka")).findFirst().orElseThrow())) @@ -123,8 +123,8 @@ class MessagingPublicSurfaceContractTest { assertThat(reachable) .as( "an empty closure would satisfy the assertion below without proving anything; the " - + "metrics registry and the reactive facade are exported on purpose") - .contains("io.micrometer:micrometer-core", "io.projectreactor:reactor-core"); + + "metrics registry is exported on purpose (the reactive facade was deleted, S7-#9c)") + .contains("io.micrometer:micrometer-core"); assertThat(reachable) .as( "both adapters were exported, so every adopter compiled against two broker SDKs at " diff --git a/messaging/messaging-transport-spi/build.gradle b/messaging/messaging-transport-spi/build.gradle index 81e3e70..55c8be5 100644 --- a/messaging/messaging-transport-spi/build.gradle +++ b/messaging/messaging-transport-spi/build.gradle @@ -4,6 +4,5 @@ plugins { dependencies { api project(':messaging:messaging-core-api') - api project(':messaging:messaging-schema-api') api project(':messaging:messaging-policy') } diff --git a/messaging/messaging-transport-spi/src/main/java/dev/caskeleton/messaging/transport/MessagingLifecycle.java b/messaging/messaging-transport-spi/src/main/java/dev/caskeleton/messaging/transport/MessagingLifecycle.java deleted file mode 100644 index e45aa2b..0000000 --- a/messaging/messaging-transport-spi/src/main/java/dev/caskeleton/messaging/transport/MessagingLifecycle.java +++ /dev/null @@ -1,67 +0,0 @@ -package dev.caskeleton.messaging.transport; - -import java.time.Duration; - -/** - * The ordered lifecycle every messaging runtime implements. - * - *

    The order in {@link ShutdownPhase} is the contract, not an implementation detail. Closing - * connections before settlements have been transmitted loses the settlements, and pausing consumers - * after draining lets fresh deliveries arrive into a runtime that is already shutting down. Each - * adapter implements the phases; none of them chooses the order. - * - *

    Implementations are driven by the Spring lifecycle rather than a JVM shutdown hook alone. A - * shutdown hook runs after the context has already begun disposing beans, so a handler mid-drain - * can find its datasource closed underneath it. - */ -public interface MessagingLifecycle { - - /** The fixed phases of an orderly shutdown, in the order they must run. */ - enum ShutdownPhase { - /** Refuse new publishes. Nothing in flight is affected. */ - STOP_PUBLISH_ADMISSION, - /** Refuse to start new delivery handlers. */ - STOP_NEW_HANDLERS, - /** Ask the broker to stop delivering. */ - PAUSE_CONSUMERS, - /** Let running handlers finish. */ - DRAIN_HANDLERS, - /** Transmit the settlements those handlers produced. */ - FLUSH_SETTLEMENTS, - /** Wait for outstanding producer confirms so publishes are not left ambiguous. */ - AWAIT_PRODUCER_CONFIRMS, - /** Return outbox leases so another relay can claim the rows immediately. */ - RELEASE_OUTBOX_LEASES, - /** Close connections and channels. */ - CLOSE_CONNECTIONS - } - - /** The default drain budget from the design. */ - Duration DEFAULT_DRAIN_DEADLINE = Duration.ofSeconds(30); - - /** - * Starts the runtime. - * - * @throws IllegalStateException when the runtime was already started - */ - void start(); - - /** - * Runs the shutdown phases in order, bounded by the drain deadline. - * - *

    Work still running at the deadline is abandoned unsettled so the broker redelivers - * it. Unconfirmed publishes are recorded as ambiguous rather than as successes. - * - * @param drainDeadline how long in-flight work may take - * @return the phase that was still running when the deadline passed, or {@link - * ShutdownPhase#CLOSE_CONNECTIONS} when the shutdown completed - */ - ShutdownPhase shutdown(Duration drainDeadline); - - /** - * Reports whether the runtime is running and accepting work. - * - * @return true between {@link #start()} and the first shutdown phase - */ - boolean isRunning(); -} diff --git a/messaging/messaging-transport-spi/src/test/java/dev/caskeleton/messaging/transport/MessagingLifecycleTest.java b/messaging/messaging-transport-spi/src/test/java/dev/caskeleton/messaging/transport/MessagingLifecycleTest.java deleted file mode 100644 index beabf82..0000000 --- a/messaging/messaging-transport-spi/src/test/java/dev/caskeleton/messaging/transport/MessagingLifecycleTest.java +++ /dev/null @@ -1,59 +0,0 @@ -package dev.caskeleton.messaging.transport; - -import static org.assertj.core.api.Assertions.assertThat; - -import dev.caskeleton.messaging.transport.MessagingLifecycle.ShutdownPhase; -import java.time.Duration; -import java.util.List; -import org.junit.jupiter.api.Test; - -class MessagingLifecycleTest { - - @Test - void publishAdmissionClosesBeforeConsumersArePaused() { - List order = List.of(ShutdownPhase.values()); - - assertThat(order.indexOf(ShutdownPhase.STOP_PUBLISH_ADMISSION)) - .isLessThan(order.indexOf(ShutdownPhase.PAUSE_CONSUMERS)); - } - - @Test - void handlersDrainBeforeTheirSettlementsAreFlushed() { - List order = List.of(ShutdownPhase.values()); - - assertThat(order.indexOf(ShutdownPhase.DRAIN_HANDLERS)) - .as("flushing before the handlers finish would lose the settlements they produce") - .isLessThan(order.indexOf(ShutdownPhase.FLUSH_SETTLEMENTS)); - } - - @Test - void connectionsCloseLastOfAll() { - ShutdownPhase[] order = ShutdownPhase.values(); - - assertThat(order[order.length - 1]) - .as("every earlier phase needs the connection that this one closes") - .isEqualTo(ShutdownPhase.CLOSE_CONNECTIONS); - } - - @Test - void producerConfirmsAreAwaitedBeforeTheConnectionGoesAway() { - List order = List.of(ShutdownPhase.values()); - - assertThat(order.indexOf(ShutdownPhase.AWAIT_PRODUCER_CONFIRMS)) - .as("a confirm that arrives after close cannot be observed, leaving the publish ambiguous") - .isLessThan(order.indexOf(ShutdownPhase.CLOSE_CONNECTIONS)); - } - - @Test - void outboxLeasesAreReleasedBeforeClosing() { - List order = List.of(ShutdownPhase.values()); - - assertThat(order.indexOf(ShutdownPhase.RELEASE_OUTBOX_LEASES)) - .isLessThan(order.indexOf(ShutdownPhase.CLOSE_CONNECTIONS)); - } - - @Test - void theDefaultDrainDeadlineMatchesTheDesign() { - assertThat(MessagingLifecycle.DEFAULT_DRAIN_DEADLINE).isEqualTo(Duration.ofSeconds(30)); - } -}