feat(messaging): 브로커 중립 메시징 플랫폼 24개 leaf 추가
messaging-superpowers-package 설계서/계획서 기반 구현. registry를 19 → 43 leaf로 확장하고 src/messaging 아래 24개 leaf를 등록. - core-api: M1 publish/consume + M2 batch·delayed·pause-resume - policy/transport-spi: 재시도 결정, DLQ orchestration, admission control, lifecycle - kafka·rabbit(Stable): contiguous commit, confirm/return 상관, 배치, 보안 설정 - pulsar·nats(Experimental): 기본 비활성, live 인증 없음을 코드로 기록 - outbox/inbox/claim-check: 트랜잭션 결합, lease, 무결성 검증 - admin: plan → approve → execute를 타입으로 강제 - 문서 9종, infra compose 7종, JMH 벤치마크 3종 검증: 아키텍처 게이트 3종 통과, 24개 leaf 전부 check 통과, messaging 테스트 604개 통과/0 실패. 미완: 계획서가 요구한 실 브로커 IT 40개 중 7개만 작성. Rabbit 13 / Outbox 6 / Inbox 4 / NATS·Pulsar·Share 5 / testkit 2 / starter·admin 3, 그리고 TLS·ACL 2개가 남음.
This commit is contained in:
DongHyeonka
@@ -0,0 +1,156 @@
|
||||
# 설정 레퍼런스
|
||||
|
||||
## Destination profile
|
||||
|
||||
```yaml
|
||||
messaging:
|
||||
destinations:
|
||||
order-events:
|
||||
broker: kafka-primary
|
||||
kind: EVENT_STREAM # ASYNC_COMMAND | DOMAIN_EVENT | INTEGRATION_EVENT
|
||||
# | WORK_QUEUE | PUBLISH_SUBSCRIBE | EVENT_STREAM | REQUEST_REPLY
|
||||
tier: M1 # M1 | M2 | M3
|
||||
physical:
|
||||
topic: order.events.v1
|
||||
schema:
|
||||
codec: application/json
|
||||
compatibility: BACKWARD_TRANSITIVE
|
||||
message-types: [order.created]
|
||||
guarantees:
|
||||
delivery: AT_LEAST_ONCE # AT_MOST_ONCE | AT_LEAST_ONCE
|
||||
ordering: KEY # NONE | DESTINATION | PARTITION | KEY
|
||||
external-side-effect: INBOX_TRANSACTIONAL
|
||||
producer:
|
||||
confirmation: REPLICATION_OR_PERSISTENCE_ACK
|
||||
timeout: 5s
|
||||
mandatory-routing: true
|
||||
idempotent: true
|
||||
consumer:
|
||||
group: order-projection
|
||||
concurrency: 6
|
||||
max-in-flight-per-ordering-unit: 1
|
||||
prefetch: 16
|
||||
handler-timeout: 30s
|
||||
manual-settlement: false
|
||||
retry:
|
||||
mode: PAUSE_PARTITION # NONE | INLINE | BLOCKING | PAUSE_PARTITION
|
||||
# | RETRY_DESTINATION | BROKER_DELAYED
|
||||
max-attempts: 3
|
||||
initial-delay: 200ms
|
||||
max-delay: 2s
|
||||
multiplier: 2.0
|
||||
jitter: true
|
||||
ordering-impact: PRESERVE # PRESERVE | ALLOW_REORDER
|
||||
dlq:
|
||||
destination: order-events-dlq
|
||||
max-redrive-count: 1
|
||||
payload:
|
||||
max-bytes: 1048576
|
||||
claim-check-threshold-bytes: 1048576
|
||||
key-resolver-configured: true
|
||||
production: true
|
||||
topology-auto-create: false
|
||||
```
|
||||
|
||||
## 기본값
|
||||
|
||||
| 설정 | 기본값 | 근거 |
|
||||
|---|---:|---|
|
||||
| logical payload 최대 | 1,048,576 bytes | portability. 초과는 Claim Check |
|
||||
| global hard 최대 | 8,388,608 bytes | 어떤 destination도 넘을 수 없는 상한 |
|
||||
| header 총 크기 | 32,768 bytes | |
|
||||
| header 개수 | 64 | |
|
||||
| header key | 128 bytes | metric tag 안전 |
|
||||
| header value | 4,096 bytes | |
|
||||
| publish timeout | 5s | |
|
||||
| handler timeout | 30s | |
|
||||
| graceful shutdown drain | 30s | |
|
||||
| 일반 destination retry | 0회 | 자동 retry는 opt-in |
|
||||
| DLQ redrive batch | 100 | 한 번의 작업이 source를 덮치지 않게 |
|
||||
| Outbox relay batch | 100 | |
|
||||
| Outbox lease | 30s | |
|
||||
| Outbox polling | 500ms | |
|
||||
| metric dimension 상한 | 200 | cardinality 폭발 방지 |
|
||||
|
||||
## Broker profile
|
||||
|
||||
### Kafka
|
||||
|
||||
```yaml
|
||||
messaging:
|
||||
brokers:
|
||||
kafka-primary:
|
||||
type: kafka
|
||||
stable: true
|
||||
production: true
|
||||
bootstrap-servers: [broker-1:9093, broker-2:9093]
|
||||
enable-idempotence: true # stable에서 필수
|
||||
acks: all # stable에서 필수
|
||||
max-in-flight-requests-per-connection: 5 # 최대 5
|
||||
delivery-timeout: 30s
|
||||
enable-auto-commit: false # 항상 금지
|
||||
tls-enabled: true # production 필수
|
||||
authentication-enabled: true # production 필수
|
||||
```
|
||||
|
||||
### RabbitMQ
|
||||
|
||||
```yaml
|
||||
messaging:
|
||||
brokers:
|
||||
rabbit-primary:
|
||||
type: rabbitmq
|
||||
stable: true
|
||||
production: true
|
||||
addresses: [rabbit-1:5671]
|
||||
publisher-confirms: true # stable에서 필수
|
||||
publisher-returns: true # stable에서 필수
|
||||
mandatory: true # stable에서 필수
|
||||
confirm-timeout: 5s
|
||||
auto-ack: false # 항상 금지
|
||||
prefetch: 16
|
||||
quorum-queues: true # durable work queue 필수
|
||||
tls-enabled: true
|
||||
authentication-enabled: true
|
||||
```
|
||||
|
||||
## 보안
|
||||
|
||||
```yaml
|
||||
messaging:
|
||||
security:
|
||||
kafka-primary:
|
||||
producer: { type: SASL_SCRAM, credential-id: kafka-producer }
|
||||
consumer: { type: SASL_SCRAM, credential-id: kafka-consumer }
|
||||
# admin은 application runtime에 설정하지 않는다
|
||||
hostname-verification: true
|
||||
access:
|
||||
publishable: [order-events]
|
||||
consumable: []
|
||||
administrable: []
|
||||
```
|
||||
|
||||
## Experimental / Optional
|
||||
|
||||
기본값은 전부 `false`다.
|
||||
|
||||
```yaml
|
||||
messaging:
|
||||
experimental:
|
||||
kafka-share: false
|
||||
pulsar: false
|
||||
nats: false
|
||||
bridge:
|
||||
spring-cloud-stream: false
|
||||
```
|
||||
|
||||
## Backpressure
|
||||
|
||||
```yaml
|
||||
messaging:
|
||||
backpressure:
|
||||
global-limit: 512
|
||||
per-destination-limit: 64 # global-limit 이하여야 한다
|
||||
```
|
||||
|
||||
`per-destination-limit > global-limit`이면 global limit이 limit이 아니게 되므로 부팅에 실패한다.
|
||||
@@ -0,0 +1,75 @@
|
||||
# 전달 보장
|
||||
|
||||
## 왜 `EXACTLY_ONCE`가 없는가
|
||||
|
||||
어떤 브로커도 **외부 side effect를 포함한** exactly-once를 제공하지 않는다.
|
||||
실제로 존재하는 것은 at-least-once 전달 + 멱등하거나 transactional한 consumer의 조합이다.
|
||||
|
||||
플랫폼이 지킬 수 없는 이름을 enum에 두면 그 책임이 눈에 보이지 않는 곳으로 밀려난다.
|
||||
그래서 `DeliveryGuarantee`는 증거가 끝나는 지점에서 멈춘다.
|
||||
|
||||
```java
|
||||
public enum DeliveryGuarantee { AT_MOST_ONCE, AT_LEAST_ONCE }
|
||||
```
|
||||
|
||||
## Publish 결과는 boolean이 아니다
|
||||
|
||||
```java
|
||||
public enum PublishCompletion { CONFIRMED, REJECTED, AMBIGUOUS }
|
||||
```
|
||||
|
||||
`REJECTED`와 `AMBIGUOUS`를 하나의 "실패"로 합치면 중복 주문이 만들어진다.
|
||||
전자는 broker가 저장하지 않았음이 **확정**되어 포기해도 안전하고, 후자는 그렇지 않다.
|
||||
|
||||
| 상황 | 결과 |
|
||||
|---|---|
|
||||
| 로컬 validation 실패 | `REJECTED`, `NOT_TRANSMITTED` |
|
||||
| broker 명시적 reject / nack | `REJECTED` |
|
||||
| confirm 수신 | `CONFIRMED` |
|
||||
| Rabbit confirm + unroutable return | `REJECTED`, `UNROUTABLE` |
|
||||
| bytes 전송 후 connection loss | `AMBIGUOUS` |
|
||||
| confirm timeout | `AMBIGUOUS` |
|
||||
| adapter가 판정 불가 | 보수적으로 `AMBIGUOUS` |
|
||||
|
||||
`PublishResult` 생성자가 이 규칙을 강제한다. `CONFIRMED`인데 broker acceptance가 없거나,
|
||||
`AMBIGUOUS`인데 confirmation level을 주장하면 **객체 생성 자체가 실패**한다.
|
||||
|
||||
## Ordering
|
||||
|
||||
```java
|
||||
public enum OrderingScope { NONE, DESTINATION, PARTITION, KEY }
|
||||
```
|
||||
|
||||
순서는 partition·key·단일 consumer의 성질이지 destination 전체의 성질이 아니다.
|
||||
`GLOBAL`이 없는 이유가 이것이다.
|
||||
|
||||
`DestinationProfileValidator`가 다음을 거부한다.
|
||||
|
||||
- `ordering=KEY`인데 key resolver 없음
|
||||
- ordered destination인데 `ALLOW_REORDER` retry
|
||||
- `orderingImpact=PRESERVE`인데 재발행형 retry(`RETRY_DESTINATION`, `BROKER_DELAYED`)
|
||||
- `ordering=DESTINATION`인데 concurrency > 1
|
||||
- ordered destination인데 ordering unit당 in-flight > 1
|
||||
|
||||
## External side effect
|
||||
|
||||
```java
|
||||
public enum ExternalSideEffectGuarantee { NONE, IDEMPOTENCY_REQUIRED, INBOX_TRANSACTIONAL }
|
||||
```
|
||||
|
||||
`INBOX_TRANSACTIONAL`만이 "DB side effect와 중복 차단이 같은 transaction에서 commit된다"를 의미한다.
|
||||
Kafka transaction은 **Kafka 안에서만** 원자적이므로 이 값과 함께 설정하면
|
||||
`KafkaTransactionProfileValidator`가 거부한다. 두 개의 독립적인 commit을 하나로 착각하게 두지 않기 위해서다.
|
||||
|
||||
## Consumer settlement 순서
|
||||
|
||||
```text
|
||||
RECEIVED → DECODING → PROCESSING → HANDLER_SUCCEEDED → SETTLEMENT_SENDING
|
||||
├→ SETTLED
|
||||
└→ SETTLEMENT_UNKNOWN
|
||||
```
|
||||
|
||||
- handler는 broker ACK API를 호출하지 않는다.
|
||||
- `Success` 이후에만 source settlement한다.
|
||||
- `SETTLEMENT_UNKNOWN`은 성공이 아니다. redelivery 가능성을 의미한다.
|
||||
- `SettlementResult` 생성자가 `SETTLED`인데 `redeliveryPossible=true`인 조합을 거부한다.
|
||||
@@ -0,0 +1,97 @@
|
||||
# Experimental 정책
|
||||
|
||||
## Stable과 Experimental의 차이
|
||||
|
||||
**Stable**은 공통 Contract Suite(`MessagingAdapterContract`)를 변경 없이 통과한 어댑터다.
|
||||
컴파일되는 어댑터가 아니라, 아래 7가지를 실제로 증명한 어댑터다.
|
||||
|
||||
```text
|
||||
publishesAndConfirms
|
||||
returnsAmbiguousWhenConfirmIsLost
|
||||
redeliversWhenSettlementIsLost
|
||||
preservesMessageIdAcrossRetryAndDlq
|
||||
keepsSourceUnsettledWhenDlqPublishFails
|
||||
rejectsOversizedPayloadBeforeTransport
|
||||
stopsAcceptingNewWorkDuringShutdown
|
||||
```
|
||||
|
||||
**Experimental**은 아직 그 증명이 끝나지 않은 어댑터다.
|
||||
|
||||
## 규칙
|
||||
|
||||
### 1. 기본 비활성
|
||||
|
||||
```yaml
|
||||
messaging.experimental.kafka-share: false
|
||||
messaging.experimental.pulsar: false
|
||||
messaging.experimental.nats: false
|
||||
```
|
||||
|
||||
활성화하지 않으면 validator가 `MessagingCapabilityUnavailableException`을 던진다.
|
||||
Contract Suite가 아직 증명 중인 어댑터가 누군가의 기본 설정 때문에 load-bearing이 되어서는 안 된다.
|
||||
|
||||
### 2. Stable 모듈이 Experimental 모듈에 의존하지 않는다
|
||||
|
||||
Gradle 의존 그래프로 강제된다. `messaging-spring-boot-starter`의 `allowed_dependencies`에
|
||||
`messaging-kafka-share-experimental`, `messaging-pulsar-experimental`,
|
||||
`messaging-nats-experimental`, `messaging-spring-cloud-stream-bridge`가 **없다**.
|
||||
|
||||
`verifyCleanArchitectureDependencies`가 위반을 빌드 실패로 만든다.
|
||||
|
||||
### 3. Core 계약을 바꾸지 않는다
|
||||
|
||||
Experimental 어댑터는 브로커의 차이를 `MessagingCapabilities`로 표현할 뿐,
|
||||
`messaging-core-api`의 타입을 바꾸지 않는다.
|
||||
|
||||
### 4. 없는 기능을 광고하지 않는다
|
||||
|
||||
| 어댑터 | 광고하지 않는 것 | 이유 |
|
||||
|---|---|---|
|
||||
| Kafka Share Group | orderedStream, keyedOrdering, replay, brokerTransaction | 경쟁 소비자 + 개별 ack는 partition 순서를 유지할 수 없다 |
|
||||
| Pulsar | brokerTransaction | Pulsar에 있지만 플랫폼 Contract Suite로 증명되지 않았다 |
|
||||
| Pulsar (Shared) | keyedOrdering | round-robin 분배 |
|
||||
| NATS JetStream | nativeDeadLetter | delivery limit 초과 시 terminate할 뿐 라우팅하지 않는다 |
|
||||
| NATS JetStream | keyedOrdering | subject 기반 모델에 per-key 순서가 없다 |
|
||||
|
||||
`false`인 capability를 요구하는 profile은 startup에서 실패한다.
|
||||
조용히 약화되지 않는다.
|
||||
|
||||
### 5. 명시적 거부
|
||||
|
||||
| 조합 | 결과 |
|
||||
|---|---|
|
||||
| Kafka Share Group + ordering != NONE | 거부 |
|
||||
| Kafka Share Group + pause/resume | `MessagingCapabilityUnavailableException` |
|
||||
| Pulsar Shared + ordering=KEY | 거부 (Key_Shared 필요) |
|
||||
| Pulsar + ordering=DESTINATION | 거부 |
|
||||
| NATS Core + AT_LEAST_ONCE | 거부 (JetStream 필요) |
|
||||
| NATS ordered consumer + 경쟁 워커 > 1 | 거부 |
|
||||
| NATS + ordering=KEY | 거부 |
|
||||
|
||||
## Spring Cloud Stream bridge
|
||||
|
||||
Experimental이 아니라 **Optional**이다. 위험이 다르다.
|
||||
|
||||
Stream은 자체 binder 설정을 소유하므로, binding이 destination profile이 모르는
|
||||
serializer·error handling·acknowledgement mode를 조용히 획득할 수 있다.
|
||||
|
||||
따라서 브리지는 **플랫폼 보장에 의존하지 않는 destination에만** 허용한다.
|
||||
|
||||
```text
|
||||
ordering scope 선언 → 거부
|
||||
retry policy 선언 → 거부
|
||||
dead letter 선언 → 거부
|
||||
```
|
||||
|
||||
이 셋 중 하나라도 필요하면 native adapter를 쓴다. 거기서만 실제로 강제되기 때문이다.
|
||||
|
||||
## 승격 조건
|
||||
|
||||
Experimental → Stable로 올리려면 전부 필요하다.
|
||||
|
||||
1. `MessagingAdapterContract` 7개 테스트를 변경 없이 통과
|
||||
2. 장애 주입(연결 끊김, confirm 유실, settlement 유실) 하에서 통과
|
||||
3. 지원 브로커 버전 범위 명시 및 CI 검증
|
||||
4. `support-matrix.md`의 capability 표 갱신
|
||||
5. ADR 작성
|
||||
6. 기본 활성화 여부에 대한 별도 결정
|
||||
@@ -0,0 +1,113 @@
|
||||
# 마이그레이션 가이드
|
||||
|
||||
## 기존 Spring Kafka / Spring AMQP 코드에서
|
||||
|
||||
### 1. topic 이름을 코드에서 제거한다
|
||||
|
||||
```java
|
||||
// before
|
||||
kafkaTemplate.send("order.events.v1", key, payload);
|
||||
|
||||
// after
|
||||
publisher.publish(orderEvents, envelope, PublishOptions.defaults());
|
||||
```
|
||||
|
||||
`MessageDestination`은 logical name만 가진다. 물리 매핑은 destination profile이 소유한다.
|
||||
`DestinationName`의 패턴이 `topic://orders` 같은 값을 거부하므로 우회할 수 없다.
|
||||
|
||||
### 2. boolean 성공 판정을 없앤다
|
||||
|
||||
```java
|
||||
// before
|
||||
try { template.send(...).get(); success(); }
|
||||
catch (Exception e) { fail(); } // REJECTED와 AMBIGUOUS를 구분하지 못한다
|
||||
|
||||
// after
|
||||
PublishResult result = ...;
|
||||
switch (result.completion()) {
|
||||
case CONFIRMED -> success();
|
||||
case REJECTED -> abandon(); // broker가 저장하지 않음이 확정
|
||||
case AMBIGUOUS -> retrySameMessageId(result); // broker가 가지고 있을 수 있음
|
||||
}
|
||||
```
|
||||
|
||||
이 구분이 없으면 confirm 유실 한 번이 중복 주문 하나가 된다.
|
||||
|
||||
### 3. auto-commit / auto-ack를 끈다
|
||||
|
||||
```yaml
|
||||
# Kafka
|
||||
enable.auto.commit: false
|
||||
# RabbitMQ
|
||||
auto-ack: false
|
||||
```
|
||||
|
||||
둘 다 validator가 강제로 거부한다. 타이머 기반 commit은 handler가 실행되기도 전에
|
||||
메시지를 처리 완료로 표시한다.
|
||||
|
||||
### 4. handler에서 ack 호출을 제거한다
|
||||
|
||||
```java
|
||||
// before
|
||||
@KafkaListener(...)
|
||||
void handle(ConsumerRecord<?,?> record, Acknowledgment ack) {
|
||||
process(record);
|
||||
ack.acknowledge(); // 실패 시 순서가 애매해진다
|
||||
}
|
||||
|
||||
// after
|
||||
CompletionStage<HandleResult> handle(MessageDelivery<OrderCreated> delivery) {
|
||||
process(delivery.message().payload());
|
||||
return completedFuture(HandleResult.success());
|
||||
}
|
||||
```
|
||||
|
||||
settlement는 플랫폼이 수행한다. "성공한 뒤에만 ack"가 각 handler의 기억이 아니라
|
||||
플랫폼 불변식이 된다.
|
||||
|
||||
### 5. 중복을 정상 상황으로 다룬다
|
||||
|
||||
at-least-once는 중복을 전제한다. 세 가지 중 하나를 고른다.
|
||||
|
||||
| 방식 | 언제 |
|
||||
|---|---|
|
||||
| handler 자체 멱등 | 자연 멱등 연산 (upsert 등) |
|
||||
| Inbox | DB side effect가 있는 경우 |
|
||||
| Kafka transaction | Kafka → Kafka 파이프라인만 |
|
||||
|
||||
`ExternalSideEffectGuarantee`에 선언한다. `INBOX_TRANSACTIONAL`과 Kafka transaction을
|
||||
동시에 설정하면 거부된다. Kafka transaction은 DB를 포함하지 않는다.
|
||||
|
||||
### 6. 큰 payload는 Claim Check로
|
||||
|
||||
broker frame 크기를 키우지 않는다. broker 메모리, replication latency,
|
||||
consumer recovery가 동시에 나빠지고, 유계·검증 가능한 실패가 무계 실패로 바뀐다.
|
||||
|
||||
1 MiB 초과는 외부 저장소로 offload하고 digest를 포함한 참조만 발행한다.
|
||||
|
||||
## DB 마이그레이션
|
||||
|
||||
```text
|
||||
V1__messaging_outbox.sql
|
||||
V2__messaging_inbox.sql
|
||||
```
|
||||
|
||||
Outbox row는 business transaction과 같은 transaction에서 쓴다.
|
||||
Inbox reservation은 handler side effect와 같은 transaction에서 쓴다.
|
||||
별도 transaction이면 각 패턴이 닫으려던 창이 그대로 열려 있다.
|
||||
|
||||
## 단계적 전환
|
||||
|
||||
1. **publish만 전환** — 기존 consumer는 그대로. wire format은 reserved header가 추가될 뿐이다.
|
||||
2. **Outbox 도입** — publish 유실 창을 닫는다.
|
||||
3. **consume 전환** — handler를 `MessageHandler`로 옮기고 ack 호출을 제거한다.
|
||||
4. **Inbox 도입** — 중복 side effect를 닫는다.
|
||||
5. **retry·DLQ 정책 선언** — 이 시점까지 자동 retry는 0회다.
|
||||
|
||||
각 단계는 독립적으로 배포 가능하고, 되돌릴 수 있다.
|
||||
|
||||
## 되돌릴 수 없는 것
|
||||
|
||||
- 한 번 발행된 message type의 wire contract
|
||||
- 이미 retention 안에 있는 메시지의 schema
|
||||
- redrive된 메시지의 `messageId` (바뀌지 않는다 — 이것이 의도다)
|
||||
@@ -0,0 +1,113 @@
|
||||
# 운영 Runbook
|
||||
|
||||
## 배포 전 체크
|
||||
|
||||
```bash
|
||||
./gradlew verifyCleanArchitectureDependencies --console=plain
|
||||
./gradlew verifyRuntimeModuleMembership --console=plain
|
||||
./gradlew verifyOneTypePerFile --console=plain
|
||||
```
|
||||
|
||||
destination profile은 startup에서 검증된다. 아래는 **부팅 실패**다.
|
||||
|
||||
- ordered destination + reorder 가능 retry
|
||||
- `ordering=KEY` + key resolver 없음
|
||||
- payload 상한 > 8,388,608 bytes
|
||||
- DLQ 자기 참조 / retry 자기 참조
|
||||
- retry·DLQ 그래프 cycle
|
||||
- 미등록 retry·DLQ destination
|
||||
- M1 destination + manual settlement
|
||||
- `AT_LEAST_ONCE` + confirmation `NONE`
|
||||
- production profile + topology auto-create
|
||||
- broker topology가 manifest와 불일치
|
||||
|
||||
## 증상별 대응
|
||||
|
||||
### publish가 AMBIGUOUS로 쏟아진다
|
||||
|
||||
broker confirm 경로 문제다. 실패가 아니다.
|
||||
|
||||
1. `PublishEvidence.transmission`이 `MAY_HAVE_BEEN_TRANSMITTED`인지 확인
|
||||
2. Kafka: `delivery.timeout.ms`, ISR 상태, leader election 확인
|
||||
3. Rabbit: confirm timeout, channel 상태 확인
|
||||
4. Outbox를 쓰고 있다면 `status='AMBIGUOUS'` row가 같은 messageId로 재시도 중이다. **정상이다.**
|
||||
5. consumer 쪽 Inbox가 중복을 흡수하는지 확인
|
||||
|
||||
`AMBIGUOUS`를 실패로 취급해 새 messageId로 재발행하지 말 것. 중복이 복구 불가능해진다.
|
||||
|
||||
### DLQ가 비어 있는데 메시지가 사라졌다
|
||||
|
||||
DLQ publish 실패 시 source는 settlement되지 않는다. 메시지는 source에 남아 재전달된다.
|
||||
|
||||
1. `msg.failure-code`가 `DEAD_LETTER_*`인 로그 확인
|
||||
2. DLQ destination이 실제로 존재하는지 (topology validation)
|
||||
3. DLQ credential에 publish 권한이 있는지
|
||||
|
||||
### consumer lag이 한 partition에서만 증가한다
|
||||
|
||||
`ContiguousPartitionOffsetTracker`가 gap에서 멈춘 것이다. 설계된 동작이다.
|
||||
|
||||
commit은 **연속** 완료 offset까지만 전진한다. offset 11이 아직 실행 중이면
|
||||
10과 12가 끝나도 watermark는 10에 머문다. 12를 commit하면 consumer가 죽었을 때 11을 잃는다.
|
||||
|
||||
1. 해당 partition의 in-flight를 확인
|
||||
2. 느린 handler를 찾는다 (`handlerTimeout` 초과 여부)
|
||||
3. 필요하면 `PAUSE_PARTITION` retry가 걸려 있는지 확인
|
||||
|
||||
### 재시도 폭풍
|
||||
|
||||
`RetryPolicy.jitter=false`인지 확인한다. jitter 없이는 같은 초에 실패한 모든 consumer가
|
||||
같은 초에 재시도한다.
|
||||
|
||||
### shutdown이 오래 걸린다
|
||||
|
||||
`GracefulShutdownCoordinator`가 in-flight를 기다리는 중이다.
|
||||
|
||||
- `inFlight()`가 0이 되면 즉시 종료
|
||||
- drain deadline(기본 30초) 초과 시 남은 작업을 **unsettled로 포기**한다 → broker가 재전달
|
||||
- draining 시작 후 새 retry attempt는 만들지 않는다
|
||||
|
||||
## Destructive 작업
|
||||
|
||||
전부 `DestructiveOperationGuard`를 통과해야 한다.
|
||||
|
||||
| 조건 | 요구 |
|
||||
|---|---|
|
||||
| admin credential | application runtime은 보유하지 않음 |
|
||||
| `AdminApproval` | 유효기간 내 |
|
||||
| dry-run | 항상 허용 |
|
||||
|
||||
### Replay
|
||||
|
||||
```text
|
||||
기본: 격리된 consumer group (replay-<requestId>)
|
||||
기존 group 대상: 승인 티켓 필수
|
||||
```
|
||||
|
||||
기존 production group으로 replay하는 것은 "다시 읽기"가 아니라 **live consumer를 되감는 것**이다.
|
||||
그 사이의 모든 것이 재처리된다.
|
||||
|
||||
### Redrive
|
||||
|
||||
```text
|
||||
dry-run으로 후보 수 확인
|
||||
→ 승인 획득
|
||||
→ batch 100건 이하로 실행
|
||||
→ republish CONFIRMED 인 것만 DLQ에서 settlement
|
||||
```
|
||||
|
||||
`redriveId`로 재구동 루프를 추적한다. 같은 메시지가 반복해서 redrive되면
|
||||
근본 원인이 해결되지 않은 것이다.
|
||||
|
||||
### Offset reset
|
||||
|
||||
`KafkaOffsetResetExecutor`는 승인 predicate를 **생성자 인자**로 받는다.
|
||||
승인 소스 없이 조립된 runtime은 물리적으로 reset을 수행할 수 없다.
|
||||
|
||||
## Topology
|
||||
|
||||
production topology는 IaC가 만들고 애플리케이션은 **검증만** 한다.
|
||||
|
||||
`TopologyValidationRuntime`은 모든 불일치를 한 번에 보고하고 startup을 실패시킨다.
|
||||
partition 수가 다르면 destination이 광고하는 ordering 보장이 달라지고,
|
||||
`min.insync.replicas`가 없으면 `acks=all`의 의미가 달라진다.
|
||||
@@ -0,0 +1,95 @@
|
||||
# Outbox · Inbox
|
||||
|
||||
## 두 패턴이 각각 무엇을 해결하는가
|
||||
|
||||
| 패턴 | 해결하는 문제 | 해결하지 않는 문제 |
|
||||
|---|---|---|
|
||||
| Transactional Outbox | DB commit과 publish 사이의 창(窓) | 중복 |
|
||||
| Inbox | 중복 delivery의 side effect | 유실 |
|
||||
|
||||
**둘 다 필요하다.** Outbox만으로는 exactly-once가 되지 않는다.
|
||||
|
||||
## Outbox
|
||||
|
||||
business transaction과 **같은 transaction**에서 row를 쓴다. 둘 다 commit되거나 둘 다 안 된다.
|
||||
|
||||
```sql
|
||||
BEGIN;
|
||||
UPDATE orders SET status = 'PLACED' WHERE id = ?;
|
||||
INSERT INTO messaging_outbox (message_id, destination, ...) VALUES (?, ?, ...);
|
||||
COMMIT;
|
||||
```
|
||||
|
||||
### relay
|
||||
|
||||
```text
|
||||
leaseBatch(100, 30s) -- lease로 다중 relay 인스턴스 안전
|
||||
→ publish (messageId 그대로)
|
||||
→ CONFIRMED → markPublished
|
||||
→ AMBIGUOUS → markAmbiguous (같은 messageId로 재시도 가능)
|
||||
→ REJECTED → markFailed
|
||||
```
|
||||
|
||||
### 핵심 규칙: ambiguous는 같은 messageId로 재시도
|
||||
|
||||
새 id를 발급하면 "전달됐을 수도 있는 메시지"가 "확실히 두 번째인 메시지"가 되어
|
||||
downstream의 어떤 중복 제거도 복구할 수 없다.
|
||||
failed로 표시하면 broker가 이미 가지고 있을 수 있는 메시지를 잃는다.
|
||||
|
||||
`message_id`를 primary key로 둔 것도 같은 이유다. 어떤 코드 경로도 실수로 새 id를 붙일 수 없다.
|
||||
|
||||
### lease
|
||||
|
||||
```text
|
||||
status IN ('PENDING','AMBIGUOUS') AND (lease_expires_at IS NULL OR lease_expires_at <= now)
|
||||
```
|
||||
|
||||
partial index `ix_messaging_outbox_claimable`이 이 쿼리를 backlog 크기에 비례하게 유지한다.
|
||||
PUBLISHED row는 retention job이 지울 때까지 쌓이기 때문이다.
|
||||
|
||||
## Inbox
|
||||
|
||||
reservation과 side effect가 **같은 transaction**이어야 한다.
|
||||
|
||||
```java
|
||||
transactions.inTransaction(() -> {
|
||||
if (!inbox.reserve(messageId, consumerId, now)) {
|
||||
return InboxOutcome.duplicate(); // 이미 처리됨
|
||||
}
|
||||
return InboxOutcome.processed(sideEffect.get());
|
||||
});
|
||||
```
|
||||
|
||||
별도 transaction으로 예약하면 Inbox가 닫으려던 바로 그 창이 다시 열린다.
|
||||
|
||||
### 복합 키
|
||||
|
||||
`PRIMARY KEY (message_id, consumer_id)`.
|
||||
|
||||
message_id만으로 중복 제거하면 같은 event를 소비하는 두 번째 consumer가
|
||||
첫 번째에 의해 억제된다. 각 consumer가 한 번씩 처리해야 한다.
|
||||
|
||||
### retention
|
||||
|
||||
broker의 최대 redelivery window보다 **길어야** 한다.
|
||||
row를 먼저 지우면 늦게 도착한 redelivery가 두 번 처리된다.
|
||||
|
||||
## Debezium CDC 대안
|
||||
|
||||
polling relay 대신 WAL을 읽는다. polling interval과 lease 경합이 사라지지만
|
||||
인프라와 그 자체의 실패 모드가 추가된다.
|
||||
|
||||
wire contract는 동일하다. `DebeziumOutboxEventRouter`가 polling relay와 같은 reserved header를
|
||||
방출하므로 consumer는 어느 쪽이 발행했는지 구분할 수 없고, 전환은 배포 결정일 뿐 계약 변경이 아니다.
|
||||
|
||||
## Claim Check
|
||||
|
||||
1 MiB 초과 payload는 broker 프레임을 키우지 않고 외부 저장소로 offload한다.
|
||||
|
||||
`ClaimCheckReference`는 digest를 **필수**로 가진다. claim check는 메시지를 서로 다른 retention과
|
||||
replication을 가진 두 시스템으로 쪼개므로, consumer는 producer가 저장한 바로 그 bytes를 받았음을
|
||||
증명할 수 있어야 한다. 그렇지 않으면 잘린 객체와 정상 객체를 구분할 수 없다.
|
||||
|
||||
`ClaimCheckIntegrityGuard`는 fetch 전에 만료를, fetch 후에 크기와 digest를 검사한다.
|
||||
digest 불일치는 `DESERIALIZATION`이 아니라 **validation** 실패로 분류한다.
|
||||
bytes가 깨진 JSON인 게 아니라, 틀린 bytes이기 때문이다.
|
||||
@@ -0,0 +1,103 @@
|
||||
# Retry · DLQ · Redrive
|
||||
|
||||
## 자동 retry는 opt-in이다
|
||||
|
||||
일반 destination의 기본값은 **retry 없음**이다. 순서를 깨거나, 멱등하지 않은 side effect를
|
||||
증폭시키거나, 이미 throttle된 downstream을 더 때리는 retry는 보이는 실패보다 나쁘다.
|
||||
|
||||
## 결정 순서
|
||||
|
||||
`DefaultRetryDecisionEngine`은 아래 순서를 위에서 아래로 평가한다.
|
||||
|
||||
```text
|
||||
1. non-retryable category → parking(DeadLetter) 또는 Reject
|
||||
2. attempt >= maxAttempts → DeadLetter
|
||||
3. PRESERVE + ordered + orderedStream capability → PauseAndRetry
|
||||
4. mode=PAUSE_PARTITION → PauseAndRetry
|
||||
5. mode=RETRY_DESTINATION + ALLOW_REORDER → PublishToRetryDestination
|
||||
6. mode=INLINE|BLOCKING → RetryInline
|
||||
7. mode=BROKER_DELAYED + delayedDelivery capability → PublishToRetryDestination
|
||||
8. 그 외 → DeadLetter
|
||||
```
|
||||
|
||||
**retryability를 attempt 예산보다 먼저** 검사한다. deserialization 실패는 payload가 바뀌지 않으므로
|
||||
재시도가 3번 더 실패할 뿐이다. 첫 delivery에서 바로 park한다.
|
||||
|
||||
**순서 보존 전략을 재발행 전략보다 먼저** 검사한다. 둘 다 설정되어 있어도 ordered destination이
|
||||
reorder 경로로 흘러내리지 않는다.
|
||||
|
||||
## 기본 non-retryable
|
||||
|
||||
`DESERIALIZATION`, `AUTHENTICATION`, `AUTHORIZATION`, `CONFIGURATION`은 자동 retry하지 않는다.
|
||||
매 redelivery마다 동일하게 실패하므로 부하만 늘어난다.
|
||||
destination profile의 `retryableCategories`로 명시적으로 뒤집을 수는 있다.
|
||||
|
||||
## Backoff
|
||||
|
||||
`min(maxDelay, initialDelay * multiplier^(attempt-1))`, 이후 full jitter.
|
||||
|
||||
full jitter는 `[0, delay]` 균등 분포다. jitter가 없으면 같은 초에 실패한 모든 consumer가
|
||||
같은 초에 재시도하고, downstream의 회복이 재시도 폭풍으로 즉시 무효화된다.
|
||||
|
||||
## Kafka: pause-and-seek vs retry topic
|
||||
|
||||
| 전략 | 순서 | 언제 |
|
||||
|---|---|---|
|
||||
| `PAUSE_PARTITION` | 유지 | ordered destination |
|
||||
| `RETRY_DESTINATION` | 깨짐 | work queue, `ALLOW_REORDER` 명시 |
|
||||
|
||||
pause-and-seek는 메시지가 로그의 자기 자리를 떠나지 않는다. partition을 멈추고, 기다리고,
|
||||
같은 offset으로 seek해 재전달한다. 뒤의 메시지도 함께 기다리며 이것이 의도된 동작이다.
|
||||
|
||||
## RabbitMQ: delayed retry queue
|
||||
|
||||
core broker에 per-message delay가 없으므로 **TTL + DLX**로 구현한다.
|
||||
retry queue의 `x-message-ttl`이 만료되면 `x-dead-letter-exchange`를 통해 work queue로 되돌아간다.
|
||||
|
||||
주의: TTL 만료는 큐 **head**에서 평가된다. 하나의 retry queue에 서로 다른 delay를 섞으면
|
||||
독립적으로 만료되지 않는다.
|
||||
|
||||
`basic.nack(requeue=true)`는 사용하지 않는다. delay 없이 큐 head로 되돌리므로 hot loop가 된다.
|
||||
|
||||
## DLQ: publish 확인 후 settlement
|
||||
|
||||
이것이 dead lettering이 데이터 손실이 되지 않게 하는 **유일한** 불변식이다.
|
||||
|
||||
```text
|
||||
DLQ envelope 생성 (원래 messageId 유지)
|
||||
→ DLQ publish
|
||||
→ CONFIRMED 이면 source settlement
|
||||
→ REJECTED / AMBIGUOUS 이면 source를 settlement하지 않음
|
||||
```
|
||||
|
||||
source를 먼저 ACK하면, DLQ publish가 실패했을 때 메시지의 사본이 **어디에도 남지 않는다**.
|
||||
broker는 이미 해제했고 DLQ는 받지 못했다.
|
||||
|
||||
AMBIGUOUS DLQ publish는 중복을 만든다. 이것이 의도된 trade다. DLQ는 사람이 읽는 곳이고
|
||||
중복은 알아볼 수 있지만, 손실은 복구할 수 없다.
|
||||
|
||||
## DLQ envelope 내용
|
||||
|
||||
reserved header에만 기록한다. payload에 넣지 않는다.
|
||||
|
||||
```text
|
||||
msg.failure-category, msg.failure-code, msg.origin-destination,
|
||||
msg.retry-attempt, msg.first-failure-at, msg.last-failure-at
|
||||
```
|
||||
|
||||
stack trace, exception message, secret header, 실제 key는 **넣지 않는다**.
|
||||
DLQ는 원본 topic보다 오래 보관되고 더 많은 사람이 읽는다.
|
||||
|
||||
## Redrive
|
||||
|
||||
M4 Admin 전용이다. `DestructiveOperationGuard`를 통과해야 한다.
|
||||
|
||||
- admin credential 필요 (application runtime은 보유하지 않는다)
|
||||
- 유효기간 내 `AdminApproval` 필요
|
||||
- dry-run은 항상 허용 (계획이 공짜여야 사람이 계획한다)
|
||||
- batch 상한 100건
|
||||
- source == target 금지
|
||||
- `redriveId`는 `messageId`와 별개다. 재구동 루프를 식별하기 위해서다.
|
||||
|
||||
redrive도 **publish → settlement** 순서다. republish가 confirm되지 않은 메시지는
|
||||
DLQ에 남는다.
|
||||
@@ -0,0 +1,92 @@
|
||||
# Messaging 보안
|
||||
|
||||
## Credential 분리
|
||||
|
||||
producer / consumer / admin은 **서로 다른 credential**이다.
|
||||
`MessageSecurityValidator`가 startup에서 강제한다.
|
||||
|
||||
```text
|
||||
producer credential == consumer credential → 실패
|
||||
admin credential == producer|consumer → 실패
|
||||
production 프로필에 admin credential 존재 → 실패
|
||||
```
|
||||
|
||||
마지막 규칙이 "애플리케이션은 topic을 purge할 수 없다"를 **구조적으로** 만든다.
|
||||
runtime이 admin 자격 증명을 아예 보유하지 않으므로, 침해된 handler가 상승시킬 대상이 없다.
|
||||
|
||||
## Production 필수 조건
|
||||
|
||||
- TLS 활성
|
||||
- TLS hostname verification 활성
|
||||
- broker authentication 활성
|
||||
- topology auto-create 비활성
|
||||
|
||||
Kafka는 추가로 `enable.idempotence=true`, `acks=all`,
|
||||
`max.in.flight.requests.per.connection <= 5`, consumer auto-commit 금지.
|
||||
|
||||
RabbitMQ는 추가로 publisher confirm, publisher return, `mandatory=true`,
|
||||
durable work queue의 quorum queue, consumer auto-ack 금지.
|
||||
|
||||
## Credential은 값이 아니라 참조다
|
||||
|
||||
`BrokerCredentialProfile`의 어떤 variant도 secret을 담지 않는다.
|
||||
식별자만 보관하고 connect 시점에 `CredentialProvider`로 해석한다.
|
||||
heap dump나 설정 출력에서 사용 가능한 credential이 나오지 않는다.
|
||||
|
||||
`CredentialIds`는 `bearer `, `sk-`, `-----begin`, `eyJ` 같은 접두사를 거부한다.
|
||||
참조가 들어갈 자리에 secret 자체를 붙여넣는 가장 흔한 사고를 막는다.
|
||||
|
||||
## Rotation
|
||||
|
||||
`CredentialRotationPlan.isDue()`는 만료 **전에** 참이 된다.
|
||||
broker가 연결을 거부하기 시작한 시점에는 이미 publish가 실패하고 consumer가 멈춰 있다.
|
||||
|
||||
rotation은 세대 교체다. `DefaultMessagingRuntimeRegistry.install()`이 새 세대를 원자적으로
|
||||
게시하고, 이전 세대는 마지막 lease가 닫힐 때까지 열려 있다가 닫힌다.
|
||||
진행 중인 publish는 시작한 연결에서 confirm을 받는다.
|
||||
|
||||
drain deadline이 이 대기를 제한한다. 없으면 lease 하나가 새면 폐기된 credential이
|
||||
무기한 열려 있고, rotation이 보안상 무의미해진다.
|
||||
|
||||
## Header
|
||||
|
||||
금지 header는 application·platform 양쪽에서 거부한다.
|
||||
|
||||
```text
|
||||
Authorization, Proxy-Authorization, Cookie, Set-Cookie,
|
||||
access_token, refresh_token, api_key, password, client_secret
|
||||
```
|
||||
|
||||
credential이 header에 들어가면 broker storage, DLQ dump, 운영 도구에 남는다.
|
||||
downstream redaction으로는 되돌릴 수 없다.
|
||||
|
||||
예약 header(`msg.*`, `traceparent`, `tracestate`, `baggage`)는 platform만 쓴다.
|
||||
application이 `msg.id`를 설정할 수 있으면 Inbox 중복 제거와 DLQ 상관관계가 의존하는
|
||||
logical identity가 호출자 제어가 된다.
|
||||
|
||||
## ACL
|
||||
|
||||
`DestinationAccessValidator`가 broker ACL **이전에** 검사한다.
|
||||
broker ACL 거부는 애플리케이션 컨텍스트가 없는 연결 수준 오류로 도착하므로
|
||||
"어느 모듈이 어디에 publish하려 했는가"가 조사 대상이 된다.
|
||||
|
||||
## 관측성 누출
|
||||
|
||||
`MessagingRedactor`는 denylist다.
|
||||
|
||||
- secret: authorization, cookie, token, password, secret, credential
|
||||
- per-message identity: messageId, correlationId, causationId, partitionKey, key, offset, deliveryTag, sequence
|
||||
- payload: payload, body, data
|
||||
- 예외 상세: exceptionMessage, stackTrace
|
||||
|
||||
identity를 지우는 이유는 두 가지다. bounded metric을 message당 하나의 series로 만들고,
|
||||
support log를 재식별 표면으로 만들기 때문이다.
|
||||
|
||||
`CardinalityGuard`는 dimension당 값 개수를 상한한다.
|
||||
cardinality 사고는 점진적이지 않다. 테스트 10건에서는 멀쩡하고 운영에서 백엔드를 죽인다.
|
||||
|
||||
## 감사
|
||||
|
||||
`MessagingAuditEvent`는 replay, redrive, offset reset, purge, delete를 기록한다.
|
||||
subject(운영자 identity), approval ticket, 그리고 redactor를 통과한 details만 담는다.
|
||||
누가 무엇을 했는지 증명하되 payload의 두 번째 사본이 되지 않는다.
|
||||
@@ -0,0 +1,116 @@
|
||||
# Messaging 지원 매트릭스
|
||||
|
||||
플랫폼이 **무엇을 보장하는지**와 **무엇을 보장하지 않는지**를 브로커별로 고정한다.
|
||||
여기 없는 조합은 지원되지 않는다.
|
||||
|
||||
## 브로커 등급
|
||||
|
||||
| 브로커 | 등급 | 인증 기준 | Stable 기능 | 제한 |
|
||||
|---|---|---|---|---|
|
||||
| Kafka | Stable | 4.2+ / 4.3.x | producer idempotence, consumer group, batch, pause/resume, replay, transaction capability | Share Group은 Experimental |
|
||||
| RabbitMQ | Stable | 4.3.x | exchange/routing, publisher confirm, mandatory return, manual ACK, quorum queue, retry queue, DLQ | stream 및 특수 plugin 미지원 |
|
||||
| Pulsar | Experimental | 4.0 LTS + 4.2 | typed publish/consume, Shared, Key_Shared, schema | transaction 미승격, 기본 비활성 |
|
||||
| NATS JetStream | Experimental | 2.14.x | stream, durable consumer, explicit ACK, dedupe, replay | native DLQ 없음(플랫폼이 대행), 기본 비활성 |
|
||||
| Artemis/JMS | Extension | 범위 밖 | adapter SPI만 | 별도 ADR + Contract Suite 통과 필요 |
|
||||
|
||||
## Capability 매트릭스
|
||||
|
||||
`MessagingCapabilities`가 런타임에 선언하는 값이다. `false`인 기능을 요구하는 destination profile은
|
||||
**startup에서 실패**하며, 조용히 약화되지 않는다.
|
||||
|
||||
| Capability | Kafka | Kafka Share | RabbitMQ | Pulsar | NATS JS |
|
||||
|---|---|---|---|---|---|
|
||||
| brokerAcknowledgement | O | O | O | O | O |
|
||||
| replicationOrPersistenceEvidence | O | O | O | O | O |
|
||||
| perMessageSettlement | O | O | O | O | O |
|
||||
| batchSettlement | O | X | X | O | O |
|
||||
| orderedStream | O | **X** | X | X | O |
|
||||
| keyedOrdering | O | **X** | X | Key_Shared만 | X |
|
||||
| replay | O | X | X | O | O |
|
||||
| delayedDelivery | X | X | retry queue로 대행 | O | X |
|
||||
| brokerTransaction | O | X | X | 미승격 | X |
|
||||
| deduplicatedPublish | O | X | X | X | O |
|
||||
| nativeDeadLetter | X | X | O | O | **X** |
|
||||
| topologyManagement | O | X | O | O | O |
|
||||
|
||||
Kafka Share Group이 ordering 전부 `X`인 것은 설계 결정이다. share group은 개별 record를
|
||||
경쟁 소비자에게 나눠주고 개별 ack하므로 partition 순서를 유지할 수 없다. ordered destination을
|
||||
share group에 설정하면 `KafkaShareProfileValidator`가 거부한다.
|
||||
|
||||
NATS JetStream의 `nativeDeadLetter=X`도 마찬가지다. JetStream은 delivery limit 초과 시 메시지를
|
||||
**terminate**할 뿐 어디로도 라우팅하지 않으므로, 플랫폼이 DLQ publish를 직접 수행한다.
|
||||
|
||||
## 기능 등급
|
||||
|
||||
| 기능 | 등급 |
|
||||
|---|---|
|
||||
| Typed Publish·Consume | Stable M1 |
|
||||
| At-least-once contract | Stable |
|
||||
| Ambiguous publish 결과 | Stable |
|
||||
| handler 성공 후 자동 settlement | Stable M1 |
|
||||
| Batch / Manual settlement / Pause·Resume / Delayed / Replay 요청 | M2 |
|
||||
| Broker transaction / partition / routing / subscription | M3 |
|
||||
| Replay 실행 / Redrive / offset reset / purge / delete | M4 Admin |
|
||||
| Kafka Share Group, Pulsar, NATS | Experimental |
|
||||
| Spring Cloud Stream bridge | Optional |
|
||||
|
||||
## 무엇이 "Stable"을 증명하는가
|
||||
|
||||
Stable 등급은 두 가지를 **모두** 통과해야 한다. `CompatibilityMatrixTest`가 이 규칙을 강제한다.
|
||||
|
||||
### 1. 공유 Contract Suite (`MessagingAdapterContract`, 7개)
|
||||
|
||||
Kafka와 RabbitMQ가 동일한 7개 테스트를 변경 없이 통과한다. 결정적 하네스를 쓰므로
|
||||
확인 유실·settlement 유실 같은 장애를 요청 시점에 재현할 수 있다.
|
||||
|
||||
### 2. 실 브로커 인증 (Testcontainers)
|
||||
|
||||
| 스위트 | 무엇을 증명하는가 |
|
||||
|---|---|
|
||||
| `KafkaBrokerIT` | `acks=all`이 실제 replication 증거를 만든다 / 잘못된 토픽은 `REJECTED` / 발행-소비 왕복에서 identity 보존 및 contiguous commit |
|
||||
| `KafkaAmbiguityChaosIT` | 브로커를 `docker pause`로 멈춘 상태의 publish가 **`AMBIGUOUS`** 로 보고된다 (broker acceptance 없음, confirmation level `NONE`, 비-retryable) |
|
||||
| `RabbitBrokerIT` | exchange가 confirm했는데 어떤 큐에도 바인딩되지 않은 publish가 **`REJECTED` + `UNROUTABLE`** 로 보고된다 |
|
||||
| `OutboxPostgresIT` | 롤백된 트랜잭션은 발행 가능한 행을 남기지 않는다 / `SKIP LOCKED` lease가 두 relay를 분리한다 / ambiguous 행이 같은 `messageId`로 재클레임된다 |
|
||||
| `InboxPostgresIT` | 재전달이 side effect를 두 번 적용하지 않는다 / 롤백은 예약도 되돌린다 |
|
||||
|
||||
Docker가 없으면 `DockerAvailability` 가드로 skip되며, 이 표의 항목은 그때 **검증되지 않은 것**으로 취급한다.
|
||||
|
||||
### 3. 장애 시나리오 커버리지 (`BrokerFailureMatrix`)
|
||||
|
||||
`NetworkFaultScenario`가 5개 시나리오와 **각각의 기대 결과**를 코드로 고정한다. 기대 결과를 어댑터별로
|
||||
두지 않는 것이 핵심이다 — 어댑터마다 다른 답을 허용하면 공유 계약이 존재할 이유가 없다.
|
||||
|
||||
| 시나리오 | 시점 | 기대 결과 | 이유 |
|
||||
|---|---|---|---|
|
||||
| `connection-refused` | 전송 전 | `REJECTED` | 바이트가 나가지 않았으므로 broker가 가질 수 없다 |
|
||||
| `connection-cut-after-write` | 전송 후 | `AMBIGUOUS` | broker가 저장했고 confirm만 유실됐을 수 있다 |
|
||||
| `confirm-timeout` | 전송 후 | `AMBIGUOUS` | timeout은 부재의 증거가 아니라 증거의 부재다 |
|
||||
| `settlement-lost` | settlement 중 | `REDELIVERED` | 미settlement 메시지는 재전달이 설계다 |
|
||||
| `high-latency` | 전송 후 | `AMBIGUOUS` | 판단 시점에는 confirm 유실과 구별할 수 없다 |
|
||||
|
||||
`CrossBrokerContractSuite`가 릴리스 게이트로 이를 강제한다. Stable 어댑터는 5개 전부를 **실 브로커에서**
|
||||
커버해야 하고, Experimental 어댑터는 `LIVE_BROKER` 커버리지를 주장할 수 없다. 커버리지는 *능력*이 아니라
|
||||
*무엇을 실제로 돌렸는지*의 기록이다.
|
||||
|
||||
### 실 브로커가 실제로 잡아낸 결함
|
||||
|
||||
이 스위트들은 장식이 아니다. 작성 과정에서 결정적 테스트가 통과하는데 실 인프라에서 실패한
|
||||
결함을 두 건 잡았다.
|
||||
|
||||
1. **Outbox `IN_FLIGHT` 고아 행** — lease 쿼리가 `PENDING`/`AMBIGUOUS`만 클레임 대상으로 봐서,
|
||||
publish 도중 죽은 relay가 남긴 행이 lease 만료 후에도 영영 회수되지 않았다.
|
||||
2. **Rabbit confirm 경합** — transport가 publish *후에* confirm을 등록해서, 연결 스레드에서
|
||||
confirm이 먼저 도착하면 유실되고 호출자가 무한 대기했다.
|
||||
|
||||
둘 다 인메모리 double이 실제보다 관대해서 통과하고 있었다.
|
||||
|
||||
## 명시적 비지원
|
||||
|
||||
- 공통 `EXACTLY_ONCE` 설정 — `DeliveryGuarantee`에 상수가 존재하지 않는다.
|
||||
- 전역 순서 — `OrderingScope`에 `GLOBAL`이 존재하지 않는다.
|
||||
- DB와 broker의 자동 원자 transaction, 기본 XA
|
||||
- Java native serialization
|
||||
- 무제한 payload·header, 무한 retry
|
||||
- 운영 application에서의 topology 파괴 작업
|
||||
- 일반 애플리케이션에 raw broker client 반환
|
||||
- DLQ publish 확인 전 source ACK
|
||||
@@ -0,0 +1,33 @@
|
||||
# Kafka 4.3.x in KRaft mode.
|
||||
#
|
||||
# Single broker on purpose: this compose file exists to reproduce the platform's Stable profile
|
||||
# locally, not to model a production cluster. The settings below are the ones the profile guard
|
||||
# enforces, so a local run fails the same way a misconfigured deployment would.
|
||||
services:
|
||||
kafka:
|
||||
image: apache/kafka:4.3.0
|
||||
container_name: messaging-kafka
|
||||
ports:
|
||||
- "9092:9092"
|
||||
environment:
|
||||
KAFKA_NODE_ID: 1
|
||||
KAFKA_PROCESS_ROLES: broker,controller
|
||||
KAFKA_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093
|
||||
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
|
||||
KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
|
||||
KAFKA_CONTROLLER_QUORUM_VOTERS: 1@localhost:9093
|
||||
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
|
||||
# acks=all is only a durability guarantee when more than one replica must acknowledge.
|
||||
# With a single broker the platform still requires acks=all; min.insync.replicas is 1 here
|
||||
# and is expected to be 2 in any environment that claims replication evidence.
|
||||
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
|
||||
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
|
||||
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
|
||||
KAFKA_MIN_INSYNC_REPLICAS: 1
|
||||
# Topology is created by infrastructure, never by the application.
|
||||
KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "/opt/kafka/bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092 >/dev/null 2>&1"]
|
||||
interval: 5s
|
||||
timeout: 10s
|
||||
retries: 20
|
||||
@@ -0,0 +1,21 @@
|
||||
# NATS 2.14.x with JetStream, Experimental tier.
|
||||
#
|
||||
# JetStream is mandatory: core NATS is fire-and-forget with no persistence and no acknowledgement,
|
||||
# so an at-least-once destination configured against it would report success for messages that were
|
||||
# never stored. The adapter's validator refuses that combination.
|
||||
services:
|
||||
nats:
|
||||
image: nats:2.14-alpine
|
||||
container_name: messaging-nats
|
||||
ports:
|
||||
- "4222:4222"
|
||||
- "8222:8222"
|
||||
command:
|
||||
- "--jetstream"
|
||||
- "--store_dir=/data"
|
||||
- "--http_port=8222"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "wget -q -O- http://localhost:8222/healthz || exit 1"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 20
|
||||
@@ -0,0 +1,27 @@
|
||||
# PostgreSQL 16 for the Outbox and Inbox.
|
||||
#
|
||||
# logical replication is enabled so the optional Debezium CDC relay can be exercised against the
|
||||
# same database the polling relay uses; the two must produce an identical wire contract.
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
container_name: messaging-postgres
|
||||
ports:
|
||||
- "5432:5432"
|
||||
environment:
|
||||
POSTGRES_DB: messaging
|
||||
POSTGRES_USER: messaging
|
||||
POSTGRES_PASSWORD: messaging
|
||||
command:
|
||||
- "postgres"
|
||||
- "-c"
|
||||
- "wal_level=logical"
|
||||
- "-c"
|
||||
- "max_replication_slots=4"
|
||||
- "-c"
|
||||
- "max_wal_senders=4"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U messaging -d messaging"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 20
|
||||
@@ -0,0 +1,17 @@
|
||||
# Pulsar 4.0 LTS, Experimental tier.
|
||||
#
|
||||
# Present so the Experimental adapter can be exercised, not because it is supported. The adapter
|
||||
# stays disabled unless backend.messaging.experimental.pulsar=true.
|
||||
services:
|
||||
pulsar:
|
||||
image: apachepulsar/pulsar:4.0.3
|
||||
container_name: messaging-pulsar
|
||||
ports:
|
||||
- "6650:6650"
|
||||
- "8080:8080"
|
||||
command: bin/pulsar standalone --no-functions-worker --no-stream-storage
|
||||
healthcheck:
|
||||
test: ["CMD", "bin/pulsar-admin", "brokers", "healthcheck"]
|
||||
interval: 10s
|
||||
timeout: 10s
|
||||
retries: 20
|
||||
@@ -0,0 +1,22 @@
|
||||
# RabbitMQ 4.3.x.
|
||||
#
|
||||
# Quorum queues are the default for durable work queues in this platform, so the classic mirroring
|
||||
# policy is deliberately absent: classic mirrored queues can lose acknowledged messages during a
|
||||
# partition, which is precisely the guarantee a durable work queue exists to provide.
|
||||
services:
|
||||
rabbitmq:
|
||||
image: rabbitmq:4.3-management
|
||||
container_name: messaging-rabbitmq
|
||||
ports:
|
||||
- "5672:5672"
|
||||
- "15672:15672"
|
||||
environment:
|
||||
RABBITMQ_DEFAULT_USER: messaging
|
||||
RABBITMQ_DEFAULT_PASS: messaging
|
||||
# Publisher confirms and returns are client-side settings; the profile guard enforces them.
|
||||
RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS: "-rabbit consumer_timeout 1800000"
|
||||
healthcheck:
|
||||
test: ["CMD", "rabbitmq-diagnostics", "-q", "check_running"]
|
||||
interval: 5s
|
||||
timeout: 10s
|
||||
retries: 20
|
||||
@@ -0,0 +1,35 @@
|
||||
# TLS material
|
||||
|
||||
Production profiles require TLS **and** hostname verification; `MessageSecurityValidator` fails
|
||||
startup without either.
|
||||
|
||||
No key material is committed here, and none should be. Certificates are issued by the deployment's
|
||||
own PKI and mounted at runtime; a keystore in a repository is a credential in a repository, and
|
||||
rotating it means a commit.
|
||||
|
||||
## Local development
|
||||
|
||||
The compose files in the sibling directories run plaintext listeners deliberately. They exist to
|
||||
reproduce the *messaging* semantics locally, not the transport security, and running them with
|
||||
`production: false` in the destination profile is what keeps the validator honest — a profile marked
|
||||
`production: true` against a plaintext broker must fail, and that is a test, not an inconvenience.
|
||||
|
||||
## Generating a local CA for TLS testing
|
||||
|
||||
```bash
|
||||
openssl req -x509 -newkey rsa:4096 -sha256 -days 30 -nodes \
|
||||
-keyout ca.key -out ca.crt -subj "/CN=messaging-local-ca"
|
||||
|
||||
openssl req -newkey rsa:4096 -nodes -keyout broker.key -out broker.csr \
|
||||
-subj "/CN=localhost"
|
||||
|
||||
openssl x509 -req -in broker.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
|
||||
-out broker.crt -days 30 -sha256 \
|
||||
-extfile <(printf "subjectAltName=DNS:localhost,IP:127.0.0.1")
|
||||
```
|
||||
|
||||
The `subjectAltName` is not optional. Hostname verification is required in production profiles, and
|
||||
a certificate without a SAN fails it — which is the correct outcome, not something to work around by
|
||||
disabling the check.
|
||||
|
||||
Generated files are ignored by `.gitignore` in this directory.
|
||||
@@ -0,0 +1,19 @@
|
||||
# Toxiproxy, for the failures that matter most.
|
||||
#
|
||||
# The platform's hardest guarantee is that a lost confirmation is reported as AMBIGUOUS rather than
|
||||
# guessed. A healthy broker will not lose one on request, so the chaos suite puts a proxy in front of
|
||||
# it and severs the connection after the record was accepted but before the acknowledgement arrives.
|
||||
services:
|
||||
toxiproxy:
|
||||
image: ghcr.io/shopify/toxiproxy:2.12.0
|
||||
container_name: messaging-toxiproxy
|
||||
ports:
|
||||
- "8474:8474" # control API
|
||||
- "19092:19092" # proxied Kafka
|
||||
- "15673:15673" # proxied RabbitMQ
|
||||
- "15433:15433" # proxied PostgreSQL
|
||||
healthcheck:
|
||||
test: ["CMD", "/toxiproxy-cli", "list"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 20
|
||||
+49
-1
@@ -431,7 +431,12 @@ configure(subprojects.findAll { it.childProjects.isEmpty() }) {
|
||||
}
|
||||
|
||||
dependencies {
|
||||
if (project.path in [':domain-core', ':application-core', ':shared-contract']) {
|
||||
// The messaging platform leaves own a broker-neutral public contract. Keeping their test
|
||||
// classpath on plain JUnit + AssertJ is what makes "messaging-core-api has no Spring
|
||||
// dependency" verifiable rather than aspirational; leaves that genuinely need a Spring
|
||||
// test context add it in their own build file.
|
||||
if (project.path in [':domain-core', ':application-core', ':shared-contract'] ||
|
||||
project.path.startsWith(':messaging:')) {
|
||||
testImplementation 'org.junit.jupiter:junit-jupiter'
|
||||
testImplementation 'org.assertj:assertj-core'
|
||||
} else {
|
||||
@@ -444,6 +449,49 @@ configure(subprojects.findAll { it.childProjects.isEmpty() }) {
|
||||
errorprone 'com.google.errorprone:error_prone_core:2.49.0' // D5 compile-time checker
|
||||
}
|
||||
|
||||
// The three messaging leaves that carry JMH benchmarks get a `jmh` source set. It is a source
|
||||
// set rather than a plugin because the benchmarks are compiled and reviewed on every build but
|
||||
// only *run* on demand: a benchmark that stops compiling is a defect, while a benchmark that
|
||||
// runs in CI is a flaky test measuring the build agent.
|
||||
if (project.path in [':messaging:messaging-kafka',
|
||||
':messaging:messaging-rabbit',
|
||||
':messaging:messaging-testkit']) {
|
||||
sourceSets {
|
||||
jmh {
|
||||
compileClasspath += sourceSets.main.output + sourceSets.test.output
|
||||
runtimeClasspath += sourceSets.main.output + sourceSets.test.output
|
||||
}
|
||||
}
|
||||
configurations {
|
||||
jmhImplementation.extendsFrom implementation, testImplementation
|
||||
jmhRuntimeOnly.extendsFrom runtimeOnly, testRuntimeOnly
|
||||
}
|
||||
dependencies {
|
||||
jmhImplementation 'org.openjdk.jmh:jmh-core:1.37'
|
||||
jmhAnnotationProcessor 'org.openjdk.jmh:jmh-generator-annprocess:1.37'
|
||||
// ErrorProne's -Werror would reject JMH's generated sources, which the platform does
|
||||
// not own and cannot fix.
|
||||
jmhAnnotationProcessor 'com.google.errorprone:error_prone_core:2.49.0'
|
||||
}
|
||||
tasks.named('compileJmhJava') {
|
||||
options.errorprone.enabled = false
|
||||
options.compilerArgs.removeAll { it == '-Werror' }
|
||||
}
|
||||
// JMH's annotation processor emits the generated harness into this source set, and its
|
||||
// generated code trips DLS_DEAD_LOCAL_STORE by design (the dead stores are how it defeats
|
||||
// dead-code elimination). Analysing code the platform neither wrote nor can fix would make
|
||||
// the gate unactionable, so the jmh source set is excluded from the bug and style checks.
|
||||
// The benchmarks themselves are still compiled, which is what catches a real breakage.
|
||||
tasks.named('spotbugsJmh') { enabled = false }
|
||||
tasks.named('checkstyleJmh') { enabled = false }
|
||||
tasks.register('jmh', JavaExec) {
|
||||
group = 'verification'
|
||||
description = 'Runs the JMH benchmarks in this leaf.'
|
||||
classpath = sourceSets.jmh.runtimeClasspath
|
||||
mainClass = 'org.openjdk.jmh.Main'
|
||||
}
|
||||
}
|
||||
|
||||
// feature-ci-quality-gates-contract §4 (D7) — the main release gate EXCLUDES the flaky
|
||||
// quarantine bucket so a quarantined test can never block merge. Quarantined tests carry
|
||||
// JUnit's built-in @Tag("quarantine"); they run separately via `quarantineTest` (non-blocking)
|
||||
|
||||
@@ -252,6 +252,285 @@
|
||||
"runtime_memberships": [
|
||||
"sample-portfolio"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "messaging-core-api",
|
||||
"gradle_path": ":messaging:messaging-core-api",
|
||||
"source_path": "src/messaging/messaging-core-api",
|
||||
"allowed_dependencies": [],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-schema-api",
|
||||
"gradle_path": ":messaging:messaging-schema-api",
|
||||
"source_path": "src/messaging/messaging-schema-api",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-schema-json",
|
||||
"gradle_path": ":messaging:messaging-schema-json",
|
||||
"source_path": "src/messaging/messaging-schema-json",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-schema-avro",
|
||||
"gradle_path": ":messaging:messaging-schema-avro",
|
||||
"source_path": "src/messaging/messaging-schema-avro",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-schema-protobuf",
|
||||
"gradle_path": ":messaging:messaging-schema-protobuf",
|
||||
"source_path": "src/messaging/messaging-schema-protobuf",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-cloudevents",
|
||||
"gradle_path": ":messaging:messaging-cloudevents",
|
||||
"source_path": "src/messaging/messaging-cloudevents",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-policy",
|
||||
"gradle_path": ":messaging:messaging-policy",
|
||||
"source_path": "src/messaging/messaging-policy",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-transport-spi",
|
||||
"gradle_path": ":messaging:messaging-transport-spi",
|
||||
"source_path": "src/messaging/messaging-transport-spi",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api",
|
||||
"messaging-policy"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-observability",
|
||||
"gradle_path": ":messaging:messaging-observability",
|
||||
"source_path": "src/messaging/messaging-observability",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-security",
|
||||
"gradle_path": ":messaging:messaging-security",
|
||||
"source_path": "src/messaging/messaging-security",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-kafka",
|
||||
"gradle_path": ":messaging:messaging-kafka",
|
||||
"source_path": "src/messaging/messaging-kafka",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api",
|
||||
"messaging-policy",
|
||||
"messaging-transport-spi",
|
||||
"messaging-observability",
|
||||
"messaging-security",
|
||||
"messaging-admin-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-kafka-share-experimental",
|
||||
"gradle_path": ":messaging:messaging-kafka-share-experimental",
|
||||
"source_path": "src/messaging/messaging-kafka-share-experimental",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-policy",
|
||||
"messaging-transport-spi",
|
||||
"messaging-kafka"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-rabbit",
|
||||
"gradle_path": ":messaging:messaging-rabbit",
|
||||
"source_path": "src/messaging/messaging-rabbit",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api",
|
||||
"messaging-policy",
|
||||
"messaging-transport-spi",
|
||||
"messaging-observability",
|
||||
"messaging-security",
|
||||
"messaging-admin-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-reliability-api",
|
||||
"gradle_path": ":messaging:messaging-reliability-api",
|
||||
"source_path": "src/messaging/messaging-reliability-api",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-outbox-jpa",
|
||||
"gradle_path": ":messaging:messaging-outbox-jpa",
|
||||
"source_path": "src/messaging/messaging-outbox-jpa",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-reliability-api",
|
||||
"messaging-policy",
|
||||
"messaging-observability"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-inbox-jpa",
|
||||
"gradle_path": ":messaging:messaging-inbox-jpa",
|
||||
"source_path": "src/messaging/messaging-inbox-jpa",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-reliability-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-claim-check",
|
||||
"gradle_path": ":messaging:messaging-claim-check",
|
||||
"source_path": "src/messaging/messaging-claim-check",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-reliability-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-admin-api",
|
||||
"gradle_path": ":messaging:messaging-admin-api",
|
||||
"source_path": "src/messaging/messaging-admin-api",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-policy"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-admin-runtime",
|
||||
"gradle_path": ":messaging:messaging-admin-runtime",
|
||||
"source_path": "src/messaging/messaging-admin-runtime",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-policy",
|
||||
"messaging-admin-api",
|
||||
"messaging-transport-spi",
|
||||
"messaging-security",
|
||||
"messaging-observability"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-pulsar-experimental",
|
||||
"gradle_path": ":messaging:messaging-pulsar-experimental",
|
||||
"source_path": "src/messaging/messaging-pulsar-experimental",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api",
|
||||
"messaging-policy",
|
||||
"messaging-transport-spi",
|
||||
"messaging-observability",
|
||||
"messaging-security",
|
||||
"messaging-admin-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-nats-experimental",
|
||||
"gradle_path": ":messaging:messaging-nats-experimental",
|
||||
"source_path": "src/messaging/messaging-nats-experimental",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api",
|
||||
"messaging-policy",
|
||||
"messaging-transport-spi",
|
||||
"messaging-observability",
|
||||
"messaging-security",
|
||||
"messaging-admin-api"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-spring-cloud-stream-bridge",
|
||||
"gradle_path": ":messaging:messaging-spring-cloud-stream-bridge",
|
||||
"source_path": "src/messaging/messaging-spring-cloud-stream-bridge",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-policy",
|
||||
"messaging-transport-spi"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-spring-boot-starter",
|
||||
"gradle_path": ":messaging:messaging-spring-boot-starter",
|
||||
"source_path": "src/messaging/messaging-spring-boot-starter",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api",
|
||||
"messaging-schema-json",
|
||||
"messaging-cloudevents",
|
||||
"messaging-policy",
|
||||
"messaging-transport-spi",
|
||||
"messaging-observability",
|
||||
"messaging-security",
|
||||
"messaging-kafka",
|
||||
"messaging-rabbit",
|
||||
"messaging-reliability-api",
|
||||
"messaging-outbox-jpa",
|
||||
"messaging-inbox-jpa",
|
||||
"messaging-claim-check",
|
||||
"messaging-admin-api",
|
||||
"messaging-admin-runtime"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
},
|
||||
{
|
||||
"id": "messaging-testkit",
|
||||
"gradle_path": ":messaging:messaging-testkit",
|
||||
"source_path": "src/messaging/messaging-testkit",
|
||||
"allowed_dependencies": [
|
||||
"messaging-core-api",
|
||||
"messaging-schema-api",
|
||||
"messaging-policy",
|
||||
"messaging-transport-spi"
|
||||
],
|
||||
"runtime_memberships": []
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -43,4 +43,24 @@
|
||||
<Source name="DefaultTypingFixture.java"/>
|
||||
</Match>
|
||||
|
||||
<!-- The messaging codec retention test measures whether a round trip retains per-message state.
|
||||
Measuring retained memory requires forcing a collection first and again at the end;
|
||||
without it the "after" reading is dominated by garbage that simply had not been collected
|
||||
yet and the test could never distinguish a leak from ordinary allocation. Scoped to the
|
||||
one test class, so an explicit gc anywhere else stays reportable. -->
|
||||
<Match>
|
||||
<Bug pattern="DM_GC"/>
|
||||
<Source name="PlatformOverheadPerformanceTest.java"/>
|
||||
</Match>
|
||||
|
||||
<!-- The outbox and inbox container tests apply the real shipped Flyway migration by reading it
|
||||
from the classpath and executing it, which is the whole point: a test that re-declared the
|
||||
schema inline would certify a schema nothing deploys. The SQL is a build artifact, not
|
||||
input, and the database is a throwaway container. Scoped to these two classes so any other
|
||||
dynamic SQL stays reportable. -->
|
||||
<Match>
|
||||
<Bug pattern="SQL_NONCONSTANT_STRING_PASSED_TO_EXECUTE"/>
|
||||
<Source name="~(Inbox|Outbox)PostgresIT.java"/>
|
||||
</Match>
|
||||
|
||||
</FindBugsFilter>
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
apply plugin: 'java-library'
|
||||
|
||||
dependencies {
|
||||
api project(':messaging:messaging-core-api')
|
||||
api project(':messaging:messaging-policy')
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
# This is a Gradle generated file for dependency locking.
|
||||
# Manual edits can break the build and are not advised.
|
||||
# This file is expected to be part of source control.
|
||||
com.github.ben-manes.caffeine:caffeine:3.2.3=annotationProcessor,testAnnotationProcessor
|
||||
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor
|
||||
com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs
|
||||
com.github.spotbugs:spotbugs:4.10.2=spotbugs
|
||||
com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs
|
||||
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor
|
||||
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor
|
||||
com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor
|
||||
com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs
|
||||
com.google.code.gson:gson:2.13.2=spotbugs
|
||||
com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_annotations:2.41.0=spotbugs
|
||||
com.google.errorprone:error_prone_annotations:2.47.0=checkstyle
|
||||
com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor
|
||||
com.google.guava:guava:33.6.0-jre=checkstyle
|
||||
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor
|
||||
com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins
|
||||
com.puppycrawl.tools:checkstyle:13.5.0=checkstyle
|
||||
commons-beanutils:commons-beanutils:1.11.0=checkstyle
|
||||
commons-collections:commons-collections:3.2.2=checkstyle
|
||||
commons-io:commons-io:2.21.0=spotbugs
|
||||
info.picocli:picocli:4.7.7=checkstyle
|
||||
io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor
|
||||
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor
|
||||
javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor
|
||||
jaxen:jaxen:2.0.0=spotbugs
|
||||
net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath
|
||||
net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs
|
||||
org.antlr:antlr4-runtime:4.13.2=checkstyle
|
||||
org.apache.bcel:bcel:6.12.0=spotbugs
|
||||
org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs
|
||||
org.apache.commons:commons-text:1.15.0=spotbugs
|
||||
org.apache.commons:commons-text:1.3=checkstyle
|
||||
org.apache.httpcomponents:httpclient:4.5.13=checkstyle
|
||||
org.apache.httpcomponents:httpcore:4.4.16=checkstyle
|
||||
org.apache.logging.log4j:log4j-api:2.25.2=spotbugs
|
||||
org.apache.logging.log4j:log4j-core:2.25.2=spotbugs
|
||||
org.apache.maven.doxia:doxia-core:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle
|
||||
org.apache.xbean:xbean-reflect:3.7=checkstyle
|
||||
org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath
|
||||
org.assertj:assertj-core:3.27.6=testCompileClasspath,testRuntimeClasspath
|
||||
org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle
|
||||
org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle
|
||||
org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle
|
||||
org.codehaus.plexus:plexus-utils:3.3.0=checkstyle
|
||||
org.dom4j:dom4j:2.2.0=spotbugs
|
||||
org.javassist:javassist:3.28.0-GA=checkstyle
|
||||
org.jspecify:jspecify:1.0.0=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath
|
||||
org.junit.jupiter:junit-jupiter-api:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-engine:6.0.1=testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-params:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-commons:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-engine:6.0.1=testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-launcher:6.0.1=testRuntimeClasspath
|
||||
org.junit:junit-bom:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit:junit-bom:6.1.0=spotbugs
|
||||
org.mockito:mockito-core:5.20.0=mockitoAgent
|
||||
org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath
|
||||
org.ow2.asm:asm-analysis:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-commons:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-tree:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-util:9.10.1=spotbugs
|
||||
org.ow2.asm:asm:9.10.1=spotbugs
|
||||
org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor
|
||||
org.reflections:reflections:0.10.2=checkstyle
|
||||
org.slf4j:slf4j-api:2.0.17=spotbugs,spotbugsSlf4j
|
||||
org.slf4j:slf4j-simple:2.0.17=checkstyle,spotbugsSlf4j
|
||||
org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs
|
||||
empty=compileClasspath,runtimeClasspath
|
||||
+45
@@ -0,0 +1,45 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* The approval that authorises a destructive messaging operation.
|
||||
*
|
||||
* <p>Approvals expire. An open-ended approval becomes a standing permission, which is the same
|
||||
* thing as no approval at all — the window is what keeps "we approved a redrive last quarter" from
|
||||
* authorising one today.
|
||||
*
|
||||
* @param ticket the change reference
|
||||
* @param approvedBy the approver's identity
|
||||
* @param approvedAt when the approval was granted
|
||||
* @param validUntil when the approval stops authorising anything
|
||||
*/
|
||||
public record AdminApproval(
|
||||
String ticket, String approvedBy, Instant approvedAt, Instant validUntil) {
|
||||
|
||||
public AdminApproval {
|
||||
Objects.requireNonNull(approvedAt, "approvedAt must not be null");
|
||||
Objects.requireNonNull(validUntil, "validUntil must not be null");
|
||||
if (ticket == null || ticket.isBlank()) {
|
||||
throw new IllegalArgumentException("ticket must not be blank");
|
||||
}
|
||||
if (approvedBy == null || approvedBy.isBlank()) {
|
||||
throw new IllegalArgumentException("approvedBy must not be blank");
|
||||
}
|
||||
if (validUntil.isBefore(approvedAt)) {
|
||||
throw new IllegalArgumentException("an approval cannot expire before it was granted");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reports whether the approval still authorises an operation.
|
||||
*
|
||||
* @param now the current instant
|
||||
* @return true while inside the window
|
||||
*/
|
||||
public boolean isValidAt(Instant now) {
|
||||
Objects.requireNonNull(now, "now must not be null");
|
||||
return !now.isBefore(approvedAt) && now.isBefore(validUntil);
|
||||
}
|
||||
}
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.MessageAuthorizationException;
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* A redrive plan that a named human has approved.
|
||||
*
|
||||
* <p>Carries {@code loopAcknowledged} separately from the approval itself. Approving a redrive of
|
||||
* 900 parked messages and approving a redrive that will re-fail 400 of them are different
|
||||
* decisions, and the second one needs the approver to have seen the number — so the plan cannot
|
||||
* execute on a loop-risking set unless that was acknowledged explicitly.
|
||||
*
|
||||
* @param plan the plan that was approved
|
||||
* @param approval the approval authorising it
|
||||
* @param loopAcknowledged whether the approver accepted the previously-redriven candidates
|
||||
*/
|
||||
public record ApprovedRedrivePlan(
|
||||
RedrivePlan plan, AdminApproval approval, boolean loopAcknowledged) {
|
||||
|
||||
public ApprovedRedrivePlan {
|
||||
Objects.requireNonNull(plan, "plan must not be null");
|
||||
Objects.requireNonNull(approval, "approval must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Refuses execution when the approval, the topology, or the loop risk no longer permits it.
|
||||
*
|
||||
* @param now the current instant
|
||||
* @param currentTopologyVersion the topology version at execution time
|
||||
* @throws MessageAuthorizationException when the plan may not execute
|
||||
*/
|
||||
public void requireExecutable(Instant now, String currentTopologyVersion) {
|
||||
Objects.requireNonNull(now, "now must not be null");
|
||||
Objects.requireNonNull(currentTopologyVersion, "currentTopologyVersion must not be null");
|
||||
|
||||
if (!approval.isValidAt(now)) {
|
||||
throw new MessageAuthorizationException(
|
||||
"APPROVAL_EXPIRED", "approval %s is not valid at %s".formatted(approval.ticket(), now));
|
||||
}
|
||||
if (!plan.topologyVersion().equals(currentTopologyVersion)) {
|
||||
throw new MessageAuthorizationException(
|
||||
"TOPOLOGY_CHANGED_SINCE_APPROVAL",
|
||||
"the plan was approved against topology %s but the broker is now at %s"
|
||||
.formatted(plan.topologyVersion(), currentTopologyVersion));
|
||||
}
|
||||
if (plan.risksALoop() && !loopAcknowledged) {
|
||||
throw new MessageAuthorizationException(
|
||||
"REDRIVE_LOOP_NOT_ACKNOWLEDGED",
|
||||
"%d of the %d candidates already failed a previous redrive; re-running them without "
|
||||
.formatted(plan.alreadyRedrivenCandidates(), plan.candidates())
|
||||
+ "fixing the cause produces a loop that looks like progress");
|
||||
}
|
||||
}
|
||||
}
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.MessageAuthorizationException;
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* A replay plan that a named human has approved.
|
||||
*
|
||||
* <p>A distinct type from {@link ReplayPlan} rather than a boolean on it. The execute method takes
|
||||
* this type, so an unapproved plan cannot reach it — the authorisation is enforced by the compiler
|
||||
* instead of by a runtime check somebody can forget to write.
|
||||
*
|
||||
* @param plan the plan that was approved
|
||||
* @param approval the approval authorising it
|
||||
*/
|
||||
public record ApprovedReplayPlan(ReplayPlan plan, AdminApproval approval) {
|
||||
|
||||
public ApprovedReplayPlan {
|
||||
Objects.requireNonNull(plan, "plan must not be null");
|
||||
Objects.requireNonNull(approval, "approval must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Refuses execution when the approval or the plan no longer applies.
|
||||
*
|
||||
* @param now the current instant
|
||||
* @param currentTopologyVersion the topology version at execution time
|
||||
* @throws MessageAuthorizationException when the approval has expired or the topology moved
|
||||
*/
|
||||
public void requireExecutable(Instant now, String currentTopologyVersion) {
|
||||
Objects.requireNonNull(now, "now must not be null");
|
||||
Objects.requireNonNull(currentTopologyVersion, "currentTopologyVersion must not be null");
|
||||
|
||||
if (!approval.isValidAt(now)) {
|
||||
throw new MessageAuthorizationException(
|
||||
"APPROVAL_EXPIRED", "approval %s is not valid at %s".formatted(approval.ticket(), now));
|
||||
}
|
||||
if (!plan.topologyVersion().equals(currentTopologyVersion)) {
|
||||
// Every number in the plan was computed against the old topology, so the approver agreed to
|
||||
// an impact estimate that no longer describes what would happen.
|
||||
throw new MessageAuthorizationException(
|
||||
"TOPOLOGY_CHANGED_SINCE_APPROVAL",
|
||||
"the plan was approved against topology %s but the broker is now at %s; the estimated "
|
||||
.formatted(plan.topologyVersion(), currentTopologyVersion)
|
||||
+ "impact no longer applies and the plan must be rebuilt");
|
||||
}
|
||||
}
|
||||
}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* What the broker actually reports for one destination.
|
||||
*
|
||||
* <p>The counterpart to {@link TopologyManifest}: the manifest is what was declared, this is what
|
||||
* exists. Kept as a separate type rather than reusing the manifest so that a comparison cannot
|
||||
* accidentally compare a manifest with itself and report success.
|
||||
*
|
||||
* @param physicalName the broker-side name
|
||||
* @param partitions the observed partition count
|
||||
* @param replicationFactor the observed replication factor
|
||||
* @param configuration the observed configuration entries
|
||||
* @param exists whether the destination is present at all
|
||||
*/
|
||||
public record DestinationTopology(
|
||||
String physicalName,
|
||||
int partitions,
|
||||
int replicationFactor,
|
||||
Map<String, String> configuration,
|
||||
boolean exists) {
|
||||
|
||||
public DestinationTopology {
|
||||
Objects.requireNonNull(configuration, "configuration must not be null");
|
||||
if (physicalName == null || physicalName.isBlank()) {
|
||||
throw new IllegalArgumentException("physicalName must not be blank");
|
||||
}
|
||||
if (exists && partitions < 1) {
|
||||
throw new IllegalArgumentException("an existing destination has at least one partition");
|
||||
}
|
||||
configuration = Map.copyOf(configuration);
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the topology for a destination the broker does not have.
|
||||
*
|
||||
* @param physicalName the broker-side name that was looked up
|
||||
* @return the absent topology
|
||||
*/
|
||||
public static DestinationTopology absent(String physicalName) {
|
||||
return new DestinationTopology(physicalName, 0, 0, Map.of(), false);
|
||||
}
|
||||
}
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
/**
|
||||
* The operations that cannot be undone.
|
||||
*
|
||||
* <p>Enumerated so the guard is exhaustive rather than a list of {@code if} statements that a new
|
||||
* operation can quietly avoid.
|
||||
*/
|
||||
public enum DestructiveOperation {
|
||||
|
||||
/** Re-read a destination from an earlier position. */
|
||||
REPLAY,
|
||||
|
||||
/** Move messages from a dead letter destination back to their source. */
|
||||
REDRIVE,
|
||||
|
||||
/** Move a consumer group's committed position. */
|
||||
OFFSET_RESET,
|
||||
|
||||
/** Discard the contents of a destination. */
|
||||
PURGE,
|
||||
|
||||
/** Remove a destination entirely. */
|
||||
DELETE_DESTINATION
|
||||
}
|
||||
+73
@@ -0,0 +1,73 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import dev.caskeleton.messaging.api.destination.DestinationName;
|
||||
import dev.caskeleton.messaging.api.error.MessageAuthorizationException;
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* The single gate every destructive messaging operation passes through.
|
||||
*
|
||||
* <p>Three conditions, all required. The caller must hold the admin credential — an application
|
||||
* runtime does not, by construction. The approval must be present and inside its validity window.
|
||||
* And a dry run is always permitted, because the way to make operators plan before they act is to
|
||||
* make planning free.
|
||||
*
|
||||
* <p>Centralised so that adding a new destructive operation means adding an enum constant, not
|
||||
* remembering to re-implement the checks.
|
||||
*/
|
||||
public final class DestructiveOperationGuard {
|
||||
|
||||
private final boolean adminCredentialPresent;
|
||||
|
||||
/**
|
||||
* Creates a guard for a runtime.
|
||||
*
|
||||
* @param adminCredentialPresent whether this runtime holds the admin credential
|
||||
*/
|
||||
public DestructiveOperationGuard(boolean adminCredentialPresent) {
|
||||
this.adminCredentialPresent = adminCredentialPresent;
|
||||
}
|
||||
|
||||
/**
|
||||
* Authorises one destructive operation.
|
||||
*
|
||||
* @param operation the operation
|
||||
* @param destination the destination affected
|
||||
* @param approval the approval, when one was supplied
|
||||
* @param dryRun whether this is a plan-only run
|
||||
* @param now the current instant
|
||||
* @throws MessageAuthorizationException when the operation is not authorised
|
||||
*/
|
||||
public void authorize(
|
||||
DestructiveOperation operation,
|
||||
DestinationName destination,
|
||||
Optional<AdminApproval> approval,
|
||||
boolean dryRun,
|
||||
Instant now) {
|
||||
Objects.requireNonNull(operation, "operation must not be null");
|
||||
Objects.requireNonNull(destination, "destination must not be null");
|
||||
Objects.requireNonNull(approval, "approval must not be null");
|
||||
Objects.requireNonNull(now, "now must not be null");
|
||||
|
||||
if (dryRun) {
|
||||
return;
|
||||
}
|
||||
if (!adminCredentialPresent) {
|
||||
throw new MessageAuthorizationException(
|
||||
"ADMIN_CREDENTIAL_REQUIRED",
|
||||
operation + " on " + destination.value() + " requires the admin credential");
|
||||
}
|
||||
AdminApproval granted =
|
||||
approval.orElseThrow(
|
||||
() ->
|
||||
new MessageAuthorizationException(
|
||||
"APPROVAL_REQUIRED",
|
||||
operation + " on " + destination.value() + " requires an approval"));
|
||||
if (!granted.isValidAt(now)) {
|
||||
throw new MessageAuthorizationException(
|
||||
"APPROVAL_EXPIRED", "approval " + granted.ticket() + " is outside its validity window");
|
||||
}
|
||||
}
|
||||
}
|
||||
+64
@@ -0,0 +1,64 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* What a redrive would do, produced before anything is moved.
|
||||
*
|
||||
* <p>{@code alreadyRedrivenCandidates} is the number that matters most. A message with a non-zero
|
||||
* redrive count has been sent back to its source before and failed again; redriving it a second
|
||||
* time without fixing the cause produces a loop that looks like progress in every dashboard.
|
||||
* Surfacing the count at plan time is what lets an operator notice before starting it.
|
||||
*
|
||||
* @param request the request this plan was built from
|
||||
* @param candidates how many messages are eligible
|
||||
* @param alreadyRedrivenCandidates how many of those have been redriven before
|
||||
* @param plannedAt when the plan was produced
|
||||
* @param topologyVersion the topology the estimate was computed against
|
||||
*/
|
||||
public record RedrivePlan(
|
||||
RedriveRequest request,
|
||||
int candidates,
|
||||
int alreadyRedrivenCandidates,
|
||||
Instant plannedAt,
|
||||
String topologyVersion) {
|
||||
|
||||
public RedrivePlan {
|
||||
Objects.requireNonNull(request, "request must not be null");
|
||||
Objects.requireNonNull(plannedAt, "plannedAt must not be null");
|
||||
if (candidates < 0) {
|
||||
throw new IllegalArgumentException("candidates must not be negative");
|
||||
}
|
||||
if (alreadyRedrivenCandidates < 0 || alreadyRedrivenCandidates > candidates) {
|
||||
throw new IllegalArgumentException(
|
||||
"alreadyRedrivenCandidates must be between 0 and the candidate count");
|
||||
}
|
||||
if (topologyVersion == null || topologyVersion.isBlank()) {
|
||||
throw new IllegalArgumentException("topologyVersion must not be blank");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reports whether this redrive would replay messages that already failed a redrive.
|
||||
*
|
||||
* @return true when any candidate has been redriven before
|
||||
*/
|
||||
public boolean risksALoop() {
|
||||
return alreadyRedrivenCandidates > 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a one-line operator-facing summary of the impact.
|
||||
*
|
||||
* @return the sanitized summary
|
||||
*/
|
||||
public String describeImpact() {
|
||||
String loop =
|
||||
risksALoop()
|
||||
? ", %d of which already failed a previous redrive".formatted(alreadyRedrivenCandidates)
|
||||
: "";
|
||||
return "redrive %d messages from %s to %s%s"
|
||||
.formatted(candidates, request.source().value(), request.target().value(), loop);
|
||||
}
|
||||
}
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import dev.caskeleton.messaging.api.destination.DestinationName;
|
||||
import java.util.Objects;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* A request to move messages from a dead letter destination back to their source.
|
||||
*
|
||||
* <p>The redrive id is a separate identifier from the message id, and both travel with the message.
|
||||
* Reusing the message id as the operation id would make "this message was redriven" and "this
|
||||
* message is a different message" indistinguishable, and an operator could not tell a redrive loop
|
||||
* from ordinary traffic.
|
||||
*
|
||||
* @param redriveId the operation identity
|
||||
* @param source the dead letter destination to drain
|
||||
* @param target the destination to publish back to
|
||||
* @param batchSize how many messages to move per pass
|
||||
* @param dryRun whether to plan without moving anything
|
||||
*/
|
||||
public record RedriveRequest(
|
||||
UUID redriveId, DestinationName source, DestinationName target, int batchSize, boolean dryRun) {
|
||||
|
||||
private static final int MAX_BATCH = 100;
|
||||
|
||||
public RedriveRequest {
|
||||
Objects.requireNonNull(redriveId, "redriveId must not be null");
|
||||
Objects.requireNonNull(source, "source must not be null");
|
||||
Objects.requireNonNull(target, "target must not be null");
|
||||
if (source.equals(target)) {
|
||||
throw new IllegalArgumentException("a redrive cannot target its own source");
|
||||
}
|
||||
if (batchSize < 1 || batchSize > MAX_BATCH) {
|
||||
throw new IllegalArgumentException("redrive batch size must be between 1 and " + MAX_BATCH);
|
||||
}
|
||||
}
|
||||
}
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.Objects;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* What a redrive actually did.
|
||||
*
|
||||
* <p>{@code stillParked} is not simply {@code candidates - moved}. A message stays parked when its
|
||||
* republish did not confirm, and the redrive deliberately leaves it there rather than settling it —
|
||||
* the DLQ-confirm-before-settle rule applies to a redrive exactly as it does to the original
|
||||
* dead-lettering, because a redrive that settles an unconfirmed republish deletes the last copy.
|
||||
*
|
||||
* @param redriveId the operation identity
|
||||
* @param candidates how many messages were eligible
|
||||
* @param moved how many were republished and settled
|
||||
* @param stillParked how many stayed on the dead letter destination
|
||||
* @param elapsed how long the redrive took
|
||||
* @param dryRun whether nothing was actually moved
|
||||
*/
|
||||
public record RedriveResult(
|
||||
UUID redriveId, int candidates, int moved, int stillParked, Duration elapsed, boolean dryRun) {
|
||||
|
||||
public RedriveResult {
|
||||
Objects.requireNonNull(redriveId, "redriveId must not be null");
|
||||
Objects.requireNonNull(elapsed, "elapsed must not be null");
|
||||
if (candidates < 0 || moved < 0 || stillParked < 0) {
|
||||
throw new IllegalArgumentException("redrive counters must not be negative");
|
||||
}
|
||||
if (moved + stillParked > candidates) {
|
||||
throw new IllegalArgumentException(
|
||||
"a redrive cannot account for more messages than it had candidates");
|
||||
}
|
||||
if (dryRun && moved > 0) {
|
||||
throw new IllegalArgumentException("a dry run must not move anything");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reports whether every candidate was accounted for.
|
||||
*
|
||||
* <p>An unaccounted message is a bug, not a partial success: it was neither republished nor left
|
||||
* parked, which means the redrive lost track of it.
|
||||
*
|
||||
* @return true when moved plus still-parked covers every candidate
|
||||
*/
|
||||
public boolean isFullyAccounted() {
|
||||
return moved + stillParked == candidates;
|
||||
}
|
||||
}
|
||||
+55
@@ -0,0 +1,55 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* What a replay would do, produced before anything is read.
|
||||
*
|
||||
* <p>The plan exists so the estimate can be reviewed. "Replay from yesterday" is a sentence; "this
|
||||
* will re-deliver 4.2 million messages into the live consumer group" is a decision, and the only
|
||||
* moment an operator can make it is before the replay starts.
|
||||
*
|
||||
* <p>{@code topologyVersion} is captured here and re-checked at execution. A plan approved against
|
||||
* one topology and executed against another is estimating a different thing entirely — a partition
|
||||
* count that changed in between invalidates every number in this record.
|
||||
*
|
||||
* @param request the request this plan was built from
|
||||
* @param estimatedMessages how many messages the window covers
|
||||
* @param plannedAt when the plan was produced
|
||||
* @param topologyVersion the topology the estimate was computed against
|
||||
* @param targetsLiveConsumerGroup whether the replay would feed the live group rather than an
|
||||
* isolated one
|
||||
*/
|
||||
public record ReplayPlan(
|
||||
ReplayRequest request,
|
||||
long estimatedMessages,
|
||||
Instant plannedAt,
|
||||
String topologyVersion,
|
||||
boolean targetsLiveConsumerGroup) {
|
||||
|
||||
public ReplayPlan {
|
||||
Objects.requireNonNull(request, "request must not be null");
|
||||
Objects.requireNonNull(plannedAt, "plannedAt must not be null");
|
||||
if (estimatedMessages < 0) {
|
||||
throw new IllegalArgumentException("estimatedMessages must not be negative");
|
||||
}
|
||||
if (topologyVersion == null || topologyVersion.isBlank()) {
|
||||
throw new IllegalArgumentException("topologyVersion must not be blank");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a one-line operator-facing summary of the impact.
|
||||
*
|
||||
* @return the sanitized summary
|
||||
*/
|
||||
public String describeImpact() {
|
||||
return "replay %s from %s: about %d messages into %s"
|
||||
.formatted(
|
||||
request.destination().value(),
|
||||
request.from(),
|
||||
estimatedMessages,
|
||||
targetsLiveConsumerGroup ? "the LIVE consumer group" : "an isolated group");
|
||||
}
|
||||
}
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import dev.caskeleton.messaging.api.destination.DestinationName;
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* A request to re-read a destination from an earlier position.
|
||||
*
|
||||
* @param replayId the operation identity
|
||||
* @param destination the destination to replay
|
||||
* @param from the replay start point
|
||||
* @param to the replay end point, when bounded
|
||||
* @param isolatedConsumerGroup whether to replay into a throwaway group
|
||||
* @param dryRun whether to plan without reading anything
|
||||
*/
|
||||
public record ReplayRequest(
|
||||
UUID replayId,
|
||||
DestinationName destination,
|
||||
Instant from,
|
||||
Optional<Instant> to,
|
||||
boolean isolatedConsumerGroup,
|
||||
boolean dryRun) {
|
||||
|
||||
public ReplayRequest {
|
||||
Objects.requireNonNull(replayId, "replayId must not be null");
|
||||
Objects.requireNonNull(destination, "destination must not be null");
|
||||
Objects.requireNonNull(from, "from must not be null");
|
||||
Objects.requireNonNull(to, "to must not be null");
|
||||
if (to.filter(end -> end.isBefore(from)).isPresent()) {
|
||||
throw new IllegalArgumentException("replay window ends before it starts");
|
||||
}
|
||||
}
|
||||
}
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.Objects;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* What a replay actually did.
|
||||
*
|
||||
* <p>Reports the delivered count against the plan's estimate. They routinely differ — retention may
|
||||
* have expired part of the window, or the stream may have grown while the plan was being approved —
|
||||
* and the difference is the operator's signal that the replay covered something other than what was
|
||||
* approved.
|
||||
*
|
||||
* @param replayId the operation identity
|
||||
* @param estimatedMessages what the plan predicted
|
||||
* @param deliveredMessages what was actually re-delivered
|
||||
* @param elapsed how long the replay took
|
||||
* @param completed whether the whole window was covered
|
||||
* @param dryRun whether nothing was actually read
|
||||
*/
|
||||
public record ReplayResult(
|
||||
UUID replayId,
|
||||
long estimatedMessages,
|
||||
long deliveredMessages,
|
||||
Duration elapsed,
|
||||
boolean completed,
|
||||
boolean dryRun) {
|
||||
|
||||
public ReplayResult {
|
||||
Objects.requireNonNull(replayId, "replayId must not be null");
|
||||
Objects.requireNonNull(elapsed, "elapsed must not be null");
|
||||
if (estimatedMessages < 0 || deliveredMessages < 0) {
|
||||
throw new IllegalArgumentException("replay counters must not be negative");
|
||||
}
|
||||
if (dryRun && deliveredMessages > 0) {
|
||||
throw new IllegalArgumentException("a dry run must not deliver anything");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reports whether the replay covered materially less than was approved.
|
||||
*
|
||||
* @return true when fewer than 90% of the estimated messages were delivered
|
||||
*/
|
||||
public boolean fellShortOfTheEstimate() {
|
||||
return completed && estimatedMessages > 0 && deliveredMessages * 10 < estimatedMessages * 9;
|
||||
}
|
||||
}
|
||||
+82
@@ -0,0 +1,82 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* One discrepancy between a declared topology and what the broker actually has.
|
||||
*
|
||||
* <p>Severity is part of the finding because the two kinds behave differently at startup. A {@link
|
||||
* Severity#BLOCKING} issue means the destination cannot deliver its declared guarantee —
|
||||
* replication factor 1 on a destination promising durability is not a warning, it is a promise the
|
||||
* platform cannot keep — so the context refuses to start. A {@link Severity#ADVISORY} issue is a
|
||||
* drift worth reporting that does not break a guarantee.
|
||||
*
|
||||
* @param destination the logical destination
|
||||
* @param attribute the topology attribute that differs
|
||||
* @param declared what the manifest declared
|
||||
* @param actual what the broker reported
|
||||
* @param severity how the platform should react
|
||||
*/
|
||||
public record TopologyIssue(
|
||||
String destination, String attribute, String declared, String actual, Severity severity) {
|
||||
|
||||
/** How the platform reacts to a topology discrepancy. */
|
||||
public enum Severity {
|
||||
/** The destination cannot deliver a declared guarantee; startup must fail. */
|
||||
BLOCKING,
|
||||
/** Drift worth reporting that does not break a guarantee. */
|
||||
ADVISORY
|
||||
}
|
||||
|
||||
public TopologyIssue {
|
||||
Objects.requireNonNull(severity, "severity must not be null");
|
||||
requireText(destination, "destination");
|
||||
requireText(attribute, "attribute");
|
||||
requireText(declared, "declared");
|
||||
requireText(actual, "actual");
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a blocking issue.
|
||||
*
|
||||
* @param destination the logical destination
|
||||
* @param attribute the differing attribute
|
||||
* @param declared the declared value
|
||||
* @param actual the observed value
|
||||
* @return the issue
|
||||
*/
|
||||
public static TopologyIssue blocking(
|
||||
String destination, String attribute, String declared, String actual) {
|
||||
return new TopologyIssue(destination, attribute, declared, actual, Severity.BLOCKING);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates an advisory issue.
|
||||
*
|
||||
* @param destination the logical destination
|
||||
* @param attribute the differing attribute
|
||||
* @param declared the declared value
|
||||
* @param actual the observed value
|
||||
* @return the issue
|
||||
*/
|
||||
public static TopologyIssue advisory(
|
||||
String destination, String attribute, String declared, String actual) {
|
||||
return new TopologyIssue(destination, attribute, declared, actual, Severity.ADVISORY);
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a one-line operator-facing description.
|
||||
*
|
||||
* @return the sanitized description
|
||||
*/
|
||||
public String describe() {
|
||||
return "%s: %s declared %s but the broker has %s"
|
||||
.formatted(destination, attribute, declared, actual);
|
||||
}
|
||||
|
||||
private static void requireText(String value, String field) {
|
||||
if (value == null || value.isBlank()) {
|
||||
throw new IllegalArgumentException(field + " must not be blank");
|
||||
}
|
||||
}
|
||||
}
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.MessagingConfigurationException;
|
||||
|
||||
/**
|
||||
* Whether the application may create broker topology, or only check it.
|
||||
*
|
||||
* <p>{@link #VALIDATE_ONLY} in production, always. An application that auto-creates topology will
|
||||
* auto-create it after a configuration typo too, and the topic it makes is indistinguishable from a
|
||||
* real one — same broker, same client, same metrics — while carrying the broker's default partition
|
||||
* count and replication factor instead of the ones the destination needs. The failure surfaces
|
||||
* weeks later as data loss on a partition that was never replicated.
|
||||
*/
|
||||
public enum TopologyManagementMode {
|
||||
|
||||
/** Compare the declared topology against the broker and fail on a mismatch. */
|
||||
VALIDATE_ONLY,
|
||||
|
||||
/** Create missing topology. Permitted outside production only. */
|
||||
CREATE_IF_MISSING;
|
||||
|
||||
/**
|
||||
* Refuses auto-creation on a production runtime.
|
||||
*
|
||||
* @param production whether this runtime is production
|
||||
* @throws MessagingConfigurationException when auto-creation is configured in production
|
||||
*/
|
||||
public void requireSafeFor(boolean production) {
|
||||
if (production && this == CREATE_IF_MISSING) {
|
||||
throw new MessagingConfigurationException(
|
||||
"AUTO_CREATE_IN_PRODUCTION",
|
||||
"topology auto-creation is not permitted in production: a mistyped destination would be "
|
||||
+ "created with the broker's default partition count and replication factor, and "
|
||||
+ "would look exactly like a correctly provisioned one");
|
||||
}
|
||||
}
|
||||
}
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* The declared shape of a destination's broker topology.
|
||||
*
|
||||
* <p>Production topology is created by infrastructure code and only <em>validated</em> by the
|
||||
* application. An application that creates topology on startup will happily create it in the wrong
|
||||
* place after a configuration mistake, and the resulting topic looks exactly like a real one.
|
||||
*
|
||||
* @param destination the logical destination
|
||||
* @param physicalName the broker-side name
|
||||
* @param partitions the expected partition count, where the broker has partitions
|
||||
* @param replicationFactor the expected replication factor
|
||||
* @param requiredConfiguration configuration entries that must match exactly
|
||||
*/
|
||||
public record TopologyManifest(
|
||||
String destination,
|
||||
String physicalName,
|
||||
int partitions,
|
||||
int replicationFactor,
|
||||
Map<String, String> requiredConfiguration) {
|
||||
|
||||
public TopologyManifest {
|
||||
Objects.requireNonNull(requiredConfiguration, "requiredConfiguration must not be null");
|
||||
if (destination == null || destination.isBlank()) {
|
||||
throw new IllegalArgumentException("destination must not be blank");
|
||||
}
|
||||
if (physicalName == null || physicalName.isBlank()) {
|
||||
throw new IllegalArgumentException("physicalName must not be blank");
|
||||
}
|
||||
if (partitions < 1) {
|
||||
throw new IllegalArgumentException("partitions must be at least 1");
|
||||
}
|
||||
if (replicationFactor < 1) {
|
||||
throw new IllegalArgumentException("replicationFactor must be at least 1");
|
||||
}
|
||||
requiredConfiguration = Map.copyOf(requiredConfiguration);
|
||||
}
|
||||
|
||||
/**
|
||||
* Compares this manifest against what the broker actually reports.
|
||||
*
|
||||
* @param actualPartitions the observed partition count
|
||||
* @param actualReplicationFactor the observed replication factor
|
||||
* @param actualConfiguration the observed configuration
|
||||
* @return the differences, empty when the topology matches
|
||||
*/
|
||||
public List<String> differencesFrom(
|
||||
int actualPartitions, int actualReplicationFactor, Map<String, String> actualConfiguration) {
|
||||
Objects.requireNonNull(actualConfiguration, "actualConfiguration must not be null");
|
||||
List<String> differences = new java.util.ArrayList<>();
|
||||
|
||||
if (actualPartitions != partitions) {
|
||||
differences.add("partitions expected " + partitions + " but found " + actualPartitions);
|
||||
}
|
||||
if (actualReplicationFactor != replicationFactor) {
|
||||
differences.add(
|
||||
"replicationFactor expected "
|
||||
+ replicationFactor
|
||||
+ " but found "
|
||||
+ actualReplicationFactor);
|
||||
}
|
||||
requiredConfiguration.forEach(
|
||||
(key, expected) -> {
|
||||
String actual = actualConfiguration.get(key);
|
||||
if (!expected.equals(actual)) {
|
||||
differences.add(key + " expected " + expected + " but found " + actual);
|
||||
}
|
||||
});
|
||||
return List.copyOf(differences);
|
||||
}
|
||||
}
|
||||
+83
@@ -0,0 +1,83 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.MessagingConfigurationException;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* The outcome of comparing every declared topology against the broker.
|
||||
*
|
||||
* <p>Reports both severities together rather than failing on the first blocking issue. An operator
|
||||
* fixing a topology wants the whole list — fixing one attribute, redeploying, and discovering the
|
||||
* next one is how a ten-minute fix becomes an afternoon.
|
||||
*
|
||||
* @param issues every discrepancy found
|
||||
* @param destinationsChecked how many destinations were compared
|
||||
*/
|
||||
public record TopologyValidationReport(List<TopologyIssue> issues, int destinationsChecked) {
|
||||
|
||||
public TopologyValidationReport {
|
||||
Objects.requireNonNull(issues, "issues must not be null");
|
||||
if (destinationsChecked < 0) {
|
||||
throw new IllegalArgumentException("destinationsChecked must not be negative");
|
||||
}
|
||||
issues = List.copyOf(issues);
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a report with nothing to fix.
|
||||
*
|
||||
* @param destinationsChecked how many destinations were compared
|
||||
* @return the clean report
|
||||
*/
|
||||
public static TopologyValidationReport clean(int destinationsChecked) {
|
||||
return new TopologyValidationReport(List.of(), destinationsChecked);
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the issues that must fail startup.
|
||||
*
|
||||
* @return the blocking issues
|
||||
*/
|
||||
public List<TopologyIssue> blocking() {
|
||||
return issues.stream()
|
||||
.filter(issue -> issue.severity() == TopologyIssue.Severity.BLOCKING)
|
||||
.toList();
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the issues worth reporting that do not break a guarantee.
|
||||
*
|
||||
* @return the advisory issues
|
||||
*/
|
||||
public List<TopologyIssue> advisory() {
|
||||
return issues.stream()
|
||||
.filter(issue -> issue.severity() == TopologyIssue.Severity.ADVISORY)
|
||||
.toList();
|
||||
}
|
||||
|
||||
/**
|
||||
* Reports whether the topology may be used.
|
||||
*
|
||||
* @return true when nothing blocking was found
|
||||
*/
|
||||
public boolean isAcceptable() {
|
||||
return blocking().isEmpty();
|
||||
}
|
||||
|
||||
/**
|
||||
* Fails startup when any blocking issue was found.
|
||||
*
|
||||
* @throws MessagingConfigurationException listing every blocking issue
|
||||
*/
|
||||
public void requireAcceptable() {
|
||||
List<TopologyIssue> blocking = blocking();
|
||||
if (blocking.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
throw new MessagingConfigurationException(
|
||||
"TOPOLOGY_MISMATCH",
|
||||
"the broker topology cannot deliver the declared guarantees: "
|
||||
+ String.join("; ", blocking.stream().map(TopologyIssue::describe).toList()));
|
||||
}
|
||||
}
|
||||
+116
@@ -0,0 +1,116 @@
|
||||
package dev.caskeleton.messaging.admin;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatCode;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import dev.caskeleton.messaging.api.destination.DestinationName;
|
||||
import dev.caskeleton.messaging.api.error.MessageAuthorizationException;
|
||||
import java.time.Instant;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.UUID;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class DestructiveOperationGuardTest {
|
||||
|
||||
private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z");
|
||||
private static final DestinationName ORDERS = new DestinationName("order-events");
|
||||
|
||||
private static final AdminApproval VALID =
|
||||
new AdminApproval("CHG-1001", "operator", NOW.minusSeconds(60), NOW.plusSeconds(3600));
|
||||
|
||||
@Test
|
||||
void anApplicationRuntimeCannotRedrive() {
|
||||
DestructiveOperationGuard guard = new DestructiveOperationGuard(false);
|
||||
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
guard.authorize(
|
||||
DestructiveOperation.REDRIVE, ORDERS, Optional.of(VALID), false, NOW))
|
||||
.isInstanceOf(MessageAuthorizationException.class)
|
||||
.hasMessageContaining("admin credential");
|
||||
}
|
||||
|
||||
@Test
|
||||
void anAdminRuntimeStillNeedsAnApproval() {
|
||||
DestructiveOperationGuard guard = new DestructiveOperationGuard(true);
|
||||
|
||||
assertThatThrownBy(
|
||||
() -> guard.authorize(DestructiveOperation.PURGE, ORDERS, Optional.empty(), false, NOW))
|
||||
.isInstanceOf(MessageAuthorizationException.class)
|
||||
.hasMessageContaining("approval");
|
||||
}
|
||||
|
||||
@Test
|
||||
void anExpiredApprovalDoesNotAuthorise() {
|
||||
DestructiveOperationGuard guard = new DestructiveOperationGuard(true);
|
||||
AdminApproval expired =
|
||||
new AdminApproval("CHG-1000", "operator", NOW.minusSeconds(7200), NOW.minusSeconds(60));
|
||||
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
guard.authorize(
|
||||
DestructiveOperation.OFFSET_RESET, ORDERS, Optional.of(expired), false, NOW))
|
||||
.isInstanceOf(MessageAuthorizationException.class)
|
||||
.hasMessageContaining("validity window");
|
||||
}
|
||||
|
||||
@Test
|
||||
void aDryRunIsAlwaysPermitted() {
|
||||
DestructiveOperationGuard guard = new DestructiveOperationGuard(false);
|
||||
|
||||
assertThatCode(
|
||||
() ->
|
||||
guard.authorize(
|
||||
DestructiveOperation.DELETE_DESTINATION, ORDERS, Optional.empty(), true, NOW))
|
||||
.doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
@Test
|
||||
void anApprovedAdminOperationIsAuthorised() {
|
||||
DestructiveOperationGuard guard = new DestructiveOperationGuard(true);
|
||||
|
||||
assertThatCode(
|
||||
() ->
|
||||
guard.authorize(
|
||||
DestructiveOperation.REPLAY, ORDERS, Optional.of(VALID), false, NOW))
|
||||
.doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
@Test
|
||||
void aRedriveCannotTargetItsOwnSource() {
|
||||
assertThatThrownBy(() -> new RedriveRequest(UUID.randomUUID(), ORDERS, ORDERS, 100, false))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void aRedriveBatchIsBoundedSoOneOperationCannotFloodTheSource() {
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new RedriveRequest(
|
||||
UUID.randomUUID(), new DestinationName("order-events-dlq"), ORDERS, 101, false))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void aTopologyManifestReportsEveryDifference() {
|
||||
TopologyManifest manifest =
|
||||
new TopologyManifest(
|
||||
"order-events", "order.events.v1", 6, 3, Map.of("min.insync.replicas", "2"));
|
||||
|
||||
assertThat(manifest.differencesFrom(3, 3, Map.of("min.insync.replicas", "1")))
|
||||
.hasSize(2)
|
||||
.anySatisfy(difference -> assertThat(difference).contains("partitions"))
|
||||
.anySatisfy(difference -> assertThat(difference).contains("min.insync.replicas"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void aMatchingTopologyReportsNoDifferences() {
|
||||
TopologyManifest manifest =
|
||||
new TopologyManifest(
|
||||
"order-events", "order.events.v1", 6, 3, Map.of("min.insync.replicas", "2"));
|
||||
|
||||
assertThat(manifest.differencesFrom(6, 3, Map.of("min.insync.replicas", "2"))).isEmpty();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
apply plugin: 'java-library'
|
||||
|
||||
dependencies {
|
||||
api project(':messaging:messaging-core-api')
|
||||
api project(':messaging:messaging-policy')
|
||||
api project(':messaging:messaging-admin-api')
|
||||
api project(':messaging:messaging-transport-spi')
|
||||
api project(':messaging:messaging-security')
|
||||
api project(':messaging:messaging-observability')
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
# This is a Gradle generated file for dependency locking.
|
||||
# Manual edits can break the build and are not advised.
|
||||
# This file is expected to be part of source control.
|
||||
com.github.ben-manes.caffeine:caffeine:3.2.3=annotationProcessor,testAnnotationProcessor
|
||||
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor
|
||||
com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs
|
||||
com.github.spotbugs:spotbugs:4.10.2=spotbugs
|
||||
com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs
|
||||
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor
|
||||
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor
|
||||
com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor
|
||||
com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs
|
||||
com.google.code.gson:gson:2.13.2=spotbugs
|
||||
com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_annotations:2.41.0=spotbugs
|
||||
com.google.errorprone:error_prone_annotations:2.47.0=checkstyle
|
||||
com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor
|
||||
com.google.guava:guava:33.6.0-jre=checkstyle
|
||||
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor
|
||||
com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins
|
||||
com.puppycrawl.tools:checkstyle:13.5.0=checkstyle
|
||||
commons-beanutils:commons-beanutils:1.11.0=checkstyle
|
||||
commons-collections:commons-collections:3.2.2=checkstyle
|
||||
commons-io:commons-io:2.21.0=spotbugs
|
||||
info.picocli:picocli:4.7.7=checkstyle
|
||||
io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor
|
||||
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor
|
||||
io.micrometer:micrometer-commons:1.16.0=runtimeClasspath,testRuntimeClasspath
|
||||
io.micrometer:micrometer-core:1.16.0=runtimeClasspath,testRuntimeClasspath
|
||||
io.micrometer:micrometer-observation:1.16.0=runtimeClasspath,testRuntimeClasspath
|
||||
javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor
|
||||
jaxen:jaxen:2.0.0=spotbugs
|
||||
net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath
|
||||
net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs
|
||||
org.antlr:antlr4-runtime:4.13.2=checkstyle
|
||||
org.apache.bcel:bcel:6.12.0=spotbugs
|
||||
org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs
|
||||
org.apache.commons:commons-text:1.15.0=spotbugs
|
||||
org.apache.commons:commons-text:1.3=checkstyle
|
||||
org.apache.httpcomponents:httpclient:4.5.13=checkstyle
|
||||
org.apache.httpcomponents:httpcore:4.4.16=checkstyle
|
||||
org.apache.logging.log4j:log4j-api:2.25.2=spotbugs
|
||||
org.apache.logging.log4j:log4j-core:2.25.2=spotbugs
|
||||
org.apache.maven.doxia:doxia-core:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle
|
||||
org.apache.xbean:xbean-reflect:3.7=checkstyle
|
||||
org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath
|
||||
org.assertj:assertj-core:3.27.6=testCompileClasspath,testRuntimeClasspath
|
||||
org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle
|
||||
org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle
|
||||
org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle
|
||||
org.codehaus.plexus:plexus-utils:3.3.0=checkstyle
|
||||
org.dom4j:dom4j:2.2.0=spotbugs
|
||||
org.hdrhistogram:HdrHistogram:2.2.2=runtimeClasspath,testRuntimeClasspath
|
||||
org.javassist:javassist:3.28.0-GA=checkstyle
|
||||
org.jspecify:jspecify:1.0.0=annotationProcessor,checkstyle,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-api:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-engine:6.0.1=testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-params:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-commons:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-engine:6.0.1=testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-launcher:6.0.1=testRuntimeClasspath
|
||||
org.junit:junit-bom:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit:junit-bom:6.1.0=spotbugs
|
||||
org.latencyutils:LatencyUtils:2.0.3=runtimeClasspath,testRuntimeClasspath
|
||||
org.mockito:mockito-core:5.20.0=mockitoAgent
|
||||
org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath
|
||||
org.ow2.asm:asm-analysis:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-commons:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-tree:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-util:9.10.1=spotbugs
|
||||
org.ow2.asm:asm:9.10.1=spotbugs
|
||||
org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor
|
||||
org.reflections:reflections:0.10.2=checkstyle
|
||||
org.slf4j:slf4j-api:2.0.17=spotbugs,spotbugsSlf4j
|
||||
org.slf4j:slf4j-simple:2.0.17=checkstyle,spotbugsSlf4j
|
||||
org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs
|
||||
empty=compileClasspath
|
||||
+78
@@ -0,0 +1,78 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
/**
|
||||
* Remembers which approvals have already been executed.
|
||||
*
|
||||
* <p>An approval authorises one execution, not a standing permission. Without this, re-running the
|
||||
* same approved redrive twice is a single command-history arrow-up away — and the second run
|
||||
* republishes messages the first one already moved, which on a destination without an inbox is
|
||||
* indistinguishable from a duplicate storm.
|
||||
*
|
||||
* <p>Claiming is atomic and returns the previous claim rather than a boolean, so a duplicate
|
||||
* attempt can tell the operator <em>when</em> it ran and by which operation id instead of just
|
||||
* refusing.
|
||||
*/
|
||||
public final class AdminOperationIdempotencyStore {
|
||||
|
||||
private final Map<String, Claim> claims = new ConcurrentHashMap<>();
|
||||
|
||||
/**
|
||||
* One recorded execution of an approval.
|
||||
*
|
||||
* @param approvalTicket the approval that was executed
|
||||
* @param operationId the operation identity that claimed it
|
||||
* @param executedAt when it ran
|
||||
*/
|
||||
public record Claim(String approvalTicket, String operationId, Instant executedAt) {
|
||||
|
||||
public Claim {
|
||||
Objects.requireNonNull(executedAt, "executedAt must not be null");
|
||||
if (approvalTicket == null || approvalTicket.isBlank()) {
|
||||
throw new IllegalArgumentException("approvalTicket must not be blank");
|
||||
}
|
||||
if (operationId == null || operationId.isBlank()) {
|
||||
throw new IllegalArgumentException("operationId must not be blank");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Claims an approval for execution.
|
||||
*
|
||||
* @param approvalTicket the approval to claim
|
||||
* @param operationId the operation attempting it
|
||||
* @param now the current instant
|
||||
* @return empty when the claim succeeded; the existing claim when it was already executed
|
||||
*/
|
||||
public Optional<Claim> claim(String approvalTicket, String operationId, Instant now) {
|
||||
Objects.requireNonNull(now, "now must not be null");
|
||||
Claim candidate = new Claim(approvalTicket, operationId, now);
|
||||
Claim existing = claims.putIfAbsent(approvalTicket, candidate);
|
||||
return Optional.ofNullable(existing);
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the recorded execution of an approval.
|
||||
*
|
||||
* @param approvalTicket the approval
|
||||
* @return the claim, when the approval has been executed
|
||||
*/
|
||||
public Optional<Claim> find(String approvalTicket) {
|
||||
return Optional.ofNullable(claims.get(approvalTicket));
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns how many approvals have been executed.
|
||||
*
|
||||
* @return the claim count
|
||||
*/
|
||||
public int size() {
|
||||
return claims.size();
|
||||
}
|
||||
}
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import dev.caskeleton.messaging.admin.DestinationTopology;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Reads the broker's current topology.
|
||||
*
|
||||
* <p>Read-only by construction. The inspector is what the application's own credential uses, and
|
||||
* that credential holds no destructive grant, so the type exposes no way to create, alter, or
|
||||
* delete — an application cannot reach a destructive operation even by mistake because there is no
|
||||
* method to reach.
|
||||
*/
|
||||
public interface BrokerTopologyInspector {
|
||||
|
||||
/**
|
||||
* Describes one destination.
|
||||
*
|
||||
* @param physicalName the broker-side name
|
||||
* @return the observed topology, absent when the broker has no such destination
|
||||
*/
|
||||
Optional<DestinationTopology> describe(String physicalName);
|
||||
|
||||
/**
|
||||
* Returns an opaque version for the broker's current topology.
|
||||
*
|
||||
* <p>Used to invalidate an approved plan whose impact estimate was computed against an earlier
|
||||
* shape. Any value that changes when the topology changes is sufficient; it is never parsed.
|
||||
*
|
||||
* @return the topology version
|
||||
*/
|
||||
String topologyVersion();
|
||||
}
|
||||
+52
@@ -0,0 +1,52 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import dev.caskeleton.messaging.admin.DestinationTopology;
|
||||
import dev.caskeleton.messaging.admin.TopologyIssue;
|
||||
import dev.caskeleton.messaging.admin.TopologyManifest;
|
||||
import dev.caskeleton.messaging.admin.TopologyValidationReport;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* Validates every declared topology in one pass and reports the whole result.
|
||||
*
|
||||
* <p>Collects all issues rather than stopping at the first blocking one. An operator fixing a
|
||||
* topology mismatch wants the complete list — discovering the next problem only after a redeploy
|
||||
* turns one fix into a sequence of them, and each redeploy is another restart of a production
|
||||
* service.
|
||||
*/
|
||||
public final class CompositeTopologyValidator {
|
||||
|
||||
private final BrokerTopologyInspector inspector;
|
||||
private final TopologyValidator validator = new TopologyValidator();
|
||||
|
||||
/**
|
||||
* Creates a validator over a broker inspector.
|
||||
*
|
||||
* @param inspector reads the broker's current topology
|
||||
*/
|
||||
public CompositeTopologyValidator(BrokerTopologyInspector inspector) {
|
||||
this.inspector = Objects.requireNonNull(inspector, "inspector must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Compares every manifest against the broker.
|
||||
*
|
||||
* @param manifests the declared topologies
|
||||
* @return the complete report
|
||||
*/
|
||||
public TopologyValidationReport validate(List<TopologyManifest> manifests) {
|
||||
Objects.requireNonNull(manifests, "manifests must not be null");
|
||||
|
||||
List<TopologyIssue> issues = new ArrayList<>();
|
||||
for (TopologyManifest manifest : manifests) {
|
||||
DestinationTopology observed =
|
||||
inspector
|
||||
.describe(manifest.physicalName())
|
||||
.orElseGet(() -> DestinationTopology.absent(manifest.physicalName()));
|
||||
issues.addAll(validator.compare(manifest, observed));
|
||||
}
|
||||
return new TopologyValidationReport(issues, manifests.size());
|
||||
}
|
||||
}
|
||||
+199
@@ -0,0 +1,199 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import dev.caskeleton.messaging.admin.ApprovedRedrivePlan;
|
||||
import dev.caskeleton.messaging.admin.ApprovedReplayPlan;
|
||||
import dev.caskeleton.messaging.admin.RedrivePlan;
|
||||
import dev.caskeleton.messaging.admin.RedriveRequest;
|
||||
import dev.caskeleton.messaging.admin.RedriveResult;
|
||||
import dev.caskeleton.messaging.admin.ReplayPlan;
|
||||
import dev.caskeleton.messaging.admin.ReplayRequest;
|
||||
import dev.caskeleton.messaging.admin.ReplayResult;
|
||||
import dev.caskeleton.messaging.admin.TopologyManifest;
|
||||
import dev.caskeleton.messaging.admin.TopologyValidationReport;
|
||||
import dev.caskeleton.messaging.api.error.MessageAuthorizationException;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
import java.util.function.Supplier;
|
||||
|
||||
/**
|
||||
* Wires plan, approval, and execution together for the non-destructive admin operations.
|
||||
*
|
||||
* <p>Execution runs four checks, in this order, and the order is the point.
|
||||
*
|
||||
* <ol>
|
||||
* <li>The approval is still inside its window.
|
||||
* <li>The topology has not changed since the plan was approved.
|
||||
* <li>The approval has not already been executed.
|
||||
* <li>Only then does anything move.
|
||||
* </ol>
|
||||
*
|
||||
* <p>The idempotency claim comes <em>before</em> the work rather than after it. Claiming afterwards
|
||||
* leaves a window where a second execution starts while the first is still running, which is
|
||||
* precisely the double-redrive this store exists to prevent.
|
||||
*/
|
||||
public final class DefaultMessagingAdminService implements MessagingAdminService {
|
||||
|
||||
private final CompositeTopologyValidator topologyValidator;
|
||||
private final BrokerTopologyInspector inspector;
|
||||
private final AdminOperationIdempotencyStore idempotency;
|
||||
private final ReplayService replayService;
|
||||
private final RedriveService redriveService;
|
||||
private final Supplier<List<TopologyManifest>> manifests;
|
||||
private final ReplayEstimator replayEstimator;
|
||||
private final RedriveEstimator redriveEstimator;
|
||||
private final Supplier<Instant> clock;
|
||||
|
||||
/**
|
||||
* Creates the admin service.
|
||||
*
|
||||
* @param topologyValidator compares declared topology against the broker
|
||||
* @param inspector reads the broker's topology version
|
||||
* @param idempotency records which approvals have been executed
|
||||
* @param replayService performs replays
|
||||
* @param redriveService performs redrives
|
||||
* @param manifests supplies the declared topologies
|
||||
* @param replayEstimator estimates a replay's message count
|
||||
* @param redriveEstimator estimates a redrive's candidates
|
||||
* @param clock supplies the current instant
|
||||
*/
|
||||
public DefaultMessagingAdminService(
|
||||
CompositeTopologyValidator topologyValidator,
|
||||
BrokerTopologyInspector inspector,
|
||||
AdminOperationIdempotencyStore idempotency,
|
||||
ReplayService replayService,
|
||||
RedriveService redriveService,
|
||||
Supplier<List<TopologyManifest>> manifests,
|
||||
ReplayEstimator replayEstimator,
|
||||
RedriveEstimator redriveEstimator,
|
||||
Supplier<Instant> clock) {
|
||||
this.topologyValidator =
|
||||
Objects.requireNonNull(topologyValidator, "topologyValidator required");
|
||||
this.inspector = Objects.requireNonNull(inspector, "inspector must not be null");
|
||||
this.idempotency = Objects.requireNonNull(idempotency, "idempotency must not be null");
|
||||
this.replayService = Objects.requireNonNull(replayService, "replayService must not be null");
|
||||
this.redriveService = Objects.requireNonNull(redriveService, "redriveService must not be null");
|
||||
this.manifests = Objects.requireNonNull(manifests, "manifests must not be null");
|
||||
this.replayEstimator = Objects.requireNonNull(replayEstimator, "replayEstimator required");
|
||||
this.redriveEstimator = Objects.requireNonNull(redriveEstimator, "redriveEstimator required");
|
||||
this.clock = Objects.requireNonNull(clock, "clock must not be null");
|
||||
}
|
||||
|
||||
@Override
|
||||
public TopologyValidationReport validateTopology() {
|
||||
return topologyValidator.validate(manifests.get());
|
||||
}
|
||||
|
||||
@Override
|
||||
public ReplayPlan planReplay(ReplayRequest request) {
|
||||
Objects.requireNonNull(request, "request must not be null");
|
||||
return new ReplayPlan(
|
||||
request,
|
||||
replayEstimator.estimate(request),
|
||||
clock.get(),
|
||||
inspector.topologyVersion(),
|
||||
!request.isolatedConsumerGroup());
|
||||
}
|
||||
|
||||
@Override
|
||||
public ReplayResult executeReplay(ApprovedReplayPlan plan) {
|
||||
Objects.requireNonNull(plan, "plan must not be null");
|
||||
Instant now = clock.get();
|
||||
ReplayRequest request = plan.plan().request();
|
||||
|
||||
plan.requireExecutable(now, inspector.topologyVersion());
|
||||
claimOrRefuse(plan.approval().ticket(), request.replayId().toString(), now);
|
||||
|
||||
Instant startedAt = clock.get();
|
||||
ReplayReport report =
|
||||
replayService.replay(
|
||||
request, Optional.of(plan.approval()), plan.approval().approvedBy(), now);
|
||||
|
||||
return new ReplayResult(
|
||||
request.replayId(),
|
||||
plan.plan().estimatedMessages(),
|
||||
report.replayed(),
|
||||
Duration.between(startedAt, clock.get()),
|
||||
true,
|
||||
report.dryRun());
|
||||
}
|
||||
|
||||
@Override
|
||||
public RedrivePlan planRedrive(RedriveRequest request) {
|
||||
Objects.requireNonNull(request, "request must not be null");
|
||||
RedriveEstimate estimate = redriveEstimator.estimate(request);
|
||||
return new RedrivePlan(
|
||||
request,
|
||||
estimate.candidates(),
|
||||
estimate.alreadyRedriven(),
|
||||
clock.get(),
|
||||
inspector.topologyVersion());
|
||||
}
|
||||
|
||||
@Override
|
||||
public RedriveResult executeRedrive(ApprovedRedrivePlan plan) {
|
||||
Objects.requireNonNull(plan, "plan must not be null");
|
||||
Instant now = clock.get();
|
||||
RedriveRequest request = plan.plan().request();
|
||||
|
||||
plan.requireExecutable(now, inspector.topologyVersion());
|
||||
claimOrRefuse(plan.approval().ticket(), request.redriveId().toString(), now);
|
||||
|
||||
Instant startedAt = clock.get();
|
||||
RedriveReport report =
|
||||
redriveService.redrive(
|
||||
request, Optional.of(plan.approval()), plan.approval().approvedBy(), now);
|
||||
|
||||
return new RedriveResult(
|
||||
request.redriveId(),
|
||||
report.candidates(),
|
||||
report.moved(),
|
||||
report.failed(),
|
||||
Duration.between(startedAt, clock.get()),
|
||||
report.dryRun());
|
||||
}
|
||||
|
||||
private void claimOrRefuse(String approvalTicket, String operationId, Instant now) {
|
||||
idempotency
|
||||
.claim(approvalTicket, operationId, now)
|
||||
.ifPresent(
|
||||
existing -> {
|
||||
throw new MessageAuthorizationException(
|
||||
"APPROVAL_ALREADY_EXECUTED",
|
||||
"approval %s was already executed at %s by operation %s; an approval authorises "
|
||||
.formatted(approvalTicket, existing.executedAt(), existing.operationId())
|
||||
+ "one execution, not a standing permission");
|
||||
});
|
||||
}
|
||||
|
||||
/** Estimates how many messages a replay would re-deliver. */
|
||||
@FunctionalInterface
|
||||
public interface ReplayEstimator {
|
||||
|
||||
/**
|
||||
* Estimates a replay's message count without reading anything.
|
||||
*
|
||||
* @param request the replay request
|
||||
* @return the estimated count
|
||||
*/
|
||||
long estimate(ReplayRequest request);
|
||||
}
|
||||
|
||||
/** How many messages a redrive would move, and how many already failed one. */
|
||||
record RedriveEstimate(int candidates, int alreadyRedriven) {}
|
||||
|
||||
/** Estimates a redrive's candidates. */
|
||||
@FunctionalInterface
|
||||
public interface RedriveEstimator {
|
||||
|
||||
/**
|
||||
* Estimates a redrive's candidates without moving anything.
|
||||
*
|
||||
* @param request the redrive request
|
||||
* @return the estimate
|
||||
*/
|
||||
RedriveEstimate estimate(RedriveRequest request);
|
||||
}
|
||||
}
|
||||
+82
@@ -0,0 +1,82 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import dev.caskeleton.messaging.admin.AdminApproval;
|
||||
import dev.caskeleton.messaging.admin.DestructiveOperation;
|
||||
import dev.caskeleton.messaging.api.destination.DestinationName;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* The operations that destroy data an application cannot recreate.
|
||||
*
|
||||
* <p>A separate interface from {@link MessagingAdminService}, and no bean for it is ever registered
|
||||
* in an application runtime. The separation is the control: an application that never receives this
|
||||
* type cannot purge a topic even if every other guard is bypassed, because the method does not
|
||||
* exist on anything it holds.
|
||||
*
|
||||
* <p>Each operation takes an {@link Approved} argument rather than an approval parameter, so the
|
||||
* authorisation cannot be forgotten at a call site — there is no way to call these without one.
|
||||
*/
|
||||
public interface DestructiveMessagingAdmin {
|
||||
|
||||
/** An authorised destructive request. */
|
||||
record Approved(
|
||||
DestructiveOperation operation,
|
||||
DestinationName destination,
|
||||
AdminApproval approval,
|
||||
long estimatedMessagesAffected) {
|
||||
|
||||
public Approved {
|
||||
Objects.requireNonNull(operation, "operation must not be null");
|
||||
Objects.requireNonNull(destination, "destination must not be null");
|
||||
Objects.requireNonNull(approval, "approval must not be null");
|
||||
if (estimatedMessagesAffected < 0) {
|
||||
throw new IllegalArgumentException("estimatedMessagesAffected must not be negative");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** What a destructive operation did. */
|
||||
record DestructiveResult(
|
||||
DestructiveOperation operation,
|
||||
DestinationName destination,
|
||||
long messagesAffected,
|
||||
Duration elapsed,
|
||||
Instant executedAt) {
|
||||
|
||||
public DestructiveResult {
|
||||
Objects.requireNonNull(operation, "operation must not be null");
|
||||
Objects.requireNonNull(destination, "destination must not be null");
|
||||
Objects.requireNonNull(elapsed, "elapsed must not be null");
|
||||
Objects.requireNonNull(executedAt, "executedAt must not be null");
|
||||
if (messagesAffected < 0) {
|
||||
throw new IllegalArgumentException("messagesAffected must not be negative");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Moves a consumer group's committed position.
|
||||
*
|
||||
* @param request the authorised request
|
||||
* @return what the reset did
|
||||
*/
|
||||
DestructiveResult resetOffset(Approved request);
|
||||
|
||||
/**
|
||||
* Discards the messages a destination currently holds.
|
||||
*
|
||||
* @param request the authorised request
|
||||
* @return what the purge did
|
||||
*/
|
||||
DestructiveResult purge(Approved request);
|
||||
|
||||
/**
|
||||
* Removes a destination entirely.
|
||||
*
|
||||
* @param request the authorised request
|
||||
* @return what the deletion did
|
||||
*/
|
||||
DestructiveResult deleteDestination(Approved request);
|
||||
}
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import dev.caskeleton.messaging.admin.ApprovedRedrivePlan;
|
||||
import dev.caskeleton.messaging.admin.ApprovedReplayPlan;
|
||||
import dev.caskeleton.messaging.admin.RedrivePlan;
|
||||
import dev.caskeleton.messaging.admin.RedriveRequest;
|
||||
import dev.caskeleton.messaging.admin.RedriveResult;
|
||||
import dev.caskeleton.messaging.admin.ReplayPlan;
|
||||
import dev.caskeleton.messaging.admin.ReplayRequest;
|
||||
import dev.caskeleton.messaging.admin.ReplayResult;
|
||||
import dev.caskeleton.messaging.admin.TopologyValidationReport;
|
||||
|
||||
/**
|
||||
* The non-destructive half of the admin plane.
|
||||
*
|
||||
* <p>Every mutating operation is split into plan and execute, and the execute methods take an
|
||||
* {@code Approved*} type. A caller cannot execute something it has not planned, because it has no
|
||||
* way to construct the argument — the plan/approve/execute sequence is enforced by the types rather
|
||||
* than by a runtime check.
|
||||
*
|
||||
* <p>Destructive operations live in {@link DestructiveMessagingAdmin}, a separate interface that an
|
||||
* application runtime never receives a bean for. Splitting them means a compromised handler that
|
||||
* somehow reaches this service still has no method that deletes anything.
|
||||
*/
|
||||
public interface MessagingAdminService {
|
||||
|
||||
/**
|
||||
* Compares every declared topology against the broker.
|
||||
*
|
||||
* @return the discrepancies found
|
||||
*/
|
||||
TopologyValidationReport validateTopology();
|
||||
|
||||
/**
|
||||
* Estimates what a replay would do, without reading anything.
|
||||
*
|
||||
* @param request the replay request
|
||||
* @return the plan, including its impact estimate
|
||||
*/
|
||||
ReplayPlan planReplay(ReplayRequest request);
|
||||
|
||||
/**
|
||||
* Executes an approved replay.
|
||||
*
|
||||
* @param plan the approved plan
|
||||
* @return what the replay actually did
|
||||
*/
|
||||
ReplayResult executeReplay(ApprovedReplayPlan plan);
|
||||
|
||||
/**
|
||||
* Estimates what a redrive would do, without moving anything.
|
||||
*
|
||||
* @param request the redrive request
|
||||
* @return the plan, including how many candidates already failed a redrive
|
||||
*/
|
||||
RedrivePlan planRedrive(RedriveRequest request);
|
||||
|
||||
/**
|
||||
* Executes an approved redrive.
|
||||
*
|
||||
* @param plan the approved plan
|
||||
* @return what the redrive actually did
|
||||
*/
|
||||
RedriveResult executeRedrive(ApprovedRedrivePlan plan);
|
||||
}
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
/**
|
||||
* What one redrive pass did.
|
||||
*
|
||||
* @param candidates how many messages were eligible
|
||||
* @param moved how many were republished and settled
|
||||
* @param failed how many did not confirm and stay parked
|
||||
* @param dryRun whether this was a plan-only run
|
||||
*/
|
||||
public record RedriveReport(int candidates, int moved, int failed, boolean dryRun) {
|
||||
|
||||
public RedriveReport {
|
||||
if (candidates < 0 || moved < 0 || failed < 0) {
|
||||
throw new IllegalArgumentException("redrive counters must not be negative");
|
||||
}
|
||||
}
|
||||
}
|
||||
+152
@@ -0,0 +1,152 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import dev.caskeleton.messaging.admin.AdminApproval;
|
||||
import dev.caskeleton.messaging.admin.DestructiveOperation;
|
||||
import dev.caskeleton.messaging.admin.DestructiveOperationGuard;
|
||||
import dev.caskeleton.messaging.admin.RedriveRequest;
|
||||
import dev.caskeleton.messaging.api.MessageId;
|
||||
import dev.caskeleton.messaging.api.publish.PublishCompletion;
|
||||
import dev.caskeleton.messaging.api.publish.PublishResult;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Moves messages from a dead letter destination back to their source.
|
||||
*
|
||||
* <p>A redrive is a publish followed by a settlement, in that order, exactly like dead lettering in
|
||||
* reverse. A message whose republish did not confirm stays in the dead letter destination: losing
|
||||
* it on the way back would be the one outcome worse than leaving it parked.
|
||||
*
|
||||
* <p>The redrive id and a redrive counter travel with each message. Without them a message that
|
||||
* fails again is indistinguishable from a new one, and a redrive loop is invisible until the dead
|
||||
* letter destination is full.
|
||||
*/
|
||||
public final class RedriveService {
|
||||
|
||||
private final DestructiveOperationGuard guard;
|
||||
private final RedriveSource source;
|
||||
private final RedrivePublisher publisher;
|
||||
private final AuditSink audit;
|
||||
|
||||
/**
|
||||
* Creates a redrive service.
|
||||
*
|
||||
* @param guard the destructive operation guard
|
||||
* @param source reads and settles dead letter messages
|
||||
* @param publisher republishes to the target destination
|
||||
* @param audit records the operation
|
||||
*/
|
||||
public RedriveService(
|
||||
DestructiveOperationGuard guard,
|
||||
RedriveSource source,
|
||||
RedrivePublisher publisher,
|
||||
AuditSink audit) {
|
||||
this.guard = Objects.requireNonNull(guard, "guard must not be null");
|
||||
this.source = Objects.requireNonNull(source, "source must not be null");
|
||||
this.publisher = Objects.requireNonNull(publisher, "publisher must not be null");
|
||||
this.audit = Objects.requireNonNull(audit, "audit must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs one redrive pass.
|
||||
*
|
||||
* @param request what to move
|
||||
* @param approval the approval, when one was supplied
|
||||
* @param subject the operator identity
|
||||
* @param now the current instant
|
||||
* @return what the pass did
|
||||
*/
|
||||
public RedriveReport redrive(
|
||||
RedriveRequest request, Optional<AdminApproval> approval, String subject, Instant now) {
|
||||
Objects.requireNonNull(request, "request must not be null");
|
||||
guard.authorize(
|
||||
DestructiveOperation.REDRIVE, request.source(), approval, request.dryRun(), now);
|
||||
|
||||
List<MessageId> candidates = source.peek(request.source(), request.batchSize());
|
||||
if (request.dryRun()) {
|
||||
return new RedriveReport(candidates.size(), 0, 0, true);
|
||||
}
|
||||
|
||||
List<MessageId> moved = new ArrayList<>();
|
||||
int failed = 0;
|
||||
for (MessageId messageId : candidates) {
|
||||
PublishResult result = publisher.republish(messageId, request.target(), request.redriveId());
|
||||
if (result.completion() == PublishCompletion.CONFIRMED) {
|
||||
source.settle(request.source(), messageId);
|
||||
moved.add(messageId);
|
||||
} else {
|
||||
failed++;
|
||||
}
|
||||
}
|
||||
|
||||
audit.record(
|
||||
new dev.caskeleton.messaging.observation.MessagingAuditEvent(
|
||||
"REDRIVE",
|
||||
subject,
|
||||
request.source().value(),
|
||||
approval.map(AdminApproval::ticket).orElse("dry-run"),
|
||||
now,
|
||||
java.util.Map.of(
|
||||
"redriveId", request.redriveId().toString(),
|
||||
"moved", Integer.toString(moved.size()),
|
||||
"failed", Integer.toString(failed))));
|
||||
|
||||
return new RedriveReport(candidates.size(), moved.size(), failed, false);
|
||||
}
|
||||
|
||||
/** Reads and settles messages on a dead letter destination. */
|
||||
public interface RedriveSource {
|
||||
|
||||
/**
|
||||
* Returns the next candidates without settling them.
|
||||
*
|
||||
* @param destination the dead letter destination
|
||||
* @param batchSize how many to return
|
||||
* @return the candidate identities
|
||||
*/
|
||||
List<MessageId> peek(
|
||||
dev.caskeleton.messaging.api.destination.DestinationName destination, int batchSize);
|
||||
|
||||
/**
|
||||
* Settles a message that has been successfully republished.
|
||||
*
|
||||
* @param destination the dead letter destination
|
||||
* @param messageId the message identity
|
||||
*/
|
||||
void settle(
|
||||
dev.caskeleton.messaging.api.destination.DestinationName destination, MessageId messageId);
|
||||
}
|
||||
|
||||
/** Republishes a dead lettered message to its target destination. */
|
||||
@FunctionalInterface
|
||||
public interface RedrivePublisher {
|
||||
|
||||
/**
|
||||
* Republishes one message under its original identity.
|
||||
*
|
||||
* @param messageId the message identity
|
||||
* @param target the destination to publish back to
|
||||
* @param redriveId the operation identity stamped on the message
|
||||
* @return the publish outcome
|
||||
*/
|
||||
PublishResult republish(
|
||||
MessageId messageId,
|
||||
dev.caskeleton.messaging.api.destination.DestinationName target,
|
||||
java.util.UUID redriveId);
|
||||
}
|
||||
|
||||
/** Records privileged operations. */
|
||||
@FunctionalInterface
|
||||
public interface AuditSink {
|
||||
|
||||
/**
|
||||
* Records one audit event.
|
||||
*
|
||||
* @param event the event
|
||||
*/
|
||||
void record(dev.caskeleton.messaging.observation.MessagingAuditEvent event);
|
||||
}
|
||||
}
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import java.util.Objects;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* What one replay did.
|
||||
*
|
||||
* @param replayId the operation identity
|
||||
* @param replayed how many messages were re-read
|
||||
* @param dryRun whether this was a plan-only run
|
||||
*/
|
||||
public record ReplayReport(UUID replayId, long replayed, boolean dryRun) {
|
||||
|
||||
public ReplayReport {
|
||||
Objects.requireNonNull(replayId, "replayId must not be null");
|
||||
if (replayed < 0) {
|
||||
throw new IllegalArgumentException("replayed must not be negative");
|
||||
}
|
||||
}
|
||||
}
|
||||
+99
@@ -0,0 +1,99 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import dev.caskeleton.messaging.admin.AdminApproval;
|
||||
import dev.caskeleton.messaging.admin.DestructiveOperation;
|
||||
import dev.caskeleton.messaging.admin.DestructiveOperationGuard;
|
||||
import dev.caskeleton.messaging.admin.ReplayRequest;
|
||||
import dev.caskeleton.messaging.observation.MessagingAuditEvent;
|
||||
import java.time.Instant;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Re-reads a destination from an earlier position.
|
||||
*
|
||||
* <p>An isolated replay reads alongside the live consumer and needs no approval, because it changes
|
||||
* nothing: a throwaway group has its own offsets. Replaying into an existing production group is a
|
||||
* different operation entirely — it rewinds a live consumer and reprocesses everything since — so
|
||||
* it goes through the destructive guard.
|
||||
*
|
||||
* <p>Making the safe form free and the destructive form approved is what keeps operators from
|
||||
* reaching for the destructive one out of convenience.
|
||||
*/
|
||||
public final class ReplayService {
|
||||
|
||||
private final DestructiveOperationGuard guard;
|
||||
private final ReplayExecutor executor;
|
||||
private final RedriveService.AuditSink audit;
|
||||
|
||||
/**
|
||||
* Creates a replay service.
|
||||
*
|
||||
* @param guard the destructive operation guard
|
||||
* @param executor performs the replay
|
||||
* @param audit records the operation
|
||||
*/
|
||||
public ReplayService(
|
||||
DestructiveOperationGuard guard, ReplayExecutor executor, RedriveService.AuditSink audit) {
|
||||
this.guard = Objects.requireNonNull(guard, "guard must not be null");
|
||||
this.executor = Objects.requireNonNull(executor, "executor must not be null");
|
||||
this.audit = Objects.requireNonNull(audit, "audit must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs one replay.
|
||||
*
|
||||
* @param request what to replay
|
||||
* @param approval the approval, when one was supplied
|
||||
* @param subject the operator identity
|
||||
* @param now the current instant
|
||||
* @return what the replay did
|
||||
*/
|
||||
public ReplayReport replay(
|
||||
ReplayRequest request, Optional<AdminApproval> approval, String subject, Instant now) {
|
||||
Objects.requireNonNull(request, "request must not be null");
|
||||
Objects.requireNonNull(approval, "approval must not be null");
|
||||
|
||||
boolean needsApproval = !request.isolatedConsumerGroup();
|
||||
guard.authorize(
|
||||
DestructiveOperation.REPLAY,
|
||||
request.destination(),
|
||||
approval,
|
||||
request.dryRun() || !needsApproval,
|
||||
now);
|
||||
|
||||
if (request.dryRun()) {
|
||||
return new ReplayReport(request.replayId(), 0, true);
|
||||
}
|
||||
|
||||
long replayed = executor.replay(request);
|
||||
|
||||
audit.record(
|
||||
new MessagingAuditEvent(
|
||||
"REPLAY",
|
||||
subject,
|
||||
request.destination().value(),
|
||||
approval.map(AdminApproval::ticket).orElse("isolated"),
|
||||
now,
|
||||
Map.of(
|
||||
"replayId", request.replayId().toString(),
|
||||
"isolated", Boolean.toString(request.isolatedConsumerGroup()),
|
||||
"replayed", Long.toString(replayed))));
|
||||
|
||||
return new ReplayReport(request.replayId(), replayed, false);
|
||||
}
|
||||
|
||||
/** Performs the replay against a broker. */
|
||||
@FunctionalInterface
|
||||
public interface ReplayExecutor {
|
||||
|
||||
/**
|
||||
* Replays a destination and returns how many messages were re-read.
|
||||
*
|
||||
* @param request the replay request
|
||||
* @return the replayed message count
|
||||
*/
|
||||
long replay(ReplayRequest request);
|
||||
}
|
||||
}
|
||||
+88
@@ -0,0 +1,88 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import dev.caskeleton.messaging.admin.TopologyManifest;
|
||||
import dev.caskeleton.messaging.api.error.MessageTopologyException;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* Validates declared topology against what the broker actually has.
|
||||
*
|
||||
* <p>Validate-only, and it fails startup rather than logging. A partition count that silently
|
||||
* differs from the manifest changes the ordering guarantee the destination advertises, and a
|
||||
* missing {@code min.insync.replicas} changes what {@code acks=all} actually means — both are the
|
||||
* kind of drift that is invisible until the incident.
|
||||
*/
|
||||
public final class TopologyValidationRuntime {
|
||||
|
||||
private final TopologyReader reader;
|
||||
|
||||
/**
|
||||
* Creates a validator over a broker reader.
|
||||
*
|
||||
* @param reader reads the observed topology
|
||||
*/
|
||||
public TopologyValidationRuntime(TopologyReader reader) {
|
||||
this.reader = Objects.requireNonNull(reader, "reader must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Validates every manifest, reporting all differences at once.
|
||||
*
|
||||
* @param manifests the declared topology
|
||||
* @throws MessageTopologyException when the broker does not match
|
||||
*/
|
||||
public void validate(List<TopologyManifest> manifests) {
|
||||
Objects.requireNonNull(manifests, "manifests must not be null");
|
||||
List<String> problems = new ArrayList<>();
|
||||
|
||||
for (TopologyManifest manifest : manifests) {
|
||||
ObservedTopology observed = reader.read(manifest.physicalName());
|
||||
if (observed == null) {
|
||||
problems.add(manifest.physicalName() + " does not exist");
|
||||
continue;
|
||||
}
|
||||
manifest
|
||||
.differencesFrom(
|
||||
observed.partitions(), observed.replicationFactor(), observed.configuration())
|
||||
.forEach(difference -> problems.add(manifest.physicalName() + ": " + difference));
|
||||
}
|
||||
|
||||
if (!problems.isEmpty()) {
|
||||
throw new MessageTopologyException(
|
||||
"TOPOLOGY_MISMATCH",
|
||||
"broker topology does not match the manifest: " + String.join("; ", problems));
|
||||
}
|
||||
}
|
||||
|
||||
/** Reads the observed topology for a physical destination. */
|
||||
@FunctionalInterface
|
||||
public interface TopologyReader {
|
||||
|
||||
/**
|
||||
* Reads one destination's topology.
|
||||
*
|
||||
* @param physicalName the broker-side name
|
||||
* @return the observed topology, or null when it does not exist
|
||||
*/
|
||||
ObservedTopology read(String physicalName);
|
||||
}
|
||||
|
||||
/**
|
||||
* What the broker reports for a destination.
|
||||
*
|
||||
* @param partitions the observed partition count
|
||||
* @param replicationFactor the observed replication factor
|
||||
* @param configuration the observed configuration
|
||||
*/
|
||||
public record ObservedTopology(
|
||||
int partitions, int replicationFactor, Map<String, String> configuration) {
|
||||
|
||||
public ObservedTopology {
|
||||
Objects.requireNonNull(configuration, "configuration must not be null");
|
||||
configuration = Map.copyOf(configuration);
|
||||
}
|
||||
}
|
||||
}
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import dev.caskeleton.messaging.admin.DestinationTopology;
|
||||
import dev.caskeleton.messaging.admin.TopologyIssue;
|
||||
import dev.caskeleton.messaging.admin.TopologyManifest;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* Compares one declared topology against what the broker reports.
|
||||
*
|
||||
* <p>Which discrepancies block is a judgement encoded here rather than left to configuration.
|
||||
* Replication factor and absence are blocking because a destination that is missing or unreplicated
|
||||
* cannot deliver the durability its profile promises. A partition count that is <em>higher</em>
|
||||
* than declared is advisory rather than blocking: extra partitions do not break durability, and
|
||||
* someone scaling a topic up deliberately should not be met with a refusal to start.
|
||||
*
|
||||
* <p>A partition count that is <em>lower</em> is blocking, because it silently reduces the
|
||||
* concurrency the destination was sized for and, on a keyed topic, changes which key lands where.
|
||||
*/
|
||||
public final class TopologyValidator {
|
||||
|
||||
/**
|
||||
* Compares a manifest against observed topology.
|
||||
*
|
||||
* @param manifest what was declared
|
||||
* @param observed what the broker reports
|
||||
* @return the discrepancies found, empty when they agree
|
||||
*/
|
||||
public List<TopologyIssue> compare(TopologyManifest manifest, DestinationTopology observed) {
|
||||
Objects.requireNonNull(manifest, "manifest must not be null");
|
||||
Objects.requireNonNull(observed, "observed must not be null");
|
||||
|
||||
List<TopologyIssue> issues = new ArrayList<>();
|
||||
String destination = manifest.destination();
|
||||
|
||||
if (!observed.exists()) {
|
||||
issues.add(
|
||||
TopologyIssue.blocking(destination, "existence", manifest.physicalName(), "absent"));
|
||||
return List.copyOf(issues);
|
||||
}
|
||||
|
||||
if (!manifest.physicalName().equals(observed.physicalName())) {
|
||||
issues.add(
|
||||
TopologyIssue.blocking(
|
||||
destination, "physicalName", manifest.physicalName(), observed.physicalName()));
|
||||
}
|
||||
|
||||
if (observed.partitions() < manifest.partitions()) {
|
||||
issues.add(
|
||||
TopologyIssue.blocking(
|
||||
destination,
|
||||
"partitions",
|
||||
Integer.toString(manifest.partitions()),
|
||||
Integer.toString(observed.partitions())));
|
||||
} else if (observed.partitions() > manifest.partitions()) {
|
||||
// Scaling a topic up is a legitimate operation; refusing to start would punish it.
|
||||
issues.add(
|
||||
TopologyIssue.advisory(
|
||||
destination,
|
||||
"partitions",
|
||||
Integer.toString(manifest.partitions()),
|
||||
Integer.toString(observed.partitions())));
|
||||
}
|
||||
|
||||
if (observed.replicationFactor() < manifest.replicationFactor()) {
|
||||
issues.add(
|
||||
TopologyIssue.blocking(
|
||||
destination,
|
||||
"replicationFactor",
|
||||
Integer.toString(manifest.replicationFactor()),
|
||||
Integer.toString(observed.replicationFactor())));
|
||||
}
|
||||
|
||||
for (Map.Entry<String, String> required : manifest.requiredConfiguration().entrySet()) {
|
||||
String actual = observed.configuration().get(required.getKey());
|
||||
if (!required.getValue().equals(actual)) {
|
||||
issues.add(
|
||||
TopologyIssue.blocking(
|
||||
destination,
|
||||
required.getKey(),
|
||||
required.getValue(),
|
||||
actual == null ? "unset" : actual));
|
||||
}
|
||||
}
|
||||
|
||||
return List.copyOf(issues);
|
||||
}
|
||||
}
|
||||
+190
@@ -0,0 +1,190 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatCode;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import dev.caskeleton.messaging.admin.AdminApproval;
|
||||
import dev.caskeleton.messaging.admin.ApprovedRedrivePlan;
|
||||
import dev.caskeleton.messaging.admin.ApprovedReplayPlan;
|
||||
import dev.caskeleton.messaging.admin.RedrivePlan;
|
||||
import dev.caskeleton.messaging.admin.RedriveRequest;
|
||||
import dev.caskeleton.messaging.admin.RedriveResult;
|
||||
import dev.caskeleton.messaging.admin.ReplayPlan;
|
||||
import dev.caskeleton.messaging.admin.ReplayRequest;
|
||||
import dev.caskeleton.messaging.admin.ReplayResult;
|
||||
import dev.caskeleton.messaging.api.destination.DestinationName;
|
||||
import dev.caskeleton.messaging.api.error.MessageAuthorizationException;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.Optional;
|
||||
import java.util.UUID;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class ApprovedPlanExecutionTest {
|
||||
|
||||
private static final Instant NOW = Instant.parse("2026-08-10T09:00:00Z");
|
||||
private static final UUID OPERATION_ID = UUID.fromString("0199aaaa-bbbb-7ccc-8ddd-eeeeffff0000");
|
||||
|
||||
private static AdminApproval approval() {
|
||||
return new AdminApproval("CHG-1042", "ops@example.com", NOW, NOW.plus(Duration.ofHours(2)));
|
||||
}
|
||||
|
||||
private static ReplayRequest replayRequest() {
|
||||
return new ReplayRequest(
|
||||
OPERATION_ID,
|
||||
new DestinationName("orders.v1"),
|
||||
NOW.minus(Duration.ofDays(1)),
|
||||
Optional.empty(),
|
||||
true,
|
||||
false);
|
||||
}
|
||||
|
||||
private static ReplayPlan replayPlan(String topologyVersion) {
|
||||
return new ReplayPlan(replayRequest(), 4_200_000, NOW, topologyVersion, false);
|
||||
}
|
||||
|
||||
private static RedriveRequest redriveRequest() {
|
||||
return new RedriveRequest(
|
||||
OPERATION_ID,
|
||||
new DestinationName("orders.v1.dlq"),
|
||||
new DestinationName("orders.v1"),
|
||||
50,
|
||||
false);
|
||||
}
|
||||
|
||||
private static RedrivePlan redrivePlan(int candidates, int alreadyRedriven) {
|
||||
return new RedrivePlan(redriveRequest(), candidates, alreadyRedriven, NOW, "v1");
|
||||
}
|
||||
|
||||
@Test
|
||||
void anExpiredApprovalCannotExecute() {
|
||||
ApprovedReplayPlan approved = new ApprovedReplayPlan(replayPlan("v1"), approval());
|
||||
|
||||
assertThatThrownBy(() -> approved.requireExecutable(NOW.plus(Duration.ofDays(1)), "v1"))
|
||||
.isInstanceOf(MessageAuthorizationException.class)
|
||||
.hasMessageContaining("CHG-1042");
|
||||
}
|
||||
|
||||
@Test
|
||||
void aTopologyChangeSinceApprovalInvalidatesThePlan() {
|
||||
ApprovedReplayPlan approved = new ApprovedReplayPlan(replayPlan("v1"), approval());
|
||||
|
||||
assertThatThrownBy(() -> approved.requireExecutable(NOW, "v2"))
|
||||
.as("every number in the plan was computed against the old topology")
|
||||
.isInstanceOf(MessageAuthorizationException.class)
|
||||
.hasMessageContaining("rebuilt");
|
||||
}
|
||||
|
||||
@Test
|
||||
void aValidApprovalOnUnchangedTopologyExecutes() {
|
||||
assertThatCode(
|
||||
() -> new ApprovedReplayPlan(replayPlan("v1"), approval()).requireExecutable(NOW, "v1"))
|
||||
.doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
@Test
|
||||
void aRedriveThatWouldLoopNeedsThatAcknowledgedExplicitly() {
|
||||
ApprovedRedrivePlan approved =
|
||||
new ApprovedRedrivePlan(redrivePlan(900, 400), approval(), false);
|
||||
|
||||
assertThatThrownBy(() -> approved.requireExecutable(NOW, "v1"))
|
||||
.isInstanceOf(MessageAuthorizationException.class)
|
||||
.hasMessageContaining("looks like progress");
|
||||
}
|
||||
|
||||
@Test
|
||||
void anAcknowledgedLoopMayProceed() {
|
||||
assertThatCode(
|
||||
() ->
|
||||
new ApprovedRedrivePlan(redrivePlan(900, 400), approval(), true)
|
||||
.requireExecutable(NOW, "v1"))
|
||||
.doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
@Test
|
||||
void aRedriveWithNoPreviouslyRedrivenCandidatesNeedsNoAcknowledgement() {
|
||||
assertThatCode(
|
||||
() ->
|
||||
new ApprovedRedrivePlan(redrivePlan(900, 0), approval(), false)
|
||||
.requireExecutable(NOW, "v1"))
|
||||
.doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
@Test
|
||||
void thePlanDescribesItsImpactBeforeAnythingRuns() {
|
||||
assertThat(replayPlan("v1").describeImpact()).contains("4200000").contains("isolated group");
|
||||
assertThat(redrivePlan(900, 400).describeImpact())
|
||||
.contains("already failed a previous redrive");
|
||||
}
|
||||
|
||||
@Test
|
||||
void aReplayIntoTheLiveGroupSaysSoInCapitals() {
|
||||
ReplayPlan live =
|
||||
new ReplayPlan(
|
||||
new ReplayRequest(
|
||||
OPERATION_ID,
|
||||
new DestinationName("orders.v1"),
|
||||
NOW.minus(Duration.ofDays(1)),
|
||||
Optional.empty(),
|
||||
false,
|
||||
false),
|
||||
4_200_000,
|
||||
NOW,
|
||||
"v1",
|
||||
true);
|
||||
|
||||
assertThat(live.describeImpact())
|
||||
.as("re-delivering millions of messages into the live group is the decision to flag")
|
||||
.contains("LIVE consumer group");
|
||||
}
|
||||
|
||||
@Test
|
||||
void anApprovalAuthorisesOneExecutionNotAStandingPermission() {
|
||||
AdminOperationIdempotencyStore store = new AdminOperationIdempotencyStore();
|
||||
|
||||
assertThat(store.claim("CHG-1042", "op-1", NOW)).isEmpty();
|
||||
assertThat(store.claim("CHG-1042", "op-2", NOW.plus(Duration.ofMinutes(5))))
|
||||
.as("an arrow-up in the shell must not re-run an approved redrive")
|
||||
.hasValueSatisfying(existing -> assertThat(existing.operationId()).isEqualTo("op-1"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void aDifferentApprovalIsClaimedIndependently() {
|
||||
AdminOperationIdempotencyStore store = new AdminOperationIdempotencyStore();
|
||||
store.claim("CHG-1042", "op-1", NOW);
|
||||
|
||||
assertThat(store.claim("CHG-1043", "op-2", NOW)).isEmpty();
|
||||
assertThat(store.size()).isEqualTo(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void aReplayResultFlagsAShortfallAgainstTheApprovedEstimate() {
|
||||
ReplayResult result =
|
||||
new ReplayResult(OPERATION_ID, 1_000_000, 100_000, Duration.ofMinutes(3), true, false);
|
||||
|
||||
assertThat(result.fellShortOfTheEstimate())
|
||||
.as("retention may have expired part of the approved window")
|
||||
.isTrue();
|
||||
}
|
||||
|
||||
@Test
|
||||
void aRedriveResultMustAccountForEveryCandidate() {
|
||||
RedriveResult accounted =
|
||||
new RedriveResult(OPERATION_ID, 100, 80, 20, Duration.ofSeconds(4), false);
|
||||
RedriveResult unaccounted =
|
||||
new RedriveResult(OPERATION_ID, 100, 80, 10, Duration.ofSeconds(4), false);
|
||||
|
||||
assertThat(accounted.isFullyAccounted()).isTrue();
|
||||
assertThat(unaccounted.isFullyAccounted())
|
||||
.as("a message that was neither moved nor left parked has been lost track of")
|
||||
.isFalse();
|
||||
}
|
||||
|
||||
@Test
|
||||
void aDryRunCannotClaimToHaveMovedAnything() {
|
||||
assertThatThrownBy(
|
||||
() -> new RedriveResult(OPERATION_ID, 100, 5, 0, Duration.ofSeconds(1), true))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
}
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThatCode;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import dev.caskeleton.messaging.admin.TopologyManifest;
|
||||
import dev.caskeleton.messaging.api.error.MessageTopologyException;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class TopologyValidationRuntimeTest {
|
||||
|
||||
private static final TopologyManifest ORDERS =
|
||||
new TopologyManifest(
|
||||
"order-events", "order.events.v1", 6, 3, Map.of("min.insync.replicas", "2"));
|
||||
|
||||
@Test
|
||||
void aMatchingTopologyValidates() {
|
||||
TopologyValidationRuntime runtime =
|
||||
new TopologyValidationRuntime(
|
||||
name ->
|
||||
new TopologyValidationRuntime.ObservedTopology(
|
||||
6, 3, Map.of("min.insync.replicas", "2")));
|
||||
|
||||
assertThatCode(() -> runtime.validate(List.of(ORDERS))).doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
@Test
|
||||
void aMissingDestinationFailsStartup() {
|
||||
TopologyValidationRuntime runtime = new TopologyValidationRuntime(name -> null);
|
||||
|
||||
assertThatThrownBy(() -> runtime.validate(List.of(ORDERS)))
|
||||
.isInstanceOf(MessageTopologyException.class)
|
||||
.hasMessageContaining("does not exist");
|
||||
}
|
||||
|
||||
@Test
|
||||
void aWeakenedReplicationSettingFailsStartup() {
|
||||
TopologyValidationRuntime runtime =
|
||||
new TopologyValidationRuntime(
|
||||
name ->
|
||||
new TopologyValidationRuntime.ObservedTopology(
|
||||
6, 3, Map.of("min.insync.replicas", "1")));
|
||||
|
||||
assertThatThrownBy(() -> runtime.validate(List.of(ORDERS)))
|
||||
.isInstanceOf(MessageTopologyException.class)
|
||||
.hasMessageContaining("min.insync.replicas");
|
||||
}
|
||||
|
||||
@Test
|
||||
void everyDifferenceIsReportedAtOnce() {
|
||||
TopologyValidationRuntime runtime =
|
||||
new TopologyValidationRuntime(
|
||||
name ->
|
||||
new TopologyValidationRuntime.ObservedTopology(
|
||||
3, 1, Map.of("min.insync.replicas", "1")));
|
||||
|
||||
assertThatThrownBy(() -> runtime.validate(List.of(ORDERS)))
|
||||
.isInstanceOf(MessageTopologyException.class)
|
||||
.hasMessageContaining("partitions")
|
||||
.hasMessageContaining("replicationFactor")
|
||||
.hasMessageContaining("min.insync.replicas");
|
||||
}
|
||||
}
|
||||
+172
@@ -0,0 +1,172 @@
|
||||
package dev.caskeleton.messaging.admin.runtime;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatCode;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import dev.caskeleton.messaging.admin.DestinationTopology;
|
||||
import dev.caskeleton.messaging.admin.TopologyIssue;
|
||||
import dev.caskeleton.messaging.admin.TopologyManagementMode;
|
||||
import dev.caskeleton.messaging.admin.TopologyManifest;
|
||||
import dev.caskeleton.messaging.admin.TopologyValidationReport;
|
||||
import dev.caskeleton.messaging.api.error.MessagingConfigurationException;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class TopologyValidatorTest {
|
||||
|
||||
private final TopologyValidator validator = new TopologyValidator();
|
||||
|
||||
private static TopologyManifest manifest() {
|
||||
return new TopologyManifest(
|
||||
"orders.v1", "orders-v1", 12, 3, Map.of("min.insync.replicas", "2"));
|
||||
}
|
||||
|
||||
private static DestinationTopology observed(
|
||||
int partitions, int replication, Map<String, String> config) {
|
||||
return new DestinationTopology("orders-v1", partitions, replication, config, true);
|
||||
}
|
||||
|
||||
@Test
|
||||
void anAgreeingTopologyProducesNoIssues() {
|
||||
assertThat(validator.compare(manifest(), observed(12, 3, Map.of("min.insync.replicas", "2"))))
|
||||
.isEmpty();
|
||||
}
|
||||
|
||||
@Test
|
||||
void anAbsentDestinationBlocksStartup() {
|
||||
List<TopologyIssue> issues =
|
||||
validator.compare(manifest(), DestinationTopology.absent("orders-v1"));
|
||||
|
||||
assertThat(issues)
|
||||
.singleElement()
|
||||
.satisfies(
|
||||
issue -> {
|
||||
assertThat(issue.attribute()).isEqualTo("existence");
|
||||
assertThat(issue.severity()).isEqualTo(TopologyIssue.Severity.BLOCKING);
|
||||
});
|
||||
}
|
||||
|
||||
@Test
|
||||
void tooFewPartitionsBlocksBecauseKeysWouldLandDifferently() {
|
||||
List<TopologyIssue> issues =
|
||||
validator.compare(manifest(), observed(6, 3, Map.of("min.insync.replicas", "2")));
|
||||
|
||||
assertThat(issues)
|
||||
.singleElement()
|
||||
.satisfies(
|
||||
issue -> assertThat(issue.severity()).isEqualTo(TopologyIssue.Severity.BLOCKING));
|
||||
}
|
||||
|
||||
@Test
|
||||
void extraPartitionsAreAdvisoryBecauseScalingUpIsLegitimate() {
|
||||
List<TopologyIssue> issues =
|
||||
validator.compare(manifest(), observed(24, 3, Map.of("min.insync.replicas", "2")));
|
||||
|
||||
assertThat(issues)
|
||||
.singleElement()
|
||||
.satisfies(
|
||||
issue -> assertThat(issue.severity()).isEqualTo(TopologyIssue.Severity.ADVISORY));
|
||||
}
|
||||
|
||||
@Test
|
||||
void tooLittleReplicationBlocksBecauseDurabilityIsPromised() {
|
||||
List<TopologyIssue> issues =
|
||||
validator.compare(manifest(), observed(12, 1, Map.of("min.insync.replicas", "2")));
|
||||
|
||||
assertThat(issues)
|
||||
.singleElement()
|
||||
.satisfies(
|
||||
issue -> {
|
||||
assertThat(issue.attribute()).isEqualTo("replicationFactor");
|
||||
assertThat(issue.severity()).isEqualTo(TopologyIssue.Severity.BLOCKING);
|
||||
});
|
||||
}
|
||||
|
||||
@Test
|
||||
void aMissingRequiredConfigurationEntryBlocks() {
|
||||
List<TopologyIssue> issues = validator.compare(manifest(), observed(12, 3, Map.of()));
|
||||
|
||||
assertThat(issues)
|
||||
.singleElement()
|
||||
.satisfies(issue -> assertThat(issue.actual()).isEqualTo("unset"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void everyIssueIsCollectedRatherThanFailingOnTheFirst() {
|
||||
List<TopologyIssue> issues = validator.compare(manifest(), observed(6, 1, Map.of()));
|
||||
|
||||
assertThat(issues)
|
||||
.as("discovering the next problem only after a redeploy turns one fix into several")
|
||||
.hasSize(3);
|
||||
}
|
||||
|
||||
@Test
|
||||
void aReportRefusesStartupWhenAnythingBlocks() {
|
||||
TopologyValidationReport report =
|
||||
new TopologyValidationReport(
|
||||
validator.compare(manifest(), observed(12, 1, Map.of("min.insync.replicas", "2"))), 1);
|
||||
|
||||
assertThat(report.isAcceptable()).isFalse();
|
||||
assertThatThrownBy(report::requireAcceptable)
|
||||
.isInstanceOf(MessagingConfigurationException.class)
|
||||
.hasMessageContaining("replicationFactor");
|
||||
}
|
||||
|
||||
@Test
|
||||
void anAdvisoryOnlyReportStillStarts() {
|
||||
TopologyValidationReport report =
|
||||
new TopologyValidationReport(
|
||||
validator.compare(manifest(), observed(24, 3, Map.of("min.insync.replicas", "2"))), 1);
|
||||
|
||||
assertThat(report.isAcceptable()).isTrue();
|
||||
assertThat(report.advisory()).hasSize(1);
|
||||
assertThatCode(report::requireAcceptable).doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
@Test
|
||||
void theCompositeValidatorChecksEveryManifest() {
|
||||
CompositeTopologyValidator composite =
|
||||
new CompositeTopologyValidator(
|
||||
new BrokerTopologyInspector() {
|
||||
@Override
|
||||
public Optional<DestinationTopology> describe(String physicalName) {
|
||||
return Optional.empty();
|
||||
}
|
||||
|
||||
@Override
|
||||
public String topologyVersion() {
|
||||
return "v1";
|
||||
}
|
||||
});
|
||||
|
||||
TopologyValidationReport report =
|
||||
composite.validate(
|
||||
List.of(
|
||||
manifest(), new TopologyManifest("payments.v1", "payments-v1", 3, 3, Map.of())));
|
||||
|
||||
assertThat(report.destinationsChecked()).isEqualTo(2);
|
||||
assertThat(report.blocking()).hasSize(2);
|
||||
}
|
||||
|
||||
@Test
|
||||
void autoCreationIsRefusedInProduction() {
|
||||
assertThatThrownBy(() -> TopologyManagementMode.CREATE_IF_MISSING.requireSafeFor(true))
|
||||
.as("a mistyped destination would be created and look exactly like a real one")
|
||||
.isInstanceOf(MessagingConfigurationException.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void autoCreationIsAllowedOutsideProduction() {
|
||||
assertThatCode(() -> TopologyManagementMode.CREATE_IF_MISSING.requireSafeFor(false))
|
||||
.doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
@Test
|
||||
void validateOnlyIsAlwaysSafe() {
|
||||
assertThatCode(() -> TopologyManagementMode.VALIDATE_ONLY.requireSafeFor(true))
|
||||
.doesNotThrowAnyException();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
apply plugin: 'java-library'
|
||||
|
||||
dependencies {
|
||||
api project(':messaging:messaging-core-api')
|
||||
api project(':messaging:messaging-reliability-api')
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
# This is a Gradle generated file for dependency locking.
|
||||
# Manual edits can break the build and are not advised.
|
||||
# This file is expected to be part of source control.
|
||||
com.github.ben-manes.caffeine:caffeine:3.2.3=annotationProcessor,testAnnotationProcessor
|
||||
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor
|
||||
com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs
|
||||
com.github.spotbugs:spotbugs:4.10.2=spotbugs
|
||||
com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs
|
||||
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor
|
||||
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor
|
||||
com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor
|
||||
com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs
|
||||
com.google.code.gson:gson:2.13.2=spotbugs
|
||||
com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_annotations:2.41.0=spotbugs
|
||||
com.google.errorprone:error_prone_annotations:2.47.0=checkstyle
|
||||
com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor
|
||||
com.google.guava:guava:33.6.0-jre=checkstyle
|
||||
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor
|
||||
com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins
|
||||
com.puppycrawl.tools:checkstyle:13.5.0=checkstyle
|
||||
commons-beanutils:commons-beanutils:1.11.0=checkstyle
|
||||
commons-collections:commons-collections:3.2.2=checkstyle
|
||||
commons-io:commons-io:2.21.0=spotbugs
|
||||
info.picocli:picocli:4.7.7=checkstyle
|
||||
io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor
|
||||
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor
|
||||
javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor
|
||||
jaxen:jaxen:2.0.0=spotbugs
|
||||
net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath
|
||||
net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs
|
||||
org.antlr:antlr4-runtime:4.13.2=checkstyle
|
||||
org.apache.bcel:bcel:6.12.0=spotbugs
|
||||
org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs
|
||||
org.apache.commons:commons-text:1.15.0=spotbugs
|
||||
org.apache.commons:commons-text:1.3=checkstyle
|
||||
org.apache.httpcomponents:httpclient:4.5.13=checkstyle
|
||||
org.apache.httpcomponents:httpcore:4.4.16=checkstyle
|
||||
org.apache.logging.log4j:log4j-api:2.25.2=spotbugs
|
||||
org.apache.logging.log4j:log4j-core:2.25.2=spotbugs
|
||||
org.apache.maven.doxia:doxia-core:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle
|
||||
org.apache.xbean:xbean-reflect:3.7=checkstyle
|
||||
org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath
|
||||
org.assertj:assertj-core:3.27.6=testCompileClasspath,testRuntimeClasspath
|
||||
org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle
|
||||
org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle
|
||||
org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle
|
||||
org.codehaus.plexus:plexus-utils:3.3.0=checkstyle
|
||||
org.dom4j:dom4j:2.2.0=spotbugs
|
||||
org.javassist:javassist:3.28.0-GA=checkstyle
|
||||
org.jspecify:jspecify:1.0.0=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath
|
||||
org.junit.jupiter:junit-jupiter-api:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-engine:6.0.1=testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-params:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-commons:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-engine:6.0.1=testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-launcher:6.0.1=testRuntimeClasspath
|
||||
org.junit:junit-bom:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit:junit-bom:6.1.0=spotbugs
|
||||
org.mockito:mockito-core:5.20.0=mockitoAgent
|
||||
org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath
|
||||
org.ow2.asm:asm-analysis:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-commons:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-tree:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-util:9.10.1=spotbugs
|
||||
org.ow2.asm:asm:9.10.1=spotbugs
|
||||
org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor
|
||||
org.reflections:reflections:0.10.2=checkstyle
|
||||
org.slf4j:slf4j-api:2.0.17=spotbugs,spotbugsSlf4j
|
||||
org.slf4j:slf4j-simple:2.0.17=checkstyle,spotbugsSlf4j
|
||||
org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs
|
||||
empty=compileClasspath,runtimeClasspath
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
package dev.caskeleton.messaging.claimcheck;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.FailureCategory;
|
||||
import dev.caskeleton.messaging.api.error.FailureDescriptor;
|
||||
import dev.caskeleton.messaging.api.error.MessagingException;
|
||||
import java.io.Serial;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* The stored payload does not match the reference the message carried.
|
||||
*
|
||||
* <p>Not retryable. A digest mismatch means the object at that key is not the object the producer
|
||||
* wrote — the key was reused, the object was overwritten, or something truncated it — and fetching
|
||||
* it again returns the same wrong bytes. Retrying would only delay the dead-letter.
|
||||
*
|
||||
* <p>Deliberately distinct from "the object is gone". An expired claim check is an operational
|
||||
* problem with a known cause and a known fix; a digest mismatch means something wrote data nobody
|
||||
* expected, and the two must not be diagnosed as one.
|
||||
*/
|
||||
public class ClaimCheckIntegrityException extends MessagingException {
|
||||
|
||||
@Serial private static final long serialVersionUID = 1L;
|
||||
|
||||
private static final FailureCategory CATEGORY = FailureCategory.POISON_MESSAGE;
|
||||
|
||||
/**
|
||||
* Creates the exception with a stable code and sanitized message.
|
||||
*
|
||||
* @param code the stable failure code
|
||||
* @param sanitizedMessage the operator-facing description
|
||||
*/
|
||||
public ClaimCheckIntegrityException(String code, String sanitizedMessage) {
|
||||
super(new FailureDescriptor(CATEGORY, code, false, sanitizedMessage, Optional.empty()));
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates the exception from an explicit descriptor.
|
||||
*
|
||||
* @param failure the sanitized failure description
|
||||
*/
|
||||
public ClaimCheckIntegrityException(FailureDescriptor failure) {
|
||||
super(failure);
|
||||
}
|
||||
}
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
package dev.caskeleton.messaging.claimcheck;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.MessageValidationException;
|
||||
import dev.caskeleton.messaging.reliability.ClaimCheckReference;
|
||||
import java.security.MessageDigest;
|
||||
import java.security.NoSuchAlgorithmException;
|
||||
import java.time.Instant;
|
||||
import java.util.HexFormat;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* Verifies a fetched claim check payload before it is handed to a codec.
|
||||
*
|
||||
* <p>A claim check turns one message into two systems that can drift. The payload store has its own
|
||||
* retention, its own replication, and its own access control, and none of them are coordinated with
|
||||
* the broker's. So a consumer that fetches bytes and decodes them without checking is trusting
|
||||
* something the message never proved.
|
||||
*
|
||||
* <p>Both checks fail closed. An expired reference is reported before the fetch, because a
|
||||
* not-found from the store is ambiguous between "reaped" and "never written". A digest mismatch is
|
||||
* reported as validation rather than deserialization, because the bytes are not corrupt JSON — they
|
||||
* are the wrong bytes.
|
||||
*/
|
||||
public final class ClaimCheckIntegrityGuard {
|
||||
|
||||
/**
|
||||
* Verifies a payload against its reference.
|
||||
*
|
||||
* @param reference the claim check reference
|
||||
* @param payload the bytes fetched from the store
|
||||
* @param now the current instant
|
||||
* @return the verified payload
|
||||
* @throws MessageValidationException when the reference expired or the digest does not match
|
||||
*/
|
||||
public byte[] verify(ClaimCheckReference reference, byte[] payload, Instant now) {
|
||||
Objects.requireNonNull(reference, "reference must not be null");
|
||||
Objects.requireNonNull(payload, "payload must not be null");
|
||||
Objects.requireNonNull(now, "now must not be null");
|
||||
|
||||
if (reference.isExpired(now)) {
|
||||
throw new MessageValidationException(
|
||||
"CLAIM_CHECK_EXPIRED",
|
||||
"the claim check payload retention expired at " + reference.expiresAt());
|
||||
}
|
||||
if (payload.length != reference.sizeBytes()) {
|
||||
throw new MessageValidationException(
|
||||
"CLAIM_CHECK_SIZE_MISMATCH",
|
||||
"expected " + reference.sizeBytes() + " bytes but read " + payload.length);
|
||||
}
|
||||
String actual = sha256(payload);
|
||||
if (!actual.equals(reference.sha256())) {
|
||||
throw new MessageValidationException(
|
||||
"CLAIM_CHECK_DIGEST_MISMATCH", "the fetched payload does not match its reference digest");
|
||||
}
|
||||
return payload.clone();
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the lowercase hex SHA-256 of a payload.
|
||||
*
|
||||
* @param payload the bytes to digest
|
||||
* @return the digest
|
||||
*/
|
||||
public static String sha256(byte[] payload) {
|
||||
try {
|
||||
return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(payload));
|
||||
} catch (NoSuchAlgorithmException exception) {
|
||||
throw new IllegalStateException("Java runtime does not provide SHA-256", exception);
|
||||
}
|
||||
}
|
||||
}
|
||||
+89
@@ -0,0 +1,89 @@
|
||||
package dev.caskeleton.messaging.claimcheck;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.MessagingConfigurationException;
|
||||
import java.time.Duration;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* When a payload is offloaded, and how long the object must outlive the message.
|
||||
*
|
||||
* <p>The retention rule is the one that matters. A claim check object deleted while its message is
|
||||
* still deliverable turns a large message into an undeliverable one — the consumer fetches, gets
|
||||
* nothing, and the message dead-letters for a reason that has nothing to do with the message. So
|
||||
* retention must exceed the broker's own retention plus the full retry and dead-letter window, and
|
||||
* the constructor refuses a configuration where it does not.
|
||||
*
|
||||
* <p>The threshold is separate from the destination's payload limit. Offloading starts well below
|
||||
* the limit, because the limit is where the broker refuses the message and the threshold is where
|
||||
* carrying it inline stops being a good idea.
|
||||
*
|
||||
* @param thresholdBytes the encoded size above which a payload is offloaded
|
||||
* @param retention how long the stored object must remain readable
|
||||
* @param brokerRetention how long the broker keeps the message
|
||||
* @param maxRedeliveryWindow the longest retry and dead-letter path a message can take
|
||||
*/
|
||||
public record ClaimCheckPolicy(
|
||||
int thresholdBytes,
|
||||
Duration retention,
|
||||
Duration brokerRetention,
|
||||
Duration maxRedeliveryWindow) {
|
||||
|
||||
/** The default offload threshold: a quarter of the portable payload limit. */
|
||||
public static final int DEFAULT_THRESHOLD_BYTES = 262_144;
|
||||
|
||||
public ClaimCheckPolicy {
|
||||
Objects.requireNonNull(retention, "retention must not be null");
|
||||
Objects.requireNonNull(brokerRetention, "brokerRetention must not be null");
|
||||
Objects.requireNonNull(maxRedeliveryWindow, "maxRedeliveryWindow must not be null");
|
||||
if (thresholdBytes < 1) {
|
||||
throw new IllegalArgumentException("thresholdBytes must be positive");
|
||||
}
|
||||
requirePositive(retention, "retention");
|
||||
requirePositive(brokerRetention, "brokerRetention");
|
||||
requirePositive(maxRedeliveryWindow, "maxRedeliveryWindow");
|
||||
|
||||
Duration required = brokerRetention.plus(maxRedeliveryWindow);
|
||||
if (retention.compareTo(required) < 0) {
|
||||
throw new MessagingConfigurationException(
|
||||
"CLAIM_CHECK_RETENTION_TOO_SHORT",
|
||||
"claim check retention of %s is below the %s the message can remain deliverable; the "
|
||||
.formatted(retention, required)
|
||||
+ "object would be reaped while a consumer can still be handed its message");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a policy for a broker retaining one day with a one-day retry path.
|
||||
*
|
||||
* @return the default policy
|
||||
*/
|
||||
public static ClaimCheckPolicy defaults() {
|
||||
return new ClaimCheckPolicy(
|
||||
DEFAULT_THRESHOLD_BYTES, Duration.ofDays(3), Duration.ofDays(1), Duration.ofDays(1));
|
||||
}
|
||||
|
||||
/**
|
||||
* Reports whether a payload of this size is offloaded.
|
||||
*
|
||||
* @param payloadBytes the encoded payload size
|
||||
* @return true when the payload travels by reference
|
||||
*/
|
||||
public boolean shouldOffload(int payloadBytes) {
|
||||
return payloadBytes > thresholdBytes;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the shortest retention this deployment allows.
|
||||
*
|
||||
* @return the minimum safe retention
|
||||
*/
|
||||
public Duration requiredRetention() {
|
||||
return brokerRetention.plus(maxRedeliveryWindow);
|
||||
}
|
||||
|
||||
private static void requirePositive(Duration value, String field) {
|
||||
if (value.isNegative() || value.isZero()) {
|
||||
throw new IllegalArgumentException(field + " must be positive");
|
||||
}
|
||||
}
|
||||
}
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
package dev.caskeleton.messaging.claimcheck;
|
||||
|
||||
import dev.caskeleton.messaging.reliability.ClaimCheckReference;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Decides whether a payload travels inline or by reference, and stores it when it does not.
|
||||
*
|
||||
* <p>The object is written <em>before</em> the message is published, and that order is the whole
|
||||
* design. Publishing first would let a consumer receive a reference to an object that does not
|
||||
* exist yet — a race that is rare in a test and routine under load, because the broker hop is
|
||||
* faster than the object store write.
|
||||
*
|
||||
* <p>Nothing here deletes on failure. If the publish is rejected the object is left behind, and the
|
||||
* retention sweep reclaims it; deleting eagerly would delete the object out from under a publish
|
||||
* that turned out to be ambiguous rather than rejected.
|
||||
*/
|
||||
public final class ClaimCheckPublisher {
|
||||
|
||||
private final ClaimCheckStore store;
|
||||
private final ClaimCheckPolicy policy;
|
||||
|
||||
/**
|
||||
* Creates a claim check publisher.
|
||||
*
|
||||
* @param store the payload store
|
||||
* @param policy the offload threshold and retention
|
||||
*/
|
||||
public ClaimCheckPublisher(ClaimCheckStore store, ClaimCheckPolicy policy) {
|
||||
this.store = Objects.requireNonNull(store, "store must not be null");
|
||||
this.policy = Objects.requireNonNull(policy, "policy must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Offloads a payload when the policy calls for it.
|
||||
*
|
||||
* @param payload the encoded payload bytes
|
||||
* @return the outcome, carrying either the inline payload or the stored reference
|
||||
*/
|
||||
public Offloaded offload(byte[] payload) {
|
||||
Objects.requireNonNull(payload, "payload must not be null");
|
||||
|
||||
if (!policy.shouldOffload(payload.length)) {
|
||||
return new Offloaded(payload.clone(), Optional.empty());
|
||||
}
|
||||
ClaimCheckReference reference = store.put(payload, policy.retention());
|
||||
// The published message carries no payload bytes at all, only the reference. Carrying both
|
||||
// would double the transfer for no benefit and let the two disagree.
|
||||
return new Offloaded(new byte[0], Optional.of(reference));
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the policy this publisher applies.
|
||||
*
|
||||
* @return the claim check policy
|
||||
*/
|
||||
public ClaimCheckPolicy policy() {
|
||||
return policy;
|
||||
}
|
||||
|
||||
/**
|
||||
* What a payload became after the offload decision.
|
||||
*
|
||||
* @param payload the payload to publish, empty when offloaded
|
||||
* @param reference the stored object's reference, present when offloaded
|
||||
*/
|
||||
@SuppressWarnings("ArrayRecordComponent")
|
||||
public record Offloaded(byte[] payload, Optional<ClaimCheckReference> reference) {
|
||||
|
||||
public Offloaded {
|
||||
Objects.requireNonNull(payload, "payload must not be null");
|
||||
Objects.requireNonNull(reference, "reference must not be null");
|
||||
payload = payload.clone();
|
||||
}
|
||||
|
||||
@Override
|
||||
public byte[] payload() {
|
||||
return payload.clone();
|
||||
}
|
||||
|
||||
/**
|
||||
* Reports whether the payload travels by reference.
|
||||
*
|
||||
* @return true when the payload was offloaded
|
||||
*/
|
||||
public boolean isOffloaded() {
|
||||
return reference.isPresent();
|
||||
}
|
||||
}
|
||||
}
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
package dev.caskeleton.messaging.claimcheck;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.MessageValidationException;
|
||||
import dev.caskeleton.messaging.reliability.ClaimCheckReference;
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Fetches an offloaded payload and verifies it before a handler ever sees it.
|
||||
*
|
||||
* <p>Verification is not optional and cannot be skipped by a caller. An object store key is a
|
||||
* string, and a message carrying the wrong one — through a bug, a replay against a rotated bucket,
|
||||
* or a deliberate tamper — fetches bytes that decode perfectly into the wrong object. The digest is
|
||||
* the only thing standing between that and a handler acting on someone else's data.
|
||||
*
|
||||
* <p>Failures are classified rather than merged. An expired reference is an operational problem
|
||||
* whose fix is a retention change; a digest mismatch means something wrote data nobody expected.
|
||||
* Both dead-letter the message, but an operator seeing one code should not have to guess which
|
||||
* happened.
|
||||
*/
|
||||
public final class ClaimCheckResolver {
|
||||
|
||||
private final ClaimCheckStore store;
|
||||
private final ClaimCheckIntegrityGuard guard = new ClaimCheckIntegrityGuard();
|
||||
|
||||
/**
|
||||
* Creates a resolver.
|
||||
*
|
||||
* @param store the payload store
|
||||
*/
|
||||
public ClaimCheckResolver(ClaimCheckStore store) {
|
||||
this.store = Objects.requireNonNull(store, "store must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the payload a message carries, fetching it when it travels by reference.
|
||||
*
|
||||
* @param inline the payload bytes the message carried, empty when it travels by reference
|
||||
* @param reference the claim check reference, when the message carried one
|
||||
* @param now the current instant
|
||||
* @return the payload the handler should see
|
||||
* @throws ClaimCheckIntegrityException when the stored object is not the one referenced
|
||||
* @throws MessageValidationException when the reference has expired
|
||||
*/
|
||||
public byte[] resolve(byte[] inline, Optional<ClaimCheckReference> reference, Instant now) {
|
||||
Objects.requireNonNull(inline, "inline must not be null");
|
||||
Objects.requireNonNull(reference, "reference must not be null");
|
||||
Objects.requireNonNull(now, "now must not be null");
|
||||
|
||||
if (reference.isEmpty()) {
|
||||
return inline.clone();
|
||||
}
|
||||
ClaimCheckReference claimCheck = reference.get();
|
||||
|
||||
if (claimCheck.isExpired(now)) {
|
||||
// Checked before fetching. A store that still returns the object past its retention would
|
||||
// otherwise hide a misconfiguration until the day the sweep caught up.
|
||||
throw new MessageValidationException(
|
||||
"CLAIM_CHECK_EXPIRED",
|
||||
"the claim check retention expired at %s; the object may already be reaped"
|
||||
.formatted(claimCheck.expiresAt()));
|
||||
}
|
||||
|
||||
return verify(claimCheck, fetch(claimCheck), now);
|
||||
}
|
||||
|
||||
private byte[] fetch(ClaimCheckReference reference) {
|
||||
byte[] fetched = store.get(reference);
|
||||
if (fetched == null) {
|
||||
throw new MessageValidationException(
|
||||
"CLAIM_CHECK_NOT_FOUND",
|
||||
"no object exists at the referenced key; it was either reaped early or never written");
|
||||
}
|
||||
return fetched;
|
||||
}
|
||||
|
||||
private byte[] verify(ClaimCheckReference reference, byte[] fetched, Instant now) {
|
||||
try {
|
||||
return guard.verify(reference, fetched, now);
|
||||
} catch (MessageValidationException validation) {
|
||||
// A size or digest mismatch is a poison message, not a validation failure to be retried:
|
||||
// fetching the same key again returns the same wrong bytes.
|
||||
if (validation.failure().code().endsWith("_MISMATCH")) {
|
||||
throw new ClaimCheckIntegrityException(
|
||||
validation.failure().code(), validation.failure().sanitizedMessage());
|
||||
}
|
||||
throw validation;
|
||||
}
|
||||
}
|
||||
}
|
||||
+32
@@ -0,0 +1,32 @@
|
||||
package dev.caskeleton.messaging.claimcheck;
|
||||
|
||||
import dev.caskeleton.messaging.reliability.ClaimCheckReference;
|
||||
import java.time.Duration;
|
||||
|
||||
/** Stores and retrieves payloads that are too large to travel through the broker. */
|
||||
public interface ClaimCheckStore {
|
||||
|
||||
/**
|
||||
* Stores a payload and returns its reference.
|
||||
*
|
||||
* @param payload the bytes to store
|
||||
* @param retention how long the object must remain readable
|
||||
* @return the reference to publish in place of the payload
|
||||
*/
|
||||
ClaimCheckReference put(byte[] payload, Duration retention);
|
||||
|
||||
/**
|
||||
* Fetches a payload, verifying it against its reference.
|
||||
*
|
||||
* @param reference the claim check reference
|
||||
* @return the stored bytes
|
||||
*/
|
||||
byte[] get(ClaimCheckReference reference);
|
||||
|
||||
/**
|
||||
* Deletes a stored payload.
|
||||
*
|
||||
* @param reference the claim check reference
|
||||
*/
|
||||
void delete(ClaimCheckReference reference);
|
||||
}
|
||||
+86
@@ -0,0 +1,86 @@
|
||||
package dev.caskeleton.messaging.claimcheck;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatCode;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.MessageValidationException;
|
||||
import dev.caskeleton.messaging.reliability.ClaimCheckReference;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Instant;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class ClaimCheckIntegrityGuardTest {
|
||||
|
||||
private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z");
|
||||
private static final byte[] PAYLOAD = "a large document".getBytes(StandardCharsets.UTF_8);
|
||||
|
||||
private final ClaimCheckIntegrityGuard guard = new ClaimCheckIntegrityGuard();
|
||||
|
||||
@Test
|
||||
void acceptsAPayloadMatchingItsReference() {
|
||||
ClaimCheckReference reference =
|
||||
reference(PAYLOAD.length, ClaimCheckIntegrityGuard.sha256(PAYLOAD));
|
||||
|
||||
assertThat(guard.verify(reference, PAYLOAD, NOW)).isEqualTo(PAYLOAD);
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsAPayloadWhoseDigestDoesNotMatch() {
|
||||
ClaimCheckReference reference =
|
||||
reference(
|
||||
PAYLOAD.length,
|
||||
ClaimCheckIntegrityGuard.sha256(
|
||||
"a different document".getBytes(StandardCharsets.UTF_8)));
|
||||
|
||||
assertThatThrownBy(() -> guard.verify(reference, PAYLOAD, NOW))
|
||||
.isInstanceOf(MessageValidationException.class)
|
||||
.hasMessageContaining("digest");
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsATruncatedPayloadBeforeHashingIt() {
|
||||
ClaimCheckReference reference =
|
||||
reference(PAYLOAD.length + 10, ClaimCheckIntegrityGuard.sha256(PAYLOAD));
|
||||
|
||||
assertThatThrownBy(() -> guard.verify(reference, PAYLOAD, NOW))
|
||||
.isInstanceOf(MessageValidationException.class)
|
||||
.hasMessageContaining("bytes");
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsAnExpiredReferenceBeforeTheFetchIsTrusted() {
|
||||
ClaimCheckReference reference =
|
||||
new ClaimCheckReference(
|
||||
"payloads/o-1",
|
||||
PAYLOAD.length,
|
||||
ClaimCheckIntegrityGuard.sha256(PAYLOAD),
|
||||
NOW.minusSeconds(1));
|
||||
|
||||
assertThatThrownBy(() -> guard.verify(reference, PAYLOAD, NOW))
|
||||
.isInstanceOf(MessageValidationException.class)
|
||||
.hasMessageContaining("retention");
|
||||
}
|
||||
|
||||
@Test
|
||||
void aReferenceRequiresALowercaseHexDigest() {
|
||||
assertThatThrownBy(
|
||||
() -> new ClaimCheckReference("payloads/o-1", 5, "NOT-A-DIGEST", NOW.plusSeconds(60)))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void theVerifiedPayloadIsACopy() {
|
||||
ClaimCheckReference reference =
|
||||
reference(PAYLOAD.length, ClaimCheckIntegrityGuard.sha256(PAYLOAD));
|
||||
|
||||
byte[] verified = guard.verify(reference, PAYLOAD, NOW);
|
||||
verified[0] = 'z';
|
||||
|
||||
assertThatCode(() -> guard.verify(reference, PAYLOAD, NOW)).doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
private static ClaimCheckReference reference(long sizeBytes, String sha256) {
|
||||
return new ClaimCheckReference("payloads/o-1", sizeBytes, sha256, NOW.plusSeconds(3600));
|
||||
}
|
||||
}
|
||||
+168
@@ -0,0 +1,168 @@
|
||||
package dev.caskeleton.messaging.claimcheck;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.MessageValidationException;
|
||||
import dev.caskeleton.messaging.reliability.ClaimCheckReference;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.HashMap;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class ClaimCheckResolverTest {
|
||||
|
||||
private static final Instant NOW = Instant.parse("2026-08-10T09:15:00Z");
|
||||
private static final byte[] PAYLOAD = "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8);
|
||||
|
||||
/** An in-memory store that can be made to return the wrong bytes on purpose. */
|
||||
private static final class FakeStore implements ClaimCheckStore {
|
||||
|
||||
private final Map<String, byte[]> objects = new HashMap<>();
|
||||
private int puts;
|
||||
|
||||
@Override
|
||||
public ClaimCheckReference put(byte[] payload, Duration retention) {
|
||||
puts++;
|
||||
String key = "claim/" + puts;
|
||||
objects.put(key, payload.clone());
|
||||
return new ClaimCheckReference(
|
||||
key, payload.length, ClaimCheckIntegrityGuard.sha256(payload), NOW.plus(retention));
|
||||
}
|
||||
|
||||
@Override
|
||||
public byte[] get(ClaimCheckReference reference) {
|
||||
return objects.get(reference.storageKey());
|
||||
}
|
||||
|
||||
@Override
|
||||
public void delete(ClaimCheckReference reference) {
|
||||
objects.remove(reference.storageKey());
|
||||
}
|
||||
|
||||
void overwrite(String key, byte[] replacement) {
|
||||
objects.put(key, replacement);
|
||||
}
|
||||
|
||||
int puts() {
|
||||
return puts;
|
||||
}
|
||||
}
|
||||
|
||||
private static byte[] filled(int size) {
|
||||
byte[] payload = new byte[size];
|
||||
java.util.Arrays.fill(payload, (byte) 'x');
|
||||
return payload;
|
||||
}
|
||||
|
||||
@Test
|
||||
void aSmallPayloadTravelsInlineAndIsNeverStored() {
|
||||
FakeStore store = new FakeStore();
|
||||
ClaimCheckPublisher publisher = new ClaimCheckPublisher(store, ClaimCheckPolicy.defaults());
|
||||
|
||||
ClaimCheckPublisher.Offloaded offloaded = publisher.offload(PAYLOAD);
|
||||
|
||||
assertThat(offloaded.isOffloaded()).isFalse();
|
||||
assertThat(store.puts()).isZero();
|
||||
}
|
||||
|
||||
@Test
|
||||
void aLargePayloadIsStoredAndTheMessageCarriesNoBytes() {
|
||||
FakeStore store = new FakeStore();
|
||||
ClaimCheckPublisher publisher = new ClaimCheckPublisher(store, ClaimCheckPolicy.defaults());
|
||||
|
||||
ClaimCheckPublisher.Offloaded offloaded =
|
||||
publisher.offload(filled(ClaimCheckPolicy.DEFAULT_THRESHOLD_BYTES + 1));
|
||||
|
||||
assertThat(offloaded.isOffloaded()).isTrue();
|
||||
assertThat(offloaded.payload().length)
|
||||
.as("carrying both would double the transfer and let the two disagree")
|
||||
.isZero();
|
||||
}
|
||||
|
||||
@Test
|
||||
void anOffloadedPayloadRoundTripsThroughTheStore() {
|
||||
FakeStore store = new FakeStore();
|
||||
ClaimCheckPolicy policy =
|
||||
new ClaimCheckPolicy(8, Duration.ofDays(3), Duration.ofDays(1), Duration.ofDays(1));
|
||||
ClaimCheckPublisher.Offloaded offloaded =
|
||||
new ClaimCheckPublisher(store, policy).offload(PAYLOAD);
|
||||
|
||||
byte[] resolved =
|
||||
new ClaimCheckResolver(store).resolve(offloaded.payload(), offloaded.reference(), NOW);
|
||||
|
||||
assertThat(resolved).isEqualTo(PAYLOAD);
|
||||
}
|
||||
|
||||
@Test
|
||||
void anInlinePayloadIsReturnedWithoutTouchingTheStore() {
|
||||
FakeStore store = new FakeStore();
|
||||
|
||||
byte[] resolved = new ClaimCheckResolver(store).resolve(PAYLOAD, Optional.empty(), NOW);
|
||||
|
||||
assertThat(resolved).isEqualTo(PAYLOAD);
|
||||
}
|
||||
|
||||
@Test
|
||||
void anObjectThatWasSwappedUnderTheReferenceIsAPoisonMessage() {
|
||||
FakeStore store = new FakeStore();
|
||||
ClaimCheckPolicy policy =
|
||||
new ClaimCheckPolicy(8, Duration.ofDays(3), Duration.ofDays(1), Duration.ofDays(1));
|
||||
ClaimCheckPublisher.Offloaded offloaded =
|
||||
new ClaimCheckPublisher(store, policy).offload(PAYLOAD);
|
||||
store.overwrite(
|
||||
offloaded.reference().orElseThrow().storageKey(),
|
||||
"{\"orderId\":\"SOMEONE-ELSE\"}".getBytes(StandardCharsets.UTF_8));
|
||||
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new ClaimCheckResolver(store)
|
||||
.resolve(offloaded.payload(), offloaded.reference(), NOW))
|
||||
.as("the digest is the only thing between a swapped object and the wrong data")
|
||||
.isInstanceOf(ClaimCheckIntegrityException.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void anIntegrityFailureIsNotRetryableBecauseTheKeyReturnsTheSameBytes() {
|
||||
ClaimCheckIntegrityException failure =
|
||||
new ClaimCheckIntegrityException("CLAIM_CHECK_DIGEST_MISMATCH", "swapped");
|
||||
|
||||
assertThat(failure.failure().retryable()).isFalse();
|
||||
}
|
||||
|
||||
@Test
|
||||
void amissingObjectIsReportedSeparatelyFromASwappedOne() {
|
||||
FakeStore store = new FakeStore();
|
||||
ClaimCheckPolicy policy =
|
||||
new ClaimCheckPolicy(8, Duration.ofDays(3), Duration.ofDays(1), Duration.ofDays(1));
|
||||
ClaimCheckPublisher.Offloaded offloaded =
|
||||
new ClaimCheckPublisher(store, policy).offload(PAYLOAD);
|
||||
store.delete(offloaded.reference().orElseThrow());
|
||||
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new ClaimCheckResolver(store)
|
||||
.resolve(offloaded.payload(), offloaded.reference(), NOW))
|
||||
.isInstanceOf(MessageValidationException.class)
|
||||
.hasMessageContaining("never written");
|
||||
}
|
||||
|
||||
@Test
|
||||
void anExpiredReferenceIsRefusedBeforeTheStoreIsEvenAsked() {
|
||||
FakeStore store = new FakeStore();
|
||||
ClaimCheckReference expired =
|
||||
new ClaimCheckReference(
|
||||
"claim/1",
|
||||
PAYLOAD.length,
|
||||
ClaimCheckIntegrityGuard.sha256(PAYLOAD),
|
||||
NOW.minus(Duration.ofHours(1)));
|
||||
|
||||
assertThatThrownBy(
|
||||
() -> new ClaimCheckResolver(store).resolve(new byte[0], Optional.of(expired), NOW))
|
||||
.isInstanceOf(MessageValidationException.class)
|
||||
.hasMessageContaining("expired");
|
||||
}
|
||||
}
|
||||
+79
@@ -0,0 +1,79 @@
|
||||
package dev.caskeleton.messaging.claimcheck;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatCode;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.MessagingConfigurationException;
|
||||
import java.time.Duration;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class ClaimCheckRetentionValidatorTest {
|
||||
|
||||
@Test
|
||||
void retentionShorterThanTheMessageLifetimeIsRefused() {
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new ClaimCheckPolicy(
|
||||
1024, Duration.ofHours(6), Duration.ofDays(1), Duration.ofDays(1)))
|
||||
.as("the object would be reaped while a consumer can still be handed its message")
|
||||
.isInstanceOf(MessagingConfigurationException.class)
|
||||
.hasMessageContaining("reaped");
|
||||
}
|
||||
|
||||
@Test
|
||||
void retentionCoveringBrokerRetentionPlusTheRetryPathIsAccepted() {
|
||||
assertThatCode(
|
||||
() ->
|
||||
new ClaimCheckPolicy(
|
||||
1024, Duration.ofDays(2), Duration.ofDays(1), Duration.ofDays(1)))
|
||||
.doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
@Test
|
||||
void theRequiredRetentionIsBrokerRetentionPlusTheRedeliveryWindow() {
|
||||
ClaimCheckPolicy policy =
|
||||
new ClaimCheckPolicy(1024, Duration.ofDays(5), Duration.ofDays(2), Duration.ofDays(1));
|
||||
|
||||
assertThat(policy.requiredRetention()).isEqualTo(Duration.ofDays(3));
|
||||
}
|
||||
|
||||
@Test
|
||||
void theDefaultsSatisfyTheirOwnRule() {
|
||||
assertThatCode(ClaimCheckPolicy::defaults).doesNotThrowAnyException();
|
||||
}
|
||||
|
||||
/** The portable payload limit the platform documents, restated here so the two cannot drift. */
|
||||
private static final int PORTABLE_PAYLOAD_LIMIT_BYTES = 1_048_576;
|
||||
|
||||
@Test
|
||||
void theOffloadThresholdSitsWellBelowThePortablePayloadLimit() {
|
||||
assertThat(ClaimCheckPolicy.DEFAULT_THRESHOLD_BYTES)
|
||||
.as(
|
||||
"offloading starts where carrying inline stops being wise, not where the broker refuses")
|
||||
.isLessThan(PORTABLE_PAYLOAD_LIMIT_BYTES);
|
||||
}
|
||||
|
||||
@Test
|
||||
void aPayloadAtTheThresholdStillTravelsInline() {
|
||||
ClaimCheckPolicy policy = ClaimCheckPolicy.defaults();
|
||||
|
||||
assertThat(policy.shouldOffload(ClaimCheckPolicy.DEFAULT_THRESHOLD_BYTES)).isFalse();
|
||||
assertThat(policy.shouldOffload(ClaimCheckPolicy.DEFAULT_THRESHOLD_BYTES + 1)).isTrue();
|
||||
}
|
||||
|
||||
@Test
|
||||
void aNonPositiveDurationIsRefused() {
|
||||
assertThatThrownBy(
|
||||
() -> new ClaimCheckPolicy(1024, Duration.ZERO, Duration.ofDays(1), Duration.ofDays(1)))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void aNonPositiveThresholdIsRefused() {
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new ClaimCheckPolicy(0, Duration.ofDays(3), Duration.ofDays(1), Duration.ofDays(1)))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
apply plugin: 'java-library'
|
||||
|
||||
dependencies {
|
||||
api project(':messaging:messaging-core-api')
|
||||
api project(':messaging:messaging-schema-api')
|
||||
|
||||
implementation 'io.cloudevents:cloudevents-api:4.0.1'
|
||||
implementation 'io.cloudevents:cloudevents-core:4.0.1'
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
# This is a Gradle generated file for dependency locking.
|
||||
# Manual edits can break the build and are not advised.
|
||||
# This file is expected to be part of source control.
|
||||
com.github.ben-manes.caffeine:caffeine:3.2.3=annotationProcessor,testAnnotationProcessor
|
||||
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor
|
||||
com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs
|
||||
com.github.spotbugs:spotbugs:4.10.2=spotbugs
|
||||
com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs
|
||||
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor
|
||||
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor
|
||||
com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor
|
||||
com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs
|
||||
com.google.code.gson:gson:2.13.2=spotbugs
|
||||
com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_annotations:2.41.0=spotbugs
|
||||
com.google.errorprone:error_prone_annotations:2.47.0=checkstyle
|
||||
com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor
|
||||
com.google.guava:guava:33.6.0-jre=checkstyle
|
||||
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor
|
||||
com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins
|
||||
com.puppycrawl.tools:checkstyle:13.5.0=checkstyle
|
||||
commons-beanutils:commons-beanutils:1.11.0=checkstyle
|
||||
commons-collections:commons-collections:3.2.2=checkstyle
|
||||
commons-io:commons-io:2.21.0=spotbugs
|
||||
info.picocli:picocli:4.7.7=checkstyle
|
||||
io.cloudevents:cloudevents-api:4.0.1=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
|
||||
io.cloudevents:cloudevents-core:4.0.1=compileClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
|
||||
io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor
|
||||
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor
|
||||
javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor
|
||||
jaxen:jaxen:2.0.0=spotbugs
|
||||
net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath
|
||||
net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs
|
||||
org.antlr:antlr4-runtime:4.13.2=checkstyle
|
||||
org.apache.bcel:bcel:6.12.0=spotbugs
|
||||
org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs
|
||||
org.apache.commons:commons-text:1.15.0=spotbugs
|
||||
org.apache.commons:commons-text:1.3=checkstyle
|
||||
org.apache.httpcomponents:httpclient:4.5.13=checkstyle
|
||||
org.apache.httpcomponents:httpcore:4.4.16=checkstyle
|
||||
org.apache.logging.log4j:log4j-api:2.25.2=spotbugs
|
||||
org.apache.logging.log4j:log4j-core:2.25.2=spotbugs
|
||||
org.apache.maven.doxia:doxia-core:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle
|
||||
org.apache.xbean:xbean-reflect:3.7=checkstyle
|
||||
org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath
|
||||
org.assertj:assertj-core:3.27.6=testCompileClasspath,testRuntimeClasspath
|
||||
org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle
|
||||
org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle
|
||||
org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle
|
||||
org.codehaus.plexus:plexus-utils:3.3.0=checkstyle
|
||||
org.dom4j:dom4j:2.2.0=spotbugs
|
||||
org.javassist:javassist:3.28.0-GA=checkstyle
|
||||
org.jspecify:jspecify:1.0.0=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath
|
||||
org.junit.jupiter:junit-jupiter-api:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-engine:6.0.1=testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-params:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-commons:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-engine:6.0.1=testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-launcher:6.0.1=testRuntimeClasspath
|
||||
org.junit:junit-bom:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit:junit-bom:6.1.0=spotbugs
|
||||
org.mockito:mockito-core:5.20.0=mockitoAgent
|
||||
org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath
|
||||
org.ow2.asm:asm-analysis:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-commons:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-tree:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-util:9.10.1=spotbugs
|
||||
org.ow2.asm:asm:9.10.1=spotbugs
|
||||
org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor
|
||||
org.reflections:reflections:0.10.2=checkstyle
|
||||
org.slf4j:slf4j-api:2.0.17=spotbugs,spotbugsSlf4j
|
||||
org.slf4j:slf4j-simple:2.0.17=checkstyle,spotbugsSlf4j
|
||||
org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs
|
||||
empty=
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
package dev.caskeleton.messaging.cloudevents;
|
||||
|
||||
/**
|
||||
* The CloudEvents extension attribute names this profile writes.
|
||||
*
|
||||
* <p>CloudEvents requires extension names to be lowercase alphanumeric, which is why these are not
|
||||
* simply the envelope field names.
|
||||
*/
|
||||
public final class CloudEventExtensions {
|
||||
|
||||
/** Carries the envelope's correlation id. */
|
||||
public static final String CORRELATION_ID = "correlationid";
|
||||
|
||||
/** Carries the envelope's causation id. */
|
||||
public static final String CAUSATION_ID = "causationid";
|
||||
|
||||
/** Carries the envelope's schema version. */
|
||||
public static final String SCHEMA_VERSION = "schemaversion";
|
||||
|
||||
/** Carries the envelope's tenant identity. */
|
||||
public static final String TENANT_CONTEXT = "tenantcontext";
|
||||
|
||||
private CloudEventExtensions() {}
|
||||
}
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
package dev.caskeleton.messaging.cloudevents;
|
||||
|
||||
import dev.caskeleton.messaging.api.MessageEnvelope;
|
||||
import dev.caskeleton.messaging.schema.EncodedMessage;
|
||||
import io.cloudevents.CloudEvent;
|
||||
import java.net.URI;
|
||||
|
||||
/**
|
||||
* Maps between the platform envelope and CloudEvents 1.0.2.
|
||||
*
|
||||
* <p>Offered for domain and integration events only. Commands and work items are not forced through
|
||||
* CloudEvents: they are internal contracts where the interoperability the specification buys does
|
||||
* not pay for the attributes it requires.
|
||||
*/
|
||||
public interface CloudEventMapper {
|
||||
|
||||
/**
|
||||
* Converts an envelope to a CloudEvent.
|
||||
*
|
||||
* @param envelope the envelope carrying an already-encoded payload
|
||||
* @param source the event source URI
|
||||
* @return the CloudEvent
|
||||
*/
|
||||
CloudEvent toCloudEvent(MessageEnvelope<?> envelope, URI source);
|
||||
|
||||
/**
|
||||
* Converts a CloudEvent back to an envelope.
|
||||
*
|
||||
* @param event the CloudEvent
|
||||
* @return an envelope whose payload is the still-encoded data
|
||||
*/
|
||||
MessageEnvelope<EncodedMessage> fromCloudEvent(CloudEvent event);
|
||||
}
|
||||
+171
@@ -0,0 +1,171 @@
|
||||
package dev.caskeleton.messaging.cloudevents;
|
||||
|
||||
import dev.caskeleton.messaging.api.CausationId;
|
||||
import dev.caskeleton.messaging.api.ContentType;
|
||||
import dev.caskeleton.messaging.api.CorrelationId;
|
||||
import dev.caskeleton.messaging.api.MessageEnvelope;
|
||||
import dev.caskeleton.messaging.api.MessageId;
|
||||
import dev.caskeleton.messaging.api.MessageType;
|
||||
import dev.caskeleton.messaging.api.ProducerId;
|
||||
import dev.caskeleton.messaging.api.SchemaVersion;
|
||||
import dev.caskeleton.messaging.api.TenantContext;
|
||||
import dev.caskeleton.messaging.api.TraceContext;
|
||||
import dev.caskeleton.messaging.api.error.MessageValidationException;
|
||||
import dev.caskeleton.messaging.api.header.MessageHeaders;
|
||||
import dev.caskeleton.messaging.schema.EncodedMessage;
|
||||
import io.cloudevents.CloudEvent;
|
||||
import io.cloudevents.CloudEventData;
|
||||
import io.cloudevents.core.builder.CloudEventBuilder;
|
||||
import io.cloudevents.core.data.BytesCloudEventData;
|
||||
import java.net.URI;
|
||||
import java.time.Instant;
|
||||
import java.time.OffsetDateTime;
|
||||
import java.time.ZoneOffset;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* The CloudEvents 1.0.2 compatible profile.
|
||||
*
|
||||
* <p>Two mapping decisions are deliberate. An event without {@code occurredAt} is rejected rather
|
||||
* than defaulted to the production instant, because {@code time} is read downstream as when the
|
||||
* fact happened, not when the platform got around to serialising it. And an event with no data maps
|
||||
* to an envelope with empty bytes, never to a Kafka null value: a tombstone deletes a key, and
|
||||
* inventing one from an absent CloudEvent payload would turn an empty notification into a deletion.
|
||||
*/
|
||||
public final class DefaultCloudEventMapper implements CloudEventMapper {
|
||||
|
||||
private static final String SPEC_CONTENT_TYPE_FALLBACK = "application/json";
|
||||
|
||||
@Override
|
||||
public CloudEvent toCloudEvent(MessageEnvelope<?> envelope, URI source) {
|
||||
Objects.requireNonNull(envelope, "envelope must not be null");
|
||||
Objects.requireNonNull(source, "source must not be null");
|
||||
|
||||
Instant occurredAt =
|
||||
envelope
|
||||
.occurredAt()
|
||||
.orElseThrow(
|
||||
() ->
|
||||
new MessageValidationException(
|
||||
"CLOUDEVENT_TIME_REQUIRED",
|
||||
"an event mapped to CloudEvents requires occurredAt"));
|
||||
|
||||
CloudEventBuilder builder =
|
||||
CloudEventBuilder.v1()
|
||||
.withId(envelope.messageId().value().toString())
|
||||
.withSource(source)
|
||||
.withType(envelope.messageType().value())
|
||||
.withTime(OffsetDateTime.ofInstant(occurredAt, ZoneOffset.UTC))
|
||||
.withDataContentType(envelope.contentType().value())
|
||||
.withExtension(
|
||||
CloudEventExtensions.SCHEMA_VERSION,
|
||||
Integer.toString(envelope.schemaVersion().value()));
|
||||
|
||||
envelope
|
||||
.correlationId()
|
||||
.ifPresent(
|
||||
value -> builder.withExtension(CloudEventExtensions.CORRELATION_ID, value.value()));
|
||||
envelope
|
||||
.causationId()
|
||||
.ifPresent(
|
||||
value ->
|
||||
builder.withExtension(
|
||||
CloudEventExtensions.CAUSATION_ID, value.value().value().toString()));
|
||||
envelope
|
||||
.tenantContext()
|
||||
.ifPresent(
|
||||
value -> builder.withExtension(CloudEventExtensions.TENANT_CONTEXT, value.tenantId()));
|
||||
|
||||
if (envelope.payload() instanceof EncodedMessage encoded) {
|
||||
encoded
|
||||
.schemaReference()
|
||||
.flatMap(reference -> reference.schemaUri())
|
||||
.ifPresent(builder::withDataSchema);
|
||||
builder.withData(BytesCloudEventData.wrap(encoded.bytes()));
|
||||
} else if (envelope.payload() instanceof byte[] bytes) {
|
||||
builder.withData(BytesCloudEventData.wrap(bytes.clone()));
|
||||
} else {
|
||||
throw new MessageValidationException(
|
||||
"CLOUDEVENT_PAYLOAD_NOT_ENCODED",
|
||||
"CloudEvents mapping requires an already-encoded payload");
|
||||
}
|
||||
|
||||
return builder.build();
|
||||
}
|
||||
|
||||
@Override
|
||||
public MessageEnvelope<EncodedMessage> fromCloudEvent(CloudEvent event) {
|
||||
Objects.requireNonNull(event, "event must not be null");
|
||||
|
||||
OffsetDateTime time = event.getTime();
|
||||
if (time == null) {
|
||||
throw new MessageValidationException(
|
||||
"CLOUDEVENT_TIME_REQUIRED", "a CloudEvent mapped to an envelope requires time");
|
||||
}
|
||||
|
||||
ContentType contentType =
|
||||
new ContentType(
|
||||
Optional.ofNullable(event.getDataContentType()).orElse(SPEC_CONTENT_TYPE_FALLBACK));
|
||||
CloudEventData data = event.getData();
|
||||
byte[] bytes = data == null ? new byte[0] : data.toBytes();
|
||||
|
||||
Instant occurredAt = time.toInstant();
|
||||
return new MessageEnvelope<>(
|
||||
new MessageId(UUID.fromString(event.getId())),
|
||||
new MessageType(event.getType()),
|
||||
new SchemaVersion(intExtension(event, CloudEventExtensions.SCHEMA_VERSION)),
|
||||
occurredAt,
|
||||
Optional.of(occurredAt),
|
||||
new ProducerId(producerFrom(event.getSource())),
|
||||
stringExtension(event, CloudEventExtensions.CORRELATION_ID).map(CorrelationId::new),
|
||||
stringExtension(event, CloudEventExtensions.CAUSATION_ID)
|
||||
.map(value -> new CausationId(new MessageId(UUID.fromString(value)))),
|
||||
contentType,
|
||||
Optional.empty(),
|
||||
Optional.empty(),
|
||||
stringExtension(event, CloudEventExtensions.TENANT_CONTEXT).map(TenantContext::new),
|
||||
TraceContext.none(),
|
||||
MessageHeaders.empty(),
|
||||
new EncodedMessage(bytes, contentType, Optional.empty()));
|
||||
}
|
||||
|
||||
private static Optional<String> stringExtension(CloudEvent event, String name) {
|
||||
return Optional.ofNullable(event.getExtension(name)).map(Object::toString);
|
||||
}
|
||||
|
||||
private static int intExtension(CloudEvent event, String name) {
|
||||
return stringExtension(event, name)
|
||||
.map(
|
||||
value -> {
|
||||
try {
|
||||
return Integer.valueOf(value);
|
||||
} catch (NumberFormatException exception) {
|
||||
throw new MessageValidationException(
|
||||
"CLOUDEVENT_SCHEMA_VERSION_INVALID",
|
||||
"schemaversion extension is not an integer",
|
||||
exception);
|
||||
}
|
||||
})
|
||||
.orElseThrow(
|
||||
() ->
|
||||
new MessageValidationException(
|
||||
"CLOUDEVENT_SCHEMA_VERSION_REQUIRED",
|
||||
"schemaversion extension is required by this profile"));
|
||||
}
|
||||
|
||||
/**
|
||||
* Derives a bounded producer id from the source URI.
|
||||
*
|
||||
* <p>The last path or scheme-specific segment is used so that a long URI does not become an
|
||||
* unbounded producer name, which would leak straight into metric tags.
|
||||
*/
|
||||
private static String producerFrom(URI source) {
|
||||
String text = source.toString();
|
||||
int separator = Math.max(text.lastIndexOf('/'), text.lastIndexOf(':'));
|
||||
String candidate =
|
||||
separator >= 0 && separator + 1 < text.length() ? text.substring(separator + 1) : text;
|
||||
return candidate.isBlank() ? "unknown" : candidate;
|
||||
}
|
||||
}
|
||||
+160
@@ -0,0 +1,160 @@
|
||||
package dev.caskeleton.messaging.cloudevents;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
import dev.caskeleton.messaging.api.ContentType;
|
||||
import dev.caskeleton.messaging.api.CorrelationId;
|
||||
import dev.caskeleton.messaging.api.MessageEnvelope;
|
||||
import dev.caskeleton.messaging.api.MessageId;
|
||||
import dev.caskeleton.messaging.api.MessageType;
|
||||
import dev.caskeleton.messaging.api.ProducerId;
|
||||
import dev.caskeleton.messaging.api.SchemaVersion;
|
||||
import dev.caskeleton.messaging.api.TenantContext;
|
||||
import dev.caskeleton.messaging.api.TraceContext;
|
||||
import dev.caskeleton.messaging.api.error.MessageValidationException;
|
||||
import dev.caskeleton.messaging.api.header.MessageHeaders;
|
||||
import dev.caskeleton.messaging.schema.EncodedMessage;
|
||||
import io.cloudevents.CloudEvent;
|
||||
import java.net.URI;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Instant;
|
||||
import java.util.Optional;
|
||||
import java.util.UUID;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class CloudEventMappingTest {
|
||||
|
||||
private static final URI SOURCE = URI.create("urn:service:order-api");
|
||||
|
||||
private final DefaultCloudEventMapper mapper = new DefaultCloudEventMapper();
|
||||
|
||||
@Test
|
||||
void mapsLogicalIdentityAndExtensions() {
|
||||
MessageEnvelope<EncodedMessage> envelope = CloudEventFixture.orderCreatedEnvelope();
|
||||
|
||||
CloudEvent event = mapper.toCloudEvent(envelope, SOURCE);
|
||||
|
||||
assertThat(event.getId()).isEqualTo(envelope.messageId().value().toString());
|
||||
assertThat(event.getType()).isEqualTo("order.created");
|
||||
assertThat(event.getExtension(CloudEventExtensions.SCHEMA_VERSION)).isEqualTo("1");
|
||||
assertThat(event.getSource()).isEqualTo(SOURCE);
|
||||
assertThat(event.getDataContentType()).isEqualTo("application/json");
|
||||
}
|
||||
|
||||
@Test
|
||||
void mapsCorrelationAndTenantAsExtensions() {
|
||||
CloudEvent event = mapper.toCloudEvent(CloudEventFixture.orderCreatedEnvelope(), SOURCE);
|
||||
|
||||
assertThat(event.getExtension(CloudEventExtensions.CORRELATION_ID)).isEqualTo("wf-1");
|
||||
assertThat(event.getExtension(CloudEventExtensions.TENANT_CONTEXT)).isEqualTo("acme");
|
||||
}
|
||||
|
||||
@Test
|
||||
void mapsOccurredAtToEventTime() {
|
||||
CloudEvent event = mapper.toCloudEvent(CloudEventFixture.orderCreatedEnvelope(), SOURCE);
|
||||
|
||||
assertThat(event.getTime()).isNotNull();
|
||||
assertThat(event.getTime().toInstant()).isEqualTo(Instant.parse("2026-08-10T09:15:00Z"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsAnEventEnvelopeWithoutOccurredAt() {
|
||||
MessageEnvelope<EncodedMessage> withoutOccurredAt =
|
||||
CloudEventFixture.envelope(Optional.empty());
|
||||
|
||||
assertThatThrownBy(() -> mapper.toCloudEvent(withoutOccurredAt, SOURCE))
|
||||
.isInstanceOf(MessageValidationException.class)
|
||||
.hasMessageContaining("occurredAt");
|
||||
}
|
||||
|
||||
@Test
|
||||
void roundTripsBackToAnEnvelopeWithoutInventingATombstone() {
|
||||
MessageEnvelope<EncodedMessage> original = CloudEventFixture.orderCreatedEnvelope();
|
||||
|
||||
MessageEnvelope<EncodedMessage> restored =
|
||||
mapper.fromCloudEvent(mapper.toCloudEvent(original, SOURCE));
|
||||
|
||||
assertThat(restored.messageId()).isEqualTo(original.messageId());
|
||||
assertThat(restored.messageType()).isEqualTo(original.messageType());
|
||||
assertThat(restored.schemaVersion()).isEqualTo(original.schemaVersion());
|
||||
assertThat(restored.correlationId()).contains(new CorrelationId("wf-1"));
|
||||
assertThat(restored.tenantContext()).contains(new TenantContext("acme"));
|
||||
assertThat(restored.payload().bytes()).isEqualTo(original.payload().bytes());
|
||||
}
|
||||
|
||||
@Test
|
||||
void aCloudEventWithNoDataBecomesAnEmptyPayloadNotANullValue() {
|
||||
CloudEvent noData =
|
||||
io.cloudevents.core.builder.CloudEventBuilder.v1()
|
||||
.withId(UUID.randomUUID().toString())
|
||||
.withSource(SOURCE)
|
||||
.withType("order.created")
|
||||
.withTime(java.time.OffsetDateTime.parse("2026-08-10T09:15:00Z"))
|
||||
.withDataContentType("application/json")
|
||||
.withExtension(CloudEventExtensions.SCHEMA_VERSION, "1")
|
||||
.build();
|
||||
|
||||
MessageEnvelope<EncodedMessage> restored = mapper.fromCloudEvent(noData);
|
||||
|
||||
assertThat(restored.payload()).isNotNull();
|
||||
assertThat(restored.payload().size()).isZero();
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsAnUnencodedPayload() {
|
||||
MessageEnvelope<String> unencoded =
|
||||
new MessageEnvelope<>(
|
||||
MessageId.newId(),
|
||||
new MessageType("order.created"),
|
||||
new SchemaVersion(1),
|
||||
Instant.parse("2026-08-10T09:15:00Z"),
|
||||
Optional.of(Instant.parse("2026-08-10T09:15:00Z")),
|
||||
new ProducerId("order-api"),
|
||||
Optional.empty(),
|
||||
Optional.empty(),
|
||||
ContentType.JSON,
|
||||
Optional.empty(),
|
||||
Optional.empty(),
|
||||
Optional.empty(),
|
||||
TraceContext.none(),
|
||||
MessageHeaders.empty(),
|
||||
"not encoded");
|
||||
|
||||
assertThatThrownBy(() -> mapper.toCloudEvent(unencoded, SOURCE))
|
||||
.isInstanceOf(MessageValidationException.class);
|
||||
}
|
||||
}
|
||||
|
||||
/** Builds CloudEvents mapping fixtures. */
|
||||
final class CloudEventFixture {
|
||||
|
||||
private static final MessageId FIXED_ID =
|
||||
new MessageId(UUID.fromString("0190f4aa-0000-7000-8000-000000000001"));
|
||||
|
||||
private CloudEventFixture() {}
|
||||
|
||||
static MessageEnvelope<EncodedMessage> orderCreatedEnvelope() {
|
||||
return envelope(Optional.of(Instant.parse("2026-08-10T09:15:00Z")));
|
||||
}
|
||||
|
||||
static MessageEnvelope<EncodedMessage> envelope(Optional<Instant> occurredAt) {
|
||||
byte[] payload = "{\"orderId\":\"o-1\"}".getBytes(StandardCharsets.UTF_8);
|
||||
return new MessageEnvelope<>(
|
||||
FIXED_ID,
|
||||
new MessageType("order.created"),
|
||||
new SchemaVersion(1),
|
||||
Instant.parse("2026-08-10T09:15:01Z"),
|
||||
occurredAt,
|
||||
new ProducerId("order-api"),
|
||||
Optional.of(new CorrelationId("wf-1")),
|
||||
Optional.empty(),
|
||||
ContentType.JSON,
|
||||
Optional.empty(),
|
||||
Optional.empty(),
|
||||
Optional.of(new TenantContext("acme")),
|
||||
TraceContext.none(),
|
||||
MessageHeaders.empty(),
|
||||
new EncodedMessage(payload, ContentType.JSON, Optional.empty()));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
apply plugin: 'java-library'
|
||||
|
||||
dependencies {
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
# This is a Gradle generated file for dependency locking.
|
||||
# Manual edits can break the build and are not advised.
|
||||
# This file is expected to be part of source control.
|
||||
com.github.ben-manes.caffeine:caffeine:3.2.3=annotationProcessor,testAnnotationProcessor
|
||||
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,testAnnotationProcessor
|
||||
com.github.spotbugs:spotbugs-annotations:4.10.2=spotbugs
|
||||
com.github.spotbugs:spotbugs:4.10.2=spotbugs
|
||||
com.github.stephenc.jcip:jcip-annotations:1.0-1=spotbugs
|
||||
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,testAnnotationProcessor
|
||||
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,testAnnotationProcessor
|
||||
com.google.auto:auto-common:1.2.2=annotationProcessor,testAnnotationProcessor
|
||||
com.google.code.findbugs:jsr305:3.0.2=checkstyle,spotbugs
|
||||
com.google.code.gson:gson:2.13.2=spotbugs
|
||||
com.google.errorprone:error_prone_annotation:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_annotations:2.41.0=spotbugs
|
||||
com.google.errorprone:error_prone_annotations:2.47.0=checkstyle
|
||||
com.google.errorprone:error_prone_annotations:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_check_api:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.errorprone:error_prone_core:2.49.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.googlejavaformat:google-java-format:1.35.0=annotationProcessor,testAnnotationProcessor
|
||||
com.google.guava:failureaccess:1.0.3=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.guava:guava:33.5.0-jre=annotationProcessor,testAnnotationProcessor
|
||||
com.google.guava:guava:33.6.0-jre=checkstyle
|
||||
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.j2objc:j2objc-annotations:3.1=annotationProcessor,checkstyle,testAnnotationProcessor
|
||||
com.google.protobuf:protobuf-java:4.33.2=annotationProcessor,testAnnotationProcessor
|
||||
com.h3xstream.findsecbugs:findsecbugs-plugin:1.14.0=spotbugsPlugins
|
||||
com.puppycrawl.tools:checkstyle:13.5.0=checkstyle
|
||||
commons-beanutils:commons-beanutils:1.11.0=checkstyle
|
||||
commons-collections:commons-collections:3.2.2=checkstyle
|
||||
commons-io:commons-io:2.21.0=spotbugs
|
||||
info.picocli:picocli:4.7.7=checkstyle
|
||||
io.github.eisop:dataflow-errorprone:3.41.0-eisop1=annotationProcessor,testAnnotationProcessor
|
||||
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,testAnnotationProcessor
|
||||
javax.inject:javax.inject:1=annotationProcessor,testAnnotationProcessor
|
||||
jaxen:jaxen:2.0.0=spotbugs
|
||||
net.bytebuddy:byte-buddy:1.17.8=testCompileClasspath,testRuntimeClasspath
|
||||
net.sf.saxon:Saxon-HE:12.9=checkstyle,spotbugs
|
||||
org.antlr:antlr4-runtime:4.13.2=checkstyle
|
||||
org.apache.bcel:bcel:6.12.0=spotbugs
|
||||
org.apache.commons:commons-lang3:3.20.0=checkstyle,spotbugs
|
||||
org.apache.commons:commons-text:1.15.0=spotbugs
|
||||
org.apache.commons:commons-text:1.3=checkstyle
|
||||
org.apache.httpcomponents:httpclient:4.5.13=checkstyle
|
||||
org.apache.httpcomponents:httpcore:4.4.16=checkstyle
|
||||
org.apache.logging.log4j:log4j-api:2.25.2=spotbugs
|
||||
org.apache.logging.log4j:log4j-core:2.25.2=spotbugs
|
||||
org.apache.maven.doxia:doxia-core:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-logging-api:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-module-xdoc:1.12.0=checkstyle
|
||||
org.apache.maven.doxia:doxia-sink-api:1.12.0=checkstyle
|
||||
org.apache.xbean:xbean-reflect:3.7=checkstyle
|
||||
org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath
|
||||
org.assertj:assertj-core:3.27.6=testCompileClasspath,testRuntimeClasspath
|
||||
org.codehaus.plexus:plexus-classworlds:2.6.0=checkstyle
|
||||
org.codehaus.plexus:plexus-component-annotations:2.1.0=checkstyle
|
||||
org.codehaus.plexus:plexus-container-default:2.1.0=checkstyle
|
||||
org.codehaus.plexus:plexus-utils:3.3.0=checkstyle
|
||||
org.dom4j:dom4j:2.2.0=spotbugs
|
||||
org.javassist:javassist:3.28.0-GA=checkstyle
|
||||
org.jspecify:jspecify:1.0.0=annotationProcessor,checkstyle,testAnnotationProcessor,testCompileClasspath
|
||||
org.junit.jupiter:junit-jupiter-api:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-engine:6.0.1=testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter-params:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.jupiter:junit-jupiter:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-commons:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-engine:6.0.1=testRuntimeClasspath
|
||||
org.junit.platform:junit-platform-launcher:6.0.1=testRuntimeClasspath
|
||||
org.junit:junit-bom:6.0.1=testCompileClasspath,testRuntimeClasspath
|
||||
org.junit:junit-bom:6.1.0=spotbugs
|
||||
org.mockito:mockito-core:5.20.0=mockitoAgent
|
||||
org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath
|
||||
org.ow2.asm:asm-analysis:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-commons:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-tree:9.10.1=spotbugs
|
||||
org.ow2.asm:asm-util:9.10.1=spotbugs
|
||||
org.ow2.asm:asm:9.10.1=spotbugs
|
||||
org.pcollections:pcollections:4.0.1=annotationProcessor,testAnnotationProcessor
|
||||
org.reflections:reflections:0.10.2=checkstyle
|
||||
org.slf4j:slf4j-api:2.0.17=spotbugs,spotbugsSlf4j
|
||||
org.slf4j:slf4j-simple:2.0.17=checkstyle,spotbugsSlf4j
|
||||
org.xmlresolver:xmlresolver:5.3.3=checkstyle,spotbugs
|
||||
empty=compileClasspath,runtimeClasspath
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* Identity of the message that directly caused this one.
|
||||
*
|
||||
* <p>Unlike {@link CorrelationId}, which spans a whole workflow, this points at exactly one
|
||||
* predecessor and forms the causal edge used when reconstructing a flow.
|
||||
*
|
||||
* @param value the predecessor message identity
|
||||
*/
|
||||
public record CausationId(MessageId value) {
|
||||
|
||||
public CausationId {
|
||||
Objects.requireNonNull(value, "causationId value must not be null");
|
||||
}
|
||||
}
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
import java.util.Locale;
|
||||
|
||||
/**
|
||||
* Media type of an encoded payload, resolved through the codec registry.
|
||||
*
|
||||
* @param value the media type, for example {@code application/json}
|
||||
*/
|
||||
public record ContentType(String value) {
|
||||
|
||||
private static final int MAX_LENGTH = 160;
|
||||
|
||||
/** The Stable default codec's content type. */
|
||||
public static final ContentType JSON = new ContentType("application/json");
|
||||
|
||||
/** Avro binary content type. */
|
||||
public static final ContentType AVRO = new ContentType("application/avro");
|
||||
|
||||
/** Protobuf binary content type. */
|
||||
public static final ContentType PROTOBUF = new ContentType("application/x-protobuf");
|
||||
|
||||
/** Opaque bytes, only reachable through the M2 raw codec. */
|
||||
public static final ContentType OCTET_STREAM = new ContentType("application/octet-stream");
|
||||
|
||||
public ContentType {
|
||||
if (value == null || value.isBlank() || value.length() > MAX_LENGTH) {
|
||||
throw new IllegalArgumentException("contentType must contain 1 to 160 characters");
|
||||
}
|
||||
if (value.indexOf('/') < 0) {
|
||||
throw new IllegalArgumentException("contentType must be a media type: " + value);
|
||||
}
|
||||
value = value.toLowerCase(Locale.ROOT);
|
||||
}
|
||||
}
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
/**
|
||||
* Workflow-scoped correlation value shared by every message of one business flow.
|
||||
*
|
||||
* @param value the correlation value, 1 to 160 characters
|
||||
*/
|
||||
public record CorrelationId(String value) {
|
||||
|
||||
private static final int MAX_LENGTH = 160;
|
||||
|
||||
public CorrelationId {
|
||||
if (value == null || value.isBlank() || value.length() > MAX_LENGTH) {
|
||||
throw new IllegalArgumentException("correlationId must contain 1 to 160 characters");
|
||||
}
|
||||
}
|
||||
}
|
||||
+146
@@ -0,0 +1,146 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
import dev.caskeleton.messaging.api.header.MessageHeaders;
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* The platform's unit of transfer: identity, provenance, routing intent, and payload.
|
||||
*
|
||||
* <p>The payload is never null. A null Kafka value is a tombstone, which is a distinct
|
||||
* broker-native operation with different retention semantics, so generalising it into "an envelope
|
||||
* with no payload" would silently turn a delete into an event on brokers that have no such concept.
|
||||
*
|
||||
* @param <T> the payload type
|
||||
* @param messageId logical identity, preserved across retry, DLQ, and redrive
|
||||
* @param messageType stable catalog type
|
||||
* @param schemaVersion payload schema revision
|
||||
* @param producedAt instant the platform created this envelope
|
||||
* @param occurredAt instant the business fact occurred; required for events
|
||||
* @param producer logical producing service
|
||||
* @param correlationId workflow correlation
|
||||
* @param causationId directly causing message
|
||||
* @param contentType codec media type
|
||||
* @param partitionKey distribution key
|
||||
* @param orderingKey ordering key
|
||||
* @param tenantContext bounded tenant identity
|
||||
* @param traceContext trace propagation values
|
||||
* @param headers bounded application headers
|
||||
* @param payload the non-null payload
|
||||
*/
|
||||
public record MessageEnvelope<T>(
|
||||
MessageId messageId,
|
||||
MessageType messageType,
|
||||
SchemaVersion schemaVersion,
|
||||
Instant producedAt,
|
||||
Optional<Instant> occurredAt,
|
||||
ProducerId producer,
|
||||
Optional<CorrelationId> correlationId,
|
||||
Optional<CausationId> causationId,
|
||||
ContentType contentType,
|
||||
Optional<String> partitionKey,
|
||||
Optional<String> orderingKey,
|
||||
Optional<TenantContext> tenantContext,
|
||||
TraceContext traceContext,
|
||||
MessageHeaders headers,
|
||||
T payload) {
|
||||
|
||||
public MessageEnvelope {
|
||||
Objects.requireNonNull(messageId, "messageId must not be null");
|
||||
Objects.requireNonNull(messageType, "messageType must not be null");
|
||||
Objects.requireNonNull(schemaVersion, "schemaVersion must not be null");
|
||||
Objects.requireNonNull(producedAt, "producedAt must not be null");
|
||||
Objects.requireNonNull(occurredAt, "occurredAt must not be null");
|
||||
Objects.requireNonNull(producer, "producer must not be null");
|
||||
Objects.requireNonNull(correlationId, "correlationId must not be null");
|
||||
Objects.requireNonNull(causationId, "causationId must not be null");
|
||||
Objects.requireNonNull(contentType, "contentType must not be null");
|
||||
Objects.requireNonNull(partitionKey, "partitionKey must not be null");
|
||||
Objects.requireNonNull(orderingKey, "orderingKey must not be null");
|
||||
Objects.requireNonNull(tenantContext, "tenantContext must not be null");
|
||||
Objects.requireNonNull(traceContext, "traceContext must not be null");
|
||||
Objects.requireNonNull(headers, "headers must not be null");
|
||||
Objects.requireNonNull(payload, "payload must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a copy of this envelope carrying a different payload representation.
|
||||
*
|
||||
* <p>Encoding, decoding, Claim Check offloading, and DLQ forwarding all need this, and every one
|
||||
* of them must keep {@link #messageId()} intact — which is exactly what this method guarantees by
|
||||
* construction.
|
||||
*
|
||||
* @param <R> the replacement payload type
|
||||
* @param replacement the new payload
|
||||
* @return an envelope with identical identity and metadata
|
||||
*/
|
||||
public <R> MessageEnvelope<R> withPayload(R replacement) {
|
||||
return new MessageEnvelope<>(
|
||||
messageId,
|
||||
messageType,
|
||||
schemaVersion,
|
||||
producedAt,
|
||||
occurredAt,
|
||||
producer,
|
||||
correlationId,
|
||||
causationId,
|
||||
contentType,
|
||||
partitionKey,
|
||||
orderingKey,
|
||||
tenantContext,
|
||||
traceContext,
|
||||
headers,
|
||||
replacement);
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a copy of this envelope carrying a different content type.
|
||||
*
|
||||
* @param replacement the new content type
|
||||
* @return an envelope with identical identity and payload
|
||||
*/
|
||||
public MessageEnvelope<T> withContentType(ContentType replacement) {
|
||||
return new MessageEnvelope<>(
|
||||
messageId,
|
||||
messageType,
|
||||
schemaVersion,
|
||||
producedAt,
|
||||
occurredAt,
|
||||
producer,
|
||||
correlationId,
|
||||
causationId,
|
||||
Objects.requireNonNull(replacement, "contentType must not be null"),
|
||||
partitionKey,
|
||||
orderingKey,
|
||||
tenantContext,
|
||||
traceContext,
|
||||
headers,
|
||||
payload);
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a copy of this envelope carrying replacement headers.
|
||||
*
|
||||
* @param replacement the new headers
|
||||
* @return an envelope with identical identity and payload
|
||||
*/
|
||||
public MessageEnvelope<T> withHeaders(MessageHeaders replacement) {
|
||||
return new MessageEnvelope<>(
|
||||
messageId,
|
||||
messageType,
|
||||
schemaVersion,
|
||||
producedAt,
|
||||
occurredAt,
|
||||
producer,
|
||||
correlationId,
|
||||
causationId,
|
||||
contentType,
|
||||
partitionKey,
|
||||
orderingKey,
|
||||
tenantContext,
|
||||
traceContext,
|
||||
Objects.requireNonNull(replacement, "headers must not be null"),
|
||||
payload);
|
||||
}
|
||||
}
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
import java.util.Objects;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* Logical identity of a message.
|
||||
*
|
||||
* <p>This value survives publish retry, broker redelivery, retry destinations, dead lettering, and
|
||||
* redrive. A new {@code MessageId} is minted only for a genuinely new business fact or command, so
|
||||
* an Inbox can use it to suppress duplicate side effects.
|
||||
*
|
||||
* @param value the UUIDv7 identity
|
||||
*/
|
||||
public record MessageId(UUID value) {
|
||||
|
||||
public MessageId {
|
||||
Objects.requireNonNull(value, "messageId value must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Mints a new logical message identity.
|
||||
*
|
||||
* @return a fresh time-ordered identity
|
||||
*/
|
||||
public static MessageId newId() {
|
||||
return new MessageId(UuidV7.next());
|
||||
}
|
||||
}
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
/**
|
||||
* Stable catalog name of a message, such as {@code order.created}.
|
||||
*
|
||||
* <p>Java class names are deliberately not usable as a message type: renaming or repackaging a
|
||||
* class must never change the wire contract.
|
||||
*
|
||||
* @param value the catalog name, 1 to 240 characters
|
||||
*/
|
||||
public record MessageType(String value) {
|
||||
|
||||
private static final int MAX_LENGTH = 240;
|
||||
|
||||
public MessageType {
|
||||
if (value == null || value.isBlank() || value.length() > MAX_LENGTH) {
|
||||
throw new IllegalArgumentException("messageType must contain 1 to 240 characters");
|
||||
}
|
||||
}
|
||||
}
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
/**
|
||||
* Logical service identity of the component that produced a message.
|
||||
*
|
||||
* <p>This is a deployment-independent service name, not a host, pod, or connection identity, so
|
||||
* that it stays a bounded value safe for metric tags.
|
||||
*
|
||||
* @param value the service name, 1 to 120 characters
|
||||
*/
|
||||
public record ProducerId(String value) {
|
||||
|
||||
private static final int MAX_LENGTH = 120;
|
||||
|
||||
public ProducerId {
|
||||
if (value == null || value.isBlank() || value.length() > MAX_LENGTH) {
|
||||
throw new IllegalArgumentException("producerId must contain 1 to 120 characters");
|
||||
}
|
||||
}
|
||||
}
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
/**
|
||||
* Monotonic schema revision of a {@link MessageType}.
|
||||
*
|
||||
* @param value the revision, starting at 1
|
||||
*/
|
||||
public record SchemaVersion(int value) {
|
||||
|
||||
public SchemaVersion {
|
||||
if (value < 1) {
|
||||
throw new IllegalArgumentException("schemaVersion must be positive");
|
||||
}
|
||||
}
|
||||
}
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
/**
|
||||
* Bounded tenant identity carried with a message.
|
||||
*
|
||||
* <p>The value is deliberately constrained to a short slug. Tenant identity is one of the few
|
||||
* envelope fields that observability code is tempted to use as a metric label, and an unbounded
|
||||
* tenant id turns that into a cardinality explosion.
|
||||
*
|
||||
* @param tenantId the tenant slug
|
||||
*/
|
||||
public record TenantContext(String tenantId) {
|
||||
|
||||
private static final Pattern VALID = Pattern.compile("[a-z0-9][a-z0-9._-]{0,63}");
|
||||
|
||||
public TenantContext {
|
||||
if (tenantId == null || !VALID.matcher(tenantId).matches()) {
|
||||
throw new IllegalArgumentException(
|
||||
"tenantId must match [a-z0-9][a-z0-9._-]{0,63}: " + tenantId);
|
||||
}
|
||||
}
|
||||
}
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* W3C trace propagation values carried with a message.
|
||||
*
|
||||
* <p>The platform creates and propagates these; handlers do not set them. Keeping them on the
|
||||
* envelope rather than only in headers means a trace survives an Outbox round trip through the
|
||||
* database, where broker headers do not exist yet.
|
||||
*
|
||||
* @param traceparent the {@code traceparent} value when a trace is active
|
||||
* @param tracestate the {@code tracestate} value when present
|
||||
* @param baggage the {@code baggage} value when present
|
||||
*/
|
||||
public record TraceContext(
|
||||
Optional<String> traceparent, Optional<String> tracestate, Optional<String> baggage) {
|
||||
|
||||
public TraceContext {
|
||||
Objects.requireNonNull(traceparent, "traceparent must not be null");
|
||||
Objects.requireNonNull(tracestate, "tracestate must not be null");
|
||||
Objects.requireNonNull(baggage, "baggage must not be null");
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a context with no active trace.
|
||||
*
|
||||
* @return an empty trace context
|
||||
*/
|
||||
public static TraceContext none() {
|
||||
return new TraceContext(Optional.empty(), Optional.empty(), Optional.empty());
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a context carrying only a trace parent.
|
||||
*
|
||||
* @param traceparent the {@code traceparent} value
|
||||
* @return a trace context
|
||||
*/
|
||||
public static TraceContext of(String traceparent) {
|
||||
return new TraceContext(Optional.of(traceparent), Optional.empty(), Optional.empty());
|
||||
}
|
||||
}
|
||||
+62
@@ -0,0 +1,62 @@
|
||||
package dev.caskeleton.messaging.api;
|
||||
|
||||
import java.security.SecureRandom;
|
||||
import java.util.UUID;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
|
||||
/**
|
||||
* Generates RFC 9562 UUIDv7 values.
|
||||
*
|
||||
* <p>The platform needs message identifiers that sort by creation time so that outbox scans, DLQ
|
||||
* listings, and broker partitions stay locality-friendly, while still being globally unique. The 12
|
||||
* bit {@code rand_a} field is used as a monotonic intra-millisecond counter instead of random bits:
|
||||
* two calls in the same millisecond are then still strictly ordered, which is what makes "same
|
||||
* logical message keeps the same id" auditable across a retry.
|
||||
*/
|
||||
public final class UuidV7 {
|
||||
|
||||
private static final SecureRandom RANDOM = new SecureRandom();
|
||||
|
||||
/** Packs the 48-bit millisecond timestamp in the high bits and the 12-bit counter in the low. */
|
||||
private static final AtomicLong STATE = new AtomicLong();
|
||||
|
||||
private static final long COUNTER_BITS = 12L;
|
||||
private static final long COUNTER_MASK = 0xFFFL;
|
||||
private static final long VERSION_7 = 0x7L;
|
||||
private static final long VARIANT_RFC9562 = 0x8000_0000_0000_0000L;
|
||||
private static final long RANDOM_B_MASK = 0x3FFF_FFFF_FFFF_FFFFL;
|
||||
|
||||
private UuidV7() {}
|
||||
|
||||
/**
|
||||
* Returns the next monotonically increasing UUIDv7.
|
||||
*
|
||||
* @return a version 7, variant 2 UUID
|
||||
*/
|
||||
public static UUID next() {
|
||||
long state = STATE.updateAndGet(UuidV7::advance);
|
||||
long timestamp = state >>> COUNTER_BITS;
|
||||
long counter = state & COUNTER_MASK;
|
||||
|
||||
long mostSignificantBits = (timestamp << 16) | (VERSION_7 << COUNTER_BITS) | counter;
|
||||
long leastSignificantBits = (RANDOM.nextLong() & RANDOM_B_MASK) | VARIANT_RFC9562;
|
||||
return new UUID(mostSignificantBits, leastSignificantBits);
|
||||
}
|
||||
|
||||
/**
|
||||
* Advances the packed state.
|
||||
*
|
||||
* <p>When the clock moved forward the counter restarts at zero. Otherwise the packed value is
|
||||
* simply incremented: that bumps the counter and, once the 12-bit counter overflows, carries into
|
||||
* the timestamp field. A backwards clock step therefore never produces a duplicate or a
|
||||
* descending id, it only borrows from the future.
|
||||
*/
|
||||
private static long advance(long previous) {
|
||||
long now = System.currentTimeMillis();
|
||||
long previousTimestamp = previous >>> COUNTER_BITS;
|
||||
if (now > previousTimestamp) {
|
||||
return now << COUNTER_BITS;
|
||||
}
|
||||
return previous + 1;
|
||||
}
|
||||
}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
import dev.caskeleton.messaging.api.destination.DestinationName;
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Batch-wide facts about one delivered batch.
|
||||
*
|
||||
* <p>{@code orderingUnit} is what makes a batch safe to hand to an ordered destination. A batch
|
||||
* drawn from two partitions cannot be settled or retried as a unit without reordering one of them,
|
||||
* so an ordered destination requires the batch to name exactly one ordering unit and the runtime
|
||||
* rejects a batch that spans more.
|
||||
*
|
||||
* @param destination the logical destination
|
||||
* @param size the number of deliveries in the batch
|
||||
* @param orderingUnit the single partition or queue the batch was drawn from, when it has one
|
||||
* @param settlableAsBatch whether the broker can settle the whole batch in one operation
|
||||
* @param receivedAt when the consumer assembled the batch
|
||||
*/
|
||||
public record BatchDeliveryMetadata(
|
||||
DestinationName destination,
|
||||
int size,
|
||||
Optional<String> orderingUnit,
|
||||
boolean settlableAsBatch,
|
||||
Instant receivedAt) {
|
||||
|
||||
public BatchDeliveryMetadata {
|
||||
Objects.requireNonNull(destination, "destination must not be null");
|
||||
Objects.requireNonNull(orderingUnit, "orderingUnit must not be null");
|
||||
Objects.requireNonNull(receivedAt, "receivedAt must not be null");
|
||||
if (size < 1) {
|
||||
throw new IllegalArgumentException("a delivered batch has at least one delivery");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reports whether this batch may be handed to a destination with strict ordering.
|
||||
*
|
||||
* @return true when the batch came from exactly one ordering unit
|
||||
*/
|
||||
public boolean isSafeForOrderedDestination() {
|
||||
return orderingUnit.isPresent();
|
||||
}
|
||||
}
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* A batch of decoded messages handed to a batch handler.
|
||||
*
|
||||
* @param <T> the payload type
|
||||
* @param deliveries the individual deliveries, in broker order
|
||||
* @param metadata batch-wide facts
|
||||
*/
|
||||
public record BatchMessageDelivery<T>(
|
||||
List<MessageDelivery<T>> deliveries, BatchDeliveryMetadata metadata) {
|
||||
|
||||
public BatchMessageDelivery {
|
||||
Objects.requireNonNull(deliveries, "deliveries must not be null");
|
||||
Objects.requireNonNull(metadata, "metadata must not be null");
|
||||
deliveries = List.copyOf(deliveries);
|
||||
if (deliveries.isEmpty()) {
|
||||
throw new IllegalArgumentException("a delivered batch is never empty");
|
||||
}
|
||||
if (deliveries.size() != metadata.size()) {
|
||||
throw new IllegalArgumentException(
|
||||
"metadata size %d does not match %d deliveries"
|
||||
.formatted(metadata.size(), deliveries.size()));
|
||||
}
|
||||
}
|
||||
}
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
import java.util.concurrent.CompletionStage;
|
||||
|
||||
/**
|
||||
* The M2 batch consume entry point.
|
||||
*
|
||||
* <p>The handler returns one {@link HandleResult} for the whole batch. On a broker that settles
|
||||
* batches atomically the runtime applies that result once; on a broker that settles per message it
|
||||
* applies the same result to each delivery. Either way the handler is not asked to reason about
|
||||
* which settlement mode it is running under.
|
||||
*
|
||||
* @param <T> the payload type
|
||||
*/
|
||||
public interface BatchMessageHandler<T> {
|
||||
|
||||
/**
|
||||
* Handles one delivered batch.
|
||||
*
|
||||
* @param batch the batch to handle
|
||||
* @return a stage completing with the outcome for the whole batch
|
||||
*/
|
||||
CompletionStage<HandleResult> handle(BatchMessageDelivery<T> batch);
|
||||
}
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* Runtime context handed to a handler alongside a delivery.
|
||||
*
|
||||
* <p>{@code shutdownRequested} is visible to handlers on purpose: during a graceful drain the
|
||||
* platform stops creating new retry attempts, and a long-running handler that can wind down early
|
||||
* shortens the drain instead of being cancelled at the deadline.
|
||||
*
|
||||
* @param handlerDeadline the instant after which the handler is considered timed out
|
||||
* @param shutdownRequested whether the consumer has begun draining
|
||||
* @param consumerId the stable, low-cardinality consumer identity
|
||||
*/
|
||||
public record DeliveryContext(
|
||||
Instant handlerDeadline, boolean shutdownRequested, String consumerId) {
|
||||
|
||||
public DeliveryContext {
|
||||
Objects.requireNonNull(handlerDeadline, "handlerDeadline must not be null");
|
||||
if (consumerId == null || consumerId.isBlank()) {
|
||||
throw new IllegalArgumentException("consumerId must not be blank");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reports whether the handler deadline has passed.
|
||||
*
|
||||
* @param now the current instant
|
||||
* @return true when the deadline has elapsed
|
||||
*/
|
||||
public boolean isExpired(Instant now) {
|
||||
return !now.isBefore(handlerDeadline);
|
||||
}
|
||||
}
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
/**
|
||||
* Broker-level delivery guarantee offered by the common contract.
|
||||
*
|
||||
* <p>There is deliberately no {@code EXACTLY_ONCE} constant. No broker delivers exactly-once across
|
||||
* an external side effect; what real systems provide is at-least-once delivery combined with an
|
||||
* idempotent or transactional consumer. Naming a guarantee the platform cannot honour would push
|
||||
* that responsibility out of sight, so the enum stops where the evidence stops.
|
||||
*/
|
||||
public enum DeliveryGuarantee {
|
||||
|
||||
/** Delivery may be lost; duplicate suppression is preferred over durability. */
|
||||
AT_MOST_ONCE,
|
||||
|
||||
/** Redelivery is possible; durability is preferred over duplicate suppression. */
|
||||
AT_LEAST_ONCE
|
||||
}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
import dev.caskeleton.messaging.api.destination.DestinationName;
|
||||
import dev.caskeleton.messaging.api.publish.BrokerPosition;
|
||||
import java.time.Instant;
|
||||
import java.util.Objects;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Transport-side facts about one delivery attempt.
|
||||
*
|
||||
* <p>The first delivery is attempt one, not zero. Off-by-one confusion here directly changes how
|
||||
* many times a poison message is replayed before it is parked, so the counting rule is fixed at the
|
||||
* contract rather than left to each adapter.
|
||||
*
|
||||
* @param destination the logical destination
|
||||
* @param deliveryAttempt the attempt number, starting at one
|
||||
* @param redelivered whether the broker flagged this as a redelivery
|
||||
* @param brokerPosition the broker coordinate when available
|
||||
* @param partitionOrQueue the ordering unit when available
|
||||
* @param consumerGroup the consumer group when applicable
|
||||
* @param receivedAt when the consumer received the delivery
|
||||
*/
|
||||
public record DeliveryMetadata(
|
||||
DestinationName destination,
|
||||
int deliveryAttempt,
|
||||
boolean redelivered,
|
||||
Optional<BrokerPosition> brokerPosition,
|
||||
Optional<String> partitionOrQueue,
|
||||
Optional<String> consumerGroup,
|
||||
Instant receivedAt) {
|
||||
|
||||
public DeliveryMetadata {
|
||||
Objects.requireNonNull(destination, "destination must not be null");
|
||||
Objects.requireNonNull(brokerPosition, "brokerPosition must not be null");
|
||||
Objects.requireNonNull(partitionOrQueue, "partitionOrQueue must not be null");
|
||||
Objects.requireNonNull(consumerGroup, "consumerGroup must not be null");
|
||||
Objects.requireNonNull(receivedAt, "receivedAt must not be null");
|
||||
if (deliveryAttempt < 1) {
|
||||
throw new IllegalArgumentException("deliveryAttempt counts the first delivery as 1");
|
||||
}
|
||||
if (deliveryAttempt == 1 && redelivered) {
|
||||
throw new IllegalArgumentException("the first delivery attempt cannot be a redelivery");
|
||||
}
|
||||
}
|
||||
}
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
/** How a handler's external side effect is protected against redelivery. */
|
||||
public enum ExternalSideEffectGuarantee {
|
||||
|
||||
/** Nothing protects the side effect; only valid where replay is harmless. */
|
||||
NONE,
|
||||
|
||||
/** The handler must make the side effect idempotent itself. */
|
||||
IDEMPOTENCY_REQUIRED,
|
||||
|
||||
/** An Inbox row and the side effect commit inside the same database transaction. */
|
||||
INBOX_TRANSACTIONAL
|
||||
}
|
||||
+60
@@ -0,0 +1,60 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
import dev.caskeleton.messaging.api.error.FailureDescriptor;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* What an M1 handler decided about a delivery.
|
||||
*
|
||||
* <p>The handler states an intent; the platform performs the settlement. That split is what keeps
|
||||
* "acknowledge only after the handler succeeded" a platform invariant rather than something each
|
||||
* handler has to remember, and it is why no variant here carries a broker acknowledgement handle.
|
||||
*/
|
||||
public sealed interface HandleResult
|
||||
permits HandleResult.Success, HandleResult.Retry, HandleResult.DeadLetter, HandleResult.Reject {
|
||||
|
||||
/** Processing succeeded; the platform may settle the source. */
|
||||
record Success() implements HandleResult {}
|
||||
|
||||
/**
|
||||
* Processing failed in a way that may succeed later.
|
||||
*
|
||||
* @param failure the sanitized failure description
|
||||
*/
|
||||
record Retry(FailureDescriptor failure) implements HandleResult {
|
||||
public Retry {
|
||||
Objects.requireNonNull(failure, "failure must not be null");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Processing failed permanently; route to the dead letter destination.
|
||||
*
|
||||
* @param failure the sanitized failure description
|
||||
*/
|
||||
record DeadLetter(FailureDescriptor failure) implements HandleResult {
|
||||
public DeadLetter {
|
||||
Objects.requireNonNull(failure, "failure must not be null");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Discard the message without dead lettering.
|
||||
*
|
||||
* @param failure the sanitized failure description
|
||||
*/
|
||||
record Reject(FailureDescriptor failure) implements HandleResult {
|
||||
public Reject {
|
||||
Objects.requireNonNull(failure, "failure must not be null");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a successful result.
|
||||
*
|
||||
* @return the success variant
|
||||
*/
|
||||
static HandleResult success() {
|
||||
return new Success();
|
||||
}
|
||||
}
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
import dev.caskeleton.messaging.api.MessageEnvelope;
|
||||
import java.util.Objects;
|
||||
|
||||
/**
|
||||
* One decoded message handed to a handler.
|
||||
*
|
||||
* @param <T> the payload type
|
||||
* @param message the decoded envelope
|
||||
* @param metadata transport-side delivery facts
|
||||
* @param context runtime context for this attempt
|
||||
*/
|
||||
public record MessageDelivery<T>(
|
||||
MessageEnvelope<T> message, DeliveryMetadata metadata, DeliveryContext context) {
|
||||
|
||||
public MessageDelivery {
|
||||
Objects.requireNonNull(message, "message must not be null");
|
||||
Objects.requireNonNull(metadata, "metadata must not be null");
|
||||
Objects.requireNonNull(context, "context must not be null");
|
||||
}
|
||||
}
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
import java.util.concurrent.CompletionStage;
|
||||
|
||||
/**
|
||||
* The M1 typed handler implemented by ordinary business code.
|
||||
*
|
||||
* <p>Handlers state an outcome and never touch broker acknowledgement APIs. Duplicate delivery is a
|
||||
* normal condition, not an error: implementations are expected to be idempotent, or to sit behind
|
||||
* the Inbox.
|
||||
*
|
||||
* @param <T> the payload type
|
||||
*/
|
||||
public interface MessageHandler<T> {
|
||||
|
||||
/**
|
||||
* Handles one delivery.
|
||||
*
|
||||
* @param delivery the decoded message and its metadata
|
||||
* @return a stage completing with the handling outcome
|
||||
*/
|
||||
CompletionStage<HandleResult> handle(MessageDelivery<T> delivery);
|
||||
}
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
/**
|
||||
* The real scope inside which message order is preserved.
|
||||
*
|
||||
* <p>There is deliberately no {@code GLOBAL} constant. Ordering is a property of a partition, a key
|
||||
* mapping, or a single consumer — never of a whole destination — and advertising a global scope
|
||||
* would promise something no partitioned broker can keep.
|
||||
*/
|
||||
public enum OrderingScope {
|
||||
|
||||
/** No order is promised. */
|
||||
NONE,
|
||||
|
||||
/** Order holds across the destination, which requires a single ordering unit. */
|
||||
DESTINATION,
|
||||
|
||||
/** Order holds inside one broker partition. */
|
||||
PARTITION,
|
||||
|
||||
/** Order holds for one key while its mapping to an ordering unit is stable. */
|
||||
KEY
|
||||
}
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
import dev.caskeleton.messaging.api.destination.DestinationName;
|
||||
import java.util.concurrent.CompletionStage;
|
||||
|
||||
/**
|
||||
* The M2 consumer flow-control entry point.
|
||||
*
|
||||
* <p>Pausing stops new deliveries; it does not abandon the ones already in flight. A paused
|
||||
* consumer stays a member of its group and keeps its assignment, which is the point: leaving the
|
||||
* group to stop consuming would trigger a rebalance and hand the work to another instance that is
|
||||
* just as overloaded.
|
||||
*/
|
||||
public interface PauseResumeController {
|
||||
|
||||
/**
|
||||
* Stops new deliveries for a destination scope.
|
||||
*
|
||||
* @param destination the logical destination
|
||||
* @param scope the partition, queue, or {@code "*"} for every assigned unit
|
||||
* @return a stage completing once no further deliveries will be dispatched
|
||||
*/
|
||||
CompletionStage<Void> pause(DestinationName destination, String scope);
|
||||
|
||||
/**
|
||||
* Resumes deliveries for a destination scope.
|
||||
*
|
||||
* @param destination the logical destination
|
||||
* @param scope the partition, queue, or {@code "*"} for every assigned unit
|
||||
* @return a stage completing once deliveries may flow again
|
||||
*/
|
||||
CompletionStage<Void> resume(DestinationName destination, String scope);
|
||||
}
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
package dev.caskeleton.messaging.api.delivery;
|
||||
|
||||
/** How duplicate processing is neutralised once a message has been delivered. */
|
||||
public enum ProcessingGuarantee {
|
||||
|
||||
/** The handler suppresses duplicate effects using the message id or a business key. */
|
||||
APPLICATION_IDEMPOTENT,
|
||||
|
||||
/** Atomicity holds only inside the transaction scope the broker itself defines. */
|
||||
BROKER_TRANSACTIONAL
|
||||
}
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
package dev.caskeleton.messaging.api.destination;
|
||||
|
||||
/** Resolves the capability snapshot for a logical destination. */
|
||||
public interface CapabilityRegistry {
|
||||
|
||||
/**
|
||||
* Returns the capabilities of a destination.
|
||||
*
|
||||
* @param destination the logical destination
|
||||
* @return the capability snapshot
|
||||
* @throws dev.caskeleton.messaging.api.error.MessagingConfigurationException when the destination
|
||||
* is not registered
|
||||
*/
|
||||
DestinationCapabilities capabilities(DestinationName destination);
|
||||
}
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
package dev.caskeleton.messaging.api.destination;
|
||||
|
||||
/**
|
||||
* The publish confirmation a destination profile demands.
|
||||
*
|
||||
* <p>This is the requested level. What the broker actually supplied is reported separately as a
|
||||
* confirmation level on the publish evidence, so a profile asking for replication evidence against
|
||||
* an adapter that can only prove a broker ack fails at startup instead of silently downgrading.
|
||||
*/
|
||||
public enum ConfirmationRequirement {
|
||||
|
||||
/** No confirmation is required; only valid for at-most-once profiles. */
|
||||
NONE,
|
||||
|
||||
/** The broker must acknowledge receipt. */
|
||||
BROKER_ACK,
|
||||
|
||||
/** The broker must acknowledge replication or persistence. */
|
||||
REPLICATION_OR_PERSISTENCE_ACK
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user