fix!: 재리뷰가 찾은 결함과 색인에서 빠졌던 발견 5건을 고친다
문제: 성숙도 리뷰 수정을 다시 읽은 재리뷰(Critical 0, 회귀 0)가 Important 27건·Minor 41건을 찾았다. 수정이 새로 만든 결함이 많았다. @SystemInvoked가 권한 규칙의 탈출구가 됐고, 데모 계정 거부가 정확히 prod일 때만 걸렸고, outbox 순서 번호가 업그레이드 때 뒤로 갔고, GraphQL union 아래 복잡도가 0이었고, 서블릿 오류 응답이 success:true로 나갔고, 호출자 deadline이 breaker를 열었다. 원래 리뷰 색인이 06 영역 발견 5건(06-10~14)을 빠뜨려 그동안 처리되지 않았다. 방법: 영역별 7묶음이 test-driven-development 절차로 실패 테스트를 먼저 쓰고 고쳤다. 결정은 재리뷰 README 4절에 적었다: 순서 번호는 V21이 기록한 전역 floor부터, 데모 계정은 local·test·demo 허용 목록, @SystemInvoked bean은 기동 실패, complete 단계 실패는 IDEMPOTENCY_COMPLETION_UNKNOWN(결과 불명), 캐시 조건부 쓰기는 별도 SCRIPT lane, strict lane 정책은 build cache 입력, 끝나지 않는 재시도 행은 원인 코드·지표·PARKED 절차. 레지스트리가 선언만 하고 아무 코드도 내보내지 않던 redis.capability.* 지표 6개는 빼고 runbook을 실제 지표(redis.lane.*) 기준으로 고쳤다. 위험·한계: V21을 V22 없이 직접 고쳤다(어느 배포에도 아직 없음). 이 브랜치의 V21을 적용한 개발 DB는 다시 만들어야 한다. 서블릿 오류 디스패치를 보안 체인이 허용한다. 발행·저장소 포트 시그니처가 바뀌어 파생 서비스 코드가 깨진다. prod에서 fixture-로 시작하는 설정값은 기동을 막는다. httpclient failure-injection lane의 main 단계 중복 실행은 이전 결정대로 남겼다. 실제 CI runner 실행은 확인하지 못했다. 검증: ci, mainCheck, :integrationCheck, -p optional-platforms ci integrationCheck performanceCheck, :performanceCheck, :releaseCheck (-PgitRevision), verify-compose-profile-contracts(18 lane) 통과. 첫 실행의 실패 2건(FindSecBugs 1, sample 동시 생성 테스트의 v4 event id)은 고친 뒤 ci·integrationCheck·mainCheck를 다시 돌려 통과했다. Refs: docs/reviews/2026-11-03-rereview/README.md 3·4·6·7절 BREAKING CHANGE: outbox 발행 포트와 저장소 포트 시그니처, 순서 번호 첫 값(V21 floor 다음), 샘플 데모 계정 프로필 허용 목록, 새 오류 코드 IDEMPOTENCY_COMPLETION_UNKNOWN, prod의 fixture- 값 거부, 기동 rule은 static·의존 없음이어야 한다.
This commit is contained in:
@@ -26,7 +26,9 @@ producing-service identity다.
|
|||||||
|
|
||||||
## Outbox
|
## Outbox
|
||||||
|
|
||||||
outbox는 canonical 하나다(2026-09-28). use case가 `IntegrationEventEncoderPort`로 만든
|
application의 발행 경로는 하나다(2026-09-28): `outbox_event` 표와 `composition:outbox-relay`. vendored
|
||||||
|
platform의 `messaging_outbox`(`outbox-inbox.md`)는 이 경로에 쓰이지 않는다 — 두 모델의 상태 대응은
|
||||||
|
`outbox-inbox.md` 머리의 표에 있다. use case가 `IntegrationEventEncoderPort`로 만든
|
||||||
`ValidatedIntegrationEvent`를 `OutboxAppendPort`로 append하고, relay는 claim한 row의 exact envelope
|
`ValidatedIntegrationEvent`를 `OutboxAppendPort`로 append하고, relay는 claim한 row의 exact envelope
|
||||||
bytes를 `IntegrationEventPublishPort`로 보낸다. 예전의 `app.outbox.canonical-transport-enabled`
|
bytes를 `IntegrationEventPublishPort`로 보낸다. 예전의 `app.outbox.canonical-transport-enabled`
|
||||||
gate와 legacy `MessageBroker → KafkaSender` 경로는 삭제됐다. 발행할 contract의 logical destination마다
|
gate와 legacy `MessageBroker → KafkaSender` 경로는 삭제됐다. 발행할 contract의 logical destination마다
|
||||||
|
|||||||
@@ -1,5 +1,25 @@
|
|||||||
# Outbox · Inbox
|
# Outbox · Inbox
|
||||||
|
|
||||||
|
> **이 문서의 Outbox는 vendored messaging platform의 outbox다**(`messaging_outbox` 표,
|
||||||
|
> `messaging-reliability`의 `OutboxRelay`). 이 템플릿 application이 쓰는 outbox는 그것이 아니다.
|
||||||
|
> application은 `outbox_event` 표(`adapter:outbound:persistence-jpa`)에 append하고, relay는
|
||||||
|
> `composition:outbox-relay`가 돌린다. 그 동작과 운영은 `docs/manual/09-outbox-events.md`,
|
||||||
|
> `src/composition/outbox-relay/README.md`, `docs/runbooks/outbox-dead-letter.md`,
|
||||||
|
> `docs/runbooks/outbox-publish-failed.md`에 있다. platform outbox는 소유자 결정으로 제품 안에 남아
|
||||||
|
> 있지만, platform outbox repository bean이 없으면 그 relay는 만들어지지 않는다(리뷰 2026-11-01 06-11).
|
||||||
|
>
|
||||||
|
> 두 모델은 같은 이름의 상태가 반대 뜻이다. 이름으로 옮기면 확정 거부는 영원히 재시도되고 응답 없는
|
||||||
|
> 메시지는 버려진다. 대응은 아래 표 하나로만 읽는다(`OutboxPublishOutcome` Javadoc과 같은 내용).
|
||||||
|
>
|
||||||
|
> | 일어난 일 | application `outbox_event` | platform `messaging_outbox` |
|
||||||
|
> |---|---|---|
|
||||||
|
> | broker가 확인했다 | `PUBLISHED` | `PUBLISHED` |
|
||||||
|
> | 응답이 없었다(저장됐을 수 있음) | `FAILED`(재시도, 예산 안 씀, 끝 없음) | `AMBIGUOUS`(재시도, 예산을 다 쓰면 `EXHAUSTED`) |
|
||||||
|
> | 지금만 거부됐다(종료·포화·credential) | `FAILED`(재시도, 예산 안 씀, 끝 없음) | `FAILED`(종단) |
|
||||||
|
> | 확정 거부됐다(같은 payload는 또 거부) | `DEAD`(종단, 수동 처분) | `FAILED`(종단) |
|
||||||
|
> | publisher가 예외를 던졌고 시도 예산을 다 썼다 | `DEAD`(종단, 수동 처분) | `EXHAUSTED`(종단) |
|
||||||
|
> | 운영자가 재시도를 멈췄다 | `PARKED`(수동 재개·skip) | 없음 |
|
||||||
|
>
|
||||||
> **이 템플릿 application의 소비 경로**: application의 `MessageHandler`와 inbox 부품(애플리케이션
|
> **이 템플릿 application의 소비 경로**: application의 `MessageHandler`와 inbox 부품(애플리케이션
|
||||||
> DataSource 위 `JdbcInboxRepository`, `PlatformTransactionManager` 위 `TransactionRunner`)은
|
> DataSource 위 `JdbcInboxRepository`, `PlatformTransactionManager` 위 `TransactionRunner`)은
|
||||||
> `adapter:inbound:messaging`(`src/adapter/inbound/messaging/README.md`)이 둔다. platform starter는 두
|
> `adapter:inbound:messaging`(`src/adapter/inbound/messaging/README.md`)이 둔다. platform starter는 두
|
||||||
|
|||||||
@@ -40,6 +40,31 @@ JDBC이고 SQL은 `jsonb` · `FOR UPDATE SKIP LOCKED` · `ON CONFLICT` · `TIMES
|
|||||||
registry edge를 주면 그 경계가 깨진다. 복사본을 고칠 때(키 길이 검사, constant-time 비교 등)는
|
registry edge를 주면 그 경계가 깨진다. 복사본을 고칠 때(키 길이 검사, constant-time 비교 등)는
|
||||||
`shared-contract`의 정본과 같은 규칙인지 함께 본다.
|
`shared-contract`의 정본과 같은 규칙인지 함께 본다.
|
||||||
|
|
||||||
|
### Publish retryability of authentication and authorization (review 2026-11-01 06-01)
|
||||||
|
|
||||||
|
이 제품 안에는 같은 category에 대한 재시도 규칙이 둘 있다. 의도다.
|
||||||
|
|
||||||
|
- category 기본값: `FailureDescriptor.defaultRetryable(AUTHENTICATION | AUTHORIZATION) = false`. 소비 쪽
|
||||||
|
재시도 엔진(`DefaultRetryDecisionEngine`)은 policy가 category를 따로 정하지 않으면 이 기본값과
|
||||||
|
descriptor의 `retryable`을 둘 다 요구한다. 그래서 인가받지 못한 메시지의 재전달은 지금도 재시도하지 않는다.
|
||||||
|
- publish 거부의 `retryable`: `KafkaPublishFailureClassifier`는 인증·인가 거부에 `retryable = true`를 붙인다.
|
||||||
|
broker가 레코드를 저장하지 않았고, credential 교체나 application보다 늦게 배포된 ACL은 몇 분 동안 모든
|
||||||
|
publish를 거부한 뒤 멈추기 때문이다. `retryable`을 믿는 publisher가 그 몇 분 동안의 레코드를 영구 보류하지
|
||||||
|
않게 하려는 것이다.
|
||||||
|
|
||||||
|
06-01 수정이 이 제품에서 바꾼 것(모두 publish 경로, 소비 경로 동작은 그대로):
|
||||||
|
|
||||||
|
- `KafkaPublishFailureClassifier`: 인증·인가 거부를 `retryable = true`로.
|
||||||
|
- `DefaultMessagePublisher`: runtime이 없을 때의 거부(`PUBLISH_RUNTIME_UNAVAILABLE`)를 `retryable = true`로.
|
||||||
|
- `KafkaMessagingTransport`: 닫힌 transport의 거부(`KAFKA_TRANSPORT_CLOSED`)를 `retryable = true`로.
|
||||||
|
- `MessagingShutdownLifecycle.PHASE`: platform 밖 relay의 순서 테스트가 참조하도록 public.
|
||||||
|
|
||||||
|
버린 대안: vendored 제품은 그대로 두고 application bridge(ACL)가 category를 보고 "인증·인가면 재시도"로
|
||||||
|
번역하는 것. 버린 이유: `retryable`은 이 제품이 "같은 레코드를 나중에 다시 보내면 받아질 수 있나"를 말하는
|
||||||
|
자리다. bridge가 category로 그 판단을 다시 하면 같은 사실을 제품과 bridge가 따로 정하고, 이 제품의
|
||||||
|
`PublishResult`를 읽는 다른 호출자는 bridge와 다른 답을 본다. 대신 이 제품 주석은 소비자 이름(application
|
||||||
|
relay, `composition:outbox-relay`)을 부르지 않는다 — "flag를 믿는 publisher"로 쓴다(재리뷰 2026-11-03 RR-05-07).
|
||||||
|
|
||||||
## Public surface 정책
|
## Public surface 정책
|
||||||
|
|
||||||
- **vendor 타입을 public signature에 노출하면 그 의존성은 `api`로 선언한다.** 숨긴다고 타입이
|
- **vendor 타입을 public signature에 노출하면 그 의존성은 `api`로 선언한다.** 숨긴다고 타입이
|
||||||
|
|||||||
+8
@@ -61,6 +61,14 @@ public record FailureDescriptor(
|
|||||||
* automatically: each of them fails identically on every redelivery, so retrying only multiplies
|
* automatically: each of them fails identically on every redelivery, so retrying only multiplies
|
||||||
* the load while the message is still poison.
|
* the load while the message is still poison.
|
||||||
*
|
*
|
||||||
|
* <p>A default, not a rule every descriptor follows. A publish rejection may carry {@code
|
||||||
|
* retryable = true} with an authentication or authorization category: the broker did not store
|
||||||
|
* the record, and a credential being rotated or an ACL deployed after the application refuses
|
||||||
|
* every publish for a few minutes and then stops (the Kafka publish classifier). The
|
||||||
|
* consumer-side retry engine requires this default as well as the flag unless its policy names
|
||||||
|
* the category, so redelivery of an unauthorised message is still not retried; only a publisher
|
||||||
|
* that reads {@link #retryable()} sees the difference.
|
||||||
|
*
|
||||||
* @param category the classification
|
* @param category the classification
|
||||||
* @return true when automatic retry is allowed by default
|
* @return true when automatic retry is allowed by default
|
||||||
*/
|
*/
|
||||||
|
|||||||
+38
-5
@@ -344,10 +344,17 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati
|
|||||||
// A decode failure is not a handler failure and must not be retried forever. The
|
// A decode failure is not a handler failure and must not be retried forever. The
|
||||||
// offset may only advance once the record exists somewhere else, so an unavailable
|
// offset may only advance once the record exists somewhere else, so an unavailable
|
||||||
// quarantine leaves it uncommitted rather than dropping the message.
|
// quarantine leaves it uncommitted rather than dropping the message.
|
||||||
if (quarantine.quarantine(record, undecodable)) {
|
// It will never decode, however often it is redriven: DESERIALIZATION, not the
|
||||||
|
// POISON_MESSAGE of a handler that ran out of attempts (review 2026-11-01 06-14).
|
||||||
|
if (quarantine.quarantine(
|
||||||
|
record,
|
||||||
|
dev.caskeleton.messaging.api.error.FailureCategory.DESERIALIZATION,
|
||||||
|
undecodable)) {
|
||||||
settlement.acknowledgeAfterQuarantine();
|
settlement.acknowledgeAfterQuarantine();
|
||||||
} else {
|
} else {
|
||||||
settlement.requeueAfterFailure(undecodable);
|
settlement.requeueAfterFailure(
|
||||||
|
undecodable,
|
||||||
|
dev.caskeleton.messaging.api.error.FailureCategory.DESERIALIZATION);
|
||||||
}
|
}
|
||||||
coordinator.release(partition);
|
coordinator.release(partition);
|
||||||
shutdown.endWork();
|
shutdown.endWork();
|
||||||
@@ -682,6 +689,7 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati
|
|||||||
retryOrQuarantine(
|
retryOrQuarantine(
|
||||||
new IllegalStateException(
|
new IllegalStateException(
|
||||||
"the handler asked for another attempt and the retry budget is spent"),
|
"the handler asked for another attempt and the retry budget is spent"),
|
||||||
|
dev.caskeleton.messaging.api.error.FailureCategory.POISON_MESSAGE,
|
||||||
delay));
|
delay));
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -715,11 +723,21 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati
|
|||||||
* that exists nowhere else — which is why a production destination must have one.
|
* that exists nowhere else — which is why a production destination must have one.
|
||||||
*/
|
*/
|
||||||
private void requeueAfterFailure(Throwable reason) {
|
private void requeueAfterFailure(Throwable reason) {
|
||||||
|
requeueAfterFailure(reason, exhaustedCategory(reason));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* {@link #requeueAfterFailure(Throwable)} under a category the caller already knows: a record
|
||||||
|
* that could not be decoded stays {@code DESERIALIZATION} when its first quarantine write was
|
||||||
|
* not confirmed and it reaches quarantine again through the retry path.
|
||||||
|
*/
|
||||||
|
private void requeueAfterFailure(
|
||||||
|
Throwable reason, dev.caskeleton.messaging.api.error.FailureCategory category) {
|
||||||
if (!terminal.compareAndSet(null, "requeue")) {
|
if (!terminal.compareAndSet(null, "requeue")) {
|
||||||
// The handler settled before it failed or before its late completion; that answer stands.
|
// The handler settled before it failed or before its late completion; that answer stands.
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
retryOrQuarantine(reason, Duration.ZERO);
|
retryOrQuarantine(reason, category, Duration.ZERO);
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -730,11 +748,13 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati
|
|||||||
* being retried — the offset may not advance past a message that exists nowhere else.
|
* being retried — the offset may not advance past a message that exists nowhere else.
|
||||||
*/
|
*/
|
||||||
private dev.caskeleton.messaging.api.settlement.SettlementResult retryOrQuarantine(
|
private dev.caskeleton.messaging.api.settlement.SettlementResult retryOrQuarantine(
|
||||||
Throwable reason, Duration requested) {
|
Throwable reason,
|
||||||
|
dev.caskeleton.messaging.api.error.FailureCategory category,
|
||||||
|
Duration requested) {
|
||||||
FailedOffset key = new FailedOffset(partition, offset);
|
FailedOffset key = new FailedOffset(partition, offset);
|
||||||
int attempt = failures.merge(key, 1, Integer::sum);
|
int attempt = failures.merge(key, 1, Integer::sum);
|
||||||
dev.caskeleton.messaging.policy.RetryPolicy retry = spec.profile().retry();
|
dev.caskeleton.messaging.policy.RetryPolicy retry = spec.profile().retry();
|
||||||
if (attempt >= retry.maxAttempts() && quarantine.quarantine(record, reason)) {
|
if (attempt >= retry.maxAttempts() && quarantine.quarantine(record, category, reason)) {
|
||||||
failures.remove(key);
|
failures.remove(key);
|
||||||
settlements.enqueue(
|
settlements.enqueue(
|
||||||
new KafkaSettlementCommand(
|
new KafkaSettlementCommand(
|
||||||
@@ -751,6 +771,19 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati
|
|||||||
return enqueueRequeue(backoff);
|
return enqueueRequeue(backoff);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The category a record that ran out of attempts is quarantined under: its last failure's own,
|
||||||
|
* when that failure is a platform one and so carries a classification — a transient one tells a
|
||||||
|
* redrive the record may pass now — and {@code POISON_MESSAGE}, the platform's name for "keeps
|
||||||
|
* failing its consumer", when it is not.
|
||||||
|
*/
|
||||||
|
private static dev.caskeleton.messaging.api.error.FailureCategory exhaustedCategory(
|
||||||
|
Throwable reason) {
|
||||||
|
return reason instanceof dev.caskeleton.messaging.api.error.MessagingException classified
|
||||||
|
? classified.category()
|
||||||
|
: dev.caskeleton.messaging.api.error.FailureCategory.POISON_MESSAGE;
|
||||||
|
}
|
||||||
|
|
||||||
private dev.caskeleton.messaging.api.settlement.SettlementResult enqueueRequeue(
|
private dev.caskeleton.messaging.api.settlement.SettlementResult enqueueRequeue(
|
||||||
Duration delay) {
|
Duration delay) {
|
||||||
retries.pauseUntil(partition, offset, delay, clock.instant());
|
retries.pauseUntil(partition, offset, delay, clock.instant());
|
||||||
|
|||||||
+8
-4
@@ -67,10 +67,14 @@ public final class KafkaPublishFailureClassifier {
|
|||||||
* <p>Authentication and authorization prove the broker did not store the record, but not that it
|
* <p>Authentication and authorization prove the broker did not store the record, but not that it
|
||||||
* will refuse it again: a credential being rotated, or an ACL deployed after the application,
|
* will refuse it again: a credential being rotated, or an ACL deployed after the application,
|
||||||
* fails every publish this way for a few minutes and then stops. Marked not retryable, a
|
* fails every publish this way for a few minutes and then stops. Marked not retryable, a
|
||||||
* publisher that trusts the flag — the application's outbox relay does — parks every record sent
|
* publisher that trusts the flag parks every record sent in those minutes. A credential that is
|
||||||
* in those minutes. A credential that is simply wrong is retried too, and the outbox then holds
|
* simply wrong is retried too, for as long as it stays wrong; the descriptor's code names it, so
|
||||||
* its rows FAILED with an alert every retry — recoverable once the credential is fixed, where
|
* a caller can tell that retry from a rotation and stop it.
|
||||||
* DEAD rows needed a manual reset each.
|
*
|
||||||
|
* <p>This flag departs from {@link FailureDescriptor#defaultRetryable} on purpose, for publish
|
||||||
|
* only: that default says an authentication or authorization failure is not retried, and it still
|
||||||
|
* governs consumption, where the retry engine requires both. See {@code src/messaging/CLAUDE.md}
|
||||||
|
* "Publish retryability of authentication and authorization".
|
||||||
*/
|
*/
|
||||||
private static boolean mayPassOnRetry(Throwable cause) {
|
private static boolean mayPassOnRetry(Throwable cause) {
|
||||||
return cause instanceof AuthenticationException || cause instanceof AuthorizationException;
|
return cause instanceof AuthenticationException || cause instanceof AuthorizationException;
|
||||||
|
|||||||
+15
-3
@@ -1,5 +1,6 @@
|
|||||||
package dev.caskeleton.messaging.kafka;
|
package dev.caskeleton.messaging.kafka;
|
||||||
|
|
||||||
|
import dev.caskeleton.messaging.api.error.FailureCategory;
|
||||||
import org.apache.kafka.clients.consumer.ConsumerRecord;
|
import org.apache.kafka.clients.consumer.ConsumerRecord;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -18,11 +19,22 @@ import org.apache.kafka.clients.consumer.ConsumerRecord;
|
|||||||
public interface KafkaQuarantinePublisher {
|
public interface KafkaQuarantinePublisher {
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Writes an undecodable record somewhere durable.
|
* Writes a record the consumer will not deliver somewhere durable, under the reason it gives.
|
||||||
*
|
*
|
||||||
|
* <p>The category is the caller's, because only the caller knows which of two very different
|
||||||
|
* things happened: the record could not be rebuilt at all ({@link
|
||||||
|
* FailureCategory#DESERIALIZATION}), or its handler failed until the attempt budget ran out —
|
||||||
|
* under the failure's own category when it carries one, {@link FailureCategory#POISON_MESSAGE}
|
||||||
|
* when it does not. Both were written as {@code POISON_MESSAGE}, so a redrive could not tell a
|
||||||
|
* message that will never decode from one an outage kept failing (review 2026-11-01 06-14).
|
||||||
|
*
|
||||||
|
* @param record the record as it arrived
|
||||||
|
* @param category why it is quarantined, written as the failure-category header
|
||||||
|
* @param reason the failure; its type is written as the failure-code header
|
||||||
* @return true when the write is confirmed, so the source offset may be committed
|
* @return true when the write is confirmed, so the source offset may be committed
|
||||||
*/
|
*/
|
||||||
boolean quarantine(ConsumerRecord<byte[], byte[]> record, Throwable reason);
|
boolean quarantine(
|
||||||
|
ConsumerRecord<byte[], byte[]> record, FailureCategory category, Throwable reason);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The default: nothing is configured, so nothing may be committed past.
|
* The default: nothing is configured, so nothing may be committed past.
|
||||||
@@ -37,7 +49,7 @@ public interface KafkaQuarantinePublisher {
|
|||||||
/** Holds the single "not configured" quarantine. */
|
/** Holds the single "not configured" quarantine. */
|
||||||
final class Unavailable {
|
final class Unavailable {
|
||||||
|
|
||||||
private static final KafkaQuarantinePublisher INSTANCE = (record, reason) -> false;
|
private static final KafkaQuarantinePublisher INSTANCE = (record, category, reason) -> false;
|
||||||
|
|
||||||
private Unavailable() {}
|
private Unavailable() {}
|
||||||
}
|
}
|
||||||
|
|||||||
+86
-3
@@ -3,6 +3,7 @@ package dev.caskeleton.messaging.kafka;
|
|||||||
import static org.assertj.core.api.Assertions.assertThat;
|
import static org.assertj.core.api.Assertions.assertThat;
|
||||||
|
|
||||||
import dev.caskeleton.messaging.api.MessageId;
|
import dev.caskeleton.messaging.api.MessageId;
|
||||||
|
import dev.caskeleton.messaging.api.error.FailureCategory;
|
||||||
import dev.caskeleton.messaging.api.header.ReservedHeaders;
|
import dev.caskeleton.messaging.api.header.ReservedHeaders;
|
||||||
import dev.caskeleton.messaging.policy.OrderingImpact;
|
import dev.caskeleton.messaging.policy.OrderingImpact;
|
||||||
import dev.caskeleton.messaging.policy.RetryMode;
|
import dev.caskeleton.messaging.policy.RetryMode;
|
||||||
@@ -258,7 +259,7 @@ class KafkaConsumerRegistrarTest {
|
|||||||
registrar(
|
registrar(
|
||||||
KafkaContractHarness.profile(),
|
KafkaContractHarness.profile(),
|
||||||
this::fail,
|
this::fail,
|
||||||
(record, reason) -> quarantined.add(record.offset()),
|
(record, category, reason) -> quarantined.add(record.offset()),
|
||||||
() -> NOW);
|
() -> NOW);
|
||||||
assign();
|
assign();
|
||||||
addRecord(10);
|
addRecord(10);
|
||||||
@@ -274,6 +275,29 @@ class KafkaConsumerRegistrarTest {
|
|||||||
.hasValue(11L);
|
.hasValue(11L);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
@DisplayName("handler가 시도 예산을 다 쓴 레코드는 그 실패의 분류로 격리하고 분류가 없으면 POISON_MESSAGE로 적는다")
|
||||||
|
void aDeliveryThatSpentItsBudgetIsQuarantinedUnderItsFailuresCategory() {
|
||||||
|
// given: a handler that fails with a plain exception every time
|
||||||
|
List<FailureCategory> categories = new ArrayList<>();
|
||||||
|
KafkaConsumerRegistrar registrar =
|
||||||
|
registrar(
|
||||||
|
KafkaContractHarness.profile(),
|
||||||
|
this::fail,
|
||||||
|
(record, category, reason) -> categories.add(category),
|
||||||
|
() -> NOW);
|
||||||
|
assign();
|
||||||
|
addRecord(10);
|
||||||
|
registrar.pollOnce(NOW);
|
||||||
|
handlers.runAll();
|
||||||
|
|
||||||
|
// when
|
||||||
|
registrar.pollOnce(NOW);
|
||||||
|
|
||||||
|
// then
|
||||||
|
assertThat(categories).containsExactly(FailureCategory.POISON_MESSAGE);
|
||||||
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
@DisplayName("handler가 요청한 requeue도 시도 횟수를 세어 예산을 쓰면 quarantine 후 commit된다")
|
@DisplayName("handler가 요청한 requeue도 시도 횟수를 세어 예산을 쓰면 quarantine 후 commit된다")
|
||||||
void aDeliveryItsHandlerKeepsRequeueingSpendsTheBudgetAndIsQuarantined() {
|
void aDeliveryItsHandlerKeepsRequeueingSpendsTheBudgetAndIsQuarantined() {
|
||||||
@@ -283,7 +307,7 @@ class KafkaConsumerRegistrarTest {
|
|||||||
registrar(
|
registrar(
|
||||||
KafkaContractHarness.profile(),
|
KafkaContractHarness.profile(),
|
||||||
this::requeue,
|
this::requeue,
|
||||||
(record, reason) -> quarantined.add(record.offset()),
|
(record, category, reason) -> quarantined.add(record.offset()),
|
||||||
() -> NOW);
|
() -> NOW);
|
||||||
assign();
|
assign();
|
||||||
addRecord(10);
|
addRecord(10);
|
||||||
@@ -470,7 +494,7 @@ class KafkaConsumerRegistrarTest {
|
|||||||
handlers,
|
handlers,
|
||||||
new GracefulShutdownCoordinator(Duration.ofSeconds(30)),
|
new GracefulShutdownCoordinator(Duration.ofSeconds(30)),
|
||||||
Duration.ofMillis(1),
|
Duration.ofMillis(1),
|
||||||
(record, reason) -> {
|
(record, category, reason) -> {
|
||||||
quarantined.add(record.offset());
|
quarantined.add(record.offset());
|
||||||
return true;
|
return true;
|
||||||
})) {
|
})) {
|
||||||
@@ -486,6 +510,65 @@ class KafkaConsumerRegistrarTest {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A record that cannot be rebuilt into an envelope will never decode, however often it is
|
||||||
|
* redriven; the one whose handler ran out of attempts may pass once the outage is over. They were
|
||||||
|
* both written as {@code POISON_MESSAGE} (review 2026-11-01 06-14).
|
||||||
|
*/
|
||||||
|
@Test
|
||||||
|
@DisplayName("envelope로 되살릴 수 없는 레코드는 시도 예산 소진과 다른 DESERIALIZATION으로 격리한다")
|
||||||
|
void anUndecodableRecordIsQuarantinedAsDeserializationNotAsPoison() {
|
||||||
|
// given
|
||||||
|
List<FailureCategory> categories = new ArrayList<>();
|
||||||
|
try (KafkaConsumerRegistrar registrar =
|
||||||
|
new KafkaConsumerRegistrar(
|
||||||
|
new TransportConsumerSpec(KafkaContractHarness.profile(), this::settleImmediately),
|
||||||
|
consumer,
|
||||||
|
handlers,
|
||||||
|
new GracefulShutdownCoordinator(Duration.ofSeconds(30)),
|
||||||
|
Duration.ofMillis(1),
|
||||||
|
(record, category, reason) -> categories.add(category))) {
|
||||||
|
assign();
|
||||||
|
addRecordWithoutIdentityHeaders(10);
|
||||||
|
registrar.pollOnce(NOW);
|
||||||
|
|
||||||
|
// when
|
||||||
|
handlers.runAll();
|
||||||
|
|
||||||
|
// then
|
||||||
|
assertThat(categories).containsExactly(FailureCategory.DESERIALIZATION);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
@DisplayName("격리 쓰기가 한 번 실패해 재시도로 넘어간 읽을 수 없는 레코드도 DESERIALIZATION으로 격리한다")
|
||||||
|
void anUndecodableRecordStaysDeserializationWhenItsQuarantineIsRetried() {
|
||||||
|
// given: the first quarantine write is not confirmed, so the record goes round the retry path
|
||||||
|
List<FailureCategory> categories = new ArrayList<>();
|
||||||
|
try (KafkaConsumerRegistrar registrar =
|
||||||
|
new KafkaConsumerRegistrar(
|
||||||
|
new TransportConsumerSpec(KafkaContractHarness.profile(), this::settleImmediately),
|
||||||
|
consumer,
|
||||||
|
handlers,
|
||||||
|
new GracefulShutdownCoordinator(Duration.ofSeconds(30)),
|
||||||
|
Duration.ofMillis(1),
|
||||||
|
(record, category, reason) -> {
|
||||||
|
categories.add(category);
|
||||||
|
return categories.size() > 1;
|
||||||
|
})) {
|
||||||
|
assign();
|
||||||
|
addRecordWithoutIdentityHeaders(10);
|
||||||
|
registrar.pollOnce(NOW);
|
||||||
|
|
||||||
|
// when
|
||||||
|
handlers.runAll();
|
||||||
|
|
||||||
|
// then
|
||||||
|
assertThat(categories)
|
||||||
|
.containsExactly(FailureCategory.DESERIALIZATION, FailureCategory.DESERIALIZATION);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
@DisplayName("one partition hitting its ceiling does not abandon the other's records")
|
@DisplayName("one partition hitting its ceiling does not abandon the other's records")
|
||||||
void oneSaturatedPartitionDoesNotAbandonAnother() {
|
void oneSaturatedPartitionDoesNotAbandonAnother() {
|
||||||
|
|||||||
+1
-1
@@ -265,7 +265,7 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM
|
|||||||
admission.complete(destination.name().value());
|
admission.complete(destination.name().value());
|
||||||
// No runtime to send through — the registry closed during shutdown, or none is installed
|
// No runtime to send through — the registry closed during shutdown, or none is installed
|
||||||
// yet. Nothing about the message was wrong, so the refusal is marked retryable: a caller that
|
// yet. Nothing about the message was wrong, so the refusal is marked retryable: a caller that
|
||||||
// trusts the flag, as the application's outbox relay does, retries it instead of parking it.
|
// trusts the flag retries it instead of parking it.
|
||||||
return refused(
|
return refused(
|
||||||
new FailureDescriptor(
|
new FailureDescriptor(
|
||||||
FailureCategory.TRANSIENT_INFRASTRUCTURE,
|
FailureCategory.TRANSIENT_INFRASTRUCTURE,
|
||||||
|
|||||||
+6
-2
@@ -1,5 +1,6 @@
|
|||||||
package dev.caskeleton.messaging.autoconfigure;
|
package dev.caskeleton.messaging.autoconfigure;
|
||||||
|
|
||||||
|
import dev.caskeleton.messaging.api.error.FailureCategory;
|
||||||
import dev.caskeleton.messaging.api.header.ReservedHeaders;
|
import dev.caskeleton.messaging.api.header.ReservedHeaders;
|
||||||
import dev.caskeleton.messaging.kafka.KafkaQuarantinePublisher;
|
import dev.caskeleton.messaging.kafka.KafkaQuarantinePublisher;
|
||||||
import java.nio.charset.StandardCharsets;
|
import java.nio.charset.StandardCharsets;
|
||||||
@@ -38,11 +39,14 @@ final class KafkaDeadLetterQuarantine implements KafkaQuarantinePublisher {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public boolean quarantine(ConsumerRecord<byte[], byte[]> record, Throwable reason) {
|
public boolean quarantine(
|
||||||
|
ConsumerRecord<byte[], byte[]> record, FailureCategory category, Throwable reason) {
|
||||||
RecordHeaders headers = new RecordHeaders(record.headers().toArray());
|
RecordHeaders headers = new RecordHeaders(record.headers().toArray());
|
||||||
headers.remove(ReservedHeaders.FAILURE_CATEGORY);
|
headers.remove(ReservedHeaders.FAILURE_CATEGORY);
|
||||||
headers.remove(ReservedHeaders.FAILURE_CODE);
|
headers.remove(ReservedHeaders.FAILURE_CODE);
|
||||||
headers.add(ReservedHeaders.FAILURE_CATEGORY, bytes("POISON_MESSAGE"));
|
headers.add(
|
||||||
|
ReservedHeaders.FAILURE_CATEGORY,
|
||||||
|
bytes(category == null ? FailureCategory.POISON_MESSAGE.name() : category.name()));
|
||||||
headers.add(
|
headers.add(
|
||||||
ReservedHeaders.FAILURE_CODE,
|
ReservedHeaders.FAILURE_CODE,
|
||||||
bytes(reason == null ? "UNKNOWN" : reason.getClass().getSimpleName()));
|
bytes(reason == null ? "UNKNOWN" : reason.getClass().getSimpleName()));
|
||||||
|
|||||||
+8
-9
@@ -29,14 +29,14 @@ import org.springframework.context.SmartLifecycle;
|
|||||||
*/
|
*/
|
||||||
public final class MessagingShutdownLifecycle implements SmartLifecycle {
|
public final class MessagingShutdownLifecycle implements SmartLifecycle {
|
||||||
|
|
||||||
/** How often the drain is re-checked. Short enough that a fast drain is not padded. */
|
|
||||||
/**
|
/**
|
||||||
* The phase this drain stops in: {@link #getPhase()}. Public so a relay outside the platform —
|
* The phase this drain stops in: {@link #getPhase()}. Public so a relay outside the platform — a
|
||||||
* the application's own outbox relay, which may not name this type in production code — can be
|
* publisher whose production code may not name this type — can be held above it by a test that
|
||||||
* held above it by a test that does name it.
|
* does name it.
|
||||||
*/
|
*/
|
||||||
public static final int PHASE = Integer.MAX_VALUE - 1024;
|
public static final int PHASE = Integer.MAX_VALUE - 1024;
|
||||||
|
|
||||||
|
/** How often the drain is re-checked. Short enough that a fast drain is not padded. */
|
||||||
private static final Duration POLL = Duration.ofMillis(50);
|
private static final Duration POLL = Duration.ofMillis(50);
|
||||||
|
|
||||||
private final MessagingAdmissionController admission;
|
private final MessagingAdmissionController admission;
|
||||||
@@ -120,11 +120,10 @@ public final class MessagingShutdownLifecycle implements SmartLifecycle {
|
|||||||
* drain ahead of the transports it is draining, because closing a connection first would strand
|
* drain ahead of the transports it is draining, because closing a connection first would strand
|
||||||
* the settlements the drain exists to flush. It sits below the outbox relays' phase for the
|
* the settlements the drain exists to flush. It sits below the outbox relays' phase for the
|
||||||
* mirror-image reason: refusing admission while a relay pass is mid-publish turns that pass's
|
* mirror-image reason: refusing admission while a relay pass is mid-publish turns that pass's
|
||||||
* rows into refusals. Two relays stand above it: the platform's own ({@code
|
* rows into refusals. The platform's own relay ({@code MessagingOutboxRelayLifecycle}, built only
|
||||||
* MessagingOutboxRelayLifecycle}, built only when a platform outbox repository exists) and the
|
* when a platform outbox repository exists) stands above it, and so must any relay outside the
|
||||||
* application's ({@code composition:outbox-relay} {@code OutboxRelayScheduler}, phase {@code
|
* platform that publishes through it: a phase above {@link #PHASE}, finishing the publish it is
|
||||||
* Integer.MAX_VALUE}), which finishes the publish it is in and releases its other claims before
|
* in before this phase runs.
|
||||||
* this phase runs.
|
|
||||||
*
|
*
|
||||||
* @return the lifecycle phase, {@link #PHASE}
|
* @return the lifecycle phase, {@link #PHASE}
|
||||||
*/
|
*/
|
||||||
|
|||||||
Reference in New Issue
Block a user