From a32b67a2ceacfe34eb8bf52bb78356183a325240 Mon Sep 17 00:00:00 2001 From: DongHyeonka Date: Tue, 3 Nov 2026 23:29:35 +0900 Subject: [PATCH] =?UTF-8?q?fix!:=20=EC=9E=AC=EB=A6=AC=EB=B7=B0=EA=B0=80=20?= =?UTF-8?q?=EC=B0=BE=EC=9D=80=20=EA=B2=B0=ED=95=A8=EA=B3=BC=20=EC=83=89?= =?UTF-8?q?=EC=9D=B8=EC=97=90=EC=84=9C=20=EB=B9=A0=EC=A1=8C=EB=8D=98=20?= =?UTF-8?q?=EB=B0=9C=EA=B2=AC=205=EA=B1=B4=EC=9D=84=20=EA=B3=A0=EC=B9=9C?= =?UTF-8?q?=EB=8B=A4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 문제: 성숙도 리뷰 수정을 다시 읽은 재리뷰(Critical 0, 회귀 0)가 Important 27건·Minor 41건을 찾았다. 수정이 새로 만든 결함이 많았다. @SystemInvoked가 권한 규칙의 탈출구가 됐고, 데모 계정 거부가 정확히 prod일 때만 걸렸고, outbox 순서 번호가 업그레이드 때 뒤로 갔고, GraphQL union 아래 복잡도가 0이었고, 서블릿 오류 응답이 success:true로 나갔고, 호출자 deadline이 breaker를 열었다. 원래 리뷰 색인이 06 영역 발견 5건(06-10~14)을 빠뜨려 그동안 처리되지 않았다. 방법: 영역별 7묶음이 test-driven-development 절차로 실패 테스트를 먼저 쓰고 고쳤다. 결정은 재리뷰 README 4절에 적었다: 순서 번호는 V21이 기록한 전역 floor부터, 데모 계정은 local·test·demo 허용 목록, @SystemInvoked bean은 기동 실패, complete 단계 실패는 IDEMPOTENCY_COMPLETION_UNKNOWN(결과 불명), 캐시 조건부 쓰기는 별도 SCRIPT lane, strict lane 정책은 build cache 입력, 끝나지 않는 재시도 행은 원인 코드·지표·PARKED 절차. 레지스트리가 선언만 하고 아무 코드도 내보내지 않던 redis.capability.* 지표 6개는 빼고 runbook을 실제 지표(redis.lane.*) 기준으로 고쳤다. 위험·한계: V21을 V22 없이 직접 고쳤다(어느 배포에도 아직 없음). 이 브랜치의 V21을 적용한 개발 DB는 다시 만들어야 한다. 서블릿 오류 디스패치를 보안 체인이 허용한다. 발행·저장소 포트 시그니처가 바뀌어 파생 서비스 코드가 깨진다. prod에서 fixture-로 시작하는 설정값은 기동을 막는다. httpclient failure-injection lane의 main 단계 중복 실행은 이전 결정대로 남겼다. 실제 CI runner 실행은 확인하지 못했다. 검증: ci, mainCheck, :integrationCheck, -p optional-platforms ci integrationCheck performanceCheck, :performanceCheck, :releaseCheck (-PgitRevision), verify-compose-profile-contracts(18 lane) 통과. 첫 실행의 실패 2건(FindSecBugs 1, sample 동시 생성 테스트의 v4 event id)은 고친 뒤 ci·integrationCheck·mainCheck를 다시 돌려 통과했다. Refs: docs/reviews/2026-11-03-rereview/README.md 3·4·6·7절 BREAKING CHANGE: outbox 발행 포트와 저장소 포트 시그니처, 순서 번호 첫 값(V21 floor 다음), 샘플 데모 계정 프로필 허용 목록, 새 오류 코드 IDEMPOTENCY_COMPLETION_UNKNOWN, prod의 fixture- 값 거부, 기동 rule은 static·의존 없음이어야 한다. --- docs/messaging/configuration-reference.md | 4 +- docs/messaging/outbox-inbox.md | 20 +++++ messaging/CLAUDE.md | 25 ++++++ .../api/error/FailureDescriptor.java | 8 ++ .../kafka/KafkaConsumerRegistrar.java | 43 +++++++-- .../kafka/KafkaPublishFailureClassifier.java | 12 ++- .../kafka/KafkaQuarantinePublisher.java | 18 +++- .../kafka/KafkaConsumerRegistrarTest.java | 89 ++++++++++++++++++- .../runtime/DefaultMessagePublisher.java | 2 +- .../KafkaDeadLetterQuarantine.java | 8 +- .../MessagingShutdownLifecycle.java | 17 ++-- 11 files changed, 218 insertions(+), 28 deletions(-) diff --git a/docs/messaging/configuration-reference.md b/docs/messaging/configuration-reference.md index 3fc19e9..07130a2 100644 --- a/docs/messaging/configuration-reference.md +++ b/docs/messaging/configuration-reference.md @@ -26,7 +26,9 @@ producing-service identity다. ## Outbox -outbox는 canonical 하나다(2026-09-28). use case가 `IntegrationEventEncoderPort`로 만든 +application의 발행 경로는 하나다(2026-09-28): `outbox_event` 표와 `composition:outbox-relay`. vendored +platform의 `messaging_outbox`(`outbox-inbox.md`)는 이 경로에 쓰이지 않는다 — 두 모델의 상태 대응은 +`outbox-inbox.md` 머리의 표에 있다. use case가 `IntegrationEventEncoderPort`로 만든 `ValidatedIntegrationEvent`를 `OutboxAppendPort`로 append하고, relay는 claim한 row의 exact envelope bytes를 `IntegrationEventPublishPort`로 보낸다. 예전의 `app.outbox.canonical-transport-enabled` gate와 legacy `MessageBroker → KafkaSender` 경로는 삭제됐다. 발행할 contract의 logical destination마다 diff --git a/docs/messaging/outbox-inbox.md b/docs/messaging/outbox-inbox.md index 6ec2d04..08f1818 100644 --- a/docs/messaging/outbox-inbox.md +++ b/docs/messaging/outbox-inbox.md @@ -1,5 +1,25 @@ # Outbox · Inbox +> **이 문서의 Outbox는 vendored messaging platform의 outbox다**(`messaging_outbox` 표, +> `messaging-reliability`의 `OutboxRelay`). 이 템플릿 application이 쓰는 outbox는 그것이 아니다. +> application은 `outbox_event` 표(`adapter:outbound:persistence-jpa`)에 append하고, relay는 +> `composition:outbox-relay`가 돌린다. 그 동작과 운영은 `docs/manual/09-outbox-events.md`, +> `src/composition/outbox-relay/README.md`, `docs/runbooks/outbox-dead-letter.md`, +> `docs/runbooks/outbox-publish-failed.md`에 있다. platform outbox는 소유자 결정으로 제품 안에 남아 +> 있지만, platform outbox repository bean이 없으면 그 relay는 만들어지지 않는다(리뷰 2026-11-01 06-11). +> +> 두 모델은 같은 이름의 상태가 반대 뜻이다. 이름으로 옮기면 확정 거부는 영원히 재시도되고 응답 없는 +> 메시지는 버려진다. 대응은 아래 표 하나로만 읽는다(`OutboxPublishOutcome` Javadoc과 같은 내용). +> +> | 일어난 일 | application `outbox_event` | platform `messaging_outbox` | +> |---|---|---| +> | broker가 확인했다 | `PUBLISHED` | `PUBLISHED` | +> | 응답이 없었다(저장됐을 수 있음) | `FAILED`(재시도, 예산 안 씀, 끝 없음) | `AMBIGUOUS`(재시도, 예산을 다 쓰면 `EXHAUSTED`) | +> | 지금만 거부됐다(종료·포화·credential) | `FAILED`(재시도, 예산 안 씀, 끝 없음) | `FAILED`(종단) | +> | 확정 거부됐다(같은 payload는 또 거부) | `DEAD`(종단, 수동 처분) | `FAILED`(종단) | +> | publisher가 예외를 던졌고 시도 예산을 다 썼다 | `DEAD`(종단, 수동 처분) | `EXHAUSTED`(종단) | +> | 운영자가 재시도를 멈췄다 | `PARKED`(수동 재개·skip) | 없음 | +> > **이 템플릿 application의 소비 경로**: application의 `MessageHandler`와 inbox 부품(애플리케이션 > DataSource 위 `JdbcInboxRepository`, `PlatformTransactionManager` 위 `TransactionRunner`)은 > `adapter:inbound:messaging`(`src/adapter/inbound/messaging/README.md`)이 둔다. platform starter는 두 diff --git a/messaging/CLAUDE.md b/messaging/CLAUDE.md index 8c8d2ac..931c282 100644 --- a/messaging/CLAUDE.md +++ b/messaging/CLAUDE.md @@ -40,6 +40,31 @@ JDBC이고 SQL은 `jsonb` · `FOR UPDATE SKIP LOCKED` · `ON CONFLICT` · `TIMES registry edge를 주면 그 경계가 깨진다. 복사본을 고칠 때(키 길이 검사, constant-time 비교 등)는 `shared-contract`의 정본과 같은 규칙인지 함께 본다. +### Publish retryability of authentication and authorization (review 2026-11-01 06-01) + +이 제품 안에는 같은 category에 대한 재시도 규칙이 둘 있다. 의도다. + +- category 기본값: `FailureDescriptor.defaultRetryable(AUTHENTICATION | AUTHORIZATION) = false`. 소비 쪽 + 재시도 엔진(`DefaultRetryDecisionEngine`)은 policy가 category를 따로 정하지 않으면 이 기본값과 + descriptor의 `retryable`을 둘 다 요구한다. 그래서 인가받지 못한 메시지의 재전달은 지금도 재시도하지 않는다. +- publish 거부의 `retryable`: `KafkaPublishFailureClassifier`는 인증·인가 거부에 `retryable = true`를 붙인다. + broker가 레코드를 저장하지 않았고, credential 교체나 application보다 늦게 배포된 ACL은 몇 분 동안 모든 + publish를 거부한 뒤 멈추기 때문이다. `retryable`을 믿는 publisher가 그 몇 분 동안의 레코드를 영구 보류하지 + 않게 하려는 것이다. + +06-01 수정이 이 제품에서 바꾼 것(모두 publish 경로, 소비 경로 동작은 그대로): + +- `KafkaPublishFailureClassifier`: 인증·인가 거부를 `retryable = true`로. +- `DefaultMessagePublisher`: runtime이 없을 때의 거부(`PUBLISH_RUNTIME_UNAVAILABLE`)를 `retryable = true`로. +- `KafkaMessagingTransport`: 닫힌 transport의 거부(`KAFKA_TRANSPORT_CLOSED`)를 `retryable = true`로. +- `MessagingShutdownLifecycle.PHASE`: platform 밖 relay의 순서 테스트가 참조하도록 public. + +버린 대안: vendored 제품은 그대로 두고 application bridge(ACL)가 category를 보고 "인증·인가면 재시도"로 +번역하는 것. 버린 이유: `retryable`은 이 제품이 "같은 레코드를 나중에 다시 보내면 받아질 수 있나"를 말하는 +자리다. bridge가 category로 그 판단을 다시 하면 같은 사실을 제품과 bridge가 따로 정하고, 이 제품의 +`PublishResult`를 읽는 다른 호출자는 bridge와 다른 답을 본다. 대신 이 제품 주석은 소비자 이름(application +relay, `composition:outbox-relay`)을 부르지 않는다 — "flag를 믿는 publisher"로 쓴다(재리뷰 2026-11-03 RR-05-07). + ## Public surface 정책 - **vendor 타입을 public signature에 노출하면 그 의존성은 `api`로 선언한다.** 숨긴다고 타입이 diff --git a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/FailureDescriptor.java b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/FailureDescriptor.java index 5c00b10..ff8dd47 100644 --- a/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/FailureDescriptor.java +++ b/messaging/messaging-core-api/src/main/java/dev/caskeleton/messaging/api/error/FailureDescriptor.java @@ -61,6 +61,14 @@ public record FailureDescriptor( * automatically: each of them fails identically on every redelivery, so retrying only multiplies * the load while the message is still poison. * + *

A default, not a rule every descriptor follows. A publish rejection may carry {@code + * retryable = true} with an authentication or authorization category: the broker did not store + * the record, and a credential being rotated or an ACL deployed after the application refuses + * every publish for a few minutes and then stops (the Kafka publish classifier). The + * consumer-side retry engine requires this default as well as the flag unless its policy names + * the category, so redelivery of an unauthorised message is still not retried; only a publisher + * that reads {@link #retryable()} sees the difference. + * * @param category the classification * @return true when automatic retry is allowed by default */ diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java index 4b85f2f..4e96018 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrar.java @@ -344,10 +344,17 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati // A decode failure is not a handler failure and must not be retried forever. The // offset may only advance once the record exists somewhere else, so an unavailable // quarantine leaves it uncommitted rather than dropping the message. - if (quarantine.quarantine(record, undecodable)) { + // It will never decode, however often it is redriven: DESERIALIZATION, not the + // POISON_MESSAGE of a handler that ran out of attempts (review 2026-11-01 06-14). + if (quarantine.quarantine( + record, + dev.caskeleton.messaging.api.error.FailureCategory.DESERIALIZATION, + undecodable)) { settlement.acknowledgeAfterQuarantine(); } else { - settlement.requeueAfterFailure(undecodable); + settlement.requeueAfterFailure( + undecodable, + dev.caskeleton.messaging.api.error.FailureCategory.DESERIALIZATION); } coordinator.release(partition); shutdown.endWork(); @@ -682,6 +689,7 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati retryOrQuarantine( new IllegalStateException( "the handler asked for another attempt and the retry budget is spent"), + dev.caskeleton.messaging.api.error.FailureCategory.POISON_MESSAGE, delay)); } @@ -715,11 +723,21 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati * that exists nowhere else — which is why a production destination must have one. */ private void requeueAfterFailure(Throwable reason) { + requeueAfterFailure(reason, exhaustedCategory(reason)); + } + + /** + * {@link #requeueAfterFailure(Throwable)} under a category the caller already knows: a record + * that could not be decoded stays {@code DESERIALIZATION} when its first quarantine write was + * not confirmed and it reaches quarantine again through the retry path. + */ + private void requeueAfterFailure( + Throwable reason, dev.caskeleton.messaging.api.error.FailureCategory category) { if (!terminal.compareAndSet(null, "requeue")) { // The handler settled before it failed or before its late completion; that answer stands. return; } - retryOrQuarantine(reason, Duration.ZERO); + retryOrQuarantine(reason, category, Duration.ZERO); } /** @@ -730,11 +748,13 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati * being retried — the offset may not advance past a message that exists nowhere else. */ private dev.caskeleton.messaging.api.settlement.SettlementResult retryOrQuarantine( - Throwable reason, Duration requested) { + Throwable reason, + dev.caskeleton.messaging.api.error.FailureCategory category, + Duration requested) { FailedOffset key = new FailedOffset(partition, offset); int attempt = failures.merge(key, 1, Integer::sum); dev.caskeleton.messaging.policy.RetryPolicy retry = spec.profile().retry(); - if (attempt >= retry.maxAttempts() && quarantine.quarantine(record, reason)) { + if (attempt >= retry.maxAttempts() && quarantine.quarantine(record, category, reason)) { failures.remove(key); settlements.enqueue( new KafkaSettlementCommand( @@ -751,6 +771,19 @@ public final class KafkaConsumerRegistrar implements TransportConsumerRegistrati return enqueueRequeue(backoff); } + /** + * The category a record that ran out of attempts is quarantined under: its last failure's own, + * when that failure is a platform one and so carries a classification — a transient one tells a + * redrive the record may pass now — and {@code POISON_MESSAGE}, the platform's name for "keeps + * failing its consumer", when it is not. + */ + private static dev.caskeleton.messaging.api.error.FailureCategory exhaustedCategory( + Throwable reason) { + return reason instanceof dev.caskeleton.messaging.api.error.MessagingException classified + ? classified.category() + : dev.caskeleton.messaging.api.error.FailureCategory.POISON_MESSAGE; + } + private dev.caskeleton.messaging.api.settlement.SettlementResult enqueueRequeue( Duration delay) { retries.pauseUntil(partition, offset, delay, clock.instant()); diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java index 0eea955..d6569fd 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaPublishFailureClassifier.java @@ -67,10 +67,14 @@ public final class KafkaPublishFailureClassifier { *

Authentication and authorization prove the broker did not store the record, but not that it * will refuse it again: a credential being rotated, or an ACL deployed after the application, * fails every publish this way for a few minutes and then stops. Marked not retryable, a - * publisher that trusts the flag — the application's outbox relay does — parks every record sent - * in those minutes. A credential that is simply wrong is retried too, and the outbox then holds - * its rows FAILED with an alert every retry — recoverable once the credential is fixed, where - * DEAD rows needed a manual reset each. + * publisher that trusts the flag parks every record sent in those minutes. A credential that is + * simply wrong is retried too, for as long as it stays wrong; the descriptor's code names it, so + * a caller can tell that retry from a rotation and stop it. + * + *

This flag departs from {@link FailureDescriptor#defaultRetryable} on purpose, for publish + * only: that default says an authentication or authorization failure is not retried, and it still + * governs consumption, where the retry engine requires both. See {@code src/messaging/CLAUDE.md} + * "Publish retryability of authentication and authorization". */ private static boolean mayPassOnRetry(Throwable cause) { return cause instanceof AuthenticationException || cause instanceof AuthorizationException; diff --git a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java index 20489c9..af09f0e 100644 --- a/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java +++ b/messaging/messaging-kafka/src/main/java/dev/caskeleton/messaging/kafka/KafkaQuarantinePublisher.java @@ -1,5 +1,6 @@ package dev.caskeleton.messaging.kafka; +import dev.caskeleton.messaging.api.error.FailureCategory; import org.apache.kafka.clients.consumer.ConsumerRecord; /** @@ -18,11 +19,22 @@ import org.apache.kafka.clients.consumer.ConsumerRecord; public interface KafkaQuarantinePublisher { /** - * Writes an undecodable record somewhere durable. + * Writes a record the consumer will not deliver somewhere durable, under the reason it gives. * + *

The category is the caller's, because only the caller knows which of two very different + * things happened: the record could not be rebuilt at all ({@link + * FailureCategory#DESERIALIZATION}), or its handler failed until the attempt budget ran out — + * under the failure's own category when it carries one, {@link FailureCategory#POISON_MESSAGE} + * when it does not. Both were written as {@code POISON_MESSAGE}, so a redrive could not tell a + * message that will never decode from one an outage kept failing (review 2026-11-01 06-14). + * + * @param record the record as it arrived + * @param category why it is quarantined, written as the failure-category header + * @param reason the failure; its type is written as the failure-code header * @return true when the write is confirmed, so the source offset may be committed */ - boolean quarantine(ConsumerRecord record, Throwable reason); + boolean quarantine( + ConsumerRecord record, FailureCategory category, Throwable reason); /** * The default: nothing is configured, so nothing may be committed past. @@ -37,7 +49,7 @@ public interface KafkaQuarantinePublisher { /** Holds the single "not configured" quarantine. */ final class Unavailable { - private static final KafkaQuarantinePublisher INSTANCE = (record, reason) -> false; + private static final KafkaQuarantinePublisher INSTANCE = (record, category, reason) -> false; private Unavailable() {} } diff --git a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java index f5580a5..edd7769 100644 --- a/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java +++ b/messaging/messaging-kafka/src/test/java/dev/caskeleton/messaging/kafka/KafkaConsumerRegistrarTest.java @@ -3,6 +3,7 @@ package dev.caskeleton.messaging.kafka; import static org.assertj.core.api.Assertions.assertThat; import dev.caskeleton.messaging.api.MessageId; +import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.header.ReservedHeaders; import dev.caskeleton.messaging.policy.OrderingImpact; import dev.caskeleton.messaging.policy.RetryMode; @@ -258,7 +259,7 @@ class KafkaConsumerRegistrarTest { registrar( KafkaContractHarness.profile(), this::fail, - (record, reason) -> quarantined.add(record.offset()), + (record, category, reason) -> quarantined.add(record.offset()), () -> NOW); assign(); addRecord(10); @@ -274,6 +275,29 @@ class KafkaConsumerRegistrarTest { .hasValue(11L); } + @Test + @DisplayName("handler가 시도 예산을 다 쓴 레코드는 그 실패의 분류로 격리하고 분류가 없으면 POISON_MESSAGE로 적는다") + void aDeliveryThatSpentItsBudgetIsQuarantinedUnderItsFailuresCategory() { + // given: a handler that fails with a plain exception every time + List categories = new ArrayList<>(); + KafkaConsumerRegistrar registrar = + registrar( + KafkaContractHarness.profile(), + this::fail, + (record, category, reason) -> categories.add(category), + () -> NOW); + assign(); + addRecord(10); + registrar.pollOnce(NOW); + handlers.runAll(); + + // when + registrar.pollOnce(NOW); + + // then + assertThat(categories).containsExactly(FailureCategory.POISON_MESSAGE); + } + @Test @DisplayName("handler가 요청한 requeue도 시도 횟수를 세어 예산을 쓰면 quarantine 후 commit된다") void aDeliveryItsHandlerKeepsRequeueingSpendsTheBudgetAndIsQuarantined() { @@ -283,7 +307,7 @@ class KafkaConsumerRegistrarTest { registrar( KafkaContractHarness.profile(), this::requeue, - (record, reason) -> quarantined.add(record.offset()), + (record, category, reason) -> quarantined.add(record.offset()), () -> NOW); assign(); addRecord(10); @@ -470,7 +494,7 @@ class KafkaConsumerRegistrarTest { handlers, new GracefulShutdownCoordinator(Duration.ofSeconds(30)), Duration.ofMillis(1), - (record, reason) -> { + (record, category, reason) -> { quarantined.add(record.offset()); return true; })) { @@ -486,6 +510,65 @@ class KafkaConsumerRegistrarTest { } } + /** + * A record that cannot be rebuilt into an envelope will never decode, however often it is + * redriven; the one whose handler ran out of attempts may pass once the outage is over. They were + * both written as {@code POISON_MESSAGE} (review 2026-11-01 06-14). + */ + @Test + @DisplayName("envelope로 되살릴 수 없는 레코드는 시도 예산 소진과 다른 DESERIALIZATION으로 격리한다") + void anUndecodableRecordIsQuarantinedAsDeserializationNotAsPoison() { + // given + List categories = new ArrayList<>(); + try (KafkaConsumerRegistrar registrar = + new KafkaConsumerRegistrar( + new TransportConsumerSpec(KafkaContractHarness.profile(), this::settleImmediately), + consumer, + handlers, + new GracefulShutdownCoordinator(Duration.ofSeconds(30)), + Duration.ofMillis(1), + (record, category, reason) -> categories.add(category))) { + assign(); + addRecordWithoutIdentityHeaders(10); + registrar.pollOnce(NOW); + + // when + handlers.runAll(); + + // then + assertThat(categories).containsExactly(FailureCategory.DESERIALIZATION); + } + } + + @Test + @DisplayName("격리 쓰기가 한 번 실패해 재시도로 넘어간 읽을 수 없는 레코드도 DESERIALIZATION으로 격리한다") + void anUndecodableRecordStaysDeserializationWhenItsQuarantineIsRetried() { + // given: the first quarantine write is not confirmed, so the record goes round the retry path + List categories = new ArrayList<>(); + try (KafkaConsumerRegistrar registrar = + new KafkaConsumerRegistrar( + new TransportConsumerSpec(KafkaContractHarness.profile(), this::settleImmediately), + consumer, + handlers, + new GracefulShutdownCoordinator(Duration.ofSeconds(30)), + Duration.ofMillis(1), + (record, category, reason) -> { + categories.add(category); + return categories.size() > 1; + })) { + assign(); + addRecordWithoutIdentityHeaders(10); + registrar.pollOnce(NOW); + + // when + handlers.runAll(); + + // then + assertThat(categories) + .containsExactly(FailureCategory.DESERIALIZATION, FailureCategory.DESERIALIZATION); + } + } + @Test @DisplayName("one partition hitting its ceiling does not abandon the other's records") void oneSaturatedPartitionDoesNotAbandonAnother() { diff --git a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java index a51f9fe..5c4341e 100644 --- a/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java +++ b/messaging/messaging-runtime-core/src/main/java/dev/caskeleton/messaging/runtime/DefaultMessagePublisher.java @@ -265,7 +265,7 @@ public final class DefaultMessagePublisher implements MessagePublisher, EncodedM admission.complete(destination.name().value()); // No runtime to send through — the registry closed during shutdown, or none is installed // yet. Nothing about the message was wrong, so the refusal is marked retryable: a caller that - // trusts the flag, as the application's outbox relay does, retries it instead of parking it. + // trusts the flag retries it instead of parking it. return refused( new FailureDescriptor( FailureCategory.TRANSIENT_INFRASTRUCTURE, diff --git a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java index 8185ef6..85ad9c2 100644 --- a/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java +++ b/messaging/messaging-spring-boot-starter-kafka/src/main/java/dev/caskeleton/messaging/autoconfigure/KafkaDeadLetterQuarantine.java @@ -1,5 +1,6 @@ package dev.caskeleton.messaging.autoconfigure; +import dev.caskeleton.messaging.api.error.FailureCategory; import dev.caskeleton.messaging.api.header.ReservedHeaders; import dev.caskeleton.messaging.kafka.KafkaQuarantinePublisher; import java.nio.charset.StandardCharsets; @@ -38,11 +39,14 @@ final class KafkaDeadLetterQuarantine implements KafkaQuarantinePublisher { } @Override - public boolean quarantine(ConsumerRecord record, Throwable reason) { + public boolean quarantine( + ConsumerRecord record, FailureCategory category, Throwable reason) { RecordHeaders headers = new RecordHeaders(record.headers().toArray()); headers.remove(ReservedHeaders.FAILURE_CATEGORY); headers.remove(ReservedHeaders.FAILURE_CODE); - headers.add(ReservedHeaders.FAILURE_CATEGORY, bytes("POISON_MESSAGE")); + headers.add( + ReservedHeaders.FAILURE_CATEGORY, + bytes(category == null ? FailureCategory.POISON_MESSAGE.name() : category.name())); headers.add( ReservedHeaders.FAILURE_CODE, bytes(reason == null ? "UNKNOWN" : reason.getClass().getSimpleName())); diff --git a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingShutdownLifecycle.java b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingShutdownLifecycle.java index 7efaa47..147c266 100644 --- a/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingShutdownLifecycle.java +++ b/messaging/messaging-spring-boot-starter/src/main/java/dev/caskeleton/messaging/autoconfigure/MessagingShutdownLifecycle.java @@ -29,14 +29,14 @@ import org.springframework.context.SmartLifecycle; */ public final class MessagingShutdownLifecycle implements SmartLifecycle { - /** How often the drain is re-checked. Short enough that a fast drain is not padded. */ /** - * The phase this drain stops in: {@link #getPhase()}. Public so a relay outside the platform — - * the application's own outbox relay, which may not name this type in production code — can be - * held above it by a test that does name it. + * The phase this drain stops in: {@link #getPhase()}. Public so a relay outside the platform — a + * publisher whose production code may not name this type — can be held above it by a test that + * does name it. */ public static final int PHASE = Integer.MAX_VALUE - 1024; + /** How often the drain is re-checked. Short enough that a fast drain is not padded. */ private static final Duration POLL = Duration.ofMillis(50); private final MessagingAdmissionController admission; @@ -120,11 +120,10 @@ public final class MessagingShutdownLifecycle implements SmartLifecycle { * drain ahead of the transports it is draining, because closing a connection first would strand * the settlements the drain exists to flush. It sits below the outbox relays' phase for the * mirror-image reason: refusing admission while a relay pass is mid-publish turns that pass's - * rows into refusals. Two relays stand above it: the platform's own ({@code - * MessagingOutboxRelayLifecycle}, built only when a platform outbox repository exists) and the - * application's ({@code composition:outbox-relay} {@code OutboxRelayScheduler}, phase {@code - * Integer.MAX_VALUE}), which finishes the publish it is in and releases its other claims before - * this phase runs. + * rows into refusals. The platform's own relay ({@code MessagingOutboxRelayLifecycle}, built only + * when a platform outbox repository exists) stands above it, and so must any relay outside the + * platform that publishes through it: a phase above {@link #PHASE}, finishing the publish it is + * in before this phase runs. * * @return the lifecycle phase, {@link #PHASE} */