The keycloak project ended with four open questions that design could not
settle. A two-VM lab was built to answer them by measurement, and this is
that material: 26 experiments, 125 raw command outputs, 22 browser captures.
Follows the import procedure in README.md.
source/ the originating repository verbatim — 78 documents, 28 SVGs,
8 manifests, plus .source-revision recording the commit
final/ the SSOT
document.md 729 lines written from the 29 experiment documents, not
concatenated: what was predicted, what was measured, and
where the measurement itself was wrong
evidence/raw 125 outputs, flattened to <experiment>__<file> because
the originals collided (01-baseline.txt appeared three
times) and the audit only globs the top level
evidence/meta one per raw file; command and exitCode are null and the
README says why rather than inventing them
evidence/browser 22 captures
assets/ three diagrams through techviz
.techviz/ their VizSpecs
A separate project rather than an addition to keycloak: the B-layer answers
that project's four questions, but the A, C and D layers are about cluster
failure, SSO and operations, and one document.md should hold one subject.
The four question records there can point here through 관계.
Recorded rather than papered over: only three of the 28 diagrams were
remade. The repository forbids hand-drawn SVG and forbids titles inside the
canvas; all 28 originals carry both, so converting them is redrawing, not
reformatting. They stay in source/ and the gap is written into the document.
verify-pipeline.py passes. audit-records.py reports no issues.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
251 lines
20 KiB
Plaintext
251 lines
20 KiB
Plaintext
# 포트가 선언한 네 쓰기와 saveHeldBy 의 자바독
|
|
src/application-core/src/main/java/dev/caskeleton/application/notification/platform/dispatch/RecipientDeliveryStorePort.java:12: Optional<RecipientDeliveryRecord> find(RecipientDeliveryId id);
|
|
src/application-core/src/main/java/dev/caskeleton/application/notification/platform/dispatch/RecipientDeliveryStorePort.java:15: RecipientDeliveryRecord save(RecipientDeliveryRecord record);
|
|
src/application-core/src/main/java/dev/caskeleton/application/notification/platform/dispatch/RecipientDeliveryStorePort.java:18: RecipientDeliveryRecord transition(
|
|
src/application-core/src/main/java/dev/caskeleton/application/notification/platform/dispatch/RecipientDeliveryStorePort.java:35: Optional<RecipientDeliveryRecord> saveHeldBy(
|
|
src/application-core/src/main/java/dev/caskeleton/application/notification/platform/dispatch/RecipientDeliveryStorePort.java:47: Optional<RecipientDeliveryRecord> transitionHeldBy(
|
|
|
|
RecipientDeliveryStorePort.java:21 /**
|
|
RecipientDeliveryStorePort.java:22 * Stores a modified job only while the given lease still holds it.
|
|
RecipientDeliveryStorePort.java:23 *
|
|
RecipientDeliveryStorePort.java:24 * <p>The claim is fenced and the renew is fenced, and for a while the *completion* was not. A
|
|
RecipientDeliveryStorePort.java:25 * worker whose lease expired during a provider call — the one place the platform deliberately
|
|
RecipientDeliveryStorePort.java:26 * spends time outside a transaction — came back and wrote its outcome with an unconditional
|
|
RecipientDeliveryStorePort.java:27 * {@code save}, over the row a new holder had already claimed and might already have dispatched.
|
|
RecipientDeliveryStorePort.java:28 * The optimistic {@code version} column did not help: it detects a concurrent edit, not a
|
|
RecipientDeliveryStorePort.java:29 * superseded writer, and the late worker's read was recent enough to win.
|
|
RecipientDeliveryStorePort.java:30 *
|
|
RecipientDeliveryStorePort.java:31 * @param record the modified job
|
|
RecipientDeliveryStorePort.java:32 * @param lease the lease the caller believes it holds
|
|
RecipientDeliveryStorePort.java:33 * @return the stored job, or empty when the lease has been superseded and nothing was written
|
|
RecipientDeliveryStorePort.java:34 */
|
|
RecipientDeliveryStorePort.java:35 Optional<RecipientDeliveryRecord> saveHeldBy(
|
|
RecipientDeliveryStorePort.java:36 RecipientDeliveryRecord record, RecipientLease lease);
|
|
|
|
# 펜싱하는 갱신문이 where 절에 넣는 것
|
|
RecipientDeliveryJpaRepository.java:56 /**
|
|
RecipientDeliveryJpaRepository.java:57 * Writes a completion projection, and only for the holder that still owns the job.
|
|
RecipientDeliveryJpaRepository.java:58 *
|
|
RecipientDeliveryJpaRepository.java:59 * <p>The counterpart of {@link #renewLease}, for the write that happens *after* the provider
|
|
RecipientDeliveryJpaRepository.java:60 * call. Everything before the submission is database work a new holder would simply redo; the
|
|
RecipientDeliveryJpaRepository.java:61 * outcome is not — writing it under a superseded lease reports one worker's result on another
|
|
RecipientDeliveryJpaRepository.java:62 * worker's attempt, and the two need not agree about whether the notification was sent.
|
|
RecipientDeliveryJpaRepository.java:63 *
|
|
RecipientDeliveryJpaRepository.java:64 * <p>Conditioned on owner and fence for the same reason as the renew: two incarnations of one
|
|
RecipientDeliveryJpaRepository.java:65 * configured worker id share the owner string, so the fence is what distinguishes them.
|
|
RecipientDeliveryJpaRepository.java:66 */
|
|
RecipientDeliveryJpaRepository.java:67 // clearAutomatically, because the caller re-reads this row immediately. A native update bypasses
|
|
RecipientDeliveryJpaRepository.java:68 // the persistence context, so without it the re-read is served from the first-level cache with
|
|
RecipientDeliveryJpaRepository.java:69 // the values this statement just replaced.
|
|
RecipientDeliveryJpaRepository.java:70 @Modifying(clearAutomatically = true, flushAutomatically = true)
|
|
RecipientDeliveryJpaRepository.java:71 @Query(
|
|
RecipientDeliveryJpaRepository.java:72 value =
|
|
RecipientDeliveryJpaRepository.java:73 "UPDATE notification_recipient_delivery "
|
|
RecipientDeliveryJpaRepository.java:74 + "SET delivery_state = :deliveryState, submission_outcome = :submissionOutcome, "
|
|
RecipientDeliveryJpaRepository.java:75 + "delivery_outcome = :deliveryOutcome, evidence_level = :evidenceLevel, "
|
|
RecipientDeliveryJpaRepository.java:76 + "ambiguous_attempt_exists = :ambiguousAttemptExists, duplicate_risk = :duplicateRisk, "
|
|
RecipientDeliveryJpaRepository.java:77 + "route_cursor = :routeCursor, attempt_count = :attemptCount, "
|
|
RecipientDeliveryJpaRepository.java:78 + "last_failure_category = :lastFailureCategory, next_dispatch_at = :nextDispatchAt, "
|
|
RecipientDeliveryJpaRepository.java:79 + "version = version + 1, updated_at = :now "
|
|
RecipientDeliveryJpaRepository.java:80 + "WHERE id = :id AND lease_owner = :owner AND lease_fence = :fence",
|
|
RecipientDeliveryJpaRepository.java:81 nativeQuery = true)
|
|
RecipientDeliveryJpaRepository.java:82 int saveProjectionHeldBy(
|
|
|
|
세 갱신문의 where 절을 나란히 :
|
|
RecipientDeliveryJpaRepository.java:80 + "WHERE id = :id AND lease_owner = :owner AND lease_fence = :fence",
|
|
RecipientDeliveryJpaRepository.java:82 int saveProjectionHeldBy(
|
|
RecipientDeliveryJpaRepository.java:105 + "WHERE id = :id AND lease_owner = :owner AND lease_fence = :fence",
|
|
RecipientDeliveryJpaRepository.java:107 int transitionHeldBy(
|
|
RecipientDeliveryJpaRepository.java:127 + "WHERE id = :id "
|
|
RecipientDeliveryJpaRepository.java:128 + "AND lease_owner = :owner "
|
|
RecipientDeliveryJpaRepository.java:129 + "AND lease_fence = :fence "
|
|
RecipientDeliveryJpaRepository.java:134 + "AND lease_until > :now",
|
|
RecipientDeliveryJpaRepository.java:136 int renewLease(
|
|
RecipientDeliveryJpaRepository.java:153 + "WHERE id = :id AND lease_owner = :owner AND lease_fence = :fence "
|
|
RecipientDeliveryJpaRepository.java:154 + "AND lease_until > :now",
|
|
RecipientDeliveryJpaRepository.java:169 + "WHERE id = :id AND lease_owner = :owner AND lease_fence = :fence",
|
|
RecipientDeliveryJpaRepository.java:181 + "WHERE delivery_state = 'DISPATCHING' AND lease_until < :threshold "
|
|
|
|
# 조건 없는 save 가 실제로 덮는 컬럼
|
|
JpaRecipientDeliveryStore.java:32 @Override
|
|
JpaRecipientDeliveryStore.java:33 public RecipientDeliveryRecord save(RecipientDeliveryRecord record) {
|
|
JpaRecipientDeliveryStore.java:34 Objects.requireNonNull(record, "record");
|
|
JpaRecipientDeliveryStore.java:35 RecipientDeliveryEntity entity =
|
|
JpaRecipientDeliveryStore.java:36 recipients
|
|
JpaRecipientDeliveryStore.java:37 .findById(record.id().value())
|
|
JpaRecipientDeliveryStore.java:38 .orElseThrow(() -> new IllegalStateException("recipient delivery no longer exists"));
|
|
JpaRecipientDeliveryStore.java:39 entity.applyProjection(
|
|
JpaRecipientDeliveryStore.java:40 record.state().name(),
|
|
JpaRecipientDeliveryStore.java:41 record.submissionOutcome().name(),
|
|
JpaRecipientDeliveryStore.java:42 record.deliveryOutcome().name(),
|
|
JpaRecipientDeliveryStore.java:43 record.evidenceLevel().name(),
|
|
JpaRecipientDeliveryStore.java:44 record.ambiguousAttemptExists(),
|
|
JpaRecipientDeliveryStore.java:45 record.duplicateRisk(),
|
|
JpaRecipientDeliveryStore.java:46 record.routeCursor(),
|
|
JpaRecipientDeliveryStore.java:47 record.attemptCount(),
|
|
JpaRecipientDeliveryStore.java:48 record.lastFailureCategory().orElse(null),
|
|
JpaRecipientDeliveryStore.java:49 record.nextDispatchAt().orElse(null),
|
|
JpaRecipientDeliveryStore.java:50 clock.instant());
|
|
JpaRecipientDeliveryStore.java:51 return mapper.toRecord(recipients.saveAndFlush(entity));
|
|
JpaRecipientDeliveryStore.java:52 }
|
|
|
|
RecipientDeliveryEntity.java:158 /** Apply a projection update produced by the dispatcher or a projector. */
|
|
RecipientDeliveryEntity.java:159 void applyProjection(
|
|
RecipientDeliveryEntity.java:160 String deliveryState,
|
|
RecipientDeliveryEntity.java:161 String submissionOutcome,
|
|
RecipientDeliveryEntity.java:162 String deliveryOutcome,
|
|
RecipientDeliveryEntity.java:163 String evidenceLevel,
|
|
RecipientDeliveryEntity.java:164 boolean ambiguousAttemptExists,
|
|
RecipientDeliveryEntity.java:165 boolean duplicateRisk,
|
|
RecipientDeliveryEntity.java:166 int routeCursor,
|
|
RecipientDeliveryEntity.java:167 int attemptCount,
|
|
RecipientDeliveryEntity.java:168 String lastFailureCategory,
|
|
RecipientDeliveryEntity.java:169 Instant nextDispatchAt,
|
|
RecipientDeliveryEntity.java:170 Instant at) {
|
|
RecipientDeliveryEntity.java:171 this.deliveryState = deliveryState;
|
|
RecipientDeliveryEntity.java:172 this.submissionOutcome = submissionOutcome;
|
|
RecipientDeliveryEntity.java:173 this.deliveryOutcome = deliveryOutcome;
|
|
RecipientDeliveryEntity.java:174 this.evidenceLevel = evidenceLevel;
|
|
RecipientDeliveryEntity.java:175 this.ambiguousAttemptExists = ambiguousAttemptExists;
|
|
RecipientDeliveryEntity.java:176 this.duplicateRisk = duplicateRisk;
|
|
RecipientDeliveryEntity.java:177 this.routeCursor = routeCursor;
|
|
RecipientDeliveryEntity.java:178 this.attemptCount = attemptCount;
|
|
RecipientDeliveryEntity.java:179 this.lastFailureCategory = lastFailureCategory;
|
|
RecipientDeliveryEntity.java:180 this.nextDispatchAt = nextDispatchAt;
|
|
RecipientDeliveryEntity.java:181 this.updatedAt = at;
|
|
RecipientDeliveryEntity.java:182 }
|
|
|
|
# 제공자 호출 뒤 결과를 쓰는 자리
|
|
DispatchOutcomeRecorder.java:14 /**
|
|
DispatchOutcomeRecorder.java:15 * Writes what an attempt proved.
|
|
DispatchOutcomeRecorder.java:16 *
|
|
DispatchOutcomeRecorder.java:17 * <p>Re-running this with the same result produces the same rows, because recovery after a crash
|
|
DispatchOutcomeRecorder.java:18 * between the provider call and the outcome write has to be a replay, not a second send.
|
|
DispatchOutcomeRecorder.java:19 */
|
|
DispatchOutcomeRecorder.java:108 boolean ambiguous = result.confirmation() == AttemptConfirmation.AMBIGUOUS;
|
|
DispatchOutcomeRecorder.java:109 RecipientDeliveryRecord updated =
|
|
DispatchOutcomeRecorder.java:110 new RecipientDeliveryRecord(
|
|
DispatchOutcomeRecorder.java:111 recipient.id(),
|
|
DispatchOutcomeRecorder.java:112 recipient.notificationId(),
|
|
DispatchOutcomeRecorder.java:113 recipient.tenantId(),
|
|
DispatchOutcomeRecorder.java:114 recipient.recipientRef(),
|
|
DispatchOutcomeRecorder.java:115 recipient.locale(),
|
|
DispatchOutcomeRecorder.java:116 recipient.timeZone(),
|
|
DispatchOutcomeRecorder.java:117 recipient.routingPlan(),
|
|
DispatchOutcomeRecorder.java:118 recipient.routeCursor(),
|
|
DispatchOutcomeRecorder.java:119 ambiguous
|
|
DispatchOutcomeRecorder.java:120 ? RecipientDeliveryState.RECONCILIATION_REQUIRED
|
|
DispatchOutcomeRecorder.java:121 : RecipientDeliveryState.DISPATCHING,
|
|
DispatchOutcomeRecorder.java:122 result.submissionOutcome(),
|
|
DispatchOutcomeRecorder.java:123 recipient.deliveryOutcome(),
|
|
DispatchOutcomeRecorder.java:124 recipient.evidenceLevel().strongerOf(result.evidenceLevel()),
|
|
DispatchOutcomeRecorder.java:125 recipient.ambiguousAttemptExists() || ambiguous,
|
|
DispatchOutcomeRecorder.java:126 recipient.duplicateRisk() || ambiguous,
|
|
DispatchOutcomeRecorder.java:127 recipient.nextDispatchAt(),
|
|
DispatchOutcomeRecorder.java:128 recipient.expiresAt(),
|
|
DispatchOutcomeRecorder.java:129 recipient.leaseOwner(),
|
|
DispatchOutcomeRecorder.java:130 recipient.leaseUntil(),
|
|
DispatchOutcomeRecorder.java:131 recipient.attemptCount() + 1,
|
|
DispatchOutcomeRecorder.java:132 result.failure().map(failure -> failure.category().name()),
|
|
DispatchOutcomeRecorder.java:133 recipient.createdAt(),
|
|
DispatchOutcomeRecorder.java:134 completedAt);
|
|
DispatchOutcomeRecorder.java:135 recipients.save(updated);
|
|
DispatchOutcomeRecorder.java:136
|
|
|
|
# 같은 흐름의 다른 쓰기는 펜싱을 쓴다
|
|
recipients\.save\(
|
|
DispatchOutcomeRecorder.java:135 recipients.save(updated);
|
|
recipients\.transition\(
|
|
NotificationAdminApplicationService.java:121 recipients.transition(
|
|
NotificationDispatchService.java:139 () -> recipients.transition(work.recipient().id(), blocked.state(), Optional.empty()));
|
|
NotificationDispatchService.java:396 recipients.transition(work.recipient().id(), state, Optional.empty());
|
|
NotificationSubmissionService.java:305 recipients.transition(job.id(), RecipientDeliveryState.CANCELED, Optional.empty());
|
|
ReconciliationService.java:108 recipients.transition(
|
|
recipients\.saveHeldBy\(
|
|
NotificationDispatchService.java:434 recipients.saveHeldBy(advanceRoute(work.recipient(), now), lease);
|
|
recipients\.transitionHeldBy\(
|
|
NotificationDispatchService.java:422 recipients.transitionHeldBy(
|
|
NotificationDispatchService.java:428 recipients.transitionHeldBy(
|
|
NotificationDispatchService.java:436 recipients.transitionHeldBy(
|
|
|
|
# 디스패치가 레코더를 부르기 직전에 하는 일
|
|
NotificationDispatchService.java:158 attempts.nextAttemptNo(work.recipient().id()),
|
|
NotificationDispatchService.java:159 clock.instant())));
|
|
NotificationDispatchService.java:160
|
|
NotificationDispatchService.java:161 if (!leases.stillHeld(lease)) {
|
|
NotificationDispatchService.java:162 // Checked immediately before the side effect, which is the last moment it can still be
|
|
NotificationDispatchService.java:163 // prevented. Everything above is database work another holder would simply redo; a provider
|
|
NotificationDispatchService.java:164 // submission is not — once it leaves, the recipient has the notification twice.
|
|
NotificationDispatchService.java:165 return;
|
|
NotificationDispatchService.java:166 }
|
|
NotificationDispatchService.java:167
|
|
NotificationDispatchService.java:168 ProviderSubmissionResult result =
|
|
NotificationDispatchService.java:169 submitOutsideTransaction(attempt, profile, contactPoint, content, work);
|
|
NotificationDispatchService.java:170
|
|
NotificationDispatchService.java:171 DeliveryAttemptRecord recorded =
|
|
NotificationDispatchService.java:172 transactions.inWrite(
|
|
NotificationDispatchService.java:173 () -> recorder.record(attempt, work.recipient(), result, clock.instant()));
|
|
NotificationDispatchService.java:174
|
|
NotificationDispatchService.java:175 RetryDecision next = retryPolicy.decide(retryContext(work, recorded, result, profile));
|
|
NotificationDispatchService.java:176 applyNextAction(work, recorded, next, clock.instant(), lease);
|
|
NotificationDispatchService.java:177 releaseLease(lease);
|
|
NotificationDispatchService.java:178 }
|
|
|
|
# 그 넷이 놓인 메서드가 자기 자바독에 적는 것
|
|
NotificationDispatchService.java:398 });
|
|
NotificationDispatchService.java:399 }
|
|
NotificationDispatchService.java:400
|
|
NotificationDispatchService.java:401 /**
|
|
NotificationDispatchService.java:402 * Applies the outcome, and only while this worker still holds the job.
|
|
NotificationDispatchService.java:403 *
|
|
NotificationDispatchService.java:404 * <p>Every write here happens *after* the provider call, which is the one stretch the platform
|
|
NotificationDispatchService.java:405 * deliberately spends outside a transaction. A lease can expire during it, another worker can
|
|
NotificationDispatchService.java:406 * claim the job, and this worker can then wake up and describe an attempt that is no longer the
|
|
NotificationDispatchService.java:407 * live one. The fenced variants make that write match nothing rather than win.
|
|
NotificationDispatchService.java:408 *
|
|
NotificationDispatchService.java:409 * <p>Losing the lease is not an error to report. The new holder owns the job and will record its
|
|
NotificationDispatchService.java:410 * own outcome; this worker's only remaining obligation is to stop.
|
|
NotificationDispatchService.java:411 */
|
|
NotificationDispatchService.java:412 private void applyNextAction(
|
|
|
|
NotificationDispatchService.java:417 RecipientLease lease) {
|
|
NotificationDispatchService.java:418 transactions.inWrite(
|
|
NotificationDispatchService.java:419 () -> {
|
|
NotificationDispatchService.java:420 switch (decision) {
|
|
NotificationDispatchService.java:421 case RetryDecision.RetryAfter retry ->
|
|
NotificationDispatchService.java:422 recipients.transitionHeldBy(
|
|
NotificationDispatchService.java:423 work.recipient().id(),
|
|
NotificationDispatchService.java:424 RecipientDeliveryState.RETRY_WAITING,
|
|
NotificationDispatchService.java:425 Optional.of(now.plus(retry.delay())),
|
|
NotificationDispatchService.java:426 lease);
|
|
NotificationDispatchService.java:427 case RetryDecision.Reconcile reconcile ->
|
|
NotificationDispatchService.java:428 recipients.transitionHeldBy(
|
|
NotificationDispatchService.java:429 work.recipient().id(),
|
|
NotificationDispatchService.java:430 RecipientDeliveryState.RECONCILIATION_REQUIRED,
|
|
NotificationDispatchService.java:431 Optional.of(reconcile.at()),
|
|
NotificationDispatchService.java:432 lease);
|
|
NotificationDispatchService.java:433 case RetryDecision.Fallback ignored ->
|
|
NotificationDispatchService.java:434 recipients.saveHeldBy(advanceRoute(work.recipient(), now), lease);
|
|
NotificationDispatchService.java:435 case RetryDecision.Stop ignored ->
|
|
NotificationDispatchService.java:436 recipients.transitionHeldBy(
|
|
NotificationDispatchService.java:437 work.recipient().id(),
|
|
NotificationDispatchService.java:438 attempt.submissionOutcome() == SubmissionOutcome.CONFIRMED_ACCEPTED
|
|
NotificationDispatchService.java:439 ? RecipientDeliveryState.COMPLETED
|
|
NotificationDispatchService.java:440 : RecipientDeliveryState.FAILED,
|
|
NotificationDispatchService.java:441 Optional.empty(),
|
|
NotificationDispatchService.java:442 lease);
|
|
NotificationDispatchService.java:443 }
|
|
NotificationDispatchService.java:444 refreshStatus(work);
|
|
NotificationDispatchService.java:445 });
|
|
|
|
# 그 순서를 고정하는 시험이 있는가
|
|
파일명에 Fenc 나 Lease 가 든 시험 파일 : 11 개
|
|
DispatchOutcomeRecorder 를 이름에 가진 시험 파일 : 0 개
|
|
펜싱 연산 이름이 나오는 시험 자리 (더블의 선언 포함) :
|
|
adapter/outbound/notification · test · LeaseRecoveryServiceTest.java:306 public Optional<RecipientDeliveryRecord> saveHeldBy(
|
|
adapter/outbound/notification · test · LeaseRecoveryServiceTest.java:313 public Optional<RecipientDeliveryRecord> transitionHeldBy(
|
|
adapter/outbound/persistence-jpa · postgresqlIntegrationTest · PostgreSqlRecipientLeaseFencingIntegrationTest.java:93 .transitionHeldBy(
|
|
adapter/outbound/persistence-jpa · postgresqlIntegrationTest · PostgreSqlRecipientLeaseFencingIntegrationTest.java:127 .transitionHeldBy(
|
|
application-core · test · PlatformFakes.java:263 public Optional<RecipientDeliveryRecord> saveHeldBy(
|
|
application-core · test · PlatformFakes.java:272 public Optional<RecipientDeliveryRecord> transitionHeldBy(
|