chore: initialize from backend template 0a6dd0e
This commit is contained in:
DongHyeonka
@@ -0,0 +1,81 @@
|
||||
# Object Storage Batch A Checkpoint
|
||||
|
||||
- Date: 2026-07-28
|
||||
- Branch: `codex/objectstorage-production-capability`
|
||||
- Worktree:
|
||||
`/home/donghyeon/workspace/clean-architecture-backend-template-objectstorage`
|
||||
- Claimed level: R0 application contract only
|
||||
- Provider readiness advanced: no
|
||||
|
||||
## Implemented scope
|
||||
|
||||
- Characterized the legacy caller-key overwrite, whole-object materialization, locator exposure,
|
||||
eager filesystem directory creation, optional S3 bucket provisioning, and Poster transaction/API
|
||||
coupling without changing those behaviors.
|
||||
- Added provider-neutral identities, opaque checked references/handles, bounded streaming
|
||||
callbacks, content identity, digest/range values, requests, receipts, outcomes, and narrow ports
|
||||
under `dev.caskeleton.application.objectstorage`.
|
||||
- Required an `ObjectOperationKey` on mutation requests and separated normal publication,
|
||||
scan-maintenance, purge-maintenance, direct, and staged privilege surfaces.
|
||||
- Added recursive contract-purity tests and an ArchUnit freeze for the one existing sample legacy
|
||||
import.
|
||||
- Marked the legacy `ObjectStoragePort` and `StoredObject` as removal boundaries without adapting
|
||||
new semantic calls back to raw keys.
|
||||
|
||||
No provider-neutral kernel, canonical namespace/control codec, local R1 provider, S3/MinIO
|
||||
qualification, sample migration, or R2 readiness claim is included.
|
||||
|
||||
## TDD evidence
|
||||
|
||||
The planned RED checks failed only for the intentionally missing types or removal annotations:
|
||||
|
||||
- `ObjectStorageIdentityContractTest`: missing identity types before Task 2 implementation.
|
||||
- `ObjectContentContractTest` and `ObjectStorageValueContractTest`: missing content/value types
|
||||
before Task 3 implementation.
|
||||
- `ObjectStoragePortContractTest`: missing request/receipt/port family before Task 4 implementation.
|
||||
- `ObjectStorageArchitectureContractTest`: missing legacy removal annotations before Task 5
|
||||
implementation.
|
||||
|
||||
An initial ArchUnit DSL compilation error was a test-authoring error, not accepted as a RED result;
|
||||
the rule was corrected and rerun.
|
||||
|
||||
## GREEN verification
|
||||
|
||||
All commands ran from `src/` and completed with `BUILD SUCCESSFUL`:
|
||||
|
||||
```bash
|
||||
./gradlew :application-core:resolveAndLockAll --write-locks
|
||||
./gradlew :application-core:verifyDependencyLocks --console=plain
|
||||
./gradlew :application-core:test --tests '*ObjectStorageIdentityContractTest' --console=plain
|
||||
./gradlew :application-core:test \
|
||||
--tests '*ObjectContentContractTest' \
|
||||
--tests '*ObjectStorageValueContractTest' --console=plain
|
||||
./gradlew :application-core:test --tests '*ObjectStoragePortContractTest' --console=plain
|
||||
./gradlew :application-core:test \
|
||||
--tests '*ObjectStorageArchitectureContractTest' --console=plain
|
||||
./gradlew :app-bootstrap:test --tests '*CleanArchitectureTest' --console=plain
|
||||
./gradlew :application-core:check --console=plain
|
||||
./gradlew verifyCleanArchitectureDependencies --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:test :sample-portfolio:test --console=plain
|
||||
```
|
||||
|
||||
The final combined legacy focused suites completed in 27 seconds. Deprecation-for-removal warnings
|
||||
are expected evidence that legacy consumers remain visible; they are not suppressed.
|
||||
|
||||
## LLM Wiki capture
|
||||
|
||||
The canonical vault required by repository policy,
|
||||
`/home/donghyeon/workspace/ai-tool/llm-wiki-private/`, and its parent
|
||||
`/home/donghyeon/workspace/ai-tool/` do not exist in this environment. Therefore the required
|
||||
`raw/branch-notes/codex-objectstorage-production-capability.md` could not be created or updated.
|
||||
No similarly named non-canonical clone was used. This exact access block is recorded in both the
|
||||
plan and design headers and here at the Batch A boundary.
|
||||
|
||||
## Remaining gates and risks
|
||||
|
||||
- External broker and REST consumers and deployed legacy data were not inspected; Gate A remains
|
||||
blocked for legacy removal or public API versioning.
|
||||
- The new contracts have no provider implementation yet.
|
||||
- The current legacy adapter retains whole-object and raw-locator behavior by design until the
|
||||
later migration batch.
|
||||
- No readiness registry row is promoted by this checkpoint.
|
||||
@@ -0,0 +1,109 @@
|
||||
# Object Storage Batch B Checkpoint
|
||||
|
||||
- Date: 2026-07-28
|
||||
- Branch: `codex/objectstorage-production-capability`
|
||||
- Worktree:
|
||||
`/home/donghyeon/workspace/clean-architecture-backend-template-objectstorage`
|
||||
- Evidence grade: repository-local non-skipping unit/contract/application-context tests
|
||||
- Advanced cards: local managed single upload R1, local managed download R1
|
||||
- R2 or production-provider readiness advanced: no
|
||||
|
||||
## Implemented scope
|
||||
|
||||
- Added deterministic data/control namespaces, opaque reference/handle codecs, canonical request
|
||||
fingerprints, frozen binding/policy revisions, and bounded operation epochs.
|
||||
- Added six strict canonical JSON control-record families with fixed field order, outer SHA-256
|
||||
envelopes, schema/size checks, corruption rejection, and checked-in golden digests.
|
||||
- Added provider-neutral publication, scan, reference, direct-session, multipart, and pending-effect
|
||||
state transitions with same-operation replay and conflicting-intent rejection.
|
||||
- Added a provider contract and `filesystem-local-dev` implementation with bounded streaming,
|
||||
immutable exclusive create, SHA-256 verification, exact inspect/version, full/range transfer,
|
||||
conditional retirement, create resolution, restrictive permissions, and path/symlink
|
||||
confinement.
|
||||
- Added single-process exact-version control CAS and restart/corruption/fault characterization.
|
||||
Logical control keys use `.record` physical leaves locally so object-store-valid prefix/leaf key
|
||||
pairs cannot collide as filesystem file/directory paths.
|
||||
- Added constructor-bound `app.object-storage` settings and compile-before-construction
|
||||
provider/destination/route/policy binding. The capability is disabled by default and
|
||||
`filesystem-local-dev` is rejected for `prod`/`production`.
|
||||
- Added disabled, unselected, invalid, selected-success, selected-construction-failure, close,
|
||||
legacy-only, and namespace-separated dual-run composition tests.
|
||||
- Added semantic routing evidence for publish, replay without producer invocation, inspect,
|
||||
full transfer, absent reference, and exact retained route lookup.
|
||||
- Added the exact nine-card readiness registry. Only local managed single upload/download are R1;
|
||||
direct, multipart, quarantine, retention, and production reconciliation remain R0.
|
||||
|
||||
## TDD and defect evidence
|
||||
|
||||
Planned RED checks failed for the intentionally absent codec/kernel/provider/settings/readiness
|
||||
types before each implementation. Additional tests found and drove these corrections:
|
||||
|
||||
- Local control keys may legally have both a leaf and a child in object storage, while a filesystem
|
||||
cannot have both `reference` and `reference/lifecycle`; local physical `.record` mapping fixed the
|
||||
collision without changing logical keys.
|
||||
- `ObjectInspectionPort.inspect` initially threw for an absent known-route reference; it now
|
||||
returns `Optional.empty()` while incomplete/corrupt evidence still fails closed.
|
||||
- The application purity test initially scanned its own test output after a full `check`; it now
|
||||
derives the production class root from a production contract type.
|
||||
- The general B7 ArchUnit rule initially classified objectstorage provider-internal SPI/control
|
||||
return values as public adapter responses. The existing negative fixture remains active, while a
|
||||
dedicated non-empty rule now checks the actual objectstorage `*Adapter` semantic boundaries.
|
||||
|
||||
No skipped Docker or external-service test is used as Batch B readiness evidence.
|
||||
|
||||
## GREEN verification
|
||||
|
||||
All commands ran from `src/` unless noted and completed with `BUILD SUCCESSFUL` after the documented
|
||||
RED/fix cycles:
|
||||
|
||||
```bash
|
||||
./gradlew :adapter:outbound:objectstorage:verifyDependencyLocks --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:test \
|
||||
--tests '*ObjectNamespaceCodecTest' \
|
||||
--tests '*ObjectRequestFingerprintCodecTest' \
|
||||
--tests '*ObjectOperationEpochTest' --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:test \
|
||||
--tests '*ObjectControlRecordCodecTest' \
|
||||
--tests '*ObjectOperationStateMachineTest' \
|
||||
--tests '*ObjectOperationKernelTest' --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:test \
|
||||
--tests '*ObjectStorageProviderContract' \
|
||||
--tests '*LocalDevObjectStorageProviderTest' \
|
||||
--tests '*LocalDevObjectStorageRecoveryTest' --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:test \
|
||||
--tests '*ObjectStorageBindingCompilerTest' \
|
||||
--tests '*ObjectStorageCapabilityConfigTest' \
|
||||
--tests '*RoutingObjectStorageAdapterTest' --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:test \
|
||||
--tests '*ObjectStorageReadinessRegistryTest' --console=plain
|
||||
./gradlew :sample-portfolio:test --console=plain
|
||||
./gradlew :application-core:check \
|
||||
:adapter:outbound:objectstorage:check --console=plain
|
||||
./gradlew :app-bootstrap:test --tests '*CleanArchitectureTest' --console=plain
|
||||
./gradlew verifyCleanArchitectureDependencies --console=plain
|
||||
```
|
||||
|
||||
The final combined application/objectstorage checkpoint completed in 23 seconds. The focused
|
||||
Clean Architecture suite and dependency verification also passed.
|
||||
|
||||
## LLM Wiki capture
|
||||
|
||||
The canonical vault required by repository policy,
|
||||
`/home/donghyeon/workspace/ai-tool/llm-wiki-private/`, and its parent
|
||||
`/home/donghyeon/workspace/ai-tool/` do not exist in this environment. Therefore the required
|
||||
`raw/branch-notes/codex-objectstorage-production-capability.md` and any derived raw documents could
|
||||
not be created or updated. No similarly named non-canonical clone was used. This exact access block
|
||||
is recorded in the plan/design status and at this Batch B boundary.
|
||||
|
||||
## Remaining gates and risks
|
||||
|
||||
- `filesystem-local-dev` has no multi-node linearizability or power-loss durability evidence and is
|
||||
forbidden in production profiles.
|
||||
- The canonical S3/MinIO provider contribution, async bounded transport, provider qualification,
|
||||
response-loss fault tests, and protected AWS evidence are not implemented.
|
||||
- Direct grants, multipart, quarantine/scan, retention/legal hold, privileged purge, reapers, and
|
||||
production reconciliation remain R0.
|
||||
- The sample Poster workflow still uses the deprecated whole-`byte[]` port and transaction-coupled
|
||||
legacy choreography. It is explicitly activated only in sample local/test configuration.
|
||||
- External API/broker consumers and deployed legacy data remain uninspected, so Gate A still blocks
|
||||
destructive migration or legacy removal.
|
||||
@@ -0,0 +1,84 @@
|
||||
# Object Storage Batch C Checkpoint
|
||||
|
||||
- Date: 2026-07-28
|
||||
- Branch: `codex/objectstorage-production-capability`
|
||||
- Worktree:
|
||||
`/home/donghyeon/workspace/clean-architecture-backend-template-objectstorage`
|
||||
- Evidence grade: repository-local tests plus digest-pinned single-node MinIO/Toxiproxy tests
|
||||
- AWS execution: not authorized; source set compiled only
|
||||
- Production-provider readiness advanced: no
|
||||
|
||||
## Implemented scope
|
||||
|
||||
- Added exact AWS S3 and MinIO provider bindings, bounded evidence descriptors, qualifier/error
|
||||
mapping, secret references, endpoint/owner/addressing validation, and selected-only lifecycle
|
||||
construction.
|
||||
- Added bounded async request/response bridges and the managed S3 put, inspect, full/range download,
|
||||
checksum, exact-version, cancellation, and content-length paths.
|
||||
- Added canonical conditional S3 control storage and operation response-loss resolution. Provider
|
||||
ETags remain adapter-private and are never exposed as logical versions.
|
||||
- Added low-level managed multipart planning, sharded immutable part ledgers, initiate-before-I/O
|
||||
state, explicit create/upload/list/complete/abort calls, and exact completion verification.
|
||||
- Added non-skipping MinIO contract/fault lanes, an AWS compile-only qualification lane, a protected
|
||||
workflow, and gate-matrix coverage.
|
||||
|
||||
The exact MinIO image is
|
||||
`minio/minio@sha256:4c4a4876193f030c81f57aabb22bcb9a73462010eb61fcab66908e03e5484af8`.
|
||||
The exact Toxiproxy image is
|
||||
`ghcr.io/shopify/toxiproxy@sha256:9378ed52a28bc50edc1350f936f518f31fa95f0d15917d6eb40b8e376d1a214e`.
|
||||
|
||||
## Exact MinIO finding
|
||||
|
||||
Real-provider tests proved an asymmetric conditional profile:
|
||||
|
||||
- `PutObject If-None-Match: *` was accepted but overwrote an existing object.
|
||||
- stale `PutObject If-Match` was rejected with HTTP 412.
|
||||
- `CompleteMultipartUpload If-None-Match: *` was accepted and overwrote an existing object.
|
||||
- checksum, HEAD, and range behavior passed the exercised contract.
|
||||
|
||||
Because immutable create and create-if-absent control CAS cannot be proven, the exact MinIO managed
|
||||
and direct mutation profiles remain `UNSUPPORTED`. The implementation does not emulate missing
|
||||
atomicity with HEAD followed by an unconditional write and does not promote a readiness card.
|
||||
|
||||
## TDD and verification
|
||||
|
||||
The task-focused RED runs first failed on the planned absent binding, bridge, conditional store,
|
||||
multipart, and qualification types. Provider qualification then found the real MinIO conditional
|
||||
behavior above; the descriptor and negative contract were changed instead of weakening the
|
||||
contract.
|
||||
|
||||
Commands completed with `BUILD SUCCESSFUL`:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:objectstorage:test \
|
||||
--tests '*S3ProviderBindingTest' \
|
||||
--tests '*S3ProviderQualifierTest' \
|
||||
--tests '*S3ProviderCompositionTest' --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:objectStorageMinioContractTest --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:objectStorageMinioFaultTest --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:objectStorageAwsQualificationTestClasses --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:check --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:verifyDependencyLocks \
|
||||
verifyCleanArchitectureDependencies --console=plain
|
||||
bash ../.github/scripts/verify-gate-matrix.sh
|
||||
```
|
||||
|
||||
The gate matrix reports 22 gates: 21 verified and the protected AWS qualification gate explicitly
|
||||
`delegated-pending`.
|
||||
|
||||
## LLM Wiki capture
|
||||
|
||||
The canonical vault `/home/donghyeon/workspace/ai-tool/llm-wiki-private/` and its parent
|
||||
`/home/donghyeon/workspace/ai-tool/` do not exist in this environment. The required
|
||||
`raw/branch-notes/codex-objectstorage-production-capability.md` and derived raw documents could not
|
||||
be created or updated. No similarly named non-canonical clone was used.
|
||||
|
||||
## Remaining risks
|
||||
|
||||
- No AWS request was executed, so there is no observed AWS provider claim.
|
||||
- The pinned MinIO topology is a local single-node container and is not production TLS,
|
||||
multi-node, durability, or linearizability evidence.
|
||||
- The detailed managed multipart fault matrix is not exhaustive enough for R2.
|
||||
- No sample migration, public API, scan/publication choreography, retention, purge, or reaper is
|
||||
included in this checkpoint.
|
||||
@@ -0,0 +1,104 @@
|
||||
# Object Storage Batch D Checkpoint
|
||||
|
||||
- Date: 2026-07-28
|
||||
- Branch: `codex/objectstorage-production-capability`
|
||||
- Worktree:
|
||||
`/home/donghyeon/workspace/clean-architecture-backend-template-objectstorage`
|
||||
- Scope: direct-transfer provider/application primitives only
|
||||
- Public endpoint: none
|
||||
- Readiness advanced: no; all direct cards remain R0
|
||||
|
||||
## Implemented scope
|
||||
|
||||
- Added direct single-upload session policy, durable prepared/issued transitions, bearer
|
||||
redaction, exact completion verification, published-version download resolution, and an
|
||||
S3-presigner lifecycle owned by the selected provider.
|
||||
- Added direct multipart durable session and part-grant families, opaque acknowledgement tokens,
|
||||
sharded part records, admission-close/expiry fencing, exact ledger validation, completion/abort
|
||||
states, response-loss resolution, and persisted terminal exact-version replay.
|
||||
- Added direct S3 initiate/discovery, exact-part presign, `ListParts` acknowledgement, conditional
|
||||
complete followed by exact HEAD verification, and abort resolution.
|
||||
- Registered direct single and multipart delegates only when their exact compiled capability is
|
||||
selected. One presigner is constructed and closed exactly once.
|
||||
- Added golden canonical envelopes for the direct session, direct multipart session, and direct
|
||||
multipart grant families.
|
||||
- Fixed `MultipartCompleteRequest` null validation so valid immutable `List.of(...)` input no longer
|
||||
throws from `contains(null)`.
|
||||
|
||||
## Qualification truth
|
||||
|
||||
The exact MinIO release cannot prove create-only PUT or create-only multipart completion, so both
|
||||
direct profiles are explicitly `UNSUPPORTED`. The direct MinIO contract/fault lanes are negative
|
||||
admission tests: they prove no bearer or multipart mutation enters an unsupported profile. No test
|
||||
skip is used as positive evidence.
|
||||
|
||||
The AWS managed/direct source sets compile, but no AWS call was made and no AWS evidence row was
|
||||
published. No inbound controller, authorization surface, CORS runtime configuration, or public
|
||||
direct API exists.
|
||||
|
||||
## Verification
|
||||
|
||||
Commands completed with `BUILD SUCCESSFUL`:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:objectstorage:test \
|
||||
--tests '*DirectTransferCoordinatorTest' \
|
||||
--tests '*PresignedGrantRedactionTest' \
|
||||
--tests '*S3DirectTransferProviderTest' \
|
||||
--tests '*ObjectControlRecordCodecTest' \
|
||||
--tests '*S3ProviderCompositionTest' --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:test \
|
||||
--tests '*DirectMultipartCoordinatorTest' \
|
||||
--tests '*DirectMultipartRaceTest' \
|
||||
--tests '*S3DirectMultipartProviderTest' \
|
||||
--tests '*ObjectControlRecordCodecTest' \
|
||||
--tests '*S3ProviderCompositionTest' --console=plain
|
||||
./gradlew \
|
||||
:adapter:outbound:objectstorage:objectStorageMinioContractTest \
|
||||
:adapter:outbound:objectstorage:objectStorageMinioFaultTest \
|
||||
--tests '*DirectTransfer*' --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:test \
|
||||
--tests '*DirectTransferCorsContractTest' --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:objectStorageAwsQualificationTestClasses --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:check --console=plain
|
||||
./gradlew :adapter:outbound:objectstorage:verifyDependencyLocks \
|
||||
verifyCleanArchitectureDependencies --console=plain
|
||||
./gradlew \
|
||||
:adapter:outbound:objectstorage:objectStorageMinioContractTest \
|
||||
:adapter:outbound:objectstorage:objectStorageMinioFaultTest --console=plain
|
||||
bash ../.github/scripts/verify-gate-matrix.sh
|
||||
./gradlew test --console=plain
|
||||
./gradlew check --console=plain
|
||||
```
|
||||
|
||||
The module `check` includes unit tests, Checkstyle, Spotless, SpotBugs, architecture, configuration
|
||||
processor, environment-key, and repository-wide policy checks. Existing test-only compiler
|
||||
warnings remain non-failing. The final repository-wide test run completed 79 tasks and the final
|
||||
repository-wide check completed 214 tasks.
|
||||
|
||||
## Deliberate limitations
|
||||
|
||||
- Issued bearer material is process-local. A restart fails closed instead of reconstructing or
|
||||
reissuing an already-issued bearer.
|
||||
- The signing clock/window is stored and bounded, but AWS SDK presigner query timing is not driven
|
||||
by the injected application clock.
|
||||
- The direct multipart recovery/race matrix covers its principal fences and completion response
|
||||
loss but is not exhaustive enough for an R2 claim.
|
||||
- Retention/Object Lock grant headers and a provider-enforced direct-single hard size ceiling are
|
||||
not qualified.
|
||||
- No public endpoint exists, so CORS evidence is a pure contract and no direct card may exceed R0
|
||||
in the current registry.
|
||||
|
||||
## Approval Gate A
|
||||
|
||||
Tasks 20–24 remain blocked until the user explicitly approves scanner ownership, the sample's
|
||||
first publication profile, the additive asynchronous API/status contract, and digest transport.
|
||||
No scan/publication/sample endpoint implementation was started.
|
||||
|
||||
## LLM Wiki capture
|
||||
|
||||
The canonical vault `/home/donghyeon/workspace/ai-tool/llm-wiki-private/` and its parent
|
||||
`/home/donghyeon/workspace/ai-tool/` do not exist in this environment. The required
|
||||
`raw/branch-notes/codex-objectstorage-production-capability.md` and derived raw documents could not
|
||||
be created or updated. No similarly named non-canonical clone was used.
|
||||
@@ -0,0 +1,94 @@
|
||||
# Object Storage Batch E Pause Checkpoint
|
||||
|
||||
- Recorded: 2026-07-29 (Asia/Seoul)
|
||||
- Branch: `codex/objectstorage-production-capability`
|
||||
- Worktree:
|
||||
`/home/donghyeon/workspace/clean-architecture-backend-template-objectstorage`
|
||||
- Status: implementation in progress; intentionally paused at the user's request
|
||||
- Evidence grade: local unit/integration/architecture evidence only; no AWS R2 evidence
|
||||
|
||||
## Implemented at this checkpoint
|
||||
|
||||
- Staged integrity verification, fake-scanner routing, publication handoff fencing, and stable
|
||||
replay receipts.
|
||||
- Additive Poster V8 dual-read schema (renumbered from branch-local V7 during JPA integration),
|
||||
upload/retirement intents, HMAC-sanitized idempotency scope,
|
||||
PostgreSQL atomic claim SPI, and forward-only migration qualification lane.
|
||||
- Short-transaction Poster image publication flow and additive locator-free `202` API under the
|
||||
AIP-122-compatible `/posters/{id}/imagePublications` collection.
|
||||
- Exact-reference/version logical retirement enqueue, lease/fence takeover, response-loss retry,
|
||||
Poster deletion survival, and disabled-by-default worker composition.
|
||||
- Isolated legacy migration contracts, report-only inspection, two-distinct-approver Ed25519
|
||||
approval verification, nonce replay boundary, and explicit maintenance-only composition.
|
||||
|
||||
## Verification completed
|
||||
|
||||
The following focused command passed after the final architecture fixes:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew \
|
||||
:sample-portfolio:spotlessApply \
|
||||
:sample-portfolio:test --tests '*PosterImagePublicationControllerWireTest' \
|
||||
:app-bootstrap:test --tests '*CleanArchitectureTest' \
|
||||
--console=plain
|
||||
```
|
||||
|
||||
The following focused suites also passed during this checkpoint:
|
||||
|
||||
```bash
|
||||
./gradlew :adapter:outbound:objectstorage:test \
|
||||
--tests '*LegacyObjectAdoptionServiceTest' \
|
||||
--tests '*LegacyAdoptionApprovalVerifierTest' \
|
||||
--tests '*ObjectStorageLegacyMigrationConfigTest' --console=plain
|
||||
|
||||
./gradlew :sample-portfolio:test \
|
||||
--tests '*DeletePosterImageRetirementTest' \
|
||||
--tests '*PosterImageRetirementCrashMatrixTest' \
|
||||
--tests '*PosterImageRetirementConfigTest' \
|
||||
--tests '*LegacyPosterImageUploadCharacterizationTest' --console=plain
|
||||
|
||||
./gradlew :sample-portfolio:test \
|
||||
--tests '*SampleApplicationContextTest' \
|
||||
:sample-portfolio:posterImageMigrationTest --console=plain
|
||||
```
|
||||
|
||||
The migration lane included
|
||||
`PosterImageRetirementQualificationTest`, which proved that an exact retirement row survives
|
||||
deletion of its Poster row.
|
||||
|
||||
## Failures found and resolved
|
||||
|
||||
- `spotlessJavaCheck` initially found formatting drift in newly changed application-core and
|
||||
persistence files. The owner-module Spotless apply tasks fixed it.
|
||||
- `SampleApplicationContextTest` initially failed because Spring's persistence exception advisor
|
||||
could not CGLIB-proxy the final `PosterImageAttachmentCasRepository`. Removing `final` fixed the
|
||||
context; the focused context suite then passed.
|
||||
- `CleanArchitectureTest` initially rejected an application-core return type from the sample domain
|
||||
and the kebab-case `image-publications` path. Conversion moved back to the application use case,
|
||||
and the endpoint changed to the repository's AIP-122-compatible `imagePublications` segment. The
|
||||
complete focused architecture suite then passed.
|
||||
|
||||
## Not yet re-run / not complete
|
||||
|
||||
- The combined Batch E checkpoint command stopped on the two architecture failures above before all
|
||||
requested root tasks could complete. The focused failing suites passed after the fixes, but
|
||||
`:sample-portfolio:check`, `verifyPublicPathSnapshot`, and the full combined Batch E command have
|
||||
not been re-run after those final fixes.
|
||||
- The complete repository `./gradlew test` and `./gradlew check` have not been re-run after the
|
||||
Batch E additions.
|
||||
- The legacy adoption runner/configuration is not yet wired to a production legacy inspector,
|
||||
permission-checked trust-key loader, or durable control-record replay-store implementation.
|
||||
- Tasks 25–30 (Batch F) have not started in this continuation.
|
||||
- Actual AWS qualification is blocked by Approval Gate B: no approved account, bucket/namespaces,
|
||||
workload roles, signed deployment attestation, or mutation/test authority was supplied.
|
||||
- No readiness card was promoted. Local/MinIO ceilings and unsupported conditional behavior remain
|
||||
unchanged.
|
||||
|
||||
## Wiki capture
|
||||
|
||||
At this isolated-branch checkpoint, the then-selected private vault
|
||||
`/home/donghyeon/workspace/ai-tool/llm-wiki-private/` was absent, so capture was blocked. The final
|
||||
main integration was later captured in the user-designated public vault at
|
||||
`raw/branch-notes/chore-main-worktree-capability-integration.md`, with the derived error note
|
||||
`raw/errors/multi-worktree-contract-drift-2026-07-31.md`.
|
||||
@@ -0,0 +1,73 @@
|
||||
# Object Storage Phase 0 Inventory
|
||||
|
||||
- Captured: 2026-07-28
|
||||
- Branch: `codex/objectstorage-production-capability`
|
||||
- Scope: repository-local source, tests, configuration, migrations, and documentation
|
||||
- Evidence grade: repository-local only; deployed data, broker subscribers, and external REST
|
||||
consumers were not inspected
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
rg -n 'application\.storage|ObjectStoragePort|StoredObject|ca-skeleton\.objectstorage|file://|s3://' \
|
||||
src docs
|
||||
rg -n 'image_key|posters/.*/image' src/sample-portfolio
|
||||
rg -n 'poster\.image-attached|StoredObjectResponse|PosterResponse|imageKey' \
|
||||
src/sample-portfolio docs
|
||||
```
|
||||
|
||||
The commands completed successfully in the isolated worktree. Results are classified below.
|
||||
Documentation hits in the Object Storage design/plan describe the migration and are not runtime
|
||||
consumers. The `s3://bucket/key-1` fixture in
|
||||
`IdempotencyStoreAdapterTest` belongs to the generic idempotency response-reference test and is not
|
||||
an Object Storage legacy-port consumer.
|
||||
|
||||
## Repository-local runtime inventory
|
||||
|
||||
| Contract/data | Producer | Repository-local consumers | Classification |
|
||||
| --- | --- | --- | --- |
|
||||
| `ObjectStoragePort` / `StoredObject` | `application-core/application/storage` | filesystem and S3 adapters, `UploadPosterImageUseCase`, `PosterController`/`PosterWebMapper` | legacy runtime contract |
|
||||
| `ca-skeleton.objectstorage.*` | `ObjectStorageSettings` / `ObjectStorageConfig` | sample runtime through its objectstorage runtime dependency | legacy runtime configuration |
|
||||
| `file://` receipt | `FilesystemObjectStorageAdapter` | `StoredObjectResponse.location` through `PosterWebMapper` | public legacy locator |
|
||||
| `s3://bucket/key` receipt | `S3ObjectStorageAdapter` | `StoredObjectResponse.location` through `PosterWebMapper` | public legacy locator |
|
||||
| `/posters/{id}/image` | `PosterController` | repository tests and the generated/public HTTP contract | legacy inbound API |
|
||||
| `StoredObjectResponse` | `PosterController` / `PosterWebMapper` | HTTP caller, with `key`, `size`, `contentType`, and `location` | legacy response DTO |
|
||||
| `PosterResponse.imageKey` | `PosterWebMapper` | list/get/create/update/publish/archive HTTP responses | legacy general response field |
|
||||
| `poster.image-attached` | `PosterEventPublisher` | no subscriber found in this repository | versionless broker event; external consumers unknown |
|
||||
| `poster.image-attached.imageKey` | `PosterImageAttached` and publisher JSON | no subscriber found in this repository | raw locator-shaped event field |
|
||||
| `poster.image_key` | Flyway V6, `PosterEntity`, persistence mapper | `Poster` aggregate and repository adapter | stored-data schema |
|
||||
| `posters/{id}/image` key | `UploadPosterImageUseCase` | aggregate `imageKey`, event payload, DB row, HTTP response | deterministic overwriteable legacy key |
|
||||
|
||||
## Executable characterization
|
||||
|
||||
The following tests pin the current behavior without approving it as the target design:
|
||||
|
||||
- `LegacyObjectStorageBehaviorTest`
|
||||
- caller-selected keys overwrite;
|
||||
- `get` returns `Optional<byte[]>` and materializes the whole object;
|
||||
- receipts expose `file://` and `s3://` locators.
|
||||
- `LegacyObjectStorageConfigTest`
|
||||
- missing backend configuration selects filesystem;
|
||||
- context creation creates the filesystem directory before the first write;
|
||||
- `autoCreateBucket=true` probes and creates a missing bucket during S3 bean construction.
|
||||
- `LegacyPosterImageUploadCharacterizationTest`
|
||||
- remote storage is called while `TransactionPort.inWrite` is active;
|
||||
- the controller calls `MultipartFile.getBytes`;
|
||||
- the response exposes raw key and provider location;
|
||||
- Poster deletion leaves the legacy object untouched.
|
||||
|
||||
## External inventory gap and Gate A
|
||||
|
||||
Repository search does not prove that the following have no deployed consumers:
|
||||
|
||||
- `POST /posters/{id}/image`;
|
||||
- `StoredObjectResponse.key` and `.location`;
|
||||
- `PosterResponse.imageKey`;
|
||||
- broker event type `poster.image-attached` and its `imageKey` payload;
|
||||
- rows already stored in `poster.image_key`;
|
||||
- filesystem/S3 objects already written under `posters/{id}/image`.
|
||||
|
||||
No deployed database, object namespace, access log, API client catalog, broker consumer group, schema
|
||||
registry, or owning team approval was inspected. Therefore removal, in-place field rename, event
|
||||
payload replacement, or legacy-object deletion remains blocked. Approval Gate A must obtain owner
|
||||
and consumer evidence and choose an additive/versioned migration contract.
|
||||
@@ -0,0 +1,176 @@
|
||||
# Fileserver configuration
|
||||
|
||||
Every key below lives under `app.fileserver-platform` (environment form
|
||||
`APP_FILESERVER_PLATFORM_*`). That namespace is the HTTP platform's alone: `app.fileserver.*`
|
||||
belongs to the R2 tabular publication capability and `app.file-export.*` to the R1 CSV export, and
|
||||
the three are deliberately separate so switching one on cannot switch on another.
|
||||
|
||||
While `app.fileserver-platform.enabled` is false none of these keys is bound at all — the
|
||||
auto-configuration that binds them is not processed — so a malformed value in a block nobody
|
||||
enabled cannot fail a startup. Once enabled, binding is strict: an unknown key under the prefix is
|
||||
refused rather than ignored. The defaults are the conservative ones: the
|
||||
capability is off, the admin plane is off, background reclamation is off, and there is no permissive
|
||||
authorization fallback. Turning the capability on is a deliberate act, and so is every surface it
|
||||
exposes.
|
||||
|
||||
## Minimum to start
|
||||
|
||||
```yaml
|
||||
ca-skeleton:
|
||||
fileserver:
|
||||
enabled: true
|
||||
instance-id: ${HOSTNAME} # writer-lease owner; must be unique per node
|
||||
storage:
|
||||
root: /var/lib/backend/files # absolute, outside any webroot or config dir
|
||||
security:
|
||||
access-policy: role-based # or supply your own FileAccessPolicy bean
|
||||
observability:
|
||||
fingerprint-key: ${FILESERVER_FINGERPRINT_KEY}
|
||||
```
|
||||
|
||||
Startup fails, rather than degrading, when any of these is missing or unsafe:
|
||||
|
||||
| Condition | Why it is fatal |
|
||||
| --- | --- |
|
||||
| `security.access-policy` left at `required` with no `FileAccessPolicy` bean | a file capability that authorizes by default is worse than one that refuses to start |
|
||||
| `observability.fingerprint-key` unset while metrics are on | an unkeyed digest of an enumerable identifier is reversible |
|
||||
| the storage root fails a mandatory capability probe | a volume that cannot create atomically, keep staging and content on one FileStore, or refuse symlinks is unsafe, not degraded |
|
||||
| `storage.publish-mode: atomic-move-required` on a volume where the probe could not prove an atomic move | the configured guarantee cannot be delivered |
|
||||
| `security.access-policy: unenforced` under a `prod` profile | a value that was convenient in development must not survive promotion |
|
||||
|
||||
## Authorization — `security`
|
||||
|
||||
| Key | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `access-policy` | `required` | `required` (supply your own bean), `role-based`, or `unenforced` |
|
||||
| `read-roles` | `ROLE_FILE_READ` | grants metadata read and download |
|
||||
| `write-roles` | `ROLE_FILE_WRITE` | grants create, append, finalize, delete, copy, move |
|
||||
| `admin-roles` | `ROLE_FILE_ADMIN` | grants reverify and force-delete, and gates `/internal/fileserver/**` at the servlet chain |
|
||||
|
||||
There is no anonymous-read switch. Every Fileserver route is authenticated by the servlet chain
|
||||
before any application policy is consulted, so such a setting could only ever have described a
|
||||
permission the transport had already refused — a configuration that reads as if it grants access
|
||||
and does not.
|
||||
|
||||
The three tiers do not inherit. An admin role cannot delete through the data plane, and a write role
|
||||
cannot reach the management plane — a role model where "can delete" implied "can force-delete" would
|
||||
make the audited plane reachable through the unaudited one.
|
||||
|
||||
`unenforced` authorizes everything and exists so a developer can exercise upload and download before
|
||||
deciding on a role model. It is refused under a production profile.
|
||||
|
||||
## Storage — `storage`
|
||||
|
||||
| Key | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `root` | `/var/lib/backend/files` | absolute path; the only place a path exists |
|
||||
| `publish-mode` | `atomic-move-preferred` | `atomic-move-required`, `atomic-move-preferred`, `metadata-pointer` |
|
||||
| `buffer-size` | `128KB` | bounds every transfer allocation; memory never scales with file size |
|
||||
| `forbidden-root-ancestors` | `/app,/etc,/usr/share/nginx/html` | roots the storage root must not live under (webroot, config dirs) |
|
||||
|
||||
`root` must be absolute. A relative root resolves against the process working directory, which is
|
||||
one path in a container and another in a test, so it is refused at binding time.
|
||||
|
||||
Three former keys are gone, pinned as constants instead: staging and content share one FileStore,
|
||||
symbolic links are never followed, and the object and its directory are synced before READY. Each
|
||||
is an invariant the atomic publish and the namespace boundary are built on — a deployment that
|
||||
could switch one off would be running a different capability under the same name and the same
|
||||
tests.
|
||||
|
||||
The storage provider has no selector either. There is exactly one implementation, and a `type` key
|
||||
with one legal value is a promise of pluggability that nothing keeps.
|
||||
|
||||
## Upload, download, transfer
|
||||
|
||||
| Key | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `upload.max-file-size` | `100MB` | hard ceiling; also drives `spring.servlet.multipart.max-file-size` |
|
||||
| `upload.max-request-size` | `110MB` | request envelope; must be at least `max-file-size` |
|
||||
| `upload.initial-reservation` | `8MB` | quota reserved when the length is unknown |
|
||||
| `upload.ttl` | `1h` | how long a resumable upload stays claimable |
|
||||
| `upload.reservation-ttl` | `24h` | how long an unsettled quota reservation survives |
|
||||
| `upload.lease-duration` | `30s` | writer lease; renewed at one third of this |
|
||||
| `upload.max-parts` | `16` | multipart part ceiling |
|
||||
| `upload.require-content-length` | `false` | refuse chunked raw uploads |
|
||||
| `download.cache-control` | `private, no-store` | emitted on every content response |
|
||||
| `download.inline-allowed` | `false` | scriptable content is always an attachment regardless |
|
||||
| `download.max-ranges` | `1` | multi-range responses are opt-in |
|
||||
| `download.max-range-bytes` | `100MB` | total bytes one ranged response may cover |
|
||||
| `download.zero-copy-enabled` | `true` | hand large plaintext responses to the kernel |
|
||||
| `download.zero-copy-minimum-bytes` | `16MB` | below this the syscall setup costs more than it saves |
|
||||
| `transfer.core-size` / `max-size` / `queue-capacity` | `8` / `32` / `64` | blocking transfer pool bounds |
|
||||
| `transfer.await-seconds` | `300` | how long a transfer may occupy a pool thread |
|
||||
|
||||
Zero copy changes no header and no status. When storage declines it — an unreadable region, an
|
||||
unsupported backend — the response is streamed instead and is byte-identical.
|
||||
|
||||
## Verification — `verification`
|
||||
|
||||
| Key | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `timeout` | `5s` | per-verifier ceiling |
|
||||
| `require-media-type-verdict` | `false` | refuse a file whose type could not be determined |
|
||||
| `inline-safe-profile` | `false` | accept scriptable content instead of quarantining it |
|
||||
|
||||
Set `inline-safe-profile: true` only when downloads are never served inline from a trusted origin.
|
||||
|
||||
## Quota and admission — `quota`
|
||||
|
||||
| Key | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `instance-upload-permits` | `16` | concurrent uploads this node admits |
|
||||
| `scope-upload-permits` | `4` | concurrent uploads one namespace admits |
|
||||
| `direct-download-permits` | `64` | concurrent non-delegated downloads |
|
||||
| `soft-high-water` | `0.70` | storage fraction at which pressure is reported |
|
||||
| `hard-high-water` | `0.85` | storage fraction at which uploads are refused |
|
||||
|
||||
When the storage fraction cannot be read, admission treats it as unknown and does not apply the
|
||||
high-water rule — a synthetic `0` would silently disable the guard, and a synthetic `1` would take
|
||||
the capability down over a failed syscall.
|
||||
|
||||
## Background reclamation — `cleanup`
|
||||
|
||||
| Key | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `enabled` | `false` | run the cleanup worker on this node |
|
||||
| `interval` | `60s` | fixed delay between batches, not fixed rate |
|
||||
| `max-items` | `100` | items one batch may claim |
|
||||
| `max-bytes` | `1GB` | bytes one batch may reclaim |
|
||||
| `retry-backoff` | `5m` | how long a failed item waits before it is due again |
|
||||
|
||||
The worker deletes physical objects, so it is off until a deployment decides otherwise. A node
|
||||
without it still queues cleanup items; another node or an operator reclaims them. An item that fails
|
||||
eight times is abandoned rather than retried forever — it stays visible to an operator, parked
|
||||
rather than discarded.
|
||||
|
||||
## Management plane — `admin`
|
||||
|
||||
| Key | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `enabled` | `false` | expose `/internal/fileserver/**` |
|
||||
| `orphan-minimum-age` | `1h` | how long an unreferenced object must exist before a scan may name it |
|
||||
|
||||
Publishing content and committing its record are two steps. Anything younger than
|
||||
`orphan-minimum-age` is assumed to be mid-commit rather than abandoned; shortening this makes
|
||||
concurrent uploads look like orphans.
|
||||
|
||||
## Front-proxy delegation — `nginx`
|
||||
|
||||
| Key | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `enabled` | `false` | emit `X-Accel-Redirect` instead of a body |
|
||||
| `internal-prefix` | `/__files/` | must be an `internal` location resolving to the content root |
|
||||
| `object-suffix` | `.bin` | layout suffix the proxy appends |
|
||||
| `minimum-size` | `16MB` | below this the application serves the transfer itself |
|
||||
|
||||
Delegation is decided only after authorization and the READY gate, so an internal redirect can only
|
||||
ever name content the caller was already allowed to read.
|
||||
|
||||
## Protocols — `tus`
|
||||
|
||||
| Key | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `enabled` | `false` | expose the tus 1.0 endpoints |
|
||||
|
||||
The HTTPbis resumable-upload draft-12 surface is experimental and documented in
|
||||
[support-matrix.md](support-matrix.md).
|
||||
@@ -0,0 +1,213 @@
|
||||
# Fileserver — deviations from the design
|
||||
|
||||
The design specification and the implementation plan are frozen documents. Where implementation
|
||||
found them under-specified or self-contradicting, the resolution is recorded here rather than by
|
||||
editing the specification, and every entry names the test that pins the decision.
|
||||
|
||||
## Resolved inconsistencies in the state machine and contracts
|
||||
|
||||
### 1. `CREATED → FAILED` has no edge in the transition table
|
||||
|
||||
A create that fails after the record exists must end in `FAILED`, but the table has no direct edge.
|
||||
The record therefore walks `CREATED → UPLOADING → FAILED`, which is also the honest reading: the
|
||||
upload had been admitted before it failed.
|
||||
|
||||
Pinned by `UploadApplicationServiceTest` (application-core).
|
||||
|
||||
### 2. `ContentKey`'s alphabet admits a leading separator
|
||||
|
||||
The design's key pattern `[a-z0-9/_-]{16,200}` matches `/etc/passwd/...`. Rejecting an absolute path
|
||||
at the value type would change a design-fixed contract, so the stricter shape check lives in
|
||||
`PhysicalPathResolver`, per §12.2 rule 1 — the only place that turns an identifier into a path.
|
||||
|
||||
Pinned by `PhysicalPathResolverTest`.
|
||||
|
||||
### 3. `VERIFYING → DELETING` has no edge
|
||||
|
||||
Deleting a file that is mid-verification has no legal transition. The lifecycle service refuses it
|
||||
with `409 FILE_NOT_READY` rather than inventing an edge, which matches the allowed-state list the
|
||||
JPA `markDeleting` statement already enforced.
|
||||
|
||||
Pinned by `FileLifecycleServiceTest`.
|
||||
|
||||
### 4. `If-Match` is specified as an ETag but the lifecycle was designed around the row version
|
||||
|
||||
The HTTP contract sends an entity tag; the metadata store guards on a numeric version. The service
|
||||
takes `Optional<String> expectedEtag` and compares against the record's strong validator, so the
|
||||
precondition a client sends is the precondition that is checked.
|
||||
|
||||
Pinned by `FileLifecycleServiceTest`.
|
||||
|
||||
### 5. The filename policy left `:` intact
|
||||
|
||||
`C:\Windows\system.ini` sanitized to `C:Windowssystem.ini` — a drive-qualified name surviving into
|
||||
display text and headers. `:` joined the structural strip set.
|
||||
|
||||
Pinned by `FileserverHardeningContractTest` and `AmbiguousFilesystemOperationDetectorTest`.
|
||||
|
||||
## Additions the design implies but does not specify
|
||||
|
||||
### 6. `fs_recovery_item`
|
||||
|
||||
§10.2 lists five core tables and none of them can hold the recovery queue, yet §29.3 requires one:
|
||||
reconciliation reports files whose bytes and metadata disagree, and holding that list in memory
|
||||
would lose exactly the cases a restart interrupted. Added in
|
||||
`V2__fileserver_recovery_and_staging_cleanup.sql` with one open item per file, so repeated sweeps
|
||||
update a worklist rather than accumulating a log.
|
||||
|
||||
Pinned by `PostgreSqlFileserverReclamationIntegrationTest`.
|
||||
|
||||
### 7. `fs_cleanup_item.upload_id`
|
||||
|
||||
A staging object is addressed by upload, not by file. Without this column a queued staging cleanup
|
||||
could name only already-published content, so a cancelled or expired upload left bytes nothing could
|
||||
find. Added in the same migration, with a check constraint that an item names exactly one target.
|
||||
|
||||
Pinned by `PostgreSqlFileserverReclamationIntegrationTest`.
|
||||
|
||||
### 8. `ContentReferenceLedger` and `StagingUploadLocator`
|
||||
|
||||
The orphan scan must ask whether a record still claims a physical object, and reconciliation must
|
||||
map a file back to the upload that last staged it. Neither question is answerable through the
|
||||
design's `FileMetadataStore` or `UploadSessionStore` as written. Rather than widen those
|
||||
design-fixed interfaces, both are narrow single-method ports.
|
||||
|
||||
Pinned by `PostgreSqlFileserverReclamationIntegrationTest` and `LocalOrphanScanAdapterTest`.
|
||||
|
||||
## Interpretations
|
||||
|
||||
### 9. Quota settlement is FIFO within a scope
|
||||
|
||||
Nothing links a reservation row to the upload that took it, and the design deliberately reclaims
|
||||
stragglers by TTL and the `STALE_QUOTA_RESERVATION` cleanup type rather than threading a reservation
|
||||
id through the upload session. `QuotaCommitGateway` therefore settles the oldest live reservation in
|
||||
the file's namespace.
|
||||
|
||||
Which row closes does not change any quota decision: enforcement sums reserved and committed bytes
|
||||
per scope and never reads an individual row. Concurrent uploads of different sizes can leave the
|
||||
reserved total transiently high or low, and it converges as each settles. Durable usage with no live
|
||||
reservation behind it — an upload that outlived its TTL — is still recorded, because a ledger that
|
||||
silently under-counts is worse than one that is briefly imprecise.
|
||||
|
||||
Pinned by `PostgreSqlFileserverReclamationIntegrationTest`.
|
||||
|
||||
### 10. Zero copy is a channel transfer, not a file handoff
|
||||
|
||||
Task 22 asks for zero copy on local files; §19 forbids a `Path` leaving the storage adapter, and §5
|
||||
of the plan forbids adding `Path` to the content store. WebFlux's zero-copy API takes a `Path`, so
|
||||
that route is closed.
|
||||
|
||||
The servlet path takes the other one: `ZeroCopyDownloadGateway` receives a `WritableByteChannel` from
|
||||
the transport and the storage adapter performs `FileChannel.transferTo` into it. That is a genuine
|
||||
kernel-level transfer with no filesystem concept leaving storage. The reactive path continues to
|
||||
stream with bounded demand.
|
||||
|
||||
Zero copy is an optimization with no observable difference: when storage declines, the response is
|
||||
streamed and is byte-identical.
|
||||
|
||||
Pinned by `LocalStorageGatewayContractTest` and `ZeroCopyEligibilityTest`.
|
||||
|
||||
### 11. The Fileserver JPA stores are gated on the capability switch
|
||||
|
||||
The metadata store, session store, quota service, queues, ledger, and staging locator carry
|
||||
`@ConditionalOnProperty(app.fileserver-platform.enabled)` even though the rest of
|
||||
`adapter:outbound:persistence-jpa` is unconditional.
|
||||
|
||||
Without the gate, every composition root that includes the persistence module built these beans —
|
||||
including `sample-portfolio`, which has no Fileserver — and each of them needs collaborators only
|
||||
the Fileserver configuration provides. That is the same rule the design states for the transport
|
||||
surface, applied to persistence: no surface appears merely because the dependency is present.
|
||||
|
||||
`FileStateMachine` is bound alongside them, in `FileserverStorageConfiguration`. It had no
|
||||
production binding at all before, which made the metadata store unconstructible in any
|
||||
component-scanned context.
|
||||
|
||||
Pinned by `SampleApplicationContextTest` (the capability off) and
|
||||
`FileserverRuntimeAssemblyTest` (the capability on).
|
||||
|
||||
### 12. Transaction boundaries are owned by the application services, and are deliberately narrow
|
||||
|
||||
The design does not say where a transaction begins. The repository does:
|
||||
`adapter:outbound:persistence-jpa` forbids a repository adapter from owning a `@Transactional`
|
||||
boundary, and `application-core` owns them through `TransactionPort`. The Fileserver follows that
|
||||
rule — every `Jpa*` store here declares no `@Transactional` of its own.
|
||||
|
||||
What is specific to this capability is how narrow the boundaries are. A boundary covers a contiguous
|
||||
run of metadata writes and **stops before every storage call**, because a filesystem operation
|
||||
inside a database transaction would hold a connection for the length of a byte transfer. The upload
|
||||
path therefore has three boundaries, not one: acquire the lease, transfer the bytes, commit the
|
||||
offset.
|
||||
|
||||
Where several stores must agree, they share one boundary:
|
||||
|
||||
| Unit | Why it is one boundary |
|
||||
| --- | --- |
|
||||
| reserve quota + insert record + create session | a reservation that outlived a failed insert holds capacity for a file that never existed |
|
||||
| READY transition + quota commit | a finished file whose reservation was never converted holds capacity until the reservation expires |
|
||||
| `markDeleting` + enqueue cleanup | a file that stopped being reachable with nothing queued to reclaim it is never collected |
|
||||
| content delete settlement: reclaim + retire record + close queue item | half of it leaves the item to be retried against content that no longer exists |
|
||||
|
||||
What this cannot make atomic is the storage/metadata seam itself — no database boundary could. That
|
||||
seam is exactly what the ambiguous-completion path and the reconciler exist for, and the one
|
||||
hand-written compensation that remains (staging creation failing after the records committed) is
|
||||
there for the same reason.
|
||||
|
||||
Pinned by `FileserverRoundTripContractTest` against real PostgreSQL; the application tests use
|
||||
`DirectTransactions`, which runs a boundary inline and counts it.
|
||||
|
||||
## Not implemented
|
||||
|
||||
### `AsyncContentStore`, `CapacityAwareContentStore`, `CopyCapableContentStore`, `DelegatedDownloadStore`
|
||||
|
||||
Four optional content-store SPIs are declared in `application-core` with no implementation. Each is
|
||||
an extension point for a backend this template does not ship:
|
||||
|
||||
- `AsyncContentStore` — for a backend whose native client is non-blocking. The local platform is
|
||||
blocking, and the reactive transport bridges to it on a dedicated I/O scheduler.
|
||||
- `CapacityAwareContentStore` — capacity is reported through `StorageHealthPort` and
|
||||
`StorageUsageProbe`, which the local platform implements.
|
||||
- `CopyCapableContentStore` — server-side copy is delivered by `CopyContentGateway`; the local
|
||||
platform has no cheaper primitive than a streamed copy.
|
||||
- `DelegatedDownloadStore` — delegation is delivered at the transport boundary by the nginx
|
||||
`X-Accel-Redirect` strategy, which needs no store participation.
|
||||
|
||||
`ContentStoreCapabilities` reports what the running store actually supports, so no unimplemented SPI
|
||||
is advertised as available.
|
||||
|
||||
## Known deviation from the repository's application-layer contract
|
||||
|
||||
### 5. Fileserver application services are not `CommandUseCase` / `QueryUseCase`
|
||||
|
||||
`src/application-core/CLAUDE.md` requires every inbound port implementation to extend
|
||||
`CommandUseCase` or `QueryUseCase` and to carry `@UseCaseCapability`, which declares its transaction
|
||||
mode, idempotency and repository access. The Fileserver instead exposes multi-method services —
|
||||
`UploadApplicationService`, `DownloadApplicationService`, `FileLifecycleService`,
|
||||
`FileserverAdminService` and their `Default*` implementations.
|
||||
|
||||
This is a real deviation, not an oversight, and it is unenforced: the ArchUnit rules
|
||||
`inbound_port_implementations_end_with_use_case` and
|
||||
`inbound_port_implementations_declare_capability` only match types that implement `UseCase`, so a
|
||||
service that never does is silently exempt. The capability contract that every other feature in
|
||||
this repository declares is therefore absent here.
|
||||
|
||||
Two things follow from it. The transaction mode of each operation is expressed only by which
|
||||
`TransactionPort` method the body happens to call, rather than declared and checked. And the
|
||||
application layer holds transport policy it would not hold if each operation were a use case with
|
||||
its own command: HTTP status codes on `FileserverErrorCode`, `Range` and conditional-request
|
||||
parsing in `api.transfer`, and `Content-Disposition` construction.
|
||||
|
||||
The status mapping in particular is a deliberate trade rather than an accident. It lives in
|
||||
`application-core` so the servlet transport, the reactive transport and the Nginx delegation path
|
||||
cannot answer the same failure with three different statuses. Moving it to the transport layer
|
||||
resolves the layering complaint and reintroduces exactly that drift, which is why this is an
|
||||
architecture decision rather than a cleanup.
|
||||
|
||||
**Status: open, deliberately unresolved in this change set.** Closing it means roughly thirty
|
||||
command/query use cases, a decision about where the shared status vocabulary lives, and a change to
|
||||
the ArchUnit rules so a service that bypasses the contract fails the build instead of being exempt
|
||||
from it. That belongs in its own ADR with its own review, and doing it inside a correctness patch
|
||||
would mix a large mechanical refactor into changes that need to be readable.
|
||||
|
||||
Nothing here is pinned by a test, because the deviation is the absence of a constraint. The next
|
||||
step is the ADR, not another test.
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
# Fileserver HTTP contract
|
||||
|
||||
Every public endpoint is listed here. `FileserverDocumentationCoverageTest` scans the controllers
|
||||
and fails if one is missing, so this file cannot silently fall behind the code.
|
||||
|
||||
## Public endpoints
|
||||
|
||||
| Method | Path | Success | Notes |
|
||||
|---|---|---|---|
|
||||
| POST | `/v1/files` | `201` READY, `202` VERIFYING | multipart single upload |
|
||||
| POST | `/v1/files:raw` | `201`, `202` | the whole request body is the file |
|
||||
| POST | `/v1/files:batch` | `200` | ordered per-part results; explicitly non-atomic |
|
||||
| GET | `/v1/files/{fileId}` | `200` | public metadata; never a content key or path |
|
||||
| GET | `/v1/files/{fileId}/content` | `200`, `206`, `304` | download |
|
||||
| HEAD | `/v1/files/{fileId}/content` | `200`, `304` | identical headers, no body |
|
||||
| DELETE | `/v1/files/{fileId}` | `202`, `204` | logical delete first |
|
||||
| POST | `/v1/files/{fileId}:copy` | `202` | create-only target |
|
||||
| POST | `/v1/files/{fileId}:move` | `200` | logical namespace change only |
|
||||
| OPTIONS | `/v1/uploads` | `204` | tus capability discovery |
|
||||
| POST | `/v1/uploads` | `201` | tus creation |
|
||||
| HEAD | `/v1/uploads/{uploadId}` | `204` | tus offset |
|
||||
| PATCH | `/v1/uploads/{uploadId}` | `204` | tus append |
|
||||
| DELETE | `/v1/uploads/{uploadId}` | `204` | tus termination |
|
||||
| POST | `/v1/experimental/draft12/uploads` | `201` | Experimental; off by default |
|
||||
| PATCH | `/v1/experimental/draft12/uploads/{uploadId}` | `204` | Experimental; off by default |
|
||||
|
||||
## Management endpoints
|
||||
|
||||
Reachable only where both `app.fileserver-platform.enabled=true` and
|
||||
`app.fileserver-platform.admin.enabled=true`, gated at the servlet chain on
|
||||
`app.fileserver-platform.security.admin-roles`, and intended for a management
|
||||
port rather than the public one.
|
||||
|
||||
| Method | Path |
|
||||
|---|---|
|
||||
| GET | `/internal/fileserver/storage-health` |
|
||||
| GET | `/internal/fileserver/capabilities` |
|
||||
| GET | `/internal/fileserver/orphans` |
|
||||
| POST | `/internal/fileserver/orphans:reconcile` |
|
||||
| POST | `/internal/fileserver/files/{fileId}:reverify` |
|
||||
| POST | `/internal/fileserver/files/{fileId}:force-delete` |
|
||||
| GET | `/internal/fileserver/uploads/incomplete` |
|
||||
| POST | `/internal/fileserver/uploads:cleanup` |
|
||||
|
||||
## Status codes
|
||||
|
||||
| Status | Condition |
|
||||
|---:|---|
|
||||
| `200` | metadata, full GET, batch result, move |
|
||||
| `201` | file or upload created |
|
||||
| `202` | verification or physical cleanup deferred |
|
||||
| `204` | append, cancel, bodyless update |
|
||||
| `206` | satisfiable Range |
|
||||
| `304` | validator matched on GET or HEAD |
|
||||
| `400` | malformed header or header combination |
|
||||
| `401` | unauthenticated |
|
||||
| `403` / `404` | denied, or hidden under the existence-hiding profile |
|
||||
| `409` | state, offset, or lease conflict |
|
||||
| `410` | expired upload resource |
|
||||
| `411` | `require-content-length` profile with no length |
|
||||
| `412` | precondition failed |
|
||||
| `413` | size or quota policy violation |
|
||||
| `415` | upload media type not accepted |
|
||||
| `416` | unsatisfiable Range; carries the real length |
|
||||
| `422` | digest, signature, or scanner rejection |
|
||||
| `429` | transfer admission or rate limit |
|
||||
| `503` | storage or scanner unavailable |
|
||||
| `504` | downstream timeout |
|
||||
| `507` | out of storage capacity |
|
||||
|
||||
## Failure body
|
||||
|
||||
Every failure answers `application/problem+json` with a stable code and its URN:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "urn:fileserver:problem:upload-offset-mismatch",
|
||||
"title": "Upload offset mismatch",
|
||||
"status": 409,
|
||||
"code": "UPLOAD_OFFSET_MISMATCH",
|
||||
"retryable": true,
|
||||
"ambiguous": false,
|
||||
"reconciliationRequired": false,
|
||||
"traceId": "..."
|
||||
}
|
||||
```
|
||||
|
||||
The server-side exception message never appears. `ambiguous` is the field a client must read before
|
||||
retrying: an ambiguous failure may already have taken effect.
|
||||
|
||||
## Header contract
|
||||
|
||||
| Header | Contract |
|
||||
|---|---|
|
||||
| `Content-Type` | client value is a claim; the verified type is stored separately |
|
||||
| `Content-Disposition` | `attachment` by default; scriptable types are never inline |
|
||||
| `Accept-Ranges` | `bytes` |
|
||||
| `Range` | single range by default; multi-range only under an explicit budget |
|
||||
| `Content-Range` | actual range on `206`; the unsatisfied form on `416` |
|
||||
| `ETag` | strong validator derived from the SHA-256 |
|
||||
| `Last-Modified` | metadata publication instant, never a filesystem timestamp |
|
||||
| `Cache-Control` | `private, no-store` by default |
|
||||
| `X-Content-Type-Options` | always `nosniff` on a download |
|
||||
| `Retry-After` | on retryable `409`, `429`, `503`, and `504` |
|
||||
| `X-Accel-Redirect` | internal only; never forwarded to a client |
|
||||
@@ -0,0 +1,128 @@
|
||||
# Fileserver runbooks
|
||||
|
||||
Each runbook names the exact metric that fires it and the exact command that resolves it. A runbook
|
||||
whose trigger is "someone noticed" is not actionable, so every one below starts from a signal.
|
||||
|
||||
## Storage full
|
||||
|
||||
**Signal** — `fileserver.quota{result="rejected"}` rising, or `507` responses appearing.
|
||||
|
||||
Storage capacity is exhausted or the high-water guard tripped. Uploads are rejected before any bytes
|
||||
are written, so nothing is corrupt; the system is refusing work it cannot complete.
|
||||
|
||||
```bash
|
||||
curl -s $ADMIN/internal/fileserver/storage-health | jq '.usedFraction, .usableBytes'
|
||||
curl -s -X POST "$ADMIN/internal/fileserver/uploads:cleanup?maxItems=500&maxBytes=10737418240"
|
||||
curl -s "$ADMIN/internal/fileserver/orphans?limit=200" | jq '[.[].sizeBytes] | add'
|
||||
```
|
||||
|
||||
Drain the cleanup backlog first — it reclaims space the system already knows is dead. Only then
|
||||
consider an orphan reconcile, and start with a dry run.
|
||||
|
||||
## Orphan growth
|
||||
|
||||
**Signal** — `fileserver.cleanup{result="skipped"}` climbing, or the orphan scan returning more
|
||||
objects each run.
|
||||
|
||||
Physical objects exist with no metadata record pointing at them. This is not immediately dangerous —
|
||||
nothing serves them — but it consumes capacity indefinitely.
|
||||
|
||||
```bash
|
||||
# Always look first. A reconcile without dryRun=false is a plan, not an action.
|
||||
curl -s -X POST "$ADMIN/internal/fileserver/orphans:reconcile" \
|
||||
-H 'content-type: application/json' -d '{"limit":100}' | jq '.candidates'
|
||||
|
||||
# Apply only the fingerprints you were just shown.
|
||||
curl -s -X POST "$ADMIN/internal/fileserver/orphans:reconcile" \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"dryRun":false,"limit":100,"maxBytes":1073741824,
|
||||
"expectedFingerprints":["<from the dry run>"],"reasonCode":"ORPHAN_GROWTH_RUNBOOK"}'
|
||||
```
|
||||
|
||||
Echoing the fingerprints is the safety property: an object that changed between the scan and the
|
||||
apply is skipped rather than deleted.
|
||||
|
||||
## Verification backlog
|
||||
|
||||
**Signal** — `fileserver.verification.queue{age_bucket="old"}` non-zero, or files sitting in
|
||||
VERIFYING.
|
||||
|
||||
A verifier is slow or unavailable. Files stay non-public, which is the correct failure direction: a
|
||||
`RETRY` verdict never becomes an `ACCEPT`.
|
||||
|
||||
```bash
|
||||
curl -s $ADMIN/internal/fileserver/capabilities | jq '.storageType'
|
||||
# Once the verifier is healthy, quarantined files can be re-examined individually.
|
||||
curl -s -X POST "$ADMIN/internal/fileserver/files/$FILE_ID:reverify"
|
||||
```
|
||||
|
||||
Do not clear the backlog by disabling verification. A file that reached READY without an accepting
|
||||
verdict cannot be distinguished later from one that was verified.
|
||||
|
||||
## NFS ambiguity
|
||||
|
||||
**Signal** — problem documents carrying `"ambiguous": true`, or
|
||||
`fileserver.transfer.interruption{reason="stale_handle"}`.
|
||||
|
||||
An operation's outcome could not be determined: the response was lost after the write or rename may
|
||||
have landed. These are never retried automatically.
|
||||
|
||||
```bash
|
||||
# The recovery queue holds the files awaiting a decision.
|
||||
curl -s "$ADMIN/internal/fileserver/uploads/incomplete?limit=100" | jq
|
||||
```
|
||||
|
||||
Reconciliation compares the physical size and digest against the record and only confirms READY when
|
||||
all four of key, size, digest, and version agree. Anything short of that is reported, never guessed.
|
||||
|
||||
## PVC remount
|
||||
|
||||
**Signal** — startup failure naming "atomic move", "same file store", or "not writable".
|
||||
|
||||
The volume was remounted somewhere the probe can no longer prove a required capability. The
|
||||
application refuses traffic rather than serving from storage it cannot publish to atomically.
|
||||
|
||||
```bash
|
||||
kubectl apply -f infra/fileserver/kubernetes/pvc-certification-job.yaml
|
||||
kubectl logs job/fileserver-pvc-certification
|
||||
```
|
||||
|
||||
Compare the printed tuple with the certified one in `docs/fileserver/storage-certification.md`. A
|
||||
mismatch in CSI driver, StorageClass, access mode, or mount options is the cause; the certification
|
||||
does not carry across it.
|
||||
|
||||
## Nginx delegation failure
|
||||
|
||||
**Signal** — `fileserver.download.delegation{delegated="true"}` with client-visible `404`s.
|
||||
|
||||
The internal location is misconfigured, so the proxy cannot resolve the redirect it was handed.
|
||||
|
||||
```bash
|
||||
# The internal prefix must resolve to the content root and must be marked `internal`.
|
||||
grep -A5 '__files' infra/fileserver/nginx/nginx.conf
|
||||
curl -s $ADMIN/internal/fileserver/capabilities | jq '.capabilities.delegatedDownload'
|
||||
```
|
||||
|
||||
Turning delegation off is a safe immediate mitigation: the application serves the transfer itself,
|
||||
slower but correct.
|
||||
|
||||
```bash
|
||||
app.fileserver-platform.nginx.enabled=false
|
||||
```
|
||||
|
||||
## Cleanup backlog
|
||||
|
||||
**Signal** — `fileserver.cleanup{result="deferred"}` rising, or reclaimed bytes flat while deletes
|
||||
continue.
|
||||
|
||||
Items are being deferred faster than they drain. The usual cause is an active writer lease still
|
||||
holding staging objects, which is correct behaviour, not a fault.
|
||||
|
||||
```bash
|
||||
curl -s "$ADMIN/internal/fileserver/uploads/incomplete?limit=100" \
|
||||
| jq '[.[] | select(.leaseUntil != null)] | length'
|
||||
curl -s -X POST "$ADMIN/internal/fileserver/uploads:cleanup?maxItems=500&maxBytes=10737418240"
|
||||
```
|
||||
|
||||
If the deferrals are all `ACTIVE_WRITER_LEASE`, the backlog resolves itself as those uploads expire.
|
||||
Never delete staging content to clear a backlog: an upload that is mid-flight will corrupt.
|
||||
@@ -0,0 +1,74 @@
|
||||
# Fileserver security model
|
||||
|
||||
## The rule everything else follows
|
||||
|
||||
Uploaded content is attacker-controlled. Every guard below exists because some part of the request
|
||||
— the filename, the declared media type, the range, the offset — is a value the caller chose.
|
||||
|
||||
## Path safety
|
||||
|
||||
A client value never becomes a path. The physical key is server-generated, and `ContentKey`'s
|
||||
character class excludes `.` entirely, so no traversal or extension-shaped segment survives
|
||||
validation. `DefaultPhysicalPathResolver` is the only place an identifier becomes a `Path`, and it
|
||||
normalizes and re-checks containment after construction rather than trusting the input.
|
||||
|
||||
Symlink refusal happens at open time, not only at construction. A parent directory can be replaced
|
||||
between the two, so a check that ran only at path-building time would be a race, not a guard.
|
||||
|
||||
## Filename handling
|
||||
|
||||
`OriginalFilenamePolicy` strips path separators, NUL, quoting characters, and the colon — the last
|
||||
because on Windows it opens both a drive reference and an NTFS alternate data stream, so a name that
|
||||
keeps it is still path-shaped after the slashes are gone. Control characters and bidirectional
|
||||
overrides are removed, dot runs collapsed, reserved device names guarded, and the result is bounded
|
||||
in UTF-8 bytes.
|
||||
|
||||
The sanitized name is display data. It is never used to build a key, and it reaches a header only
|
||||
through `ContentDispositionFactory`, which restricts the ASCII form and percent-encodes the UTF-8
|
||||
form.
|
||||
|
||||
## Content type
|
||||
|
||||
The client's `Content-Type` is stored as a claim. The verified type comes from the verification
|
||||
pipeline, and only the verified type is served. A claimed type that contradicts the content is
|
||||
quarantined rather than corrected.
|
||||
|
||||
Scriptable types are never served inline, whatever the caller asked for: serving stored HTML or SVG
|
||||
inline from an upload origin is a stored cross-site scripting primitive. Every download also carries
|
||||
`X-Content-Type-Options: nosniff`.
|
||||
|
||||
## Verification precedence
|
||||
|
||||
`REJECT > QUARANTINE > RETRY > ACCEPT`. A verifier that times out or throws is `RETRY`, never a
|
||||
silent pass, and an empty verifier chain answers `RETRY` rather than accepting. A file becomes
|
||||
publicly readable only after an `ACCEPT`.
|
||||
|
||||
## Range safety
|
||||
|
||||
The range budget is enforced before content is opened, so a request naming many ranges is rejected
|
||||
without amplifying into storage work. An unsatisfiable range answers `416` with the real length and
|
||||
opens nothing.
|
||||
|
||||
## Authorization
|
||||
|
||||
Every public operation calls the injected `FileAccessPolicy` before any quota reservation or storage
|
||||
mutation, so a denial leaves no record, no reservation, and no staging object. Startup refuses to
|
||||
run a production profile with an allow-all policy.
|
||||
|
||||
## Delegation
|
||||
|
||||
`X-Accel-Redirect` is emitted only after authorization and the READY gate, and only for a full,
|
||||
unconditional response. The internal prefix must be an `internal` Nginx location; the front proxy
|
||||
also strips any client-supplied delegation header so a caller cannot name an internal object.
|
||||
|
||||
## Telemetry
|
||||
|
||||
No metric label, span attribute, or audit record carries a file id, upload id, filename, path, or
|
||||
user id. Where correlation is needed the value is a keyed HMAC fingerprint — keyed because the
|
||||
identifier space is enumerable and an unkeyed digest of it is reversible by brute force.
|
||||
|
||||
## Ambiguous failures
|
||||
|
||||
A failure whose operation may already have taken effect is reported as ambiguous and is never
|
||||
retryable. On a network filesystem a lost response is indistinguishable from a rejection at the
|
||||
socket level, so anything not provably safe is treated as ambiguous and sent to reconciliation.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Storage certification
|
||||
|
||||
## Why a certification is per-volume
|
||||
|
||||
Atomic rename, same-file-store guarantees, and symlink refusal are properties of a specific
|
||||
filesystem behind a specific mount — not of "Kubernetes" or "a PVC". Change the CSI driver, the
|
||||
StorageClass, the access mode, the backend, or the mount options and any of them can differ. A
|
||||
certification that does not name all five is not transferable.
|
||||
|
||||
## What is certified
|
||||
|
||||
| Property | Why it matters |
|
||||
|---|---|
|
||||
| Same file store for staging and content | A rename across stores is a copy, so publication stops being atomic. |
|
||||
| Atomic rename | The publish path's default strategy. |
|
||||
| Atomic create (`O_EXCL`) | Makes a publish create-only rather than a silent overwrite. |
|
||||
| Symlink refusal | Stops a replaced parent from redirecting a write outside the root. |
|
||||
| Ranged read | The download contract depends on it. |
|
||||
|
||||
## Running the certification
|
||||
|
||||
```bash
|
||||
kubectl apply -f infra/fileserver/kubernetes/pvc-certification-job.yaml
|
||||
kubectl logs job/fileserver-pvc-certification
|
||||
```
|
||||
|
||||
The job writes a machine-readable result to the claim itself, carrying the full tuple:
|
||||
|
||||
```json
|
||||
{
|
||||
"kubernetesVersion": "...",
|
||||
"csiDriver": "...",
|
||||
"storageClass": "...",
|
||||
"accessMode": "ReadWriteOnce",
|
||||
"backend": "ext2/ext3",
|
||||
"mountOptions": "rw,relatime",
|
||||
"atomicMove": true,
|
||||
"sameFileStore": true,
|
||||
"atomicCreate": true
|
||||
}
|
||||
```
|
||||
|
||||
The job fails closed: a volume whose staging and content areas are on different stores is not
|
||||
certified, because its publish would silently degrade to a copy.
|
||||
|
||||
## Network filesystems
|
||||
|
||||
```bash
|
||||
docker compose -f infra/fileserver/nfs/compose.yml up -d
|
||||
FILESERVER_NFS_TESTS=true ./gradlew :adapter:outbound:fileserver:test
|
||||
```
|
||||
|
||||
The mount is `hard`, deliberately. A `soft` mount converts a slow server into a short write, which
|
||||
is exactly the corruption this design refuses to accept.
|
||||
|
||||
## Startup enforcement
|
||||
|
||||
`FileserverStartupValidator` re-runs the probe at boot and refuses to accept traffic when a required
|
||||
capability is missing — `ATOMIC_MOVE_REQUIRED` on a filesystem that cannot prove an atomic move
|
||||
fails closed rather than degrading silently.
|
||||
|
||||
## Adding a new store
|
||||
|
||||
Extend `ContentStoreContract` and pass it. A prose claim of compatibility is not accepted; the
|
||||
contract is executable precisely so a future object-storage adapter has to demonstrate the same
|
||||
offset, digest, and create-only behaviour the local store does.
|
||||
@@ -0,0 +1,83 @@
|
||||
# Fileserver support matrix
|
||||
|
||||
A support level here is a claim about evidence, not about intent. Every row names the CI job that
|
||||
produces that evidence; `FileserverDocumentationCoverageTest` fails the build if a row names a job
|
||||
that does not exist, so a level can never outlive the test that justified it.
|
||||
|
||||
## Levels
|
||||
|
||||
| Level | What it means |
|
||||
|---|---|
|
||||
| Stable | Certified on every pull request. Contract changes are breaking changes. |
|
||||
| Beta | Certified nightly. The contract may still change with a deprecation notice. |
|
||||
| Limited | Certified on the release gate only, under stated constraints. |
|
||||
| Compatibility | Accepted but not optimized; known caveats are listed inline. |
|
||||
| Experimental | Off by default, unratified upstream, may change without notice. |
|
||||
|
||||
## Runtime profiles
|
||||
|
||||
| Profile | Level | CI job |
|
||||
|---|---|---|
|
||||
| Local filesystem (ext4) content store | Stable | `fileserver-local-ext4-contract` |
|
||||
| Spring MVC transport (raw, multipart, batch, download) | Stable | `fileserver-http-contract` |
|
||||
| Spring WebFlux transport | Experimental | `fileserver-http-contract` |
|
||||
| Path, filename, range, and problem-detail hardening | Stable | `fileserver-security-suite` |
|
||||
| Bounded-memory transfer | Stable | `fileserver-bounded-memory` |
|
||||
| Application and architecture invariants | Stable | `fileserver-unit-and-architecture` |
|
||||
| Runtime assembly (the capability starts with the flag on) | Stable | `fileserver-unit-and-architecture` |
|
||||
| tus 1.0 resumable uploads | Stable | `fileserver-http-contract` |
|
||||
| Crash-recovery matrix | Beta | `fileserver-process-kill-matrix` |
|
||||
| NFSv4 ambiguity handling | Beta | `fileserver-nfs-ambiguity` |
|
||||
| Large-file and slow-client performance | Beta | `fileserver-large-file-performance` |
|
||||
| Multi-instance writer lease | Beta | `fileserver-multi-instance-lease` |
|
||||
| Kubernetes ReadWriteOnce PVC | Limited | `fileserver-pvc-certification` (manifest checks in CI; cluster run is operator-driven) |
|
||||
| Nginx `X-Accel-Redirect` delegation | Limited | `fileserver-http-contract` |
|
||||
| Telemetry sensitive-data suppression | Stable | `fileserver-sensitive-telemetry-scan` |
|
||||
| Documentation and support-claim coverage | Stable | `fileserver-documentation-gate` |
|
||||
| Full release verification | Stable | `fileserver-full-verification` |
|
||||
| HTTP resumable uploads draft-12 | Experimental | `fileserver-http-contract` |
|
||||
|
||||
### Why WebFlux is Experimental, not Stable
|
||||
|
||||
The reactive router, handlers and readers are now wired: `FileserverReactiveConfiguration`
|
||||
contributes the scheduler, the handlers and a `RouterFunction` bean under
|
||||
`@ConditionalOnWebApplication(type = REACTIVE)` plus the platform master switch. Previously nothing
|
||||
built them at all, so "Stable" described the source tree rather than a running server.
|
||||
|
||||
It stays `Experimental` because the shipped composition cannot select it. `adapter:inbound:web`
|
||||
also puts `DispatcherServlet` on the classpath — deliberately, so adding `spring-webflux` does not
|
||||
drag a second embedded server onto the runtime — and Boot's application-type deduction therefore
|
||||
resolves SERVLET. A fork that removes the servlet stack and adds a reactive server gets working
|
||||
routes without editing any Fileserver code; the shipped template does not exercise that path.
|
||||
|
||||
Raising it to Stable requires a contract job that drives the routes over a running reactive server
|
||||
rather than through direct construction.
|
||||
|
||||
## Explicitly not claimed
|
||||
|
||||
These have no job, and therefore no claim:
|
||||
|
||||
- An automated Kubernetes cluster result. `fileserver-pvc-certification` validates the manifest on
|
||||
every release and applies it only when a release cluster is configured; without one it warns and
|
||||
records that nothing was certified. The cluster tuple is produced by an operator and read from
|
||||
[storage-certification.md](storage-certification.md).
|
||||
|
||||
- Kubernetes ReadWriteMany PVC. Concurrent writers across nodes are not certified.
|
||||
- Windows NTFS as a production storage root. The filename policy strips the characters NTFS
|
||||
reserves, but no job certifies the publish path there.
|
||||
- Object storage as a content store. The contract exists (`ContentStoreContract`) but no adapter
|
||||
implements it yet.
|
||||
- Server-side malware scanning. The verification pipeline has the port and the verdict precedence;
|
||||
no scanner is shipped.
|
||||
|
||||
## Where the rest is written down
|
||||
|
||||
- [configuration.md](configuration.md) — every `app.fileserver-platform.*` key, its default, and the
|
||||
conditions that fail startup rather than degrade.
|
||||
- [design-deviations.md](design-deviations.md) — where the implementation departs from the frozen
|
||||
design, why, and the test that pins each decision.
|
||||
- [http-contract.md](http-contract.md) — the wire contract.
|
||||
- [security.md](security.md) — the threat model and what enforces each control.
|
||||
- [operations.md](operations.md) — runbooks, each starting from a metric.
|
||||
- [storage-certification.md](storage-certification.md) — how a volume is certified.
|
||||
- [upgrade-guide.md](upgrade-guide.md) — what changes between versions.
|
||||
@@ -0,0 +1,82 @@
|
||||
# Fileserver upgrade guide
|
||||
|
||||
## Enabling the capability
|
||||
|
||||
The Fileserver ships off. Nothing is registered — no endpoint, no thread pool, no metric — until it
|
||||
is enabled explicitly.
|
||||
|
||||
```yaml
|
||||
ca-skeleton:
|
||||
fileserver:
|
||||
enabled: true
|
||||
instance-id: ${HOSTNAME}
|
||||
default-namespace: default
|
||||
observability:
|
||||
fingerprint-key: ${FILESERVER_FINGERPRINT_KEY}
|
||||
```
|
||||
|
||||
`instance-id` must be unique per instance: it is the writer-lease owner, and two nodes sharing one
|
||||
would both believe they hold the same lease.
|
||||
|
||||
`fingerprint-key` is required and has no default. Startup fails without it rather than falling back
|
||||
to an unkeyed digest, which would be reversible for an enumerable identifier space.
|
||||
|
||||
## Optional surfaces
|
||||
|
||||
Each is a separate switch, and each defaults to off:
|
||||
|
||||
```yaml
|
||||
ca-skeleton:
|
||||
fileserver:
|
||||
admin:
|
||||
enabled: false # management plane; intended for a management port
|
||||
tus:
|
||||
enabled: false # tus 1.0 Stable
|
||||
httpbis-draft12:
|
||||
enabled: false # Experimental; unratified, may change without notice
|
||||
nginx:
|
||||
enabled: false # front-proxy delegation; needs a validated internal location
|
||||
```
|
||||
|
||||
## Database schema
|
||||
|
||||
The metadata schema is installed as a capability migration and starts inactive:
|
||||
|
||||
```
|
||||
V1__create_fileserver_metadata.sql → capability_schema_registry: jpa-fileserver-metadata-v1
|
||||
```
|
||||
|
||||
Activate it deliberately. Enabling the capability without an activated schema fails at startup
|
||||
rather than at the first upload.
|
||||
|
||||
## Choosing a publish mode
|
||||
|
||||
| Mode | When |
|
||||
|---|---|
|
||||
| `atomic-move-preferred` | Default. Uses an atomic rename when the probe proves one, else a metadata pointer. |
|
||||
| `atomic-move-required` | Fail closed. Refuses to start on storage that cannot prove an atomic move. |
|
||||
| `metadata-pointer` | For storage without atomic rename; publication is the metadata commit. |
|
||||
|
||||
Pick `atomic-move-required` when the storage is certified and you want a misconfiguration to surface
|
||||
at boot rather than at publish time.
|
||||
|
||||
## Behaviour that will surprise you
|
||||
|
||||
- **A delete answers `202`, not `204`, when content still exists.** The file is already unreadable;
|
||||
the physical reclaim is deferred. Treating `202` as a failure will produce spurious retries.
|
||||
- **A batch upload answers `200` even when parts failed.** The batch is explicitly non-atomic, and a
|
||||
single status could not report a partial outcome honestly. Read `results[].problem`.
|
||||
- **An ambiguous failure must not be retried.** Check `"ambiguous": true` in the problem document.
|
||||
- **`If-Match` takes the strong ETag, not a version number.** A client can only assert about the
|
||||
representation it was actually served.
|
||||
- **Inline rendering is refused for scriptable types** even when the caller asks for it.
|
||||
|
||||
## Verifying an upgrade
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew verifyCleanArchitectureDependencies --console=plain
|
||||
./gradlew :application-core:check :adapter:inbound:web:check \
|
||||
:adapter:outbound:fileserver:check --console=plain
|
||||
./gradlew :app-bootstrap:test --tests '*Fileserver*' --console=plain
|
||||
```
|
||||
@@ -0,0 +1,228 @@
|
||||
# HTTP Client Platform — Configuration Reference
|
||||
|
||||
Every outbound call resolves exactly one **Named Client Profile**. The whole capability lives under
|
||||
the `app.httpclient` prefix: profiles under `app.httpclient.clients[N]`, Dynamic Target policies
|
||||
under `app.httpclient.dynamic-targets[N]`.
|
||||
|
||||
Design §30.1 forbids a production profile from inheriting large framework defaults. Anything a
|
||||
production deployment must decide has either no default or an unusable one, and
|
||||
`HttpClientStartupValidator` fails the context rather than guessing.
|
||||
|
||||
## The master switch
|
||||
|
||||
| Property | Type | Default | Environment |
|
||||
|---|---|---|---|
|
||||
| `app.httpclient.enabled` | boolean | `false` | `APP_HTTPCLIENT_ENABLED` |
|
||||
|
||||
Off is the shipped state and it is a structural one. `HttpClientPlatformAutoConfiguration` lives in
|
||||
a package the composition root's component scan excludes, so while the switch is absent or false the
|
||||
class is never processed and neither is anything it imports: no property is bound, and no transport
|
||||
provider, connection pool, TLS context, credential, thread, gateway or actuator endpoint exists. A
|
||||
malformed HTTP client setting cannot fail the startup of a deployment that never wanted outbound
|
||||
HTTP.
|
||||
|
||||
Anything that is not exactly `true` — `yes`, `1`, blank — leaves the platform off. Turning it on
|
||||
with no client declared is a startup failure carrying `HTTPCLIENT_ACTIVE_WITHOUT_CLIENTS`: a
|
||||
platform with nothing to call still holds transport providers and gateways no caller can reach.
|
||||
|
||||
## Declaring clients from the environment
|
||||
|
||||
Clients are an indexed list carrying their own `name`, not a map keyed by name. A map key becomes a
|
||||
segment of the environment variable and the relaxed binder normalises it, so `payment-api` and
|
||||
`payment_api` would arrive as one entry with nothing said about the one that was lost. Both a
|
||||
duplicate name and a name that collides once normalised fail startup.
|
||||
|
||||
```dotenv
|
||||
APP_HTTPCLIENT_ENABLED=true
|
||||
|
||||
APP_HTTPCLIENT_CLIENTS_0_NAME=payment
|
||||
APP_HTTPCLIENT_CLIENTS_0_BASE_URL=https://payment.example
|
||||
APP_HTTPCLIENT_CLIENTS_0_ALLOWED_HOSTS_0=payment.example
|
||||
APP_HTTPCLIENT_CLIENTS_0_ALLOWED_PORTS_0=443
|
||||
APP_HTTPCLIENT_CLIENTS_0_REQUEST_MAX_BODY_BYTES=1048576
|
||||
APP_HTTPCLIENT_CLIENTS_0_TLS_PROFILE_ID=payment
|
||||
|
||||
APP_HTTPCLIENT_DYNAMIC_TARGETS_0_NAME=webhook
|
||||
APP_HTTPCLIENT_DYNAMIC_TARGETS_0_ALLOWED_SCHEMES_0=https
|
||||
```
|
||||
|
||||
`docs/httpclient/env-fields.yaml` is the registry of accepted variable names. It is
|
||||
derived from the settings record and held to it in both directions, and the platform refuses to
|
||||
start on an `APP_HTTPCLIENT_` variable that is not in it — so
|
||||
`APP_HTTPCLIENT_CLIENTS_0_TIMEUOT_TOTAL_CALL` fails startup instead of silently leaving the client
|
||||
on its default budget. Unknown keys supplied through a configuration file rather than the
|
||||
environment are refused by strict binding for the same reason.
|
||||
|
||||
Only `APP_HTTPCLIENT_ENABLED` appears in `src/.env` and `docs/registries/env-keys.yaml`. It is the
|
||||
one key with a deployment-independent value; templating an indexed client in `application.yml` would
|
||||
materialise a nameless client in every deployment, which the aggregate validation refuses.
|
||||
|
||||
## `app.httpclient.clients[N]`
|
||||
|
||||
| Property | Type | Default | Notes |
|
||||
|---|---|---|---|
|
||||
| `name` | string | — | Required, unique, and distinct from every other name once normalised for the environment |
|
||||
| `mode` | `TRUSTED` \| `DYNAMIC` | `TRUSTED` | A dynamic profile may not carry a default credential |
|
||||
| `base-url` | URI | — | Required for a trusted profile; no userinfo, no query |
|
||||
| `allowed-hosts` | list | empty | Required in production |
|
||||
| `allowed-ports` | list | empty | Compared against the effective port |
|
||||
| `api` | `REST_CLIENT` \| `WEB_CLIENT` | `REST_CLIENT` | Decides blocking or reactive runtime |
|
||||
| `transport` | `APACHE` \| `JDK` \| `REACTOR_NETTY` \| `JETTY` \| `SIMPLE` | `APACHE` | `SIMPLE` is rejected in production |
|
||||
| `protocols` | list | `HTTP_1_1` | The default transport is Apache, whose classic client is HTTP/1.1 only; a profile that wants HTTP/2 declares it together with a transport that can deliver it. `HTTP_3` requires the experimental acknowledgement |
|
||||
| `experimental-acknowledgement` | string | — | Must equal `I_ACCEPT_HTTP3_EXPERIMENTAL_SEMANTICS` |
|
||||
|
||||
### `pool`
|
||||
|
||||
| Property | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `max-total-connections` | `50` | Socket ceiling for the runtime |
|
||||
| `max-connections-per-route` | `25` | Per-upstream ceiling |
|
||||
| `max-pending-acquires` | `100` | Waiting-request memory ceiling |
|
||||
| `pending-acquire-timeout` | `200ms` | Pool or stream wait ceiling |
|
||||
| `max-idle-time` | `30s` | Idle eviction |
|
||||
| `max-life-time` | `5m` | Picks up DNS, load-balancer, and certificate changes |
|
||||
| `validate-after-inactivity` | `5s` | Stale and half-open detection |
|
||||
| `eviction-interval` | `15s` | Background cleanup |
|
||||
| `shutdown-timeout` | `5s` | Drain deadline before forced close |
|
||||
| `requires-route-pool` | `false` | Set when route-scoped limits are mandatory; the JDK transport then refuses the profile |
|
||||
| `requires-bounded-pending-queue` | `false` | Same, for a bounded pending queue |
|
||||
|
||||
### `timeout`
|
||||
|
||||
| Property | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `dns` | `300ms` | Hostname resolution |
|
||||
| `connect` | `500ms` | Socket connect |
|
||||
| `tls-handshake` | `1s` | TLS and ALPN |
|
||||
| `proxy-connect` | `500ms` | Proxy socket or CONNECT |
|
||||
| `request-write-idle` | `1s` | No progress writing the request |
|
||||
| `response-header` | `2s` | Until final response headers |
|
||||
| `read-idle` | `3s` | Between response chunks |
|
||||
| `total-call` | `4s` | The whole logical call, including retry backoff |
|
||||
| `streaming-idle` | `30s` | Silence on a long-lived stream |
|
||||
|
||||
`total-call` must not be shorter than `connect` or `response-header`; the validator emits
|
||||
`INVALID_TIMEOUT_BUDGET` otherwise.
|
||||
|
||||
### `redirect`, `request`, `response`
|
||||
|
||||
| Property | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `redirect.enabled` | `false` | Engine redirect handling is always off; the platform follows hops itself |
|
||||
| `redirect.max-hops` | `0` | Enabling redirects with zero hops is a configuration error |
|
||||
| `redirect.allow-cross-origin` | `false` | When enabled, credentials are stripped on the hop |
|
||||
| `request.max-body-bytes` | `0` | Required in production |
|
||||
| `request.compression` | `false` | |
|
||||
| `response.max-wire-bytes` | `5242880` | Bytes on the wire |
|
||||
| `response.max-decoded-bytes` | `10485760` | Bytes after decoding; hard maximum is 64 MiB |
|
||||
| `response.allowed-content-types` | JSON + problem+json | Empty means "any" |
|
||||
|
||||
### `authentication`
|
||||
|
||||
| Property | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `type` | `NONE` | One of the design §20.1 methods |
|
||||
| `registration-id` | — | Required for OAuth2 |
|
||||
| `scopes` | empty | Part of the token cache key |
|
||||
| `audience` | — | Part of the token cache key |
|
||||
| `header-name` | — | Required for `API_KEY_HEADER`; must be on the allowlist |
|
||||
| `secret-reference` | — | Resolved by the deployment's secret loader, never a literal |
|
||||
|
||||
### `retry`
|
||||
|
||||
| Property | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `policy` | `none` | Named policy for reporting |
|
||||
| `max-attempts` | `1` | Attempts, not retries |
|
||||
| `base-backoff` | `50ms` | |
|
||||
| `max-backoff` | `200ms` | |
|
||||
| `jitter` | `FULL` | `NONE` \| `FULL` \| `DECORRELATED` |
|
||||
| `retry-after` | `HONOR` | `HONOR` \| `IGNORE` \| `CAP` |
|
||||
| `budget` | — | Shared token bucket name |
|
||||
|
||||
### `tls`
|
||||
|
||||
| Property | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `profile-id` | — | Required in production; the only TLS identifier the actuator exposes |
|
||||
| `protocols` | `TLSv1.3, TLSv1.2` | Anything else is rejected |
|
||||
| `hostname-verification` | `true` | Setting it false fails startup |
|
||||
| `trust-all` | `false` | Exists only so the unsafe intent is rejectable; nothing acts on `true` |
|
||||
| `allow-plain-http` | `false` | Plaintext fallback fails startup in production |
|
||||
| `trust-material-reference` | — | Custom CA, resolved by the secret loader |
|
||||
| `key-material-reference` | — | Client certificate for mTLS |
|
||||
|
||||
### `proxy` and `observability`
|
||||
|
||||
| Property | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `proxy.enabled` | `false` | |
|
||||
| `proxy.host` / `proxy.port` / `proxy.type` | — / `0` / `HTTP` | |
|
||||
| `proxy.credential-provider` | — | Proxy authentication is separate from target authentication |
|
||||
| `proxy.connect-timeout` | `500ms` | Recorded as its own metric |
|
||||
| `proxy.import-ambient-no-proxy` | `false` | Ambient `NO_PROXY` never widens a validated profile |
|
||||
| `observability.operation-name-required` | `true` | |
|
||||
| `observability.full-url-recording` | `false` | |
|
||||
| `observability.body-logging` | `false` | |
|
||||
|
||||
## `app.httpclient.dynamic-targets[N]`
|
||||
|
||||
| Property | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `name` | — | Required, unique, and subject to the same normalisation rule as a client name |
|
||||
| `allowed-schemes` | `https` | |
|
||||
| `allowed-ports` | `443` | |
|
||||
| `allowed-host-suffixes` | empty | |
|
||||
| `allowed-hosts` | empty | Empty means "any host that survives address validation" |
|
||||
| `max-redirect-hops` | `0` | Each hop repeats the full validation flow |
|
||||
| `trace-propagation` | `false` | Off by default for dynamic targets |
|
||||
| `blocked-cidrs` | empty | Organisation-defined internal ranges |
|
||||
|
||||
## Startup violation codes
|
||||
|
||||
`TRUSTED_BASE_URL_REQUIRED`, `BASE_URL_USERINFO_FORBIDDEN`, `BASE_URL_QUERY_FORBIDDEN`,
|
||||
`PLAINTEXT_PRODUCTION_TARGET`, `ALLOWED_HOST_MISMATCH`, `ALLOWED_PORT_MISMATCH`,
|
||||
`REDIRECT_POLICY_INVALID`, `REDIRECT_CROSS_ORIGIN_CREDENTIAL_POLICY_REQUIRED`,
|
||||
`INVALID_TIMEOUT_BUDGET`, `RESPONSE_HARD_MAXIMUM_EXCEEDED`, `PRODUCTION_SIMPLE_FACTORY_FORBIDDEN`,
|
||||
`JDK_FINE_GRAINED_POOL_UNSUPPORTED`, `HTTP3_STABLE_FORBIDDEN`,
|
||||
`DYNAMIC_TARGET_TRANSPORT_UNSUPPORTED`, `DYNAMIC_DEFAULT_CREDENTIAL_FORBIDDEN`,
|
||||
`OAUTH2_REGISTRATION_REQUIRED`, `API_KEY_HEADER_NAME_REQUIRED`, `TRUST_ALL_FORBIDDEN`,
|
||||
`HOSTNAME_VERIFICATION_REQUIRED`, `PLAINTEXT_FALLBACK_FORBIDDEN`, `TLS_PROTOCOL_FORBIDDEN`,
|
||||
`RETRY_BACKOFF_REQUIRED`, `MISSING_PRODUCTION_SETTING`, `DUPLICATE_CLIENT_NAME`,
|
||||
`HTTPCLIENT_ACTIVE_WITHOUT_CLIENTS`, `DYNAMIC_BASE_URL_REQUIRED`,
|
||||
`DYNAMIC_TARGET_PROXY_UNSUPPORTED`, `REACTIVE_AUTHENTICATION_UNSUPPORTED`,
|
||||
`HTTP2_REQUIRED_TRANSPORT_UNSUPPORTED`, `POOL_ROUTE_EXCEEDS_TOTAL`,
|
||||
`TLS_PROTOCOL_SET_REQUIRED`, `REACTIVE_REDIRECT_UNSUPPORTED`,
|
||||
`RETRY_POLICY_CONTRADICTS_ATTEMPTS`, `FULL_URL_RECORDING_FORBIDDEN`, `BODY_LOGGING_FORBIDDEN`,
|
||||
`DNS_TIMEOUT_UNSUPPORTED`, `PROXY_CREDENTIAL_UNSUPPORTED`, `PROXY_AMBIENT_NO_PROXY_UNSUPPORTED`.
|
||||
|
||||
The last three name settings the platform binds but cannot yet honour. Neither the Apache classic
|
||||
client nor the JDK client exposes a DNS-resolution timeout, and no proxy-credential path exists, so
|
||||
a non-default value is refused rather than accepted and ignored. Leaving the defaults alone is
|
||||
unaffected — only a deliberate, unmet request fails.
|
||||
|
||||
Three of these are about a guarantee that used to be silently unmet rather than refused:
|
||||
|
||||
- `HTTP2_REQUIRED_TRANSPORT_UNSUPPORTED` — declaring `protocols: [HTTP_2]` alone states that HTTP/2
|
||||
is required. Only `REACTOR_NETTY` can be configured to offer H2 and nothing else; the JDK client
|
||||
treats it as a preference and negotiates HTTP/1.1, and Apache's classic client is HTTP/1.1 only.
|
||||
- `POOL_ROUTE_EXCEEDS_TOTAL` — a per-route ceiling above the total is incoherent, and on Reactor,
|
||||
where the per-route knob is the only one that exists, it silently becomes the effective limit.
|
||||
- `TLS_PROTOCOL_SET_REQUIRED` — an empty `tls.protocols` used to pass and then let the JVM choose,
|
||||
so emptying the list to "tighten" a profile loosened it.
|
||||
- `REACTIVE_REDIRECT_UNSUPPORTED` — engine redirect following is disabled on every transport and
|
||||
only the blocking stack has a coordinator that follows hops with per-hop re-validation. A
|
||||
`WEB_CLIENT` profile with `redirect.enabled=true` did not follow redirects; the caller received the
|
||||
3xx as an ordinary response. Refused until the reactive coordinator exists.
|
||||
- `RETRY_POLICY_CONTRADICTS_ATTEMPTS` — `retry.policy` was read by nothing on the execution path, so
|
||||
the actuator could report `none` for a profile retrying three times. The two settings must now
|
||||
agree: `policy: none` requires `max-attempts: 1`, and any other policy requires more than one.
|
||||
- `FULL_URL_RECORDING_FORBIDDEN` / `BODY_LOGGING_FORBIDDEN` — both settings were bindable and inert.
|
||||
Recording an expanded URL puts path identifiers and query strings into unbounded metric tags;
|
||||
recording bodies puts someone else's data into logs. Representable so the intent is rejectable,
|
||||
refused under a production profile.
|
||||
|
||||
`DYNAMIC_TARGET_PROXY_UNSUPPORTED` is worth spelling out: a forward proxy resolves the hostname on
|
||||
its own side, so the addresses this platform validated and pinned are not the addresses the
|
||||
connection reaches. The SSRF defence would be present, correct, and bypassed — so the combination is
|
||||
refused rather than served with a guarantee it cannot keep.
|
||||
@@ -0,0 +1,179 @@
|
||||
# HTTP Client platform — Java field path to environment variable template.
|
||||
#
|
||||
# The SSOT is HttpClientPlatformSettings. HttpClientEnvironmentKeys derives this list from the
|
||||
# record tree at runtime, HttpClientPlatformEnvManifestTest fails when the two disagree in either
|
||||
# direction, and the platform refuses to start on an APP_HTTPCLIENT_ variable that is not here. So a
|
||||
# field added with no entry, an entry whose field was renamed, and a misspelled variable in a
|
||||
# deployment are all failures rather than silence.
|
||||
#
|
||||
# `N` and `M` are list indices, not literals: `N` for the outermost list, `M` for a list inside it.
|
||||
# `app.httpclient.clients[N].base-url` is set as APP_HTTPCLIENT_CLIENTS_0_BASE_URL for the first
|
||||
# client, and `clients[N].allowed-hosts[M]` as APP_HTTPCLIENT_CLIENTS_0_ALLOWED_HOSTS_0.
|
||||
#
|
||||
# Only APP_HTTPCLIENT_ENABLED is registered in docs/registries/env-keys.yaml and shipped in
|
||||
# src/.env: it is the only key with a deployment-independent value, and it is the only one the
|
||||
# three-way verifyEnvKeys gate can express. Everything below is per deployment and is set directly
|
||||
# in the environment — templating an indexed client in application.yml would materialise a nameless
|
||||
# client in every deployment, which the settings' aggregate validation refuses.
|
||||
#
|
||||
# This file lives beside the HTTP Client documentation rather than in docs/registries, which is a
|
||||
# fail-closed catalog of exactly eight contract registries with a fixed row schema
|
||||
# (owner_branch/compatibility_impact/required_test per row). A field-to-variable mapping does not
|
||||
# have that shape, and admitting it would have meant loosening a gate rather than satisfying one.
|
||||
#
|
||||
# Secrets are referenced, never carried: authentication.secret-reference, tls.*-material-reference
|
||||
# and proxy.credential-provider name material that a secret backend resolves. Putting the material
|
||||
# itself in one of these variables defeats the indirection they exist for.
|
||||
fields:
|
||||
- field: enabled
|
||||
env: APP_HTTPCLIENT_ENABLED
|
||||
- field: clients[N].name
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_NAME
|
||||
- field: clients[N].mode
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_MODE
|
||||
- field: clients[N].base-url
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_BASE_URL
|
||||
- field: clients[N].allowed-hosts[M]
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_ALLOWED_HOSTS_M
|
||||
- field: clients[N].allowed-ports[M]
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_ALLOWED_PORTS_M
|
||||
- field: clients[N].api
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_API
|
||||
- field: clients[N].transport
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TRANSPORT
|
||||
- field: clients[N].protocols[M]
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_PROTOCOLS_M
|
||||
- field: clients[N].pool.max-total-connections
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_MAX_TOTAL_CONNECTIONS
|
||||
- field: clients[N].pool.max-connections-per-route
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_MAX_CONNECTIONS_PER_ROUTE
|
||||
- field: clients[N].pool.max-pending-acquires
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_MAX_PENDING_ACQUIRES
|
||||
- field: clients[N].pool.pending-acquire-timeout
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_PENDING_ACQUIRE_TIMEOUT
|
||||
- field: clients[N].pool.max-idle-time
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_MAX_IDLE_TIME
|
||||
- field: clients[N].pool.max-life-time
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_MAX_LIFE_TIME
|
||||
- field: clients[N].pool.validate-after-inactivity
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_VALIDATE_AFTER_INACTIVITY
|
||||
- field: clients[N].pool.eviction-interval
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_EVICTION_INTERVAL
|
||||
- field: clients[N].pool.shutdown-timeout
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_SHUTDOWN_TIMEOUT
|
||||
- field: clients[N].pool.requires-route-pool
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_REQUIRES_ROUTE_POOL
|
||||
- field: clients[N].pool.requires-bounded-pending-queue
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_POOL_REQUIRES_BOUNDED_PENDING_QUEUE
|
||||
- field: clients[N].timeout.dns
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TIMEOUT_DNS
|
||||
- field: clients[N].timeout.connect
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TIMEOUT_CONNECT
|
||||
- field: clients[N].timeout.tls-handshake
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TIMEOUT_TLS_HANDSHAKE
|
||||
- field: clients[N].timeout.proxy-connect
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TIMEOUT_PROXY_CONNECT
|
||||
- field: clients[N].timeout.request-write-idle
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TIMEOUT_REQUEST_WRITE_IDLE
|
||||
- field: clients[N].timeout.response-header
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TIMEOUT_RESPONSE_HEADER
|
||||
- field: clients[N].timeout.read-idle
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TIMEOUT_READ_IDLE
|
||||
- field: clients[N].timeout.total-call
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TIMEOUT_TOTAL_CALL
|
||||
- field: clients[N].timeout.streaming-idle
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TIMEOUT_STREAMING_IDLE
|
||||
- field: clients[N].redirect.enabled
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_REDIRECT_ENABLED
|
||||
- field: clients[N].redirect.max-hops
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_REDIRECT_MAX_HOPS
|
||||
- field: clients[N].redirect.allow-cross-origin
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_REDIRECT_ALLOW_CROSS_ORIGIN
|
||||
- field: clients[N].request.max-body-bytes
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_REQUEST_MAX_BODY_BYTES
|
||||
- field: clients[N].request.compression
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_REQUEST_COMPRESSION
|
||||
- field: clients[N].response.max-wire-bytes
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_RESPONSE_MAX_WIRE_BYTES
|
||||
- field: clients[N].response.max-decoded-bytes
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_RESPONSE_MAX_DECODED_BYTES
|
||||
- field: clients[N].response.allowed-content-types[M]
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_RESPONSE_ALLOWED_CONTENT_TYPES_M
|
||||
- field: clients[N].authentication.type
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_AUTHENTICATION_TYPE
|
||||
- field: clients[N].authentication.registration-id
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_AUTHENTICATION_REGISTRATION_ID
|
||||
- field: clients[N].authentication.scopes[M]
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_AUTHENTICATION_SCOPES_M
|
||||
- field: clients[N].authentication.audience
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_AUTHENTICATION_AUDIENCE
|
||||
- field: clients[N].authentication.header-name
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_AUTHENTICATION_HEADER_NAME
|
||||
- field: clients[N].authentication.secret-reference
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_AUTHENTICATION_SECRET_REFERENCE
|
||||
- field: clients[N].retry.policy
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_RETRY_POLICY
|
||||
- field: clients[N].retry.max-attempts
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_RETRY_MAX_ATTEMPTS
|
||||
- field: clients[N].retry.base-backoff
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_RETRY_BASE_BACKOFF
|
||||
- field: clients[N].retry.max-backoff
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_RETRY_MAX_BACKOFF
|
||||
- field: clients[N].retry.jitter
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_RETRY_JITTER
|
||||
- field: clients[N].retry.retry-after
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_RETRY_RETRY_AFTER
|
||||
- field: clients[N].retry.budget
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_RETRY_BUDGET
|
||||
- field: clients[N].observability.operation-name-required
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_OBSERVABILITY_OPERATION_NAME_REQUIRED
|
||||
- field: clients[N].observability.full-url-recording
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_OBSERVABILITY_FULL_URL_RECORDING
|
||||
- field: clients[N].observability.body-logging
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_OBSERVABILITY_BODY_LOGGING
|
||||
- field: clients[N].tls.profile-id
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TLS_PROFILE_ID
|
||||
- field: clients[N].tls.protocols[M]
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TLS_PROTOCOLS_M
|
||||
- field: clients[N].tls.hostname-verification
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TLS_HOSTNAME_VERIFICATION
|
||||
- field: clients[N].tls.trust-all
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TLS_TRUST_ALL
|
||||
- field: clients[N].tls.allow-plain-http
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TLS_ALLOW_PLAIN_HTTP
|
||||
- field: clients[N].tls.trust-material-reference
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TLS_TRUST_MATERIAL_REFERENCE
|
||||
- field: clients[N].tls.key-material-reference
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_TLS_KEY_MATERIAL_REFERENCE
|
||||
- field: clients[N].proxy.enabled
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_PROXY_ENABLED
|
||||
- field: clients[N].proxy.host
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_PROXY_HOST
|
||||
- field: clients[N].proxy.port
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_PROXY_PORT
|
||||
- field: clients[N].proxy.type
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_PROXY_TYPE
|
||||
- field: clients[N].proxy.credential-provider
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_PROXY_CREDENTIAL_PROVIDER
|
||||
- field: clients[N].proxy.connect-timeout
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_PROXY_CONNECT_TIMEOUT
|
||||
- field: clients[N].proxy.import-ambient-no-proxy
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_PROXY_IMPORT_AMBIENT_NO_PROXY
|
||||
- field: clients[N].experimental-acknowledgement
|
||||
env: APP_HTTPCLIENT_CLIENTS_N_EXPERIMENTAL_ACKNOWLEDGEMENT
|
||||
- field: dynamic-targets[N].name
|
||||
env: APP_HTTPCLIENT_DYNAMIC_TARGETS_N_NAME
|
||||
- field: dynamic-targets[N].allowed-schemes[M]
|
||||
env: APP_HTTPCLIENT_DYNAMIC_TARGETS_N_ALLOWED_SCHEMES_M
|
||||
- field: dynamic-targets[N].allowed-ports[M]
|
||||
env: APP_HTTPCLIENT_DYNAMIC_TARGETS_N_ALLOWED_PORTS_M
|
||||
- field: dynamic-targets[N].allowed-host-suffixes[M]
|
||||
env: APP_HTTPCLIENT_DYNAMIC_TARGETS_N_ALLOWED_HOST_SUFFIXES_M
|
||||
- field: dynamic-targets[N].allowed-hosts[M]
|
||||
env: APP_HTTPCLIENT_DYNAMIC_TARGETS_N_ALLOWED_HOSTS_M
|
||||
- field: dynamic-targets[N].max-redirect-hops
|
||||
env: APP_HTTPCLIENT_DYNAMIC_TARGETS_N_MAX_REDIRECT_HOPS
|
||||
- field: dynamic-targets[N].trace-propagation
|
||||
env: APP_HTTPCLIENT_DYNAMIC_TARGETS_N_TRACE_PROPAGATION
|
||||
- field: dynamic-targets[N].blocked-cidrs[M]
|
||||
env: APP_HTTPCLIENT_DYNAMIC_TARGETS_N_BLOCKED_CIDRS_M
|
||||
@@ -0,0 +1,64 @@
|
||||
# Migrating from `RestTemplate`
|
||||
|
||||
`RestTemplate` is permitted only inside `…httpclient.migration`; `RestTemplateBoundaryTest` enforces
|
||||
that. New retry, Dynamic Target, and HTTP/3 capabilities are deliberately unreachable from the
|
||||
migration path — a caller that wants them moves to a Named Client Profile.
|
||||
|
||||
## 1. Audit before changing anything
|
||||
|
||||
```java
|
||||
RestTemplateInventory inventory = new RestTemplateInventoryScanner().scan(existingTemplate);
|
||||
```
|
||||
|
||||
The inventory reports the request factory, message converters, interceptors, error handler, and URI
|
||||
template handler, plus findings:
|
||||
|
||||
| Code | Severity | Meaning |
|
||||
|---|---|---|
|
||||
| `SIMPLE_REQUEST_FACTORY` | blocking | no connection pool; unsupported in production |
|
||||
| `NO_MESSAGE_CONVERTERS` | blocking | the template cannot encode or decode a body |
|
||||
| `NO_INTERCEPTORS` | warning | confirm where correlation and timeouts are applied |
|
||||
| `TIMEOUTS_NOT_INTROSPECTABLE` | informational | declare timeouts explicitly on the target profile |
|
||||
|
||||
## 2. Bridge without changing behaviour
|
||||
|
||||
```java
|
||||
RestClient client = new RestTemplateToRestClientAdapter().adaptChecked(existingTemplate);
|
||||
```
|
||||
|
||||
`adaptChecked` refuses to migrate a template with a blocking finding. The bridge carries the
|
||||
existing converters, interceptors, error handler, and URI handler across, so this step changes the
|
||||
API and nothing else.
|
||||
|
||||
## 3. Move to a Named Client Profile
|
||||
|
||||
Turn the platform on with `APP_HTTPCLIENT_ENABLED=true` — it ships off, and while it is off none of
|
||||
the settings below are bound — then declare the upstream as `app.httpclient.clients[N]` with its
|
||||
`name` and an explicit base URL, transport, timeouts, pool, body limits, authentication, retry
|
||||
policy, redirect policy, and TLS profile. Startup validation will tell you exactly which of those is
|
||||
missing. See `docs/httpclient/configuration-reference.md` for the environment form.
|
||||
|
||||
## 4. Move to a typed client
|
||||
|
||||
```java
|
||||
@HttpClientProfile("payment")
|
||||
@HttpExchange("/payments")
|
||||
public interface PaymentClient {
|
||||
|
||||
@PostExchange
|
||||
@HttpOperationPolicy(
|
||||
name = "create-payment",
|
||||
idempotency = OperationIdempotency.IDEMPOTENCY_KEY_REQUIRED,
|
||||
retryPolicy = "payment-write")
|
||||
PaymentResponse create(
|
||||
@RequestHeader("Idempotency-Key") String idempotencyKey, @RequestBody PaymentRequest request);
|
||||
}
|
||||
```
|
||||
|
||||
The interface fails startup validation unless it declares a profile, gives every method a stable
|
||||
operation name and an explicit idempotency, supplies a key parameter when the operation requires
|
||||
one, keeps a single execution model, and does not enable retry on a non-idempotent write.
|
||||
|
||||
## 5. Retire the template
|
||||
|
||||
Once no production package references `RestTemplate`, `RestTemplateBoundaryTest` keeps it that way.
|
||||
@@ -0,0 +1,121 @@
|
||||
# HTTP Client Platform — Operations Runbook
|
||||
|
||||
## Metrics
|
||||
|
||||
| Metric | Meaning |
|
||||
|---|---|
|
||||
| `http.client.requests` | Physical attempt timer (Spring standard name, kept deliberately) |
|
||||
| `http.client.logical.calls` | User-visible logical call timer |
|
||||
| `http.client.attempts` | Attempt counter |
|
||||
| `http.client.retry.count` | Retries by reason |
|
||||
| `http.client.retry.exhausted` | Retry budget exhausted |
|
||||
| `http.client.ambiguous` | Ambiguous outcomes |
|
||||
| `http.client.timeout` | Timeouts by stage |
|
||||
| `http.client.request.bytes` | Request wire bytes |
|
||||
| `http.client.response.bytes` | Response bytes |
|
||||
| `http.client.active` | In-flight attempts |
|
||||
| `http.client.pool.connections` | Leased and available connections |
|
||||
| `http.client.pool.pending` | Pool waiters |
|
||||
| `http.client.pool.acquire.duration` | Pool wait time |
|
||||
| `http.client.dns.duration` | DNS time |
|
||||
| `http.client.connect.duration` | Connect time |
|
||||
| `http.client.tls.duration` | TLS time |
|
||||
| `http.client.circuit.state` | Circuit state |
|
||||
| `http.client.bulkhead.rejected` | Bulkhead rejections |
|
||||
| `http.client.rate_limit.rejected` | Local rate-limit rejections |
|
||||
| `http.client.oauth.refresh` | Token refresh outcomes |
|
||||
| `http.client.ssrf.rejected` | Dynamic target rejections |
|
||||
|
||||
`http.client.requests` counts attempts and `http.client.logical.calls` counts user calls. When they
|
||||
diverge, retries are absorbing failures — which is the first thing to look at during an incident.
|
||||
|
||||
## Reading an incident
|
||||
|
||||
| Symptom | Likely cause | Where to look |
|
||||
|---|---|---|
|
||||
| logical calls fine, attempts spiking | upstream degraded, retries absorbing it | `http.client.retry.count` by reason |
|
||||
| `http.client.ambiguous` non-zero | non-idempotent writes reaching `SENT_NO_RESPONSE` | reconcile with the upstream; consider an idempotency key |
|
||||
| pool pending climbing | pool too small or upstream slow | `http.client.pool.acquire.duration`, `pool.connections` |
|
||||
| circuit open | sustained upstream failure | `http.client.circuit.state`; local rejections do not open it |
|
||||
| `http.client.ssrf.rejected` non-zero | a caller is submitting internal URLs | Dynamic Target policy and audit trail |
|
||||
|
||||
## Actuator
|
||||
|
||||
`GET /actuator/httpclients` reports profile name, runtime generation, state, transport, API,
|
||||
protocols, active leases, pool ceiling, credential type, TLS profile id, redirect flag, retry policy,
|
||||
and capability warnings. Base URL, credentials, trust store paths, and resolved IPs are deliberately
|
||||
absent: an actuator endpoint is reachable by more people than a secret store is.
|
||||
|
||||
## Rotation
|
||||
|
||||
Certificates and secrets rotate by building a new runtime generation and swapping the registry
|
||||
pointer, never by mutating a live client. A connection pool holds sockets established under the
|
||||
previous identity, so replacing material without replacing the pool leaves live connections
|
||||
authenticated by a certificate that is meant to be gone.
|
||||
|
||||
```text
|
||||
build new generation → validate → atomic swap → new calls use it
|
||||
old generation → DRAINING → in-flight calls finish → no new retries → forced close at the drain deadline
|
||||
```
|
||||
|
||||
## Shutdown
|
||||
|
||||
```text
|
||||
RUNNING → DRAINING
|
||||
new logical calls refused or routed to the new generation
|
||||
in-flight attempts complete
|
||||
new retries refused
|
||||
shutdown timeout
|
||||
remaining calls cancelled
|
||||
pool closed
|
||||
```
|
||||
|
||||
## Retry ownership
|
||||
|
||||
Exactly one of the application client, an external SDK, or the service mesh may own retries.
|
||||
Two owners multiply traffic during an incident. Record the owner per upstream and check it whenever
|
||||
a mesh retry policy changes.
|
||||
|
||||
## Error model
|
||||
|
||||
Every outbound failure is one of these stable types. The type is derived from the classified failure
|
||||
category, not from whatever the engine happened to throw, so it means the same thing on Apache, JDK,
|
||||
and Reactor Netty. Each carries `HttpFailureMetadata`: client, operation, method, URI **template**,
|
||||
evidence, replayability, stage, retryability, attempt, elapsed, remaining deadline, status, trace id
|
||||
— and nothing else.
|
||||
|
||||
| Exception | Raised when | Retryable |
|
||||
|---|---|---|
|
||||
| `HttpConfigurationException` | profile, operation, or capability configuration is invalid | never |
|
||||
| `HttpTargetRejectedException` | target URI, host, port, header, or address policy refused the request | never |
|
||||
| `HttpDnsException` | hostname resolution failed or timed out | yes, inside budget |
|
||||
| `HttpPoolAcquireTimeoutException` | no connection or stream within the pending-acquire budget | yes, inside budget |
|
||||
| `HttpConnectException` | socket connect failed | yes, inside budget |
|
||||
| `HttpProxyException` | proxy connect, CONNECT tunnel, or proxy auth failed | yes, inside budget |
|
||||
| `HttpTlsException` | TLS handshake failed | only a transient handshake timeout |
|
||||
| `HttpRequestWriteException` | request headers or body could not be fully written | only when safely idempotent |
|
||||
| `HttpResponseTimeoutException` | final headers or a body chunk did not arrive in time | only when safely idempotent |
|
||||
| `HttpResponseTruncatedException` | the response ended before the body was complete | only when safely idempotent and undelivered |
|
||||
| `HttpRemoteErrorException` | non-success status without a problem document | per the status rules |
|
||||
| `HttpProblemDetailException` | non-success status with a bounded RFC 9457 document | per the status rules |
|
||||
| `HttpRedirectRejectedException` | a hop violated hop count, origin, method, or replay policy | never |
|
||||
| `HttpAuthenticationException` | credential materialization or refresh failed | never |
|
||||
| `HttpSerializationException` | request encoding or response decoding failed | never |
|
||||
| `HttpResponseTooLargeException` | wire or decoded bytes exceeded the profile limit | never |
|
||||
| `HttpDeadlineExceededException` | the effective deadline was reached | never |
|
||||
| `HttpCircuitOpenException` | the upstream circuit is open | never |
|
||||
| `HttpBulkheadRejectedException` | no attempt or logical admission permit was available | never |
|
||||
| `HttpRateLimitRejectedException` | the local attempt rate limit or retry budget rejected the attempt | never |
|
||||
| `HttpAmbiguousExecutionException` | a non-idempotent request was sent and the outcome is unknown | never — reconcile instead |
|
||||
|
||||
## Traces
|
||||
|
||||
```text
|
||||
http.client.operation logical internal span
|
||||
└─ http.client.request attempt 1 CLIENT span
|
||||
└─ http.client.request attempt 2 CLIENT span
|
||||
```
|
||||
|
||||
W3C Trace Context is propagated with a Baggage allowlist. Dynamic Targets do not propagate trace
|
||||
context by default. Retry reason and evidence are recorded as span events; credentials and remote
|
||||
error bodies are never recorded as attributes.
|
||||
@@ -0,0 +1,55 @@
|
||||
# HTTP Client Platform — Performance Baseline
|
||||
|
||||
The certification lane asserts **resource bounds**, not throughput targets. Its purpose is to prove
|
||||
that a failing upstream, a large body, or a rotation cannot consume unbounded memory, connections,
|
||||
threads, or upstream traffic. Nothing here becomes a runtime adaptive default: every bound comes
|
||||
from an explicit profile setting.
|
||||
|
||||
## How to run
|
||||
|
||||
```bash
|
||||
# structural bounds only (default; still executes every test)
|
||||
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest --console=plain
|
||||
|
||||
# full certification, including machine-dependent bounds
|
||||
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest \
|
||||
-Pperformance.assertions.enabled=true --console=plain
|
||||
|
||||
# JMH benchmarks
|
||||
./gradlew :adapter:outbound:httpclient:jmh --console=plain
|
||||
```
|
||||
|
||||
Machine-dependent assertions are reported as explicitly skipped when the flag is absent — the lane
|
||||
never silently degrades into a pass.
|
||||
|
||||
## Certified bounds
|
||||
|
||||
| Test | Bound | Kind |
|
||||
|---|---|---|
|
||||
| `RetryStormBudgetTest` | 10 000 logical calls against a failing upstream produce at most 11 000 physical attempts at a 10 % budget | structural |
|
||||
| `LargeBodyResourceTest` | a 32 MiB streaming download consumes every byte without buffering the payload on the heap | structural + machine-dependent heap bound |
|
||||
| `PoolSaturationPerformanceTest` | 24 concurrent calls against a 4-connection pool all reach a terminal outcome; none hang | structural |
|
||||
| `Http2StreamSaturationTest` | 32 concurrent reactive streams share a 2-connection pool and complete | structural |
|
||||
| `OAuthRefreshContentionTest` | 100 genuinely concurrent callers produce exactly one token request | structural |
|
||||
| `RuntimeRotationDrainTest` | 50 rotations close all 50 retired generations and leave no drain thread | structural |
|
||||
|
||||
## Recording a baseline
|
||||
|
||||
When certifying a deployment, record alongside the numbers: the exact command, the commit, hardware,
|
||||
JVM flags, the profile YAML under test, p50/p95/p99/max, peak heap, peak direct memory, thread count,
|
||||
connection count, physical attempt count, and error count. A latency figure without its profile and
|
||||
hardware is not a baseline; it is an anecdote.
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Command | _fill in at certification time_ |
|
||||
| Commit | _fill in_ |
|
||||
| Hardware / JVM | _fill in_ |
|
||||
| Profile under test | _fill in_ |
|
||||
| p50 / p95 / p99 / max | _fill in_ |
|
||||
| Peak heap / direct memory | _fill in_ |
|
||||
| Threads / connections | _fill in_ |
|
||||
| Physical attempts / errors | _fill in_ |
|
||||
|
||||
The table is intentionally left unfilled in the repository: publishing numbers measured on a build
|
||||
agent as if they were a certified baseline would be worse than having none.
|
||||
@@ -0,0 +1,47 @@
|
||||
# HTTP Client Platform — Release Checklist
|
||||
|
||||
A release is complete when each item below is demonstrated by a command, not by review.
|
||||
|
||||
## Gates
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:httpclient:test --console=plain
|
||||
./gradlew :adapter:outbound:httpclient:httpClientStableContractTest --console=plain
|
||||
./gradlew :adapter:outbound:httpclient:httpClientSecurityTest --console=plain
|
||||
./gradlew :adapter:outbound:httpclient:httpClientBlockHoundTest --console=plain
|
||||
./gradlew :adapter:outbound:httpclient:spring62CompatibilityTest --console=plain
|
||||
./gradlew :adapter:outbound:httpclient:spring70CompatibilityTest --console=plain
|
||||
./gradlew :adapter:outbound:httpclient:httpClientFailureInjectionTest --console=plain # needs Docker
|
||||
./gradlew :adapter:outbound:httpclient:httpClientPerformanceTest \
|
||||
-Pperformance.assertions.enabled=true --console=plain
|
||||
./gradlew verifyCleanArchitectureDependencies --console=plain
|
||||
./gradlew :app-bootstrap:test --tests '*CleanArchitectureTest' --console=plain
|
||||
python3 ../scripts/verify-httpclient-docs.py
|
||||
```
|
||||
|
||||
## Completion criteria (design §33)
|
||||
|
||||
- [ ] Typed clients are the default entry point; H2 and H3 are separately authorised.
|
||||
- [ ] H1–H4 cannot bypass timeout, host, TLS, auth, size, or observation policy.
|
||||
- [ ] Apache, JDK, and Reactor produce identical result and exception metadata.
|
||||
- [ ] Pool, DNS, connect, TLS, and retry backoff all fit inside the effective deadline.
|
||||
- [ ] Every extra attempt is explained by idempotency, replayability, evidence, deadline, and budget.
|
||||
- [ ] Non-idempotent `SENT_NO_RESPONSE` surfaces as `HttpAmbiguousExecutionException`.
|
||||
- [ ] Pool and buffers are reclaimed after unread bodies, decode errors, cancels, and size rejections.
|
||||
- [ ] OAuth2 refresh is single-flight and 401 replay happens at most once.
|
||||
- [ ] Trust-all and hostname-verification bypass fail at startup.
|
||||
- [ ] Canonicalisation, DNS/IP validation, redirect revalidation, and egress control all pass.
|
||||
- [ ] No transparent retry occurs after the first delivered byte.
|
||||
- [ ] No platform code blocks a Reactor event loop, proven by a BlockHound self-check.
|
||||
- [ ] The negotiated wire protocol matches what the support matrix claims per transport.
|
||||
- [ ] Logical calls and attempts are separate metrics with no forbidden label.
|
||||
- [ ] DNS, pool, TLS, reset, partial response, and HTTP/2 GOAWAY are reproducible.
|
||||
- [ ] Thread, heap, direct memory, pool, and retry budget bounds hold.
|
||||
- [ ] The support matrix, configuration reference, security guide, runbook, and migration guide match the code.
|
||||
|
||||
## Experimental
|
||||
|
||||
Jetty HTTP/3 stays Experimental until `Http3CapabilityReport` reports QUIC and TLS 1.3 and the
|
||||
contract subset it declares passes in a dedicated environment. It is never auto-configured by the
|
||||
Stable starter.
|
||||
@@ -0,0 +1,87 @@
|
||||
# HTTP Client Platform — Repository Adaptation Contract
|
||||
|
||||
**Design source:** `httpclient-superpowers-package/docs/superpowers/specs/2026-08-08-httpclient-platform-design.md`
|
||||
**Plan source:** `httpclient-superpowers-package/docs/superpowers/plans/2026-08-08-httpclient-platform-implementation-plan.md`
|
||||
|
||||
The design package states its own adaptation rule:
|
||||
|
||||
> 실제 Backend Skeleton 저장소가 제공되지 않았으므로 package 경로와 Gradle 구조는 설계서의 명시적
|
||||
> 구현 가정이다. 구현 전 저장소의 기존 convention과 root package에 맞춰 경로만 조정하고 공개 계약과
|
||||
> 정책 의미론은 유지한다.
|
||||
|
||||
This file is the single record of *how* the design's assumed layout was mapped onto this repository.
|
||||
Only paths, build DSL, and composition-root ownership changed. Public contracts, policy order, and
|
||||
error semantics are implemented exactly as specified.
|
||||
|
||||
## 1. Why the module layout differs
|
||||
|
||||
The design assumes a greenfield library with 19 Gradle projects under `modules/httpclient/`.
|
||||
This repository is a Clean Architecture template whose **fail-closed registry**
|
||||
(`src/config/architecture/modules.json`, enforced by `src/settings.gradle` and
|
||||
`verifyCleanArchitectureDependencies`) declares **exactly 19 leaf identities**. Creating 19 more
|
||||
Gradle projects would violate HARD-STOP #5 in `AGENTS.md`.
|
||||
|
||||
Therefore the design's 19 library modules become **package boundaries inside the registered leaf**
|
||||
`:adapter:outbound:httpclient`, with two exceptions driven by this repository's own rules:
|
||||
|
||||
| Design module | Repository home | Reason |
|
||||
|---|---|---|
|
||||
| `httpclient-spring-boot-starter` | `:app-bootstrap` (`dev.caskeleton.bootstrap.autoconfigure.httpclient`) | This repository's composition root owns wiring and canonical activation; an adapter leaf must not auto-configure itself. |
|
||||
| `httpclient-testkit` | `:adapter:outbound:httpclient` `src/testkit/java/**/testkit` | The design forbids production modules depending on the testkit; a source set whose dependencies are declared only on the test configurations gives the same guarantee without a new Gradle project. It is its own source set rather than part of `test` because three lanes consume it — `test`, `httpClientPerformanceTest` and `jmh` — and reaching into `sourceSets.test.output` from `jmh` compiled under Gradle but could not be modelled by an IDE, which classifies a source set as test source only when a `Test` task runs its output and forbids main source from reading test source. `PlatformClasses` excludes the source set's output so the boundary rules keep meaning production classes. |
|
||||
|
||||
The package boundary is enforced by ArchUnit rules (`PublicApiArchitectureTest`,
|
||||
`HttpClientModuleBoundaryTest`) that reproduce the design's module dependency table.
|
||||
|
||||
## 2. Package mapping
|
||||
|
||||
Root package: `io.backend.skeleton.httpclient` → `dev.caskeleton.adapter.outbound.httpclient`.
|
||||
|
||||
| Design module | Design package | Repository package |
|
||||
|---|---|---|
|
||||
| `httpclient-core-api` | `…httpclient.api` (+ `.body`, `.error`, `.operation`, `.result`) | `dev.caskeleton.adapter.outbound.httpclient.api` (+ same subpackages) |
|
||||
| `httpclient-profile` | `…httpclient.profile` | `…outbound.httpclient.profile` |
|
||||
| `httpclient-transport-spi` | `…httpclient.transport` | `…outbound.httpclient.transport` |
|
||||
| `httpclient-transport-apache` | `…httpclient.apache` | `…outbound.httpclient.apache` |
|
||||
| `httpclient-transport-jdk` | `…httpclient.jdk` | `…outbound.httpclient.jdk` |
|
||||
| `httpclient-restclient` | `…httpclient.restclient` | `…outbound.httpclient.restclient` |
|
||||
| `httpclient-resilience` | `…httpclient.resilience` | `…outbound.httpclient.resilience` |
|
||||
| `httpclient-auth` | `…httpclient.auth` | `…outbound.httpclient.auth` |
|
||||
| `httpclient-security` | `…httpclient.security` | `…outbound.httpclient.security` |
|
||||
| `httpclient-observability` | `…httpclient.observation` | `…outbound.httpclient.observation` |
|
||||
| `httpclient-transport-reactor-netty` | `…httpclient.reactor` | `…outbound.httpclient.reactor` |
|
||||
| `httpclient-webclient` | `…httpclient.webclient` | `…outbound.httpclient.webclient` |
|
||||
| `httpclient-service-client` | `…httpclient.service` | `…outbound.httpclient.service` |
|
||||
| `httpclient-dynamic-target` | `…httpclient.dynamic` | `…outbound.httpclient.dynamic` |
|
||||
| `httpclient-resttemplate-migration` | `…httpclient.migration` | `…outbound.httpclient.migration` |
|
||||
| `httpclient-spring7-service-groups` | `…httpclient.spring7` | `…outbound.httpclient.spring7` |
|
||||
| `httpclient-jetty-http3-experimental` | `…httpclient.http3` | `…outbound.httpclient.http3` |
|
||||
| `httpclient-spring-boot-starter` | `…httpclient.autoconfigure` | `dev.caskeleton.bootstrap.autoconfigure.httpclient` |
|
||||
| `httpclient-testkit` | `…httpclient.testkit` | `…outbound.httpclient.testkit` (`testkit` source set) |
|
||||
|
||||
## 3. Other deliberate substitutions
|
||||
|
||||
| Design assumption | Repository reality | Adaptation |
|
||||
|---|---|---|
|
||||
| Gradle Kotlin DSL, `build-logic` convention plugin | Groovy DSL, root `build.gradle` conventions, `LockMode.STRICT` dependency locking | Dependencies declared in `src/adapter/outbound/httpclient/build.gradle`; `gradle.lockfile` regenerated. |
|
||||
| Spring Framework 6.2 baseline with 7.0 compatibility | Spring Boot 4.0.0 / Spring Framework 7.0 is the repository baseline | Common code targets the Spring 6.2 **API surface** (no 6.2-only or 7.0-only classes in common packages). The Spring 7 HTTP Service Group integration stays isolated in `…httpclient.spring7`, exactly as the design requires. |
|
||||
| `settings.gradle.kts` module registration | Fail-closed registry | No registry change; leaf identity, gradle path, allowed dependencies unchanged. |
|
||||
| Design §6.2 grades Apache HttpClient 5 as HTTP/2-capable | Spring's blocking factory drives Apache's **classic** client, which is HTTP/1.1 only; HTTP/2 lives in Apache's async client | `ApacheBlockingTransportProvider` declares HTTP/1.1 and rejects an HTTP/2 profile at startup. Blocking HTTP/2 is served by the JDK transport, measured by `NegotiatedProtocolContractTest`. |
|
||||
| Design §28.1 names WireMock for stateful fixtures | WireMock's Jetty modules bind a different Jetty 12 ABI than the Boot-managed one this module already needs for HTTP/3, and fail at server start | `StatefulUpstream` provides path-keyed stateful responses on the existing fixture server; the WireMock dependency was removed rather than worked around with a shaded jar |
|
||||
| Per-task `git commit` | `AGENTS.md`: commit policy is `human-only` | Implementation is delivered unstaged; commits are the human's action. This is the only plan step intentionally not executed, and it is recorded here. |
|
||||
| `docs/httpclient/**`, `.github/workflows/httpclient-*.yml`, `scripts/verify-httpclient-docs.py` | Repository already owns `docs/` and `.github/workflows/` | Created at the same repository-relative paths. |
|
||||
|
||||
## 4. What is unchanged from the design
|
||||
|
||||
- H1 / H2 / H3 / H4 exposure rules and the forbidden native-engine signatures.
|
||||
- `ExecutionEvidence`, `BodyReplayability`, `OperationIdempotency`, `AttemptStage`, `FailureCategory`.
|
||||
- `HttpOperation`, `HttpCallResult`, `BodySource`, `ResponseType`, `BlockingStreamingResponse`.
|
||||
- The complete stable exception hierarchy and `HttpFailureMetadata` redaction rules.
|
||||
- Named Client Profile schema, startup validation codes, and operation override direction.
|
||||
- Effective deadline formula, attempt budget, and streaming setup/idle split.
|
||||
- Retry eligibility inputs, the ordered decision table, retry budget, and backoff rules.
|
||||
- Circuit → Rate Limiter → Bulkhead attempt order and logical admission placement.
|
||||
- OAuth2 cache key, single-flight refresh, and the 401 replay-at-most-once rule.
|
||||
- TLS allow/forbid lists and permanent-failure classification.
|
||||
- Dynamic Target canonicalization → all-answer DNS validation → pinning → redirect revalidation.
|
||||
- Low-cardinality tag allowlist, forbidden labels, trace and logging rules.
|
||||
- Runtime generation swap and drain semantics.
|
||||
@@ -0,0 +1,71 @@
|
||||
# Retry and Ambiguity
|
||||
|
||||
The platform never decides a retry from the HTTP method alone (design D-09). A second attempt
|
||||
happens only when idempotency, body replayability, execution evidence, deadline, and retry budget
|
||||
all permit it.
|
||||
|
||||
## Execution evidence
|
||||
|
||||
| Evidence | Meaning | Typical cause |
|
||||
|---|---|---|
|
||||
| `NOT_SENT` | Proven that the server never received the request | profile rejection, pool timeout, DNS failure, connect failure, pre-request TLS failure, HTTP/2 `REFUSED_STREAM` |
|
||||
| `SENT_NO_RESPONSE` | Some or all of the request was written, no final header arrived | partial write, response-header timeout, connection reset |
|
||||
| `RESPONSE_RECEIVED` | Final headers arrived, whatever the status | 2xx, 4xx, 5xx, redirect |
|
||||
| `PARTIAL_RESPONSE` | Headers and part of the body arrived | reset during decode, interrupted stream |
|
||||
|
||||
`NOT_SENT` is only produced by a stage failure that proves it. A generic engine I/O error is never
|
||||
upgraded to `NOT_SENT`, because that is exactly how a timeout becomes a duplicate payment.
|
||||
|
||||
## Body replayability
|
||||
|
||||
| Body | Replayability |
|
||||
|---|---|
|
||||
| immutable `byte[]` | `REPLAYABLE` |
|
||||
| DTO plus a deterministic codec | `REPLAYABLE` |
|
||||
| reopenable file or resource supplier | `REOPENABLE` |
|
||||
| a single `InputStream` instance | `ONE_SHOT` |
|
||||
| publisher factory | as declared |
|
||||
| publisher instance | `ONE_SHOT` |
|
||||
| multipart | the weakest part |
|
||||
|
||||
## Decision order
|
||||
|
||||
`DefaultRetryEligibilityEngine` evaluates in this order, and a later rule can never re-enable
|
||||
something an earlier one forbade:
|
||||
|
||||
1. attempts exhausted → `RetryDenied.maxAttempts()`
|
||||
2. retry budget empty → `RetryDenied.budgetExhausted()`
|
||||
3. body not replayable → `RetryDenied.bodyNotReplayable()`
|
||||
4. first byte already delivered → `RetryDenied.responseAlreadyDelivered()`
|
||||
5. runtime draining → `RetryDenied.runtimeDraining()`
|
||||
6. remaining deadline below the minimum attempt budget → `RetryDenied.deadline()`
|
||||
7. permanent failure category → `RetryDenied.permanentFailure(...)`
|
||||
8. `SENT_NO_RESPONSE` on an operation that is not safely idempotent → `AmbiguousFailure`
|
||||
9. status- and failure-specific rules
|
||||
|
||||
## Status rules
|
||||
|
||||
| Status | Decision |
|
||||
|---|---|
|
||||
| 408 | retry inside deadline and budget |
|
||||
| 425 | at most one retry, first attempt only |
|
||||
| 429 | retry inside `Retry-After`, deadline, and budget |
|
||||
| 401 | one refresh-and-replay, safe replayable operations only |
|
||||
| 500 | denied unless the upstream registered it as transient **and** the operation is safely idempotent |
|
||||
| 502, 503, 504 | retry for safely idempotent operations; ambiguous otherwise |
|
||||
| other 4xx | denied |
|
||||
|
||||
## Ambiguity
|
||||
|
||||
A non-idempotent request that reached `SENT_NO_RESPONSE` raises
|
||||
`HttpAmbiguousExecutionException`. It is a third answer on purpose: retrying may duplicate a side
|
||||
effect, and reporting a plain failure would tell the caller the request did not happen, which may
|
||||
be false. The caller reconciles, usually by querying the upstream or replaying with an idempotency
|
||||
key.
|
||||
|
||||
## Budget and backoff
|
||||
|
||||
Retry tokens come from a per-upstream token bucket sized as a fraction of real traffic, so a failing
|
||||
upstream cannot be flooded by retries from a healthy fleet. Backoff is exponential with full or
|
||||
decorrelated jitter, bounded by `max-backoff`, by `Retry-After`, and by the remaining deadline. No
|
||||
connection and no bulkhead permit is held while a backoff is waiting.
|
||||
@@ -0,0 +1,75 @@
|
||||
# HTTP Client Platform — Security Guide
|
||||
|
||||
## What the platform owns
|
||||
|
||||
`Authorization`, `Proxy-Authorization`, `Host`, `Content-Length`, `Transfer-Encoding`,
|
||||
`Traceparent`, `Tracestate`, `Baggage`, and (unless a profile opts in) `Cookie` are platform-owned.
|
||||
A caller cannot set them. `Idempotency-Key` is accepted only when the operation declares it. Any
|
||||
header name or value containing CR or LF is rejected before the request is built.
|
||||
|
||||
## Target policy
|
||||
|
||||
A trusted profile accepts only a profile-relative URI template. An absolute URI is rejected rather
|
||||
than sanitised: varying the destination is what H3 is for, and H3 has its own policy, credentials,
|
||||
and address validation. Template variables are encoded per component, so a value containing `/`,
|
||||
`?`, or `#` cannot change the shape of the request.
|
||||
|
||||
## TLS
|
||||
|
||||
Allowed: TLS 1.2 and 1.3, hostname verification, the JVM trust store, a per-profile custom CA, a
|
||||
per-profile client certificate, mTLS, SNI and ALPN, and certificate rotation through a new runtime
|
||||
generation.
|
||||
|
||||
Forbidden and unrepresentable: a trust-all trust manager, disabled hostname verification, ignoring
|
||||
certificate errors, automatically trusting a production self-signed certificate, falling back to
|
||||
plaintext after an HTTPS failure, and writing key material into configuration or logs.
|
||||
|
||||
Unknown CA, hostname mismatch, expired certificate, revoked certificate, protocol mismatch, and a
|
||||
missing client certificate are permanent. Only a transient handshake timeout may be retried, inside
|
||||
the deadline.
|
||||
|
||||
## Dynamic Target (SSRF)
|
||||
|
||||
Every hop — the first one included — runs the whole flow:
|
||||
|
||||
1. strict URI parse
|
||||
2. scheme allowlist
|
||||
3. reject userinfo and invalid ports
|
||||
4. IDNA-canonicalise the host
|
||||
5. host allowlist or suffix policy
|
||||
6. resolve **every** A and AAAA answer
|
||||
7. normalise each address, including IPv4-mapped IPv6
|
||||
8. reject loopback, link-local, RFC1918, ULA, carrier-grade NAT, unspecified, multicast, cloud
|
||||
metadata, and organisation-defined ranges
|
||||
9. pin the connection to the approved addresses through the same validated resolver
|
||||
10. apply response size and content policy
|
||||
11. repeat for each redirect
|
||||
|
||||
Any forbidden address in the answer set rejects the whole target. Validating only the first answer
|
||||
would let a host that resolves to one public and one private address through.
|
||||
|
||||
Dynamic profiles inherit no API key, OAuth token, Cookie, or default header, and no Cookie jar is
|
||||
created. A specific host may be granted a credential only through an explicitly registered
|
||||
`DynamicCredentialBinding`.
|
||||
|
||||
Application-level validation is not sufficient on its own. A network control — Kubernetes
|
||||
NetworkPolicy, service-mesh egress policy, firewall, or proxy ACL — is an operational completion
|
||||
requirement.
|
||||
|
||||
## Redirects
|
||||
|
||||
Disabled by default. Engine redirect handling is off in every transport so the platform can
|
||||
re-validate each hop. 307 and 308 preserve method and body and are therefore allowed only for a
|
||||
replayable body. Cross-origin hops are refused unless the profile opts in, and when they are
|
||||
allowed `Authorization`, `Proxy-Authorization`, `Cookie`, and API-key headers are stripped.
|
||||
|
||||
## Observability
|
||||
|
||||
Allowed tags: `clientName`, `operationName`, `method`, `uriTemplate`, `status`, `outcome`,
|
||||
`transport`, `protocol`, `timeoutType`, `retryReason`, `evidence`, `circuitState`.
|
||||
|
||||
Rejected outright: full URL, query parameters, path variable values, user ID, raw tenant ID,
|
||||
resolved IP, API key, token, Cookie, idempotency key, request or response body, exception message.
|
||||
|
||||
Failures are logged once, structured, at the end of a logical call. Retry attempts are DEBUG or span
|
||||
events. URLs appear only as templates.
|
||||
@@ -0,0 +1,52 @@
|
||||
# Streaming and Large Bodies
|
||||
|
||||
## Response lifecycle
|
||||
|
||||
A blocking streaming download returns `BlockingStreamingResponse`, never a bare `InputStream`.
|
||||
Closing is idempotent and always releases the connection — after a full read, a partial read, a
|
||||
decode failure, or a size rejection. The status is validated before any body byte is delivered, so a
|
||||
failed download never becomes a half-consumed stream the caller has to reason about.
|
||||
|
||||
A reactive download emits bounded `DataBuffer` values. Buffers are released on completion, error,
|
||||
and cancellation; a dropped buffer is direct memory nobody returns.
|
||||
|
||||
Wire bytes and decoded bytes are bounded independently, because a compressed payload passes a wire
|
||||
check and then expands. Limits are enforced while reading, not after buffering.
|
||||
|
||||
## The first-byte boundary
|
||||
|
||||
```text
|
||||
response headers received
|
||||
→ nothing delivered yet
|
||||
→ a read-only operation may still be retried
|
||||
→ first InputStream read or first Flux onNext
|
||||
→ transparent retry is permanently disabled
|
||||
```
|
||||
|
||||
`FirstByteDeliveryGuard` latches once and never resets. Retrying after delivery would replay a
|
||||
stream the caller has already partly consumed, producing duplicated or reordered data that no
|
||||
downstream code can detect.
|
||||
|
||||
## Request bodies
|
||||
|
||||
A reopenable body is opened once per attempt, which is what makes it replayable; reusing the
|
||||
previous stream would silently send an empty body on the retry. A one-shot stream or publisher
|
||||
instance is never retried. `ReactiveBodySource` takes a publisher *factory* rather than a publisher
|
||||
so a reactive body can honestly declare itself replayable.
|
||||
|
||||
A multipart body is exactly as replayable as its weakest part.
|
||||
|
||||
## Server-sent events
|
||||
|
||||
Three budgets stay separate:
|
||||
|
||||
- `setupDeadline` — establishing the stream
|
||||
- `streamingIdleTimeout` — silence once it is open
|
||||
- `maxStreamDuration` — optional total lifetime
|
||||
|
||||
Applying the request-shaped `total-call` timeout to an SSE subscription would terminate a perfectly
|
||||
healthy stream on schedule, so it is not applied.
|
||||
|
||||
`Last-Event-ID` is opt-in. Replaying from an id is only correct when the producer guarantees it;
|
||||
sending it blindly can skip or duplicate events. Reconnects consume the retry budget like any other
|
||||
physical attempt, and cancelling the subscription stops both the stream and any pending reconnect.
|
||||
@@ -0,0 +1,88 @@
|
||||
# HTTP Client Platform — Support Matrix
|
||||
|
||||
Grades follow design §6 and §29. A row is **Stable** only when the cross-transport contract suite
|
||||
proves it; anything the suite cannot prove is **Experimental** and says so.
|
||||
|
||||
## Spring API
|
||||
|
||||
| API | Grade | Role | Constraint |
|
||||
|---|---|---|---|
|
||||
| `RestClient` | Stable | Blocking execution | Bounded concurrency and an effective deadline are mandatory |
|
||||
| `WebClient` | Stable | Reactive, streaming, SSE | No blocking work on the event loop |
|
||||
| HTTP Service Client (`@HttpExchange`) | Default | Declarative typed client | Operation metadata is mandatory |
|
||||
| `RestTemplate` | Migration only | Moving existing calls | No new profile or feature |
|
||||
| Generic Exchange (H2) | Restricted | Dynamic method, path, body | Base URL and policy are immutable |
|
||||
| Dynamic Target (H3) | Restricted | User-supplied URL | Separate SSRF policy; inherits no credential |
|
||||
| Native engine | Internal | Engine-specific configuration | Never an application-facing API |
|
||||
|
||||
## Transports
|
||||
|
||||
| Transport | Blocking | Reactive | HTTP/1.1 | HTTP/2 | HTTP/3 | Grade | Verified by |
|
||||
|---|---:|---:|---:|---:|---:|---|---|
|
||||
| Apache HttpClient 5 (classic) | yes | no | yes | **no** | no | Stable (blocking default) | `httpClientStableContractTest`, `NegotiatedProtocolContractTest` |
|
||||
| JDK HttpClient | yes | `sendAsync` | yes | yes (TLS/ALPN) | no | Stable (lightweight, blocking HTTP/2) | `NegotiatedProtocolContractTest` |
|
||||
| Reactor Netty | limited | yes | yes | yes | experimental | Stable (reactive default) | `NegotiatedProtocolContractTest` |
|
||||
| Jetty | facade | yes | yes | yes | yes | **Experimental** | `Http3OptInTest` only |
|
||||
| Simple request factory | yes | no | limited | no | no | Local test only | rejected in production by `ClientProfileValidator` |
|
||||
|
||||
### Apache is HTTP/1.1 here, and why
|
||||
|
||||
Design §6.2 grades Apache HttpClient 5 as HTTP/2-capable, and the library is — in its **async**
|
||||
client. Spring's `HttpComponentsClientHttpRequestFactory` drives the **classic** client, which
|
||||
speaks HTTP/1.1 only. `NegotiatedProtocolContractTest` measures this rather than assuming it: the
|
||||
classic client fails outright against a prior-knowledge h2c server.
|
||||
|
||||
So `ApacheBlockingTransportProvider.capabilities()` declares HTTP/1.1, and a profile that pairs
|
||||
Apache with `HTTP_2` is rejected at startup instead of quietly running HTTP/1.1 while this table
|
||||
claims otherwise. **Blocking HTTP/2 is served by the JDK transport**; reactive HTTP/2 by Reactor
|
||||
Netty. Both are measured from the client after a real TLS handshake, not read from configuration.
|
||||
|
||||
The JDK transport declares `routeScopedPool=false`, `boundedPendingAcquireQueue=false`, and
|
||||
`dynamicTargetStable=false`. A profile that needs any of those is rejected at startup rather than
|
||||
served with weaker guarantees. Choosing between Apache and JDK is therefore a real trade: Apache
|
||||
gives route-scoped pooling and Dynamic Target pinning, JDK gives HTTP/2.
|
||||
|
||||
## Capability gates
|
||||
|
||||
| Capability | Gate |
|
||||
|---|---|
|
||||
| Dynamic Target (H3) | Apache and Reactor Netty only; JDK and Jetty are rejected |
|
||||
| HTTP/3 | `experimentalAcknowledgement` must equal `I_ACCEPT_HTTP3_EXPERIMENTAL_SEMANTICS` |
|
||||
| Cross-origin redirect | opt-in per profile; credentials are stripped on the hop |
|
||||
| Retry | evidence-based; never enabled by HTTP method alone |
|
||||
|
||||
## CI matrix
|
||||
|
||||
| Profile | Frequency | Release gate | Task |
|
||||
|---|---|---|---|
|
||||
| Spring Framework 7.0 (repository baseline) | every PR | required | `spring70CompatibilityTest` |
|
||||
| Spring Framework 6.2 API surface | every PR | required | `spring62CompatibilityTest` |
|
||||
| Apache HC5 + RestClient | every PR | required | `httpClientStableContractTest -Phttpclient.contract.transports=apache` |
|
||||
| JDK HttpClient + RestClient | every PR | required | `httpClientStableContractTest -Phttpclient.contract.transports=jdk` |
|
||||
| Reactor Netty + WebClient | every PR | required | `httpClientStableContractTest -Phttpclient.contract.transports=reactor` |
|
||||
| SSRF / cardinality suite | every PR | required | `httpClientSecurityTest` |
|
||||
| Toxiproxy fault suite | nightly, release | required | `httpClientFailureInjectionTest` |
|
||||
| Event-loop blocking (BlockHound) | every PR | required | `httpClientBlockHoundTest` |
|
||||
| Performance certification | nightly, release | required | `httpClientPerformanceTest -Pperformance.assertions.enabled=true` |
|
||||
| Jetty HTTP/3 | nightly | Experimental, non-blocking | `test -Phttp3.tests.enabled=true` |
|
||||
|
||||
### Known limitation of the Spring 6.2 lane
|
||||
|
||||
This repository's Spring Boot 4.0 baseline pins Spring Framework 7, so a real 6.2 runtime cannot be
|
||||
resolved here. `spring62CompatibilityTest` therefore verifies the **API surface**: the common
|
||||
packages must not reference any Spring 7-only type, and `org.springframework.web.service.registry`
|
||||
is confined to `…httpclient.spring7`. Executing the suite against an actual 6.2 distribution
|
||||
requires a host project on that line. This limitation is stated rather than hidden behind a passing
|
||||
check.
|
||||
|
||||
|
||||
## What the suites do not prove
|
||||
|
||||
Stated so the matrix is read as a measurement rather than an aspiration.
|
||||
|
||||
| Gap | Why | What is proven instead |
|
||||
|---|---|---|
|
||||
| HTTP/2 frame injection (`REFUSED_STREAM`, arbitrary `GOAWAY`) | The fixture server exposes no frame-level control, and a purpose-built h2 server is a larger dependency than the guarantee is worth here | `Http2EvidenceMapperTest` proves the frame → evidence mapping, and `NegotiatedProtocolContractTest` proves h2 is really negotiated |
|
||||
| Netty buffer-leak detection | Netty reports a leak when an unreferenced buffer is collected, which the suite does not force | `NettyLeakDetectionExtension` asserts the PARANOID detector is live and reports nothing; explicit release assertions in the streaming suites are the primary guarantee |
|
||||
| Spring 6.2 runtime | This repository's Boot 4.0 baseline pins Spring 7 | `spring62CompatibilityTest` confines the common packages to the 6.2 API surface |
|
||||
| Performance latency baseline | Numbers measured on a build agent are not a certification | `httpClientPerformanceTest` asserts structural bounds unconditionally; latency and heap bounds run under `-Pperformance.assertions.enabled=true` |
|
||||
@@ -0,0 +1,49 @@
|
||||
# Command policy
|
||||
|
||||
`src/adapter/outbound/cache-redis/src/main/resources/redis-sdk/redis-command-policy.yml` is the
|
||||
single source of truth for what this SDK is willing to do with each Redis command. Official server
|
||||
metadata decides what a command *is*; this file decides what we allow.
|
||||
|
||||
A command that is not classified there is refused. Adding a command therefore means editing that
|
||||
file, not writing code — and the edit is where the risk decision is made and reviewed.
|
||||
|
||||
## Fields
|
||||
|
||||
| Field | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `risk` | required | `R1` routine, `R2` needs an explicit permit, `R3` administrative, `R4` never allowed |
|
||||
| `support` | required | `TYPED`, `ADVANCED_TYPED`, `RAW_ONLY`, `ADMIN_ONLY`, `VERSION_GATED`, `BLOCKED` |
|
||||
| `minimum-version` | `7.2` | lowest server version that carries the command |
|
||||
| `access` | derived from `support` | which ACL account may issue it |
|
||||
| `blocking` | `false` | occupies its connection until the server replies |
|
||||
| `optional-block` | `false` | the command also has a non-blocking form; only `XREAD` and `XREADGROUP` carry it |
|
||||
| `read-only` | `false` | never mutates the dataset |
|
||||
| `retry-safe` | `read-only` | may be retried after a failure that could have reached the server |
|
||||
| `may-be-ambiguous` | `!read-only` | a failure may leave the outcome unknown |
|
||||
| `timeout-profile` | derived | `FAST`, `COLLECTION`, `ADMIN`, `BLOCKING` |
|
||||
| `key-spec` | `1 1 1` | where the keys are, or `none`, or `movable` |
|
||||
| `required-policy` | – | the permit policy an R2 command demands |
|
||||
|
||||
## Rules the catalog enforces
|
||||
|
||||
- An R2 `ADVANCED_TYPED` command must name the permit policy it requires. There is no R2 command
|
||||
that anyone may issue without an issued permit.
|
||||
- An R4 command must be `BLOCKED`, and an R3 command must be `ADMIN_ONLY`. The type system refuses
|
||||
the other combinations at load time.
|
||||
- A `BLOCKED` command carries no ACL account, so no path in the SDK can reach it.
|
||||
- A blocking command must use the `BLOCKING` timeout profile, and its request must declare a bounded
|
||||
server block — unless it also declares `optional-block`, which only the two stream reads do.
|
||||
- Deprecated command names stay `BLOCKED` even when the SDK offers their behaviour. The typed
|
||||
sorted-set ranges issue `ZRANGE ... BYSCORE|BYLEX|REV`, not `ZRANGEBYSCORE`, so what the guard was
|
||||
told and what reaches the wire are the same command.
|
||||
|
||||
## Where each support level is reachable from
|
||||
|
||||
| Support | Reachable from |
|
||||
| --- | --- |
|
||||
| `TYPED` | the typed operations, no permit |
|
||||
| `ADVANCED_TYPED` | the typed operations, with the named permit |
|
||||
| `VERSION_GATED` | a capability bean that exists only when the probe found the feature |
|
||||
| `RAW_ONLY` | `sdk.raw`, and only with a deployment-registered approval |
|
||||
| `ADMIN_ONLY` | `sdk.admin`, read-only diagnostics only |
|
||||
| `BLOCKED` | nowhere |
|
||||
@@ -0,0 +1,75 @@
|
||||
# Operating the Redis SDK
|
||||
|
||||
## What the metrics can and cannot tell you
|
||||
|
||||
Every observation carries the command family, the deployment mode, and latency. None carries a key,
|
||||
a field, a member, or a value — not because they would be large, but because a metric dimension
|
||||
built from caller data is unbounded cardinality and, for most deployments, tenant identity in a
|
||||
dashboard.
|
||||
|
||||
That means you can answer "which command family is slow" and "which one is failing", and you cannot
|
||||
answer "which key is hot" from metrics. Use the admin plane's `SLOWLOG` projection for the first
|
||||
question and `MEMORY USAGE` on a specific key for the second.
|
||||
|
||||
## The failures worth alerting on
|
||||
|
||||
| Signal | What it means | What to do |
|
||||
| --- | --- | --- |
|
||||
| `RedisCommandRejectedException` | the SDK refused before sending | a caller exceeded a declared bound; the reason names which one |
|
||||
| `RedisCrossSlotException` | a multi-key command spans slots | the keys need a shared hash tag |
|
||||
| `RedisAmbiguousExecutionException` | a write may or may not have applied | reconcile; the SDK will not retry it |
|
||||
| `RedisCapabilityUnavailableException` | the server lacks the feature | a capability bean was constructed by hand, or the probe result changed |
|
||||
| `SentinelFailoverObserver.ambiguousWriteCount` | non-idempotent writes lost to a promotion | each one needs reconciling; the count is the workload |
|
||||
| `ClusterTopologyObserver.reshardingObserved` | `ASK`/`TRYAGAIN` seen | a slot migration is in progress; latency will be uneven until it ends |
|
||||
|
||||
## Things the SDK will never do for you
|
||||
|
||||
- Retry a non-idempotent write after a timeout. `ExecutionCertainty.AMBIGUOUS_FAILURE` is reported,
|
||||
not resolved.
|
||||
- Follow a cross-slot multi-key command by splitting it. It is refused instead.
|
||||
- Read a whole collection, stream, or index. Every read declares a bound.
|
||||
- Load a Lua script or a function library at request time. Both are deployment actions.
|
||||
- Send a command it cannot classify.
|
||||
- Tell you that an acknowledged write was lost. See below — this one is not a limitation you can
|
||||
work around in application code.
|
||||
|
||||
## The write loss the client cannot see
|
||||
|
||||
Set these on every Redis node that can ever be a primary:
|
||||
|
||||
```
|
||||
min-replicas-to-write 1
|
||||
min-replicas-max-lag 1
|
||||
```
|
||||
|
||||
Without them a Sentinel promotion silently destroys acknowledged writes, and this is measured, not
|
||||
theoretical. In `LiveRedisSentinelPromotionTest` on the 7.4 lane, Sentinel promoted the replica and
|
||||
did not demote the old primary for **eleven seconds**. The client stayed connected to a primary that
|
||||
had already been replaced, wrote, and was told `+OK` **2,086 times**. Every one of those writes was
|
||||
discarded when the old primary resynced. Exactly one command failed.
|
||||
|
||||
Nothing on the client can detect this. The server answered, so the driver recorded a success, the
|
||||
SDK recorded `CONFIRMED_SUCCESS`, and the caller was told the write landed. No metric here counts
|
||||
it, `SentinelFailoverObserver` cannot count it, and no retry policy helps — there was no failure to
|
||||
react to. A second run of the same promotion produced sixteen thousand writes, **zero** exceptions,
|
||||
and the same silent loss.
|
||||
|
||||
With the two settings, the identical promotion lost **one** write and refused 2,020 with
|
||||
`NOREPLICAS`, which the SDK reports as a definite, non-ambiguous failure the caller can act on. That
|
||||
is the whole difference: an outage you can see instead of data you cannot.
|
||||
|
||||
The residual window is `min-replicas-max-lag` wide and cannot be closed by configuration alone. A
|
||||
write that must survive a promotion under any circumstances needs `WAIT` after it, at the cost of a
|
||||
round trip to the replica — decide that per write, not globally.
|
||||
|
||||
## Blocking work
|
||||
|
||||
Blocking pops and blocking stream reads run on a dedicated connection lane. If those saturate, the
|
||||
symptom is blocking calls timing out while ordinary traffic is healthy — that is the lane doing its
|
||||
job, not a fault. Size the blocking pool to the number of concurrent consumers, not to request rate.
|
||||
|
||||
## Pub/Sub
|
||||
|
||||
At-most-once. A subscriber that reconnects misses whatever arrived while it was gone, and there is
|
||||
no replay. Durable business events belong in a stream with a consumer group, which is at-least-once
|
||||
and therefore requires idempotent consumers.
|
||||
@@ -0,0 +1,159 @@
|
||||
# Redis SDK support matrix
|
||||
|
||||
This file is a gate, not a summary. `RedisSupportMatrixTest` parses the tables below and fails when
|
||||
the SDK grows a package or a capability that is not listed, so a module cannot ship without someone
|
||||
stating its minimum version, its topology support, and what it does not do.
|
||||
|
||||
Design: `docs/superpowers/specs/2026-08-07-redis-wrapper-typed-api-design.md`.
|
||||
Delivery status and the decisions behind each module: `docs/superpowers/plans/2026-08-07-redis-wrapper-typed-api-status.md`.
|
||||
|
||||
## Modules
|
||||
|
||||
| Module | Minimum Redis | Topology | Risk exposure | Sync | Reactive | Known limitations |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| `api` | 7.2 | all | none | n/a | n/a | contract only; no driver types |
|
||||
| `api/key` | 7.2 | all | none | n/a | n/a | slot tags must be low-cardinality |
|
||||
| `api/codec` | 7.2 | all | none | n/a | n/a | no Java native serialization |
|
||||
| `api/command` | 7.2 | all | none | n/a | n/a | permits never widen the ACL account |
|
||||
| `api/error` | 7.2 | all | none | n/a | n/a | failure metadata carries no key or value |
|
||||
| `api/operations` | 7.2 | all | none | n/a | n/a | contract only |
|
||||
| `api/reactive` | 7.2 | all | none | n/a | n/a | Reactor confined to this package |
|
||||
| `lettuce` | 7.2 | all | R1–R2 | yes | yes | pinned to Lettuce 6.8.2 |
|
||||
| `lettuce/codec` | 7.2 | all | none | yes | yes | UTF-8 and byte array codecs only |
|
||||
| `lettuce/command` | 7.2 | all | R1–R2 | yes | yes | policy catalog is the only command authority |
|
||||
| `lettuce/connection` | 7.2 | all | none | yes | yes | five lanes; blocking work never shares the regular lane |
|
||||
| `lettuce/observability` | 7.2 | all | none | yes | yes | command family only, never a key |
|
||||
| `lettuce/operations` | 7.2 | all | R1–R2 | yes | yes | hash field TTL needs 7.4; sharded pub/sub needs 7.0; stream deletion needs 8.2 |
|
||||
| `config` | 7.2 | all | none | n/a | n/a | permit provenance is HMAC-signed per process |
|
||||
| `cluster` | 7.2 | cluster | none | n/a | n/a | slot arithmetic only; no redirect following |
|
||||
| `programmability` | 7.2 | all | R2 | yes | no | transactions never roll back; scripts return one bulk reply; `FUNCTION LOAD` is admin-plane |
|
||||
| `raw` | 7.2 | all | R2 | yes | no | `RAW_ONLY` commands only; movable key specs unapprovable |
|
||||
| `admin` | 7.2 | all | R3 read-only | yes | no | replies are projected; no destructive command exists |
|
||||
| `extensions` | 8.0 | all | none | yes | no | shared command runner; every extension declares its key |
|
||||
| `extensions/json` | 8.0 | all | R1–R2 | yes | no | narrow JSONPath grammar; documents exchanged as text |
|
||||
| `extensions/search` | 8.0 | all | R2 | yes | no | index names namespaced by the SDK; no drop index |
|
||||
| `extensions/timeseries` | 8.0 | all | R1–R2 | yes | no | retention mandatory at creation |
|
||||
| `extensions/probabilistic` | 8.0 | all | R1–R2 | yes | no | every answer is approximate by construction |
|
||||
|
||||
## Capabilities
|
||||
|
||||
| Capability | Minimum Redis | Gate | Bean when absent |
|
||||
| --- | --- | --- | --- |
|
||||
| `SHARDED_PUBSUB` | 7.0 | probe and catalog minimum | none |
|
||||
| `FUNCTIONS` | 7.0 | probe and catalog minimum | none |
|
||||
| `HASH_FIELD_EXPIRATION` | 7.4 | probe and catalog minimum | none |
|
||||
| `HASH_FIELD_EXPIRATION_COMBINED` | 8.0 | probe and catalog minimum | none |
|
||||
| `STREAM_ACKNOWLEDGE_DELETE` | 8.2 | probe and catalog minimum | none |
|
||||
| `STREAM_NEGATIVE_ACKNOWLEDGE` | 8.8 | probe and catalog minimum | none, and no bean exists yet |
|
||||
| `JSON` | 8.0 | probe is authoritative | none |
|
||||
| `SEARCH` | 8.0 | probe is authoritative | none |
|
||||
| `TIME_SERIES` | 8.0 | probe is authoritative | none |
|
||||
| `PROBABILISTIC` | 8.0 | probe is authoritative | none |
|
||||
|
||||
## Certified versions
|
||||
|
||||
A version is certified by its lane producing evidence, not by the version number being newer. An
|
||||
evidence claim here must name the test class that produced it; `RedisSupportMatrixTest` fails the
|
||||
build on a row that claims anything else, so "verified" cannot be written into this table without a
|
||||
test behind it.
|
||||
|
||||
All three lanes have now run on 7.4. The other declared versions are declared, not certified:
|
||||
nothing in this repository has executed against 7.2 or 8.2.
|
||||
|
||||
| Topology | Versions declared | Evidence status |
|
||||
| --- | --- | --- |
|
||||
| Standalone | 7.2, 7.4, 8.2 | `RedisTopologyContractTest`, `LiveRedisGuardrailTest` on 7.4 |
|
||||
| Sentinel | 7.4, 8.2 | `RedisTopologyContractTest`, `LiveRedisSentinelPromotionTest` on 7.4 |
|
||||
| Cluster | 7.4, 8.2 | `RedisTopologyContractTest`, `LiveRedisClusterTest` on 7.4 |
|
||||
|
||||
### What the standalone ACL run established
|
||||
|
||||
`RedisTopologyContractTest` runs the four accounts in `infra/redis-sdk/acl` against a live server and
|
||||
asserts that each `CommandAccess` level grants exactly what the command policy catalog says it may
|
||||
issue. Writing it found five defects that no amount of reading the files would have surfaced:
|
||||
|
||||
1. A Redis ACL file accepts neither comments nor line continuations — the original files did not load
|
||||
at all, and the server refused to start.
|
||||
2. The advanced account granted `SMEMBERS` and `SORT`, both `RAW_ONLY` and therefore the raw gateway
|
||||
account's alone.
|
||||
3. The ordinary account granted `SORT_RO` for the same reason.
|
||||
4. The ordinary account could not run `PUBLISH`, `SUBSCRIBE`, or `PING`, all classified `TYPED`.
|
||||
5. The ordinary account could not run `MULTI`, `EXEC`, `UNWATCH`, or `DISCARD`, also `TYPED`.
|
||||
|
||||
6. The admin account was missing twelve read-only diagnostics the catalog exposes — the `OBJECT`,
|
||||
`PUBSUB`, `XINFO`, `FUNCTION LIST`/`STATS`, and `CLUSTER KEYSLOT` subcommands.
|
||||
|
||||
7. The cursor-scan reply budget was sized to the requested `COUNT`, which Redis treats as a hint —
|
||||
a real `HSCAN COUNT 500` came back with 501 entries and the SDK refused a correct reply.
|
||||
|
||||
Points 2 and 3 are the ones that matter: the account is the last enforcement boundary, so an account
|
||||
wider than the catalog silently removes the second control the design relies on.
|
||||
|
||||
### What the standalone guardrail run established
|
||||
|
||||
`LiveRedisGuardrailTest` wires the real guard, catalog, and typed operations to a live server —
|
||||
the first time `LettuceRedisCommandGateway`, the one class that encodes commands, runs under the
|
||||
SDK's own contracts rather than against the in-memory stand-in. It carries the plan's datasets: a
|
||||
value at the 1 MiB ceiling, a hundred-thousand-field hash, hundred-thousand-member set and sorted
|
||||
set, a twenty-thousand-element list, a stream trimmed to 1,000 while twenty thousand entries are
|
||||
appended, and a five-hundred-command batch.
|
||||
|
||||
The assertions are about limits holding, not throughput. A guardrail test that measured absolute
|
||||
speed would fail on a loaded laptop and teach nobody anything.
|
||||
|
||||
### What the Sentinel promotion run established
|
||||
|
||||
`LiveRedisSentinelPromotionTest` forces one real promotion and asserts several independent claims
|
||||
about it. Every write carries a token unique to the run, so the list on the promoted primary is a
|
||||
verbatim record of what happened and each per-call verdict can be checked against it.
|
||||
|
||||
It found the most serious defect in this delivery, and it is not in the SDK's code:
|
||||
|
||||
> **A superseded primary keeps acknowledging writes.** Sentinel promoted the replica at
|
||||
> `05:56:12.503` and did not demote the old primary until `05:56:23.529` — eleven seconds in which
|
||||
> the client, still connected, wrote and was told `+OK` **2,086 times**. Every one of those writes
|
||||
> was discarded when the old primary resynced from the new one. Exactly **one** command failed. No
|
||||
> client-side signal exists for this: the server answered, so the driver, the SDK, and the caller
|
||||
> all correctly recorded a success.
|
||||
|
||||
`SentinelFailoverObserver` counts *ambiguous* writes, and its documentation used to call those "the
|
||||
ones an operator has to reconcile". That was wrong by three orders of magnitude, and the class now
|
||||
says so.
|
||||
|
||||
What closes the window is on the server, not the client. Re-running the identical promotion with
|
||||
`min-replicas-to-write 1` and `min-replicas-max-lag 1` configured cut acknowledged-and-discarded
|
||||
writes from **2,086 to 1**: the orphaned primary refused 2,020 writes with `NOREPLICAS`, which the
|
||||
SDK translates to a definite, non-ambiguous failure the caller can act on. Both settings are now in
|
||||
the lane, and `acknowledgedWriteLossIsBounded` ties the tolerated loss to the configured lag window,
|
||||
so removing them makes the count jump by an order of magnitude and fails the test.
|
||||
|
||||
That assertion then caught a second version of the same mistake within a day of being written. The
|
||||
first guarded run passed; the second failed with 2,099 lost writes, because the setting had been
|
||||
written into the lane's `primary` service only. The two data nodes swap roles on every failover, so
|
||||
a guardrail applied to whichever one happens to start as primary stops applying the moment the lane
|
||||
does the thing it exists to do. Both nodes now take their whole configuration from one definition.
|
||||
Three consecutive promotions in both directions since: 0, 0, and 1 acknowledged write lost.
|
||||
|
||||
The run also found a translator defect. A promotion closed the channel under an in-flight `RPUSH`
|
||||
and the driver raised a bare `RedisException`, which matched no branch and fell through to a generic
|
||||
failure reported as *definitely did not run*. Nothing about an unrecognised failure supports that
|
||||
claim, and a caller who believes it retries a non-idempotent write. The fallback now treats an
|
||||
unclassified write failure as ambiguous.
|
||||
|
||||
### What the Cluster run established
|
||||
|
||||
`LiveRedisClusterTest` checks the part of `sdk.cluster` that is pure client-side arithmetic against
|
||||
the server that has the last word. The calculator agreed with `CLUSTER KEYSLOT` on every entry of a
|
||||
corpus built from the brace rules a hand-written implementation gets wrong — an empty tag `{}`,
|
||||
`foo{}{bar}`, `foo{{bar}}zap`, an unclosed brace, `}{`, the empty key, and non-ASCII keys — and the
|
||||
rendered-key invariant holds: the slot the SDK computes from a tag alone equals the slot the server
|
||||
computes from the whole rendered key.
|
||||
|
||||
Cross-slot refusal was checked in both directions, because a guard stricter than the cluster costs
|
||||
availability for no reason and a looser one sends requests that cannot succeed. The same key pair
|
||||
the guard refuses is the pair the server answers `CROSSSLOT` for.
|
||||
|
||||
Redirects were observed rather than assumed: a `MOVED` names the slot the client computed, and a
|
||||
slot put into a real `MIGRATING`/`IMPORTING` state answers `ASK` for an absent key and `TRYAGAIN`
|
||||
for a multi-key request that straddles the migration. The lane restores the slot to `STABLE`
|
||||
afterwards, so a run leaves the cluster as it found it.
|
||||
@@ -0,0 +1,62 @@
|
||||
# Redis and client upgrade gate
|
||||
|
||||
Changing the Redis server version or the Lettuce version is not a dependency bump. Both change what
|
||||
commands exist, what they reply, and what an ACL account is allowed to do — all three are things this
|
||||
SDK encodes as fixed decisions. The checks below must pass before either version moves, and each one
|
||||
exists because skipping it produces a specific failure that only shows up in production.
|
||||
|
||||
## 1. Command metadata diff
|
||||
|
||||
Run the catalog drift check against the new server. Every command the server reports must be
|
||||
classified in `src/adapter/outbound/cache-redis/src/main/resources/redis-sdk/redis-command-policy.yml`.
|
||||
|
||||
*Why:* an unclassified command is refused by `CommandPolicyGuard`, so a server that grew a command
|
||||
does not create a hole — but a command whose **risk changed upstream** and is still classified R1
|
||||
here does. The diff is what surfaces that.
|
||||
|
||||
## 2. ACL regression
|
||||
|
||||
Re-run `ACL DRYRUN` for every account against every command the SDK can issue, using
|
||||
`RedisAdminOperations.aclDryRun`.
|
||||
|
||||
*Why:* a permit never widens an ACL account, so the account is the last boundary. A new server
|
||||
version that moved a command into a different ACL category silently turns a working call into a
|
||||
runtime refusal on the first request that needs it.
|
||||
|
||||
## 3. Serializer golden bytes
|
||||
|
||||
Compare the encoded form of every registered codec against the stored golden bytes.
|
||||
|
||||
*Why:* a value written by the old version must still decode after the upgrade. A codec change that
|
||||
looks harmless in a round-trip test is not harmless against data already in the instance.
|
||||
|
||||
## 4. Support matrix
|
||||
|
||||
Update `docs/redis/support-matrix.md`. `RedisSupportMatrixTest` fails when a module or capability is
|
||||
missing, and the certified-version table must not claim a version until its topology lane has
|
||||
actually run.
|
||||
|
||||
## 5. Topology suite
|
||||
|
||||
Run the standalone, Sentinel, and Cluster lanes declared in `infra/redis-sdk/`. A version is
|
||||
certified by the lane passing, not by the version number being newer.
|
||||
|
||||
*Why:* failover certainty and cross-slot behaviour are the two things the in-memory fixture cannot
|
||||
prove. `ExecutionCertainty` and `RedisSlotCalculator` are classification and arithmetic; whether the
|
||||
driver actually behaves that way during a promotion or a resharding is only observable on a real
|
||||
topology.
|
||||
|
||||
## 6. Rollback
|
||||
|
||||
Before the upgrade, record the previous server version, the previous Lettuce version, and the
|
||||
`SCRIPT LOAD` digests of every registered script. A rollback is not complete until the digests
|
||||
resolve again on the restored version.
|
||||
|
||||
*Why:* digests are cached per process and invalidated by `SCRIPT FLUSH` and by restarts. A rollback
|
||||
that leaves a process holding digests the restored server does not know produces `NOSCRIPT` on
|
||||
every scripted call until the cache is dropped.
|
||||
|
||||
## What this gate does not cover
|
||||
|
||||
Data migration. Nothing here moves or reshapes stored values; a change that alters what is stored,
|
||||
rather than how it is addressed, needs its own plan.
|
||||
@@ -0,0 +1,120 @@
|
||||
# Registry: Repository Access Capabilities
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/capabilities.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# Owner branch: feature-repository-access-permission-contract
|
||||
# Last updated: 2026-06-05
|
||||
#
|
||||
# Notes
|
||||
# - capability는 사용자 권한이 아니라 application use case가 infrastructure capability를
|
||||
# 사용할 수 있는지에 대한 계약 (feature-repository-access-permission-contract).
|
||||
# - enforcement default = ArchUnit annotation-based rule. compile-time annotation processor는
|
||||
# alternative. runtime AOP는 forbidden.
|
||||
# - capability 제거는 항상 breaking change. 추가는 additive (registry row 동반 시).
|
||||
# - annotation 표기 (as-built, F1/F2 reconciled 2026-06-05): 코드 SSOT는 단일
|
||||
# `@UseCaseCapability` (TYPE target, typed attribute). 노트 D2/D11의 flat
|
||||
# `@UseCaseRepositoryAccess(Capability[])` 모델은 superseded. 7 capability ↔ as-built 매핑:
|
||||
# READ_REPOSITORY/WRITE_REPOSITORY → repositoryAccess, TRANSACTION_REQUIRED → transactionMode,
|
||||
# EXTERNAL_OUTBOUND_ALLOWED → externalOutboundAllowed, SENSITIVE_READ → sensitiveRead,
|
||||
# BULK_WRITE → bulkWrite, CROSS_TENANT_ADMIN → crossTenantAdmin.
|
||||
# 각 row의 annotation: 필드는 아래에서 as-built 표기로 정합됨.
|
||||
|
||||
capabilities:
|
||||
|
||||
# source: feature-repository-access-permission-contract — 판정 기준 "Required capability: READ_REPOSITORY"
|
||||
# source: feature-application-port-usecase-contract — "read-only query use case는 readOnly 와 READ_REPOSITORY capability만 선언 가능"
|
||||
- name: READ_REPOSITORY
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(repositoryAccess = READ_REPOSITORY)"
|
||||
semantics: "use case가 read-only repository operation을 호출하는 것을 허용. query use case의 기본 capability. write/sensitive/bulk 작업은 별도 capability 선언이 없으면 forbidden."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:read-repository-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — 판정 기준 "Required capability: WRITE_REPOSITORY"
|
||||
# source: feature-application-port-usecase-contract — "write use case는 transactionMode, idempotency, repositoryAccess를 명시해야 함"
|
||||
- name: WRITE_REPOSITORY
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(repositoryAccess = WRITE_REPOSITORY)"
|
||||
semantics: "use case가 mutating repository operation(insert/update/delete)을 호출하는 것을 허용. 단일/소량 write 기준이며 batch size > 100은 BULK_WRITE 별도 선언 필요. read-only use case에서 이 capability 없이 write repository 접근하면 fail."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:write-repository-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — decisions 2026-05-22
|
||||
# "SENSITIVE_READ marker = registry-managed metadata table (entity FQN + field name 단위)"
|
||||
- name: SENSITIVE_READ
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(sensitiveRead = true)"
|
||||
semantics: "PII/credential 등 sensitive field를 읽는 use case가 선언해야 하는 capability. marker는 registry-managed metadata table(entity FQN + field name 단위)에서 lookup. domain annotation 또는 JPA entity annotation 형태는 forbidden(domain에 framework 의존 회피). pseudonymized data read는 documented 시에만 예외 허용."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:sensitive-read-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — decisions 2026-05-22
|
||||
# "BULK_WRITE threshold = N > 100 또는 batch size > 100. 미만은 일반 WRITE_REPOSITORY로 충분"
|
||||
- name: BULK_WRITE
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(bulkWrite = true)"
|
||||
semantics: "단일 transaction 내 N > 100 또는 batch size > 100 mutating operation을 수행하는 use case가 선언해야 하는 capability. 이 미만이면 일반 WRITE_REPOSITORY로 충분. lock 점유 시간, pool 영향, retry 비용이 큰 작업을 명시화."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: WRITE_REPOSITORY
|
||||
threshold: 100
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:bulk-write-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — decisions 2026-05-22
|
||||
# "TRANSACTION_REQUIRED는 application-port branch의 TransactionPort contract와 연결되어야 하며 Spring @Transactional 직접 import로 충족하지 않음"
|
||||
# source: feature-application-port-usecase-contract — TransactionPort Contract
|
||||
- name: TRANSACTION_REQUIRED
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(transactionMode = WRITE | READ_ONLY | REQUIRES_NEW)"
|
||||
semantics: "use case가 TransactionPort(또는 TransactionalUseCaseRunner)를 통해 transactional boundary를 갖는 것을 강제. Spring @Transactional의 application package 직접 import는 forbidden. infrastructure가 Spring transaction implementation을 제공하고 application은 port만 호출."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:transaction-required-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — decisions 2026-05-22
|
||||
# "EXTERNAL_OUTBOUND_ALLOWED 분류 = outbox row INSERT는 in-process(불요), polling publisher의 broker publish는 outbound(필요)"
|
||||
# source: feature-application-port-usecase-contract — "outbound adapter 호출 use case에 EXTERNAL_OUTBOUND_ALLOWED가 없으면 실패"
|
||||
- name: EXTERNAL_OUTBOUND_ALLOWED
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(externalOutboundAllowed = true)"
|
||||
semantics: "use case가 외부 HTTP/message broker로 outbound 호출을 발생시키는 것을 허용. outbox claim 분류: outbox row INSERT는 in-process이므로 본 capability 불요. polling publisher의 broker publish는 outbound이므로 필요. domain event without transport detail은 outbound 호출이 아니므로 별도 분류."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:external-outbound-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — decisions 2026-05-22
|
||||
# "CROSS_TENANT_ADMIN capability를 capability vocabulary에 추가 (tenant branch feature-tenant-context-policy와 cross-link)"
|
||||
- name: CROSS_TENANT_ADMIN
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(crossTenantAdmin = true)"
|
||||
semantics: "tenant 경계를 넘어 데이터에 접근/변경하는 admin use case가 선언해야 하는 capability. tenant-context-policy의 cross-tenant 정책과 cross-link되어야 하며, 단일 tenant 범위 use case에서 이 capability를 선언하면 review에서 reject. SENSITIVE_READ가 동반될 가능성이 높지만 자동 결합은 아님."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:cross-tenant-admin-capability
|
||||
|
||||
# Row count verification
|
||||
# - feature-repository-access-permission-contract 판정 기준 "Required capability" 표에 명시된 7개:
|
||||
# READ_REPOSITORY, WRITE_REPOSITORY, SENSITIVE_READ, BULK_WRITE, TRANSACTION_REQUIRED,
|
||||
# EXTERNAL_OUTBOUND_ALLOWED, CROSS_TENANT_ADMIN.
|
||||
# - source에 명시되지 않은 capability는 본 registry에 추가하지 않음 (추측 금지).
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,918 @@
|
||||
# Registry: Error Codes
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/error-codes.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# Category enum owner: feature-operational-error-observability-foundation
|
||||
# Last updated: 2026-05-22
|
||||
# Note: 이 파일은 Phase B 산출물. Phase C2(ca-tmpl 실 코드)에서 generated Java constants의 source.
|
||||
#
|
||||
# Schema (per row):
|
||||
# code: UPPER_SNAKE_CASE
|
||||
# category: VALIDATION | AUTH | AUTHZ | NOT_FOUND | CONFLICT |
|
||||
# RATE_LIMIT | TRANSIENT_DEPENDENCY | PERMANENT_DEPENDENCY |
|
||||
# DATA_INTEGRITY | INTERNAL
|
||||
# http_status: int (async-only failures use 500 placeholder)
|
||||
# retryable: bool
|
||||
# retry_after_seconds: int | null (RATE_LIMIT/TRANSIENT 권고 backoff)
|
||||
# owner_branch: source branch (raw/branch-notes/feature-*.md)
|
||||
# owner_layer: presentation | application | domain | infrastructure | crosscut
|
||||
# client_safe_message: no token / no principal raw / no internal path / no stack trace
|
||||
# log_level: ERROR | WARN | INFO
|
||||
# runbook_link: runbook://area/scenario OR null (client-error만 null 허용)
|
||||
# compatibility_impact: none | additive | behavior-change | breaking
|
||||
# required_test: owning contract test identifier
|
||||
#
|
||||
# Runbook policy (operational-runbook-contract L80):
|
||||
# retryable=false + category ∈ {AUTH, AUTHZ, RATE_LIMIT, INTERNAL,
|
||||
# TRANSIENT_DEPENDENCY, PERMANENT_DEPENDENCY} ⇒ runbook_link 필수.
|
||||
# VALIDATION/NOT_FOUND/CONFLICT/DATA_INTEGRITY는 client-error로 runbook 면제 가능.
|
||||
# retryable=true 인 모든 row는 runbook_link 필수.
|
||||
|
||||
errors:
|
||||
|
||||
# ============================================================
|
||||
# AUTH (feature-security-operational-baseline / Decision Matrix)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-security-operational-baseline L82 — "token 누락 | 401 | AUTH_TOKEN_MISSING | AUTH"
|
||||
- code: AUTH_TOKEN_MISSING
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication required"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://auth/token-missing"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L83 — "token malformed (parse fail) | 401 | AUTH_TOKEN_MALFORMED | AUTH"
|
||||
- code: AUTH_TOKEN_MALFORMED
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://auth/token-malformed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L84 — "token expired (clock skew tolerance 60s 초과) | 401 | AUTH_TOKEN_EXPIRED | AUTH"
|
||||
- code: AUTH_TOKEN_EXPIRED
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication expired"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://auth/token-expired"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L85 — "invalid signature | 401 | AUTH_TOKEN_INVALID_SIGNATURE | AUTH"
|
||||
- code: AUTH_TOKEN_INVALID_SIGNATURE
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/token-invalid-signature"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L86 — "issuer mismatch | 401 | AUTH_ISSUER_MISMATCH | AUTH"
|
||||
- code: AUTH_ISSUER_MISMATCH
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/issuer-mismatch"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L87 — "audience mismatch | 401 | AUTH_AUDIENCE_MISMATCH | AUTH"
|
||||
- code: AUTH_AUDIENCE_MISMATCH
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/audience-mismatch"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L88 — "unknown kid (JWKS 미캐시) | 401 + Retry-After 5s | AUTH_KID_UNKNOWN | AUTH"
|
||||
- code: AUTH_KID_UNKNOWN
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: true # 2026-06-01: false→true. JWKS 키 회전 중 unknown kid 는 ~5s 후 JWKS refresh 로 해소 가능(transient). retry_after_seconds=5 + client_safe_message "please retry" 와 정합. 키 고정 정책으로 전환 시 false 복귀.
|
||||
retry_after_seconds: 5
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed, please retry"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://auth/kid-unknown"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L89 — "JWKS endpoint outage ... | AUTH_JWKS_UNAVAILABLE | TRANSIENT_DEPENDENCY"
|
||||
- code: AUTH_JWKS_UNAVAILABLE
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 30
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Authentication service temporarily unavailable"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/jwks-unavailable"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L90 — "claim mapping failure ... | 401 | AUTH_CLAIM_MAPPING_FAILED | AUTH"
|
||||
- code: AUTH_CLAIM_MAPPING_FAILED
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/claim-mapping-failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# ============================================================
|
||||
# AUTHZ (feature-security-operational-baseline)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-security-operational-baseline L91 — "valid token + 권한 부족 | 403 | AUTHZ_INSUFFICIENT_PERMISSION | AUTHZ"
|
||||
- code: AUTHZ_INSUFFICIENT_PERMISSION
|
||||
category: AUTHZ
|
||||
http_status: 403
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Permission denied"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://authz/insufficient-permission"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:authz-category
|
||||
|
||||
# source: feature-security-operational-baseline L92 — "valid token + tenant cross-access | 403 | AUTHZ_TENANT_MISMATCH | AUTHZ"
|
||||
- code: AUTHZ_TENANT_MISMATCH
|
||||
category: AUTHZ
|
||||
http_status: 403
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Permission denied"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://authz/tenant-mismatch"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:authz-category
|
||||
|
||||
# ============================================================
|
||||
# INTERNAL (feature-security-operational-baseline + container-runtime)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-security-operational-baseline L93 — "public path misconfiguration ... | 500 + P1 alert | INTERNAL_AUTH_MISCONFIGURATION | INTERNAL"
|
||||
- code: INTERNAL_AUTH_MISCONFIGURATION
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: crosscut
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/public-path-misconfiguration"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-container-runtime-contract L113 — "JVM OutOfMemoryError → ExitOnOutOfMemoryError로 137 exit, log에 error.code=JVM_OOM 명시"
|
||||
- code: JVM_OOM
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-container-runtime-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://runtime/jvm-oom"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:container-runtime-oom
|
||||
|
||||
# ============================================================
|
||||
# DB / Persistence (feature-persistence-failure-baseline / SQLState Matrix)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-persistence-failure-baseline L85 — "08* | all | TRANSIENT_DEPENDENCY | DB_UNAVAILABLE | true"
|
||||
- code: DB_UNAVAILABLE
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 5
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Service temporarily unavailable"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://db/unavailable"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L86 — "40001 | Postgres/MySQL | CONFLICT | DB_SERIALIZATION_FAILURE | true"
|
||||
- code: DB_SERIALIZATION_FAILURE
|
||||
category: CONFLICT
|
||||
http_status: 409
|
||||
retryable: true
|
||||
retry_after_seconds: 1
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request conflicted with another transaction, please retry"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://db/serialization-failure"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L87 — "40P01 | Postgres | CONFLICT | DB_DEADLOCK | true (backoff)"
|
||||
- code: DB_DEADLOCK
|
||||
category: CONFLICT
|
||||
http_status: 409
|
||||
retryable: true
|
||||
retry_after_seconds: 1
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request conflicted, please retry"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://db/deadlock"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L88 — "23502 | Postgres | DATA_INTEGRITY | DB_NULL_VIOLATION | false"
|
||||
- code: DB_NULL_VIOLATION
|
||||
category: DATA_INTEGRITY
|
||||
http_status: 409
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request violates a required field constraint"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L89 — "23503 | Postgres | DATA_INTEGRITY | DB_FK_VIOLATION | false"
|
||||
- code: DB_FK_VIOLATION
|
||||
category: DATA_INTEGRITY
|
||||
http_status: 409
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request references missing resource"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L90 — "23505 | Postgres | CONFLICT | DB_UNIQUE_VIOLATION | false (business mapping)"
|
||||
- code: DB_UNIQUE_VIOLATION
|
||||
category: CONFLICT
|
||||
http_status: 409
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Resource already exists"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L91 — "23514 | Postgres | DATA_INTEGRITY | DB_CHECK_VIOLATION | false"
|
||||
- code: DB_CHECK_VIOLATION
|
||||
category: DATA_INTEGRITY
|
||||
http_status: 409
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request violates a value constraint"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L92 — "25P03 | Postgres | TRANSIENT_DEPENDENCY | DB_IDLE_IN_TX_TIMEOUT | true"
|
||||
- code: DB_IDLE_IN_TX_TIMEOUT
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 2
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Service temporarily unavailable"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://db/idle-in-tx-timeout"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L93 — "57014 | Postgres | TRANSIENT_DEPENDENCY | DB_QUERY_CANCELED | false"
|
||||
- code: DB_QUERY_CANCELED
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request was canceled, please retry later"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://db/query-canceled"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# ============================================================
|
||||
# Rate limit / Idempotency (feature-rate-limit-idempotency-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract — rate limit response/log 기준 / Retry-After header 기준 (scope L29, L33)
|
||||
- code: RATE_LIMIT_EXCEEDED
|
||||
category: RATE_LIMIT
|
||||
http_status: 429
|
||||
retryable: true
|
||||
retry_after_seconds: 1
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Too many requests, please retry after the indicated interval"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://rate-limit/exceeded"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:rate-limit
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract L71 — "200ms 초과 시 409 IDEMPOTENT_IN_FLIGHT (retryable=false, client는 polling)"
|
||||
- code: IDEMPOTENT_IN_FLIGHT
|
||||
category: CONFLICT
|
||||
http_status: 409
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
owner_layer: application
|
||||
client_safe_message: "A previous identical request is still being processed, please poll for result"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:idempotency
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract L72 — "fingerprint mismatch (same key + different body) = 422 IDEMPOTENT_REQUEST_MISMATCH"
|
||||
- code: IDEMPOTENT_REQUEST_MISMATCH
|
||||
category: VALIDATION
|
||||
http_status: 422
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
owner_layer: application
|
||||
client_safe_message: "Idempotency key reused with different request body"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:idempotency
|
||||
|
||||
# ============================================================
|
||||
# File / Resource (feature-file-resource-handling-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-file-resource-handling-contract L69 — "spring.servlet.multipart.max-file-size 10MB ... Spring 단의 enforcement가 실패 시 envelope 응답 보장" / 테스트 계약 "oversized upload가 generic 500으로 처리되면 실패"
|
||||
- code: UPLOAD_SIZE_EXCEEDED
|
||||
category: VALIDATION
|
||||
http_status: 413
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-file-resource-handling-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Uploaded file exceeds maximum size"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:file-upload
|
||||
|
||||
# source: feature-file-resource-handling-contract L72 — "allowed content-type allowlist starting set ..."
|
||||
- code: UPLOAD_CONTENT_TYPE_REJECTED
|
||||
category: VALIDATION
|
||||
http_status: 415
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-file-resource-handling-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Uploaded content type is not allowed"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:file-upload
|
||||
|
||||
# source: feature-file-resource-handling-contract — Decisionized Work Items "path traversal | normalized storage key only ... | traversal test"
|
||||
- code: PATH_TRAVERSAL_DETECTED
|
||||
category: VALIDATION
|
||||
http_status: 400
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-file-resource-handling-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Invalid file path"
|
||||
log_level: ERROR
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:file-upload
|
||||
|
||||
# source: feature-file-resource-handling-contract L73 — "streaming download backpressure = response timeout 60s, max stream 100MB. 초과 시 truncate + ERROR log"
|
||||
- code: DOWNLOAD_STREAMING_FAILURE
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 5
|
||||
owner_branch: feature-file-resource-handling-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Download failed, please retry"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://file/download-streaming-failure"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:file-download
|
||||
|
||||
# ============================================================
|
||||
# API contract transport-standard codes (feature-api-contract-baseline)
|
||||
# ============================================================
|
||||
# NOTE: feature-api-contract-baseline owns the transport-shape failure
|
||||
# classification (D8 413/414, D9 406/415, D12 405, D15 412). These rows mirror
|
||||
# dev.caskeleton.shared.error.OperationalError; the D11 status-mapping
|
||||
# consistency test (owner: this branch, producer) fails the build when a code's
|
||||
# registry http_status and the enum httpStatus() drift apart.
|
||||
|
||||
# source: feature-api-contract-baseline.md D12 — "405 Method Not Allowed + Allow header 의무"
|
||||
- code: METHOD_NOT_ALLOWED
|
||||
category: VALIDATION
|
||||
http_status: 405
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "HTTP method not allowed for this resource"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# source: feature-api-contract-baseline.md D9 — "406 Not Acceptable = 응답 표현 협상 실패"
|
||||
- code: NOT_ACCEPTABLE
|
||||
category: VALIDATION
|
||||
http_status: 406
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "No acceptable representation for the requested Accept header"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# source: feature-api-contract-baseline.md D15 — "If-Match mismatch 시 412 Precondition Failed"
|
||||
- code: PRECONDITION_FAILED
|
||||
category: CONFLICT
|
||||
http_status: 412
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Resource was modified by another request; refetch and retry"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# source: feature-api-contract-baseline.md D8 — "request size limit 실패 분류 (413)"
|
||||
- code: PAYLOAD_TOO_LARGE
|
||||
category: VALIDATION
|
||||
http_status: 413
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Request payload is too large"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# source: feature-api-contract-baseline.md D8 형제 — "URI 길이 실패 분류 (414)"
|
||||
# NOTE: enforcement is Tomcat/gateway-owned (rejected before Spring dispatch);
|
||||
# this row + code exist for status-mapping consistency. End-to-end 414 contract
|
||||
# test is `planned` (gateway/Tomcat maxHttpHeaderSize 8KB boundary).
|
||||
- code: URI_TOO_LONG
|
||||
category: VALIDATION
|
||||
http_status: 414
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Request URI is too long"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# source: feature-api-contract-baseline.md D9 — "415 Unsupported Media Type = 요청 본문 format 미지원"
|
||||
- code: UNSUPPORTED_MEDIA_TYPE
|
||||
category: VALIDATION
|
||||
http_status: 415
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Request Content-Type is not supported"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# ============================================================
|
||||
# Tenant (feature-tenant-context-policy)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-tenant-context-policy L71 — "tenant 미지원 모드에서 X-Tenant-Id 헤더 수신 시 400 TENANT_NOT_SUPPORTED (filter 단계)"
|
||||
- code: TENANT_NOT_SUPPORTED
|
||||
category: VALIDATION
|
||||
http_status: 400
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-tenant-context-policy
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Tenant context is not supported by this deployment"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:tenant-policy
|
||||
|
||||
# ============================================================
|
||||
# Validation / Business rule (feature-business-rule-validation-contract)
|
||||
# ============================================================
|
||||
# NOTE: business-rule-validation branch는 mapping 규칙 SSOT (syntax→VALIDATION,
|
||||
# policy→AUTHZ/CONFLICT, invariant→CONFLICT/VALIDATION, persistence→PERSISTENCE/CONFLICT)
|
||||
# 이며 구체 code는 example로 VALIDATION_EMAIL_FORMAT만 등장
|
||||
# (feature-operational-error-observability-foundation L110). 실제 도메인별 code는
|
||||
# Phase D(도메인 feature 적용) 시 본 registry에 추가.
|
||||
|
||||
# source: feature-operational-error-observability-foundation L110 — "code: VALIDATION_EMAIL_FORMAT, // registry-registered code" (validation field error JSON shape example)
|
||||
- code: VALIDATION_EMAIL_FORMAT
|
||||
category: VALIDATION
|
||||
http_status: 400
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Invalid email format"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:validation-envelope
|
||||
|
||||
# ============================================================
|
||||
# Cache (feature-cache-consistency-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-cache-consistency-contract — Decisionized Work Items "Redis unavailable | degrade only if declared | fail-fast for required cache | generic INTERNAL | unavailable mapping" / 테스트 "Redis unavailable이 degrade 가능 여부 없이 INTERNAL로 처리되면 실패"
|
||||
- code: CACHE_UNAVAILABLE
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 2
|
||||
owner_branch: feature-cache-consistency-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Service temporarily unavailable"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://cache/unavailable"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:cache-consistency
|
||||
|
||||
# source: feature-cache-consistency-contract L70 — "stampede 방지 default = single-instance Caffeine local lock, multi-instance HPA 시 Redisson RLock distributed mutex" / 테스트 "동일 key에 대해 동시 cache miss 시 backend 호출이 1회로 제한되는지 verify (stampede). 미충족 시 실패"
|
||||
- code: CACHE_STAMPEDE_LOCK_TIMEOUT
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 1
|
||||
owner_branch: feature-cache-consistency-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Service temporarily unavailable"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://cache/stampede-lock-timeout"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:cache-consistency
|
||||
|
||||
# ============================================================
|
||||
# Outbound HTTP (feature-outbound-http-client-baseline)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-outbound-http-client-baseline L70 — "outbound HTTP timeout default = connect 2s / read 5s / global call 10s" + scope "timeout/connect/DNS failure 분류" / 테스트 "upstream timeout은 retryable dependency failure로 분류되어야 함"
|
||||
- code: DEPENDENCY_TIMEOUT
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 504
|
||||
retryable: true
|
||||
retry_after_seconds: 2
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service did not respond in time, please retry"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://dependency/timeout"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# source: feature-outbound-http-client-baseline — scope "timeout/connect/DNS failure 분류" + L70 connect=2s timeout
|
||||
- code: DEPENDENCY_CONNECT_FAILED
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 2
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service unreachable, please retry"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://dependency/connect-failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# source: feature-outbound-http-client-baseline — scope "timeout/connect/DNS failure 분류"
|
||||
- code: DEPENDENCY_DNS_FAILED
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 5
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service unreachable, please retry"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://dependency/dns-failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# source: feature-outbound-http-client-baseline — scope "upstream 4xx/5xx 분류" / 테스트 "401/403은 credential/scope/config 문제로 분류되어야 함"
|
||||
- code: DEPENDENCY_4XX_CLIENT
|
||||
category: PERMANENT_DEPENDENCY
|
||||
http_status: 502
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service rejected the request"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://dependency/4xx-client"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# source: feature-outbound-http-client-baseline — scope "upstream 4xx/5xx 분류"
|
||||
- code: DEPENDENCY_5XX_SERVER
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 502
|
||||
retryable: true
|
||||
retry_after_seconds: 2
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service error, please retry"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://dependency/5xx-server"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# source: feature-outbound-http-client-baseline L69 — "circuit breaker metric은 dependency.name, dependency.type, outcome까지만 tag로 허용" + Decisionized "circuit breaker | Resilience4j optional env"
|
||||
- code: DEPENDENCY_CIRCUIT_OPEN
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 10
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service temporarily unavailable, please retry later"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://dependency/circuit-open"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# ============================================================
|
||||
# Outbox (feature-domain-event-outbox-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-domain-event-outbox-contract L67 — "outbox row status enum = PENDING / IN_FLIGHT / PUBLISHED / FAILED / DEAD" + scope "publish 실패 분류" / 판정 "publish 실패가 retry/DLQ/log/runbook 기준 없이 삼켜지면 실패"
|
||||
- code: OUTBOX_PUBLISH_FAILED
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 500
|
||||
retryable: true
|
||||
retry_after_seconds: 30
|
||||
owner_branch: feature-domain-event-outbox-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://outbox/publish-failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbox-publish
|
||||
|
||||
# source: feature-domain-event-outbox-contract L67 — outbox status enum "DEAD" / Outbox Defaults "DLQ | background-job branch owner"
|
||||
- code: OUTBOX_DEAD_LETTER
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-domain-event-outbox-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://outbox/dead-letter"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbox-dlq
|
||||
|
||||
# ============================================================
|
||||
# Background job / Async (feature-background-job-async-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-background-job-async-contract — Decisionized "saturation | bounded executor + rejection log" / L72 "saturation policy default = AbortPolicy" / 테스트 "executor rejection이 structured log 없이 발생하면 실패"
|
||||
- code: JOB_EXECUTOR_REJECTED
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 5
|
||||
owner_branch: feature-background-job-async-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Service temporarily unavailable"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://job/executor-rejected"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:async-saturation
|
||||
|
||||
# source: feature-background-job-async-contract L69 — "기본 backoff는 exponential backoff with jitter, max attempts 3, DLQ after exhausted attempts" + scope "shutdown 중 job 처리 기준" / L73 graceful shutdown ≤19s
|
||||
- code: JOB_TIMEOUT
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 500
|
||||
retryable: true
|
||||
retry_after_seconds: 10
|
||||
owner_branch: feature-background-job-async-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://job/timeout"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:async-timeout
|
||||
|
||||
# source: feature-background-job-async-contract L69 — "DLQ after exhausted attempts" + Decisionized "retry/DLQ | exp backoff jitter, max 3, DLQ exhausted | ... | infinite retry"
|
||||
- code: JOB_DEAD_LETTER
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-background-job-async-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://job/dead-letter"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:async-dlq
|
||||
|
||||
# ============================================================
|
||||
# Distributed Lock (feature-distributed-lock-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-distributed-lock-contract D7 — "lock 획득 실패/timeout 의 error code =
|
||||
# LOCK_ACQUISITION_TIMEOUT (category CONFLICT, retryable true, client_safe true) + metric
|
||||
# lock.acquisition" / D5 — "try-lock + 유한 waitTime + lease(TTL) 필수, 무한 blocking 금지".
|
||||
# category CONFLICT 는 기존 enum 재사용; retryable=true — 락 보유자가 임계 구역을 빠져나오면
|
||||
# 동일 요청 재시도로 해소된다(transient contention). DB_DEADLOCK / DB_SERIALIZATION_FAILURE 와
|
||||
# 같은 retryable CONFLICT 계열(409). 본 코드는 distributedLockProvider 획득 timeout 전용이며
|
||||
# cache stampede lock 의 CACHE_STAMPEDE_LOCK_TIMEOUT(cache-consistency, TRANSIENT_DEPENDENCY 503)
|
||||
# 과 의미가 구분된다 — 후자는 캐시 백엔드 의존성 timeout, 전자는 분산 상호배제 contention.
|
||||
- code: LOCK_ACQUISITION_TIMEOUT
|
||||
category: CONFLICT
|
||||
http_status: 409
|
||||
retryable: true
|
||||
retry_after_seconds: 1
|
||||
owner_branch: feature-distributed-lock-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Resource is busy, please retry"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://lock/acquisition-timeout"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:lock-acquisition-timeout
|
||||
|
||||
# ============================================================
|
||||
# Migration / Startup (feature-migration-startup-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-migration-startup-contract L71 — "startup exit code 표준 = ... migration 실패=70 ..." + Decisionized "startup failure log | structured log with startup.phase, error.code, error.category"
|
||||
- code: MIGRATION_FAILED
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-migration-startup-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://migration/failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:migration-startup
|
||||
|
||||
# source: feature-migration-startup-contract L71 — "startup exit code 표준 = env 누락/malformed=78 ..." / 테스트 "required env 누락 시 startup이 성공하면 실패"
|
||||
- code: STARTUP_VALIDATION_FAILED
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-migration-startup-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://startup/validation-failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:migration-startup
|
||||
|
||||
# source: feature-migration-startup-contract L71 — "startup exit code 표준 = ... required adapter disabled=72" / 테스트 "disabled required adapter로 app이 뜨면 실패"
|
||||
- code: REQUIRED_ADAPTER_DISABLED
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-migration-startup-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://startup/required-adapter-disabled"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:migration-startup
|
||||
|
||||
# source: feature-integration-adapter-templates §구현 가이드 §4 (Layer 3) + §Audit A2.
|
||||
# Runtime-lifecycle fail-fast for an invoke against a DISABLED optional adapter
|
||||
# (Kafka/Redis/Slack/Google Email). Deliberately distinct from the startup-lifecycle
|
||||
# REQUIRED_ADAPTER_DISABLED above (exit 72): a runtime invoke ≠ a startup validation,
|
||||
# so reusing the startup code would conflate two lifecycles (A2 resolution — new
|
||||
# runtime code owned by this branch). retryable=false: the adapter stays disabled
|
||||
# until redeploy, so retrying the same call never clears it.
|
||||
- code: ADAPTER_DISABLED
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-integration-adapter-templates
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://adapter/adapter-disabled"
|
||||
compatibility_impact: none
|
||||
required_test: adapter-contract:adapter-disabled-runtime-call
|
||||
|
||||
# source: feature-migration-startup-contract L71 — "startup exit code 표준 = ... profile mismatch=71" / 테스트 "prod profile에서 local-only 설정이 켜지면 실패"
|
||||
- code: PROFILE_MISMATCH
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-migration-startup-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://startup/profile-mismatch"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:migration-startup
|
||||
|
||||
# ============================================================
|
||||
# Management / Actuator (feature-management-actuator-security-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-management-actuator-security-contract — Exposure Policy "env/configprops | forbidden" "heapdump/threaddump | forbidden unless break-glass runbook" "shutdown | forbidden" / 테스트 "prod에서 env/configprops endpoint가 노출되면 실패"
|
||||
- code: ACTUATOR_FORBIDDEN
|
||||
category: AUTHZ
|
||||
http_status: 403
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-management-actuator-security-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Permission denied"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://management/actuator-forbidden"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:management-actuator
|
||||
@@ -0,0 +1,220 @@
|
||||
# Registry: HTTP Headers
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/headers.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# Last updated: 2026-05-22
|
||||
#
|
||||
# Conventions:
|
||||
# - HTTP header name: kebab-case (X-Request-Id, X-Tenant-Id)
|
||||
# - W3C standard headers: lowercase (traceparent, tracestate)
|
||||
# - mdc_key: snake_case (foundation SSOT)
|
||||
# - envelope_meta_field: camelCase (envelope SSOT)
|
||||
|
||||
headers:
|
||||
# source: feature-operational-error-observability-foundation.md L97
|
||||
# "request_id | inbound filter (생성 또는 X-Request-Id 헤더) | response header X-Request-Id"
|
||||
- name: X-Request-Id
|
||||
direction: both
|
||||
type: ulid
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: request_id
|
||||
envelope_meta_field: requestId
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:envelope-headers
|
||||
|
||||
# source: feature-api-contract-baseline.md L67
|
||||
# "X-Api-Version은 실험/compatibility 보조 header이며 path version과 충돌하면 path가 우선"
|
||||
- name: X-Api-Version
|
||||
direction: inbound
|
||||
type: string
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:api-versioning
|
||||
|
||||
# source: feature-api-contract-baseline.md L77 / feature-rate-limit-idempotency-contract.md L66-67
|
||||
# "idempotency header 이름은 Idempotency-Key" / "기본 scope는 (authenticatedPrincipal, idempotencyKey, useCaseName)"
|
||||
- name: Idempotency-Key
|
||||
direction: inbound
|
||||
type: string
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:idempotency-replay
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract.md L85 / foundation L85
|
||||
# "RATE_LIMIT | ... | 429 | true (Retry-After 이후)" / "retry-after 기준 없이 429를 반환하면 실패"
|
||||
- name: Retry-After
|
||||
direction: outbound
|
||||
type: duration-seconds
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:rate-limit-headers
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract.md scope L26 (rate limit response/log 기준)
|
||||
# rate-limit 응답 표면 (limit/remaining/reset 3종은 표준 rate-limit signaling)
|
||||
- name: X-RateLimit-Limit
|
||||
direction: outbound
|
||||
type: numeric
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:rate-limit-headers
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract.md scope L26 (rate limit response/log 기준)
|
||||
- name: X-RateLimit-Remaining
|
||||
direction: outbound
|
||||
type: numeric
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:rate-limit-headers
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract.md scope L26 (rate limit response/log 기준)
|
||||
- name: X-RateLimit-Reset
|
||||
direction: outbound
|
||||
type: rfc3339-date
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:rate-limit-headers
|
||||
|
||||
# source: feature-api-compatibility-deprecation-contract.md L87
|
||||
# "deprecation marker | OpenAPI deprecated: true + branch note | response header optional"
|
||||
- name: Deprecation
|
||||
direction: outbound
|
||||
type: rfc3339-date
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-api-compatibility-deprecation-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:deprecation-marker
|
||||
|
||||
# source: feature-api-compatibility-deprecation-contract.md L87
|
||||
# "deprecation marker | OpenAPI deprecated: true + branch note | response header optional" (RFC 8594 Sunset)
|
||||
- name: Sunset
|
||||
direction: outbound
|
||||
type: rfc3339-date
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-api-compatibility-deprecation-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:deprecation-marker
|
||||
|
||||
# source: feature-distributed-tracing-contract.md L64, L85
|
||||
# "propagation header는 W3C traceparent default" / "HTTP | traceparent, tracestate (W3C)"
|
||||
- name: traceparent
|
||||
direction: both
|
||||
type: string
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: trace_id
|
||||
envelope_meta_field: traceId
|
||||
owner_branch: feature-distributed-tracing-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:trace-propagation
|
||||
|
||||
# source: feature-distributed-tracing-contract.md L66, L85
|
||||
# "propagation format = W3C traceparent + tracestate only. B3 propagation은 forbidden"
|
||||
- name: tracestate
|
||||
direction: both
|
||||
type: comma-separated
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-distributed-tracing-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:trace-propagation
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L100
|
||||
# "correlation_id | inbound header X-Correlation-Id 또는 생성 | HTTP X-Correlation-Id, message header correlation_id"
|
||||
- name: X-Correlation-Id
|
||||
direction: both
|
||||
type: ulid
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: correlation_id
|
||||
envelope_meta_field: correlationId
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:envelope-headers
|
||||
|
||||
# source: feature-tenant-context-policy.md L69, L101 (foundation)
|
||||
# "tenant resolution 우선순위 = ... (2) 명시적 X-Tenant-Id 헤더 (admin/internal API only)" /
|
||||
# "tenant_id | tenant context (활성 시) | downstream HTTP X-Tenant-Id (with allowlist)"
|
||||
- name: X-Tenant-Id
|
||||
direction: both
|
||||
type: ulid
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: tenant_id
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-tenant-context-policy
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:tenant-header-policy
|
||||
|
||||
# source: feature-security-operational-baseline.md L66
|
||||
# "JWT Resource Server를 baseline security model로 둠" (Bearer token via Authorization header)
|
||||
- name: Authorization
|
||||
direction: inbound
|
||||
type: bearer-token
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:jwt-resource-server
|
||||
|
||||
# source: feature-security-operational-baseline.md L83-90 (AuthN/AuthZ Decision Matrix)
|
||||
# 401 응답 시 WWW-Authenticate (Bearer realm/error) — Spring Security JWT Resource Server 표준 challenge header
|
||||
- name: WWW-Authenticate
|
||||
direction: outbound
|
||||
type: string
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:jwt-resource-server
|
||||
@@ -0,0 +1,294 @@
|
||||
# Registry: MDC / Log Keys
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/mdc-keys.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# MDC SSOT: feature-operational-error-observability-foundation
|
||||
# Last updated: 2026-05-22
|
||||
#
|
||||
# Conventions:
|
||||
# - MDC key naming: snake_case (foundation L93 "snake_case 강제. camelCase / dot.case 금지.")
|
||||
# - cardinality_safe_for_metric=true 인 key만 metric tag로 사용 가능
|
||||
# - foundation L93-102 표 "MDC Key Standard (final)" 6개가 core SSOT
|
||||
|
||||
mdc_keys:
|
||||
# source: feature-operational-error-observability-foundation.md L97
|
||||
# "request_id | inbound filter (생성 또는 X-Request-Id 헤더) | response header X-Request-Id"
|
||||
- key: request_id
|
||||
type: ulid
|
||||
source: inbound_filter
|
||||
required_in: [request, dependency, security, application]
|
||||
http_header_mapping: X-Request-Id
|
||||
envelope_field: meta.requestId
|
||||
propagation: [http, async, message]
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-mdc-keys
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L98
|
||||
# "trace_id | Micrometer Tracing | W3C traceparent header"
|
||||
- key: trace_id
|
||||
type: string
|
||||
source: observation_context
|
||||
required_in: [request, dependency, application]
|
||||
http_header_mapping: traceparent
|
||||
envelope_field: meta.traceId
|
||||
propagation: [http, async, message]
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-mdc-keys
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L99
|
||||
# "span_id | Micrometer Tracing | W3C traceparent"
|
||||
# NOTE: background-job-async-contract L71 "span_id는 Micrometer Observation context에서 자동 전파(MDC explicit copy 불필요)"
|
||||
- key: span_id
|
||||
type: string
|
||||
source: observation_context
|
||||
required_in: [request, dependency]
|
||||
http_header_mapping: traceparent
|
||||
envelope_field: null
|
||||
propagation: [http, async]
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-mdc-keys
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L100
|
||||
# "correlation_id | inbound header X-Correlation-Id 또는 생성 | HTTP X-Correlation-Id, message header correlation_id"
|
||||
- key: correlation_id
|
||||
type: ulid
|
||||
source: inbound_filter
|
||||
required_in: [request, dependency, application]
|
||||
http_header_mapping: X-Correlation-Id
|
||||
envelope_field: meta.correlationId
|
||||
propagation: [http, async, message]
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-mdc-keys
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L101 + feature-tenant-context-policy.md L70
|
||||
# "tenant_id | tenant context (활성 시) | downstream HTTP X-Tenant-Id (with allowlist)" /
|
||||
# "tenant ID format = opaque ULID (26 chars Crockford base32)"
|
||||
# NOTE: tenant L73 "tenant_id ULID 원본은 metric tag에 직접 사용 금지"
|
||||
- key: tenant_id
|
||||
type: ulid
|
||||
source: security_context
|
||||
required_in: [request, dependency, security, audit]
|
||||
http_header_mapping: X-Tenant-Id
|
||||
envelope_field: null
|
||||
propagation: [http, async, message]
|
||||
owner_branch: feature-tenant-context-policy
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:tenant-leakage
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L102
|
||||
# "user_principal | security context (pseudonymized only) | log only, headers forbidden"
|
||||
- key: user_principal
|
||||
type: string
|
||||
source: security_context
|
||||
required_in: [security, audit]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-mdc-keys
|
||||
|
||||
# ── log type extensions (log-management-contract L101-109 "Log Type별 필수 필드") ──
|
||||
|
||||
# source: feature-log-management-contract.md L105 "request | request_id, trace_id, method, uri_template, status, duration_ms"
|
||||
# NOTE: application-port-usecase-contract / business 측 operation 식별자 (uri_template과 별도 application-set)
|
||||
- key: operation
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [application, dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L105 "request | request_id, trace_id, method, uri_template, status, duration_ms"
|
||||
- key: method
|
||||
type: string
|
||||
source: inbound_filter
|
||||
required_in: [request]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L105 "request | request_id, trace_id, method, uri_template, status, duration_ms"
|
||||
# NOTE: metrics L86 "status_code | 7 (1xx-5xx + ok/other)" — bounded
|
||||
- key: status
|
||||
type: numeric
|
||||
source: inbound_filter
|
||||
required_in: [request]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L105-106 "request | ... duration_ms" / "dependency | ... duration_ms"
|
||||
- key: duration_ms
|
||||
type: numeric
|
||||
source: application_set
|
||||
required_in: [request, dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L106 "dependency | dependency_name, dependency_type, duration_ms, outcome, error_code"
|
||||
# NOTE: metrics L88 "dependency_name | 50" — bounded
|
||||
- key: dependency_name
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L106 "dependency | dependency_name, dependency_type, ..."
|
||||
- key: dependency_type
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L106 + metrics L91 "outcome (resilience4j) | 5 (SUCCESS/FAILURE/CIRCUIT_OPEN/TIMEOUT/REJECTED)"
|
||||
- key: outcome
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L106 "dependency | ... error_code (실패 시)"
|
||||
# NOTE: metrics L89 "error_code | 100 — error registry row 상한과 정합" — bounded
|
||||
- key: error_code
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L107
|
||||
# "security | event_type, user_principal (pseudonymized), source_ip (anonymized — last octet zeroed)"
|
||||
- key: event_type
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [security, audit]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L107 "security | ... source_ip (anonymized — last octet zeroed)"
|
||||
# NOTE: metrics L93 "high-cardinality 금지 tag: ... ip_address"
|
||||
- key: source_ip_anon
|
||||
type: string
|
||||
source: inbound_filter
|
||||
required_in: [security]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L108 "audit | actor, action, target, before_hash, after_hash, occurred_at"
|
||||
- key: actor
|
||||
type: string
|
||||
source: security_context
|
||||
required_in: [audit]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L108 "audit | actor, action, target, ..."
|
||||
- key: action
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [audit]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L108 "audit | actor, action, target, ..."
|
||||
- key: target
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [audit]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
@@ -0,0 +1,845 @@
|
||||
# Registry: Metrics
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/metrics.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# Owner branch: feature-metrics-alerting-contract
|
||||
# Last updated: 2026-05-22
|
||||
#
|
||||
# Notes
|
||||
# - Naming: Micrometer dot.case + unit suffix (.seconds | .bytes | .total).
|
||||
# - Tag cardinality bounds are SSOT of feature-metrics-alerting-contract "Cardinality Bounds" table.
|
||||
# - High-cardinality tags forbidden globally: user_id, request_id, raw_url, raw_query,
|
||||
# raw_header_value, ip_address. These MUST NOT appear in any row.
|
||||
# - tenant_id label is bounded mapping table id OR cohort bucket only (ULID raw forbidden).
|
||||
# - error_code tag cardinality_limit follows error-codes.yaml row count (max 100).
|
||||
|
||||
metrics:
|
||||
|
||||
# === HTTP server (inbound) ===
|
||||
# source: feature-metrics-alerting-contract — Metric/Alert Defaults
|
||||
# "HTTP metric | http.server.requests with method/status/uri-template | raw URL or user id tag"
|
||||
- name: http.server.requests
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: method
|
||||
cardinality_limit: 8
|
||||
allowed_values: [GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS, OTHER]
|
||||
- name: status
|
||||
cardinality_limit: 7
|
||||
allowed_values: [1xx, 2xx, 3xx, 4xx, 5xx, ok, other]
|
||||
- name: uri_template
|
||||
cardinality_limit: 200
|
||||
validation: must_be_template_not_raw_uri
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p1: "error_rate > 5% for 5m OR > 10% for 1m"
|
||||
p2: "error_rate > 1% for 10m"
|
||||
p3: "error_rate > 0.1% for 1h"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [method, status, uri_template]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — P1/P2/P3 정량 기준 (HTTP latency p99)
|
||||
# "P1: p99 > 5s 5분 / P2: p99 > 1s 10분 / P3: p99 > 500ms 30분"
|
||||
- name: http.server.requests.latency
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: method
|
||||
cardinality_limit: 8
|
||||
allowed_values: [GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS, OTHER]
|
||||
- name: uri_template
|
||||
cardinality_limit: 200
|
||||
validation: must_be_template_not_raw_uri
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p1: "p99 > 5s for 5m"
|
||||
p2: "p99 > 1s for 10m"
|
||||
p3: "p99 > 500ms for 30m"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [method, uri_template, duration_ms]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === HTTP client (outbound dependency) ===
|
||||
# source: feature-metrics-alerting-contract — Metric/Alert Defaults
|
||||
# "dependency metric | dependency.client.requests with dependency.name/type/outcome | endpoint with secret tag"
|
||||
# source: feature-outbound-http-client-baseline — "circuit breaker metric은 dependency.name, dependency.type, outcome까지만 tag로 허용"
|
||||
- name: dependency.client.requests
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: dependency_name
|
||||
cardinality_limit: 50
|
||||
- name: dependency_type
|
||||
cardinality_limit: 10
|
||||
allowed_values: [http, grpc, db, cache, queue, broker, other]
|
||||
- name: outcome
|
||||
cardinality_limit: 5
|
||||
allowed_values: [SUCCESS, FAILURE, CIRCUIT_OPEN, TIMEOUT, REJECTED]
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p1: "required dep unavailable for 2m"
|
||||
p2: "optional dep degraded for 5m"
|
||||
p3: "spike alert (10x baseline)"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [dependency_name, dependency_type, outcome, duration_ms]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-outbound-http-client-baseline — decisions
|
||||
# "retry/circuit breaker 기본 라이브러리는 Resilience4j"
|
||||
# source: feature-metrics-alerting-contract — "retry/CB minimum: resilience4j.retry.calls{outcome}"
|
||||
- name: resilience4j.retry.calls
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: name
|
||||
cardinality_limit: 50
|
||||
- name: outcome
|
||||
cardinality_limit: 5
|
||||
allowed_values: [SUCCESS, FAILURE, CIRCUIT_OPEN, TIMEOUT, REJECTED]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "retry exhaustion rate > 1% for 10m"
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
log_field_mapping: [dependency_name, outcome, retry_attempt]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — "resilience4j.circuitbreaker.state"
|
||||
- name: resilience4j.circuitbreaker.state
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: name
|
||||
cardinality_limit: 50
|
||||
- name: state
|
||||
cardinality_limit: 6
|
||||
allowed_values: [CLOSED, OPEN, HALF_OPEN, DISABLED, FORCED_OPEN, METRICS_ONLY]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "state == OPEN for required dependency for 2m"
|
||||
p2: "state == OPEN for optional dependency for 5m"
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
log_field_mapping: [dependency_name]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — "resilience4j.circuitbreaker.calls{outcome}"
|
||||
- name: resilience4j.circuitbreaker.calls
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: name
|
||||
cardinality_limit: 50
|
||||
- name: outcome
|
||||
cardinality_limit: 5
|
||||
allowed_values: [SUCCESS, FAILURE, CIRCUIT_OPEN, TIMEOUT, REJECTED]
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p2: "CIRCUIT_OPEN rate > 1% for 10m"
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
log_field_mapping: [dependency_name, outcome, duration_ms]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === DB connection pool ===
|
||||
# source: feature-persistence-failure-baseline — Hikari Alert Threshold
|
||||
# "pool wait p99 > 100ms 5분 지속 → P2 / pool exhaustion (active = max) > 1분 → P1"
|
||||
# source: feature-metrics-alerting-contract — "hikaricp.connections.acquire{outcome='timeout'} p99 > 100ms"
|
||||
- name: hikaricp.connections.acquire
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: pool
|
||||
cardinality_limit: 5
|
||||
- name: outcome
|
||||
cardinality_limit: 3
|
||||
allowed_values: [SUCCESS, TIMEOUT, FAILURE]
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p1: "pool exhaustion (active == max) for 1m"
|
||||
p2: "acquire p99 > 100ms for 5m"
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
log_field_mapping: [pool, outcome]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-persistence-failure-baseline — In scope "Hikari metric 노출 기준" + Hikari Alert Threshold
|
||||
- name: hikaricp.connections.usage
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: pool
|
||||
cardinality_limit: 5
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p2: "usage p99 elevated > 10m"
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
log_field_mapping: [pool]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-persistence-failure-baseline — In scope "Hikari metric 노출 기준"
|
||||
- name: hikaricp.connections.active
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: pool
|
||||
cardinality_limit: 5
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "active == max for 1m"
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
log_field_mapping: [pool]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — Histogram Buckets/Percentile "DB query: same"
|
||||
- name: db.query.duration
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: operation
|
||||
cardinality_limit: 20
|
||||
allowed_values: [select, insert, update, delete, batch, ddl, other]
|
||||
- name: outcome
|
||||
cardinality_limit: 3
|
||||
allowed_values: [SUCCESS, FAILURE, TIMEOUT]
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p2: "p99 > 1s for 10m"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [operation, outcome, duration_ms]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Background job / async executor ===
|
||||
# source: feature-background-job-async-contract — Decisionized Work Items "saturation policy"
|
||||
# "AbortPolicy default (core=10, max=50, queue=200)"
|
||||
- name: executor.saturation
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: executor_name
|
||||
cardinality_limit: 10
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "queue size > 80% capacity for 5m"
|
||||
p1: "rejection rate > 0 for 1m"
|
||||
owner_branch: feature-background-job-async-contract
|
||||
log_field_mapping: [executor_name]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-background-job-async-contract — Decisionized Work Items "saturation | bounded executor + rejection log"
|
||||
- name: executor.rejected.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: executor_name
|
||||
cardinality_limit: 10
|
||||
- name: policy
|
||||
cardinality_limit: 3
|
||||
allowed_values: [AbortPolicy, CallerRunsPolicy, DiscardPolicy]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "rejection_count > 0 for 1m"
|
||||
owner_branch: feature-background-job-async-contract
|
||||
log_field_mapping: [executor_name, policy]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-background-job-async-contract — Decisionized Work Items "retry/DLQ | exp backoff jitter, max 3, DLQ exhausted"
|
||||
- name: job.retry.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: job_name
|
||||
cardinality_limit: 50
|
||||
- name: outcome
|
||||
cardinality_limit: 4
|
||||
allowed_values: [SUCCESS, RETRY, EXHAUSTED, DLQ]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "EXHAUSTED rate > 1% for 10m"
|
||||
owner_branch: feature-background-job-async-contract
|
||||
log_field_mapping: [job_name, outcome, retry_attempt]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-background-job-async-contract — "DLQ after exhausted attempts"
|
||||
- name: job.dlq.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: job_name
|
||||
cardinality_limit: 50
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "DLQ rate sustained > 0 for 5m"
|
||||
owner_branch: feature-background-job-async-contract
|
||||
log_field_mapping: [job_name]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Distributed lock ===
|
||||
# source: feature-distributed-lock-contract D7 — "metric lock.acquisition (tag: outcome =
|
||||
# acquired/timeout/error) — 신규 제안" / D5 — try-lock + 유한 waitTime + lease(TTL). 분산
|
||||
# 상호배제(distributedLockProvider) 획득 시도 결과를 센다. key 는 tag 로 넣지 않는다
|
||||
# (무한 cardinality — 위 전역 금지 규칙). timeout outcome 은 LOCK_ACQUISITION_TIMEOUT 발생과 1:1.
|
||||
- name: lock.acquisition
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: outcome
|
||||
cardinality_limit: 3
|
||||
allowed_values: [acquired, timeout, error]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "timeout rate > 5% for 10m"
|
||||
owner_branch: feature-distributed-lock-contract
|
||||
log_field_mapping: [outcome]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-distributed-lock-contract §Edge / D5 (SI-LOCK-C5) — "lease 만료 후 unlock →
|
||||
# ConcurrentModificationException — 삼킴 금지, 로그+metric 후 정상 흐름 복귀". Counts releases that
|
||||
# found the lease already expired (the JdbcLock row was reclaimed by another instance before
|
||||
# the holder called close()). A sustained nonzero rate means lease TTL is shorter than real
|
||||
# critical-section duration — raise APP/lease TTL or shorten the protected work. Not an
|
||||
# acquisition outcome, hence a separate counter from lock.acquisition.
|
||||
- name: lock.lease.expired
|
||||
type: counter
|
||||
unit: total
|
||||
tags: []
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "lease-expired rate sustained > 0 for 10m"
|
||||
owner_branch: feature-distributed-lock-contract
|
||||
log_field_mapping: []
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Outbox publisher ===
|
||||
# source: feature-domain-event-outbox-contract — Outbox Defaults
|
||||
# "DB outbox table with eventId, aggregateId, eventType, payload, occurredAt, status, attemptCount, nextAttemptAt"
|
||||
- name: outbox.publisher.published.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: event_type
|
||||
cardinality_limit: 50
|
||||
- name: outcome
|
||||
cardinality_limit: 4
|
||||
allowed_values: [PUBLISHED, FAILED, DEAD, IN_FLIGHT]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "FAILED rate > 1% for 10m"
|
||||
owner_branch: feature-domain-event-outbox-contract
|
||||
log_field_mapping: [event_type, outcome, event_id]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-domain-event-outbox-contract — row status enum PENDING/IN_FLIGHT/PUBLISHED/FAILED/DEAD
|
||||
- name: outbox.publisher.lag
|
||||
type: gauge
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: event_type
|
||||
cardinality_limit: 50
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "lag > 60s for 10m"
|
||||
p1: "lag > 300s for 5m"
|
||||
owner_branch: feature-domain-event-outbox-contract
|
||||
log_field_mapping: [event_type]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-domain-event-outbox-contract — row status enum + Outbox Defaults attemptCount
|
||||
- name: outbox.pending.size
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: status
|
||||
cardinality_limit: 5
|
||||
allowed_values: [PENDING, IN_FLIGHT, PUBLISHED, FAILED, DEAD]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "PENDING size growing for 10m"
|
||||
owner_branch: feature-domain-event-outbox-contract
|
||||
log_field_mapping: [status]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Cache ===
|
||||
# source: feature-cache-consistency-contract — Decisionized Work Items "cache pattern | cache-aside default"
|
||||
- name: cache.gets.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: cache_name
|
||||
cardinality_limit: 50
|
||||
- name: result
|
||||
cardinality_limit: 3
|
||||
allowed_values: [hit, miss, error]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p3: "hit_ratio < baseline 0.5x for 1h"
|
||||
owner_branch: feature-cache-consistency-contract
|
||||
log_field_mapping: [cache_name, result]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-cache-consistency-contract — "invalidation = after-commit only", "invalidation 실패가 조용히 무시되면 실패"
|
||||
- name: cache.invalidations.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: cache_name
|
||||
cardinality_limit: 50
|
||||
- name: outcome
|
||||
cardinality_limit: 3
|
||||
allowed_values: [SUCCESS, FAILURE, SKIPPED]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "FAILURE rate > 0 for 5m"
|
||||
owner_branch: feature-cache-consistency-contract
|
||||
log_field_mapping: [cache_name, outcome]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: redis-production-capability — optional bounded cache-only L1
|
||||
- name: cache.local.requests.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: cache_name
|
||||
cardinality_limit: 50
|
||||
- name: result
|
||||
cardinality_limit: 4
|
||||
allowed_values: [hit, miss, error, bypass]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p3: "bypass or error rate above baseline for 15m"
|
||||
owner_branch: redis-production-capability
|
||||
log_field_mapping: [cache_name, result]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: redis-production-capability — local stale-age bound
|
||||
- name: cache.local.entry.age.seconds
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: cache_name
|
||||
cardinality_limit: 50
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p3: "p99 approaches configured local TTL for 30m"
|
||||
owner_branch: redis-production-capability
|
||||
log_field_mapping: [cache_name]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: redis-production-capability — bounded invalidation and generation reconciliation
|
||||
- name: cache.local.maintenance.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: cache_name
|
||||
cardinality_limit: 50
|
||||
- name: event
|
||||
cardinality_limit: 14
|
||||
allowed_values:
|
||||
- evict_cardinality
|
||||
- evict_weight
|
||||
- evict_ttl
|
||||
- evict_invalidation
|
||||
- flush_invalidation
|
||||
- reconcile_generation_changed
|
||||
- reconcile_unchanged
|
||||
- reconcile_error
|
||||
- subscriber_disconnected
|
||||
- subscriber_overflow
|
||||
- subscriber_malformed
|
||||
- subscriber_publish_success
|
||||
- subscriber_publish_error
|
||||
- other
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "reconcile_error, subscriber_overflow, or sustained disconnects for 5m"
|
||||
owner_branch: redis-production-capability
|
||||
log_field_mapping: [cache_name, event]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: redis-production-capability Task 16 — closed semantic operation outcomes
|
||||
- name: redis.capability.operations.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: capability
|
||||
cardinality_limit: 6
|
||||
allowed_values: [cache, rate_limit, idempotency, efficiency_lease, session, runtime]
|
||||
- name: role
|
||||
cardinality_limit: 3
|
||||
allowed_values: [cache, coordination, session]
|
||||
- name: operation
|
||||
cardinality_limit: 24
|
||||
allowed_values:
|
||||
- lookup
|
||||
- record
|
||||
- invalidate
|
||||
- refresh_claim
|
||||
- refresh_release
|
||||
- rate_evaluate
|
||||
- idempotency_claim
|
||||
- idempotency_start
|
||||
- idempotency_renew
|
||||
- idempotency_complete
|
||||
- idempotency_fail
|
||||
- idempotency_release
|
||||
- idempotency_inspect
|
||||
- lease_acquire
|
||||
- lease_inspect
|
||||
- lease_renew
|
||||
- lease_release
|
||||
- session_create
|
||||
- session_inspect
|
||||
- session_save
|
||||
- session_touch
|
||||
- session_revoke
|
||||
- session_rotate
|
||||
- route_command
|
||||
- name: redis_outcome
|
||||
cardinality_limit: 15
|
||||
allowed_values:
|
||||
- success
|
||||
- hit
|
||||
- miss
|
||||
- denied
|
||||
- contended
|
||||
- conflict
|
||||
- incompatible
|
||||
- unavailable
|
||||
- overloaded
|
||||
- closed
|
||||
- indeterminate
|
||||
- stale
|
||||
- skipped
|
||||
- tombstoned
|
||||
- absolute_expired
|
||||
- name: certainty
|
||||
cardinality_limit: 3
|
||||
allowed_values: [definite, not_applied, indeterminate]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "required coordination/session unavailable or indeterminate mutation sustained for 2m"
|
||||
p2: "optional cache unavailable or overloaded above baseline for 5m"
|
||||
owner_branch: redis-production-capability
|
||||
log_field_mapping: [capability, role, operation, redis_outcome, certainty]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: redis-production-capability Task 16 — monotonic semantic operation duration
|
||||
- name: redis.capability.duration.seconds
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: capability
|
||||
cardinality_limit: 6
|
||||
allowed_values: [cache, rate_limit, idempotency, efficiency_lease, session, runtime]
|
||||
- name: role
|
||||
cardinality_limit: 3
|
||||
allowed_values: [cache, coordination, session]
|
||||
- name: operation
|
||||
cardinality_limit: 24
|
||||
allowed_values:
|
||||
- lookup
|
||||
- record
|
||||
- invalidate
|
||||
- refresh_claim
|
||||
- refresh_release
|
||||
- rate_evaluate
|
||||
- idempotency_claim
|
||||
- idempotency_start
|
||||
- idempotency_renew
|
||||
- idempotency_complete
|
||||
- idempotency_fail
|
||||
- idempotency_release
|
||||
- idempotency_inspect
|
||||
- lease_acquire
|
||||
- lease_inspect
|
||||
- lease_renew
|
||||
- lease_release
|
||||
- session_create
|
||||
- session_inspect
|
||||
- session_save
|
||||
- session_touch
|
||||
- session_revoke
|
||||
- session_rotate
|
||||
- route_command
|
||||
- name: redis_outcome
|
||||
cardinality_limit: 15
|
||||
allowed_values:
|
||||
- success
|
||||
- hit
|
||||
- miss
|
||||
- denied
|
||||
- contended
|
||||
- conflict
|
||||
- incompatible
|
||||
- unavailable
|
||||
- overloaded
|
||||
- closed
|
||||
- indeterminate
|
||||
- stale
|
||||
- skipped
|
||||
- tombstoned
|
||||
- absolute_expired
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p2: "p99 approaches the configured command or caller deadline for 10m"
|
||||
owner_branch: redis-production-capability
|
||||
log_field_mapping: [capability, role, operation, redis_outcome]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: redis-production-capability Task 16 — admission rejected before command ownership
|
||||
- name: redis.capability.admission.rejected.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: role
|
||||
cardinality_limit: 3
|
||||
allowed_values: [cache, coordination, session]
|
||||
- name: admission
|
||||
cardinality_limit: 2
|
||||
allowed_values: [rejected_saturated, rejected_closed]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "required role rejection sustained above zero for 2m"
|
||||
p2: "optional cache saturation sustained for 5m"
|
||||
owner_branch: redis-production-capability
|
||||
log_field_mapping: [role, admission]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: redis-production-capability Task 16 — bounded admitted command count observation
|
||||
- name: redis.capability.inflight.total
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: role
|
||||
cardinality_limit: 3
|
||||
allowed_values: [cache, coordination, session]
|
||||
- name: state
|
||||
cardinality_limit: 3
|
||||
allowed_values: [idle, active, saturated]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "saturated series remains nonzero for 5m"
|
||||
owner_branch: redis-production-capability
|
||||
log_field_mapping: [role, state]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: redis-production-capability Task 16 — observations of exact sanitized RoleHealth
|
||||
- name: redis.capability.readiness.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: capability
|
||||
cardinality_limit: 5
|
||||
allowed_values: [cache, rate_limit, idempotency, efficiency_lease, session]
|
||||
- name: role
|
||||
cardinality_limit: 3
|
||||
allowed_values: [cache, coordination, session]
|
||||
- name: state
|
||||
cardinality_limit: 3
|
||||
allowed_values: [available, unavailable, overloaded]
|
||||
- name: reason
|
||||
cardinality_limit: 11
|
||||
allowed_values:
|
||||
- command_unavailable
|
||||
- route_closed
|
||||
- semantic_probe_succeeded
|
||||
- semantic_read_write_failed
|
||||
- semantic_program_acl_denied
|
||||
- semantic_program_failed
|
||||
- server_version_unsupported
|
||||
- semantic_probe_in_progress
|
||||
- semantic_observation_stale
|
||||
- command_saturated
|
||||
- recent_command_failure
|
||||
- name: requirement
|
||||
cardinality_limit: 2
|
||||
allowed_values: [optional, required]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "required coordination/session unavailable for 2m"
|
||||
p2: "optional cache unavailable or overloaded for 5m"
|
||||
owner_branch: redis-production-capability
|
||||
log_field_mapping: [capability, role, state, reason, requirement]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: redis-production-capability Task 16 — bounded router shutdown drain result
|
||||
- name: redis.capability.lifecycle.drain.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: role
|
||||
cardinality_limit: 3
|
||||
allowed_values: [cache, coordination, session]
|
||||
- name: drain_outcome
|
||||
cardinality_limit: 3
|
||||
allowed_values: [drained, forced_after_timeout, interrupted]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "required role forced_after_timeout or interrupted during shutdown"
|
||||
p2: "optional cache forced close during shutdown"
|
||||
owner_branch: redis-production-capability
|
||||
log_field_mapping: [role, drain_outcome]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Log appender ===
|
||||
# source: feature-log-management-contract — Sampling Policy (final)
|
||||
# "async appender overflow default: drop oldest INFO/DEBUG with counter metric (log.appender.dropped.total)"
|
||||
- name: log.appender.dropped.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: appender
|
||||
cardinality_limit: 5
|
||||
- name: level
|
||||
cardinality_limit: 2
|
||||
allowed_values: [INFO, DEBUG]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "dropped > 0 sustained for 10m"
|
||||
owner_branch: feature-log-management-contract
|
||||
log_field_mapping: [appender, level]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Distributed tracing ===
|
||||
# source: feature-distributed-tracing-contract — decisions
|
||||
# "trace sampling rate default = prod 1%, staging 10%, dev/local 100%"
|
||||
- name: tracing.sampling.rate
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: profile
|
||||
cardinality_limit: 4
|
||||
allowed_values: [prod, staging, dev, local]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p3: "effective rate deviates from configured for 1h"
|
||||
owner_branch: feature-distributed-tracing-contract
|
||||
log_field_mapping: [profile]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === JVM baseline ===
|
||||
# source: feature-metrics-alerting-contract — In scope "JVM/process metric"
|
||||
- name: jvm.memory.used
|
||||
type: gauge
|
||||
unit: bytes
|
||||
tags:
|
||||
- name: area
|
||||
cardinality_limit: 2
|
||||
allowed_values: [heap, nonheap]
|
||||
- name: id
|
||||
cardinality_limit: 10
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "heap used / max > 0.85 for 10m"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [area, id]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — In scope "JVM/process metric"
|
||||
- name: jvm.gc.pause
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: action
|
||||
cardinality_limit: 10
|
||||
- name: cause
|
||||
cardinality_limit: 10
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p2: "p99 pause > 500ms for 10m"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [action, cause]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — In scope "JVM/process metric"
|
||||
- name: jvm.threads.live
|
||||
type: gauge
|
||||
unit: total
|
||||
tags: []
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p3: "thread count > 2x baseline for 30m"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: []
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — In scope "JVM/process metric" (process uptime)
|
||||
- name: process.uptime
|
||||
type: gauge
|
||||
unit: seconds
|
||||
tags: []
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "uptime reset unexpectedly < 60s (crash loop signal)"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: []
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
@@ -0,0 +1,118 @@
|
||||
# Repository owner test: dev.caskeleton.bootstrap.contract.ContractRegistrySchemaGovernanceTest
|
||||
# Owner Gradle path: :app-bootstrap:test
|
||||
# Semantic owner test: dev.caskeleton.adapter.outbound.objectstorage.readiness.ObjectStorageReadinessRegistryTest
|
||||
# Semantic owner Gradle path: :adapter:outbound:objectstorage:test
|
||||
schema_version: 1
|
||||
claims:
|
||||
- card_id: object-storage-managed-upload-single
|
||||
provider_type: filesystem-local-dev
|
||||
provider_version: jdk-21
|
||||
destination_profile: local-managed-integrity
|
||||
claimed_level: R1
|
||||
evidence_revision: batch-b-local-r1
|
||||
evidence_expires_on: ""
|
||||
required_tasks:
|
||||
- ":adapter:outbound:objectstorage:check"
|
||||
limitations:
|
||||
- single-process control CAS only; no multi-node linearizability
|
||||
- fsync and atomic move tests do not prove power-loss durability
|
||||
- local development provider is forbidden in production profiles
|
||||
|
||||
- card_id: object-storage-managed-upload-multipart
|
||||
provider_type: filesystem-local-dev
|
||||
provider_version: jdk-21
|
||||
destination_profile: local-unimplemented
|
||||
claimed_level: R0
|
||||
evidence_revision: batch-b-contract-r0
|
||||
evidence_expires_on: ""
|
||||
required_tasks:
|
||||
- ":application-core:check"
|
||||
limitations:
|
||||
- multipart publication protocol is not implemented
|
||||
|
||||
- card_id: object-storage-managed-download
|
||||
provider_type: filesystem-local-dev
|
||||
provider_version: jdk-21
|
||||
destination_profile: local-managed-integrity
|
||||
claimed_level: R1
|
||||
evidence_revision: batch-b-local-r1
|
||||
evidence_expires_on: ""
|
||||
required_tasks:
|
||||
- ":adapter:outbound:objectstorage:check"
|
||||
limitations:
|
||||
- local functional full and range reads are not production-provider qualification
|
||||
- no multi-node or power-loss durability claim
|
||||
|
||||
- card_id: object-storage-direct-upload-single
|
||||
provider_type: filesystem-local-dev
|
||||
provider_version: jdk-21
|
||||
destination_profile: local-unimplemented
|
||||
claimed_level: R0
|
||||
evidence_revision: batch-b-contract-r0
|
||||
evidence_expires_on: ""
|
||||
required_tasks:
|
||||
- ":application-core:check"
|
||||
limitations:
|
||||
- direct grant provider and public inbound endpoint are not implemented
|
||||
|
||||
- card_id: object-storage-direct-upload-multipart
|
||||
provider_type: filesystem-local-dev
|
||||
provider_version: jdk-21
|
||||
destination_profile: local-unimplemented
|
||||
claimed_level: R0
|
||||
evidence_revision: batch-b-contract-r0
|
||||
evidence_expires_on: ""
|
||||
required_tasks:
|
||||
- ":application-core:check"
|
||||
limitations:
|
||||
- direct multipart session and public inbound endpoint are not implemented
|
||||
|
||||
- card_id: object-storage-direct-download
|
||||
provider_type: filesystem-local-dev
|
||||
provider_version: jdk-21
|
||||
destination_profile: local-unimplemented
|
||||
claimed_level: R0
|
||||
evidence_revision: batch-b-contract-r0
|
||||
evidence_expires_on: ""
|
||||
required_tasks:
|
||||
- ":application-core:check"
|
||||
limitations:
|
||||
- direct download grant and public inbound endpoint are not implemented
|
||||
|
||||
- card_id: object-storage-quarantine-publication
|
||||
provider_type: filesystem-local-dev
|
||||
provider_version: jdk-21
|
||||
destination_profile: local-unimplemented
|
||||
claimed_level: R0
|
||||
evidence_revision: batch-b-contract-r0
|
||||
evidence_expires_on: ""
|
||||
required_tasks:
|
||||
- ":application-core:check"
|
||||
limitations:
|
||||
- scanner handoff and verdict fencing are not implemented
|
||||
|
||||
- card_id: object-storage-retention
|
||||
provider_type: filesystem-local-dev
|
||||
provider_version: jdk-21
|
||||
destination_profile: local-unimplemented
|
||||
claimed_level: R0
|
||||
evidence_revision: batch-b-contract-r0
|
||||
evidence_expires_on: ""
|
||||
required_tasks:
|
||||
- ":application-core:check"
|
||||
limitations:
|
||||
- retention and legal-hold provider enforcement are not implemented
|
||||
- privileged purge composition remains intentionally empty
|
||||
|
||||
- card_id: object-storage-reconciliation
|
||||
provider_type: filesystem-local-dev
|
||||
provider_version: jdk-21
|
||||
destination_profile: local-unimplemented
|
||||
claimed_level: R0
|
||||
evidence_revision: batch-b-contract-r0
|
||||
evidence_expires_on: ""
|
||||
required_tasks:
|
||||
- ":application-core:check"
|
||||
limitations:
|
||||
- local create resolution is single-process functional evidence only
|
||||
- production response-loss and multi-node reconciliation are not implemented
|
||||
@@ -0,0 +1,323 @@
|
||||
# Registry: Secrets Classification
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/secrets-classification.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# Owner branch: feature-secrets-config-source-contract
|
||||
# Last updated: 2026-05-22
|
||||
#
|
||||
# Conventions:
|
||||
# - 3-tier classification (feature-secrets-config-source-contract 2026-05-22):
|
||||
# public-config | sensitive-config | secret
|
||||
# - `secret` rows: prod_default 항상 null. dev fake 식별자는 `__LOCAL_DEV_` prefix
|
||||
# (feature-secrets-config-source-contract 2026-05-22: "dev/local sentinel value prefix = __LOCAL_DEV_").
|
||||
# - prod profile에서 `__LOCAL_DEV_` prefix 발견 시 startup fail
|
||||
# (feature-secrets-config-source-contract 2026-05-22).
|
||||
# - Masking 기본 = `full_except_last_4` (feature-secrets-config-source-contract 2026-05-22:
|
||||
# "full mask except last 4 chars for non-secret tokens"). 진짜 secret(password/private key)은 `full`.
|
||||
# - Naming suffix는 보조 신호 (feature-secrets-config-source-contract: "_TOKEN, _KEY, _PASSWORD").
|
||||
# - public-config 항목은 env-keys.yaml에서 직접 정의되며 본 파일에는 reference row만 둠.
|
||||
|
||||
secrets:
|
||||
|
||||
# === Tier 3: secret (true secret — password/private-key/HMAC-salt) ===
|
||||
|
||||
- name: APP_DATASOURCE_PASSWORD
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# "DB credential은 dual-bind 60s" + "__LOCAL_DEV_FAKE_DB_PASSWORD" 예시
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: dual-bind-60s
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:db-password-no-leak-in-actuator
|
||||
|
||||
- name: APP_SECURITY_JWT_SIGNING_KEY
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# "JWT signing key는 24h overlap window 유지 (security branch와 cross-link)"
|
||||
# + feature-security-operational-baseline "rotation overlap window = 새 kid 도입 → 24h 동안 old kid 병행"
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: overlap-24h
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:jwt-signing-key-rotation-overlap
|
||||
|
||||
- name: APP_SECURITY_OAUTH_CLIENT_SECRET
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# "secret classification은 ... naming pattern은 보조(suffix _TOKEN, _KEY, _PASSWORD)"
|
||||
# + feature-security-operational-baseline "JWT Resource Server를 baseline security model" (OAuth 자격 증명 분류)
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:oauth-client-secret-no-leak
|
||||
|
||||
- name: APP_EXTERNAL_API_KEY
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# "external API key는 application restart 시 reload"
|
||||
# (per-dependency suffix는 adapter 등록 시 추가; 본 row는 baseline 분류 정의)
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full_except_last_4
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:external-api-key-no-leak
|
||||
|
||||
- name: APP_CACHE_REDIS_PASSWORD
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# 3-tier classification "secret" + feature-integration-adapter-templates "Redis | disabled optional module"
|
||||
# (Redis enabled + auth 사용 시 secret으로 분류)
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:redis-password-no-leak
|
||||
|
||||
- name: APP_CACHE_REDIS_TRUST_PEM
|
||||
# Public CA bundle content, but integrity-sensitive and supplied by the mounted environment.
|
||||
classification: sensitive-config
|
||||
source: mounted-env
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: redis-production-capability
|
||||
masking_rule: full
|
||||
compatibility_impact: additive
|
||||
required_test: secrets-contract:redis-trust-reference-no-leak
|
||||
|
||||
- name: APP_CACHE_REDIS_KEY_HMAC_SECRET
|
||||
# Stable cache-key HMAC material. It is distinct from the Redis authentication credential.
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: dual-read-restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: codex-phase-a-ci-recovery
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:redis-key-hmac-no-leak
|
||||
|
||||
- name: APP_RATE_LIMIT_REDIS_PASSWORD
|
||||
# Dedicated coordination-role Redis credential. It is never inherited from cache Redis.
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: redis-distributed-rate-limit
|
||||
masking_rule: full
|
||||
compatibility_impact: additive
|
||||
required_test: secrets-contract:rate-limit-redis-password-no-leak
|
||||
|
||||
- name: APP_RATE_LIMIT_REDIS_TRUST_PEM
|
||||
# Coordination-role CA bundle content; integrity-sensitive but not credential material.
|
||||
classification: sensitive-config
|
||||
source: mounted-env
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: redis-production-capability
|
||||
masking_rule: full
|
||||
compatibility_impact: additive
|
||||
required_test: secrets-contract:rate-limit-redis-trust-reference-no-leak
|
||||
|
||||
- name: APP_RATE_LIMIT_REDIS_KEY_HMAC_SECRET
|
||||
# Stable private-key derivation material for rate-limit subjects and policy revisions.
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: dual-read-restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: redis-distributed-rate-limit
|
||||
masking_rule: full
|
||||
compatibility_impact: additive
|
||||
required_test: secrets-contract:rate-limit-redis-key-hmac-no-leak
|
||||
|
||||
- name: APP_SESSION_REDIS_PASSWORD
|
||||
# Dedicated session-role ACL credential; never shared implicitly with cache or coordination.
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: redis-production-capability
|
||||
masking_rule: full
|
||||
compatibility_impact: additive
|
||||
required_test: secrets-contract:session-redis-password-no-leak
|
||||
|
||||
- name: APP_SESSION_REDIS_TRUST_PEM
|
||||
# Session-role CA bundle content; integrity-sensitive but not credential material.
|
||||
classification: sensitive-config
|
||||
source: mounted-env
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: redis-production-capability
|
||||
masking_rule: full
|
||||
compatibility_impact: additive
|
||||
required_test: secrets-contract:session-redis-trust-reference-no-leak
|
||||
|
||||
- name: APP_SESSION_REDIS_KEY_HMAC_SECRET
|
||||
# Stable private derivation material for pseudonymous Redis session keys.
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: dual-read-restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: redis-production-capability
|
||||
masking_rule: full
|
||||
compatibility_impact: additive
|
||||
required_test: secrets-contract:session-redis-key-hmac-no-leak
|
||||
|
||||
- name: APP_IDEMPOTENCY_REDIS_KEY_HMAC_SECRET
|
||||
# Owner-safe request-replay keys must not expose tenant/scope/request identifiers.
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: cold-cutover-restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: redis-production-capability-completion
|
||||
masking_rule: full
|
||||
compatibility_impact: additive
|
||||
required_test: secrets-contract:idempotency-redis-key-hmac-no-leak
|
||||
|
||||
- name: APP_LEASE_REDIS_KEY_HMAC_SECRET
|
||||
# Efficiency-lease resource and owner scopes use a dedicated derivation key.
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: cold-cutover-restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: redis-production-capability-completion
|
||||
masking_rule: full
|
||||
compatibility_impact: additive
|
||||
required_test: secrets-contract:lease-redis-key-hmac-no-leak
|
||||
|
||||
- name: APP_PRIVACY_PSEUDONYMIZATION_SALT
|
||||
# source: feature-data-retention-privacy-contract 2026-05-22
|
||||
# "pseudonymization key = HMAC-SHA-256 with rotating salt. salt rotation interval = 90일.
|
||||
# rotation 시 old salt 90일 retain (lookup 가능)."
|
||||
# + feature-tenant-context-policy "tenant identifier는 raw PII가 아니어야 하며 ... pseudonymized id"
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: salt-rotation-90d
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-data-retention-privacy-contract
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:pseudonymization-salt-rotation
|
||||
|
||||
# === Tier 2: sensitive-config (token-bearing URL or id with exposure restriction) ===
|
||||
|
||||
- name: APP_NOTIFICATION_SLACK_WEBHOOK_URL
|
||||
# source: feature-integration-adapter-templates 2026-05-22
|
||||
# "Slack | disabled optional module | notification failure policy"
|
||||
# Slack webhook URL은 token을 path에 포함하므로 sensitive-config (URL 형태이지만 secret과 동급 취급)
|
||||
classification: sensitive-config
|
||||
source: secret-manager
|
||||
rotation_policy: manual
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full_except_last_4
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:slack-webhook-no-leak
|
||||
|
||||
- name: APP_SECURITY_GOOGLE_OAUTH_CLIENT_ID
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# "sensitive-config" tier (id이지만 노출 제한)
|
||||
# + feature-integration-adapter-templates "Google Email | disabled optional module"
|
||||
classification: sensitive-config
|
||||
source: mounted-env
|
||||
rotation_policy: manual
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full_except_last_4
|
||||
compatibility_impact: behavior-change
|
||||
required_test: secrets-contract:google-oauth-client-id-masked
|
||||
|
||||
- name: APP_DATASOURCE_USERNAME
|
||||
# source: feature-secrets-config-source-contract 2026-05-22 — "sensitive-config" tier
|
||||
# (DB user는 password와 함께 노출되면 위험하므로 sensitive-config)
|
||||
classification: sensitive-config
|
||||
source: mounted-env
|
||||
rotation_policy: dual-bind-60s
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full_except_last_4
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:datasource-username-masked-in-actuator
|
||||
|
||||
- name: APP_DATASOURCE_URL
|
||||
# source: feature-secrets-config-source-contract 2026-05-22 — JDBC URL은 host/db 포함하므로 sensitive-config
|
||||
# (env-keys.yaml에서는 public-config 처리; 본 파일에서는 노출 통제 관점에서 sensitive로 재분류 — masking 기준 명시 목적)
|
||||
classification: sensitive-config
|
||||
source: mounted-env
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full_except_last_4
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:datasource-url-masked-in-actuator
|
||||
|
||||
# === Tier 1: public-config (reference only — full row in env-keys.yaml) ===
|
||||
|
||||
- name: APP_PROFILE
|
||||
# source: feature-env-driven-runtime-configuration — public-config tier reference
|
||||
classification: public-config
|
||||
source: application-yml
|
||||
owner_branch: feature-env-driven-runtime-configuration
|
||||
masking_rule: none
|
||||
reference: env-keys.yaml#APP_PROFILE
|
||||
|
||||
- name: APP_NAME
|
||||
# source: feature-env-driven-runtime-configuration — public-config tier reference
|
||||
classification: public-config
|
||||
source: application-yml
|
||||
owner_branch: feature-env-driven-runtime-configuration
|
||||
masking_rule: none
|
||||
reference: env-keys.yaml#APP_NAME
|
||||
|
||||
- name: SERVER_PORT
|
||||
# source: feature-env-driven-runtime-configuration — Spring native, public-config tier reference
|
||||
classification: public-config
|
||||
source: application-yml
|
||||
owner_branch: feature-env-driven-runtime-configuration
|
||||
masking_rule: none
|
||||
reference: env-keys.yaml#SERVER_PORT
|
||||
|
||||
- name: SPRING_PROFILES_ACTIVE
|
||||
# source: feature-env-driven-runtime-configuration — Spring native, public-config tier reference
|
||||
classification: public-config
|
||||
source: application-yml
|
||||
owner_branch: feature-env-driven-runtime-configuration
|
||||
masking_rule: none
|
||||
reference: env-keys.yaml#SPRING_PROFILES_ACTIVE
|
||||
|
||||
- name: OTEL_EXPORTER_OTLP_ENDPOINT
|
||||
# source: feature-distributed-tracing-contract — public-config tier reference
|
||||
classification: public-config
|
||||
source: application-yml
|
||||
owner_branch: feature-distributed-tracing-contract
|
||||
masking_rule: none
|
||||
reference: env-keys.yaml#OTEL_EXPORTER_OTLP_ENDPOINT
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — ADAPTER_DISABLED (런타임 어댑터 비활성화 호출)
|
||||
category: INTERNAL
|
||||
error_codes: [ADAPTER_DISABLED]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: ADAPTER_DISABLED (`runbook://adapter/adapter-disabled`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 500 with `error.code=ADAPTER_DISABLED`
|
||||
- Code invoked an optional adapter (Kafka/Redis/Slack/Email) that is disabled in this deployment
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check adapter name in log (`adapter_name` field)
|
||||
- Review deployment config — which optional adapters are enabled?
|
||||
|
||||
## Action
|
||||
|
||||
- Enable the adapter in deployment configuration (env flag)
|
||||
- Or update application logic to skip disabled-adapter paths
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to deployment team if adapter should be enabled but isn't
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_AUDIENCE_MISMATCH (대상 불일치)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_AUDIENCE_MISMATCH]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_AUDIENCE_MISMATCH (`runbook://auth/audience-mismatch`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_AUDIENCE_MISMATCH`
|
||||
- Token `aud` claim does not include this service's expected audience
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check token `aud` claim value
|
||||
- Compare against configured `spring.security.oauth2.resourceserver.jwt.audiences`
|
||||
|
||||
## Action
|
||||
|
||||
- Verify client is requesting tokens scoped to the correct audience
|
||||
- Update audience configuration if service identifier changed
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to auth-platform team if misconfiguration is system-wide
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_CLAIM_MAPPING_FAILED (클레임 매핑 실패)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_CLAIM_MAPPING_FAILED]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_CLAIM_MAPPING_FAILED (`runbook://auth/claim-mapping-failed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_CLAIM_MAPPING_FAILED`
|
||||
- Token validated but required claims (sub, roles, tenant) missing or unexpected type
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Inspect token payload claims via logs
|
||||
- Check claim extractor configuration
|
||||
|
||||
## Action
|
||||
|
||||
- Verify IdP token template includes required claims
|
||||
- Update claim mapping configuration if IdP schema changed
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to auth-platform team if IdP changed claim schema
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_ISSUER_MISMATCH (발급자 불일치)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_ISSUER_MISMATCH]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_ISSUER_MISMATCH (`runbook://auth/issuer-mismatch`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_ISSUER_MISMATCH`
|
||||
- Token `iss` claim does not match configured expected issuer
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Compare token `iss` against `spring.security.oauth2.resourceserver.jwt.issuer-uri`
|
||||
- Check if IdP environment changed
|
||||
|
||||
## Action
|
||||
|
||||
- Update issuer config if IdP migrated
|
||||
- Reject tokens from unexpected issuers
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to platform-security if unexpected issuer detected
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_JWKS_UNAVAILABLE (JWKS 엔드포인트 장애)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [AUTH_JWKS_UNAVAILABLE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_JWKS_UNAVAILABLE (`runbook://auth/jwks-unavailable`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=AUTH_JWKS_UNAVAILABLE`
|
||||
- All authentication failing; JWKS refresh attempts failing
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check IdP JWKS endpoint health: `curl -sf https://<idp-host>/.well-known/jwks.json`
|
||||
- Check network connectivity from app pods to IdP
|
||||
|
||||
## Action
|
||||
|
||||
- Enable cached JWKS fallback if available
|
||||
- Coordinate with IdP team for restoration
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 page: IdP team immediately if JWKS endpoint unreachable > 2 minutes
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_KID_UNKNOWN (키 ID 미인식)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_KID_UNKNOWN]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_KID_UNKNOWN (`runbook://auth/kid-unknown`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_KID_UNKNOWN`, `retryable=true`
|
||||
- Token `kid` header not present in cached JWKS
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check if IdP key rotation occurred recently
|
||||
- Verify JWKS cache TTL and refresh timing
|
||||
|
||||
## Action
|
||||
|
||||
- Force JWKS cache refresh
|
||||
- Confirm new key is published in IdP JWKS endpoint
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to IdP team if new kid not appearing in JWKS after 10 minutes
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — INTERNAL_AUTH_MISCONFIGURATION (공개 경로 설정 오류)
|
||||
category: INTERNAL
|
||||
error_codes: [INTERNAL_AUTH_MISCONFIGURATION]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: INTERNAL_AUTH_MISCONFIGURATION (`runbook://auth/public-path-misconfiguration`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 500 with `error.code=INTERNAL_AUTH_MISCONFIGURATION`
|
||||
- Security filter misconfiguration detected at runtime
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `verifyPublicPathSnapshot` output in CI
|
||||
- Review recent changes to `SecurityConfig` or `application.yml` public path list
|
||||
|
||||
## Action
|
||||
|
||||
- Revert misconfigured public path change
|
||||
- Run `./gradlew verifyPublicPathSnapshot` to compare snapshot
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 immediate: if auth bypass is possible due to misconfiguration
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_TOKEN_EXPIRED (토큰 만료)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_TOKEN_EXPIRED]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_TOKEN_EXPIRED (`runbook://auth/token-expired`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_TOKEN_EXPIRED`
|
||||
- Spike may indicate clock skew or long-lived token usage
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `exp` claim vs server clock
|
||||
- Check NTP sync on token-issuing host
|
||||
|
||||
## Action
|
||||
|
||||
- Client must refresh tokens before expiry
|
||||
- Verify clock skew tolerance is configured (default 60s)
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate if spike is widespread or clock drift is confirmed
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_TOKEN_INVALID_SIGNATURE (서명 검증 실패)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_TOKEN_INVALID_SIGNATURE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_TOKEN_INVALID_SIGNATURE (`runbook://auth/token-invalid-signature`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_TOKEN_INVALID_SIGNATURE`
|
||||
- `log_level=ERROR` — may indicate forged tokens or wrong signing key
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check if JWKS endpoint returned a new key set
|
||||
- Check for token forgery attempts in logs
|
||||
|
||||
## Action
|
||||
|
||||
- Verify JWKS key IDs match token headers
|
||||
- Alert security team if forgery suspected
|
||||
|
||||
## Escalation
|
||||
|
||||
- Immediate P1 escalation if forgery indicators present
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_TOKEN_MALFORMED (토큰 파싱 실패)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_TOKEN_MALFORMED]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_TOKEN_MALFORMED (`runbook://auth/token-malformed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 responses with `error.code=AUTH_TOKEN_MALFORMED`
|
||||
- Token present but fails JWT parse (not 3-part, non-base64, etc.)
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Inspect raw Authorization header value in logs
|
||||
- Check if token generation tooling has a bug
|
||||
|
||||
## Action
|
||||
|
||||
- Identify source of malformed tokens
|
||||
- Fix or update client token generation
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate if spike suggests infrastructure issue
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_TOKEN_MISSING (인증 토큰 누락)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_TOKEN_MISSING]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_TOKEN_MISSING (`runbook://auth/token-missing`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 responses with `error.code=AUTH_TOKEN_MISSING`
|
||||
- Client missing Authorization header or Bearer token
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check request logs for missing Authorization header
|
||||
- Verify client SDK configuration
|
||||
|
||||
## Action
|
||||
|
||||
- Confirm API clients are sending Authorization header
|
||||
- Check gateway/proxy configuration for header stripping
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate if widespread or affecting critical workflows
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: Runbook — JWT key rotation 시 인증 실패 spike
|
||||
category: AUTH
|
||||
error_codes: [AUTH_TOKEN_EXPIRED, AUTH_KID_UNKNOWN, AUTH_JWKS_UNAVAILABLE, AUTH_TOKEN_INVALID_SIGNATURE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-05-22
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: JWT key rotation 시 인증 실패 spike
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `auth_401_error_rate_high` 또는 `jwks_refresh_failure_spike`
|
||||
- alert payload 필수 field: `operation`, `error.code`, `error.category`, `runbook_link`, `dependency_name`
|
||||
- 임계: 401 error rate > 5% 5분 지속 OR JWKS refresh failure count > 10건/분
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. JWKS endpoint health check: `curl -sf https://<idp-host>/.well-known/jwks.json | jq '.keys | length'`
|
||||
2. log query에서 `error.code` 분포 확인 — `AUTH_KID_UNKNOWN` 비중이 높으면 rotation 원인 강력 시사
|
||||
3. IdP rotation schedule 확인 (직전 24h 내 rotation 이벤트가 있었는지)
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- JWKS cache TTL을 짧게(예: 60s) 강제하여 새 kid 전파 가속
|
||||
- 새 kid가 JWKS에 publish되어 있는지 확인. 누락이면 IdP에 republish 요청
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query (Loki/CloudWatch): `{service="auth"} | error.category="AUTH" | dependency_name="jwks-endpoint"`
|
||||
- metric panel: `auth_jwks_cache_hit_ratio`, `auth_jwks_refresh_failure_total`, `auth_kid_unknown_total`
|
||||
- trace: 실패한 request 1건에서 `traceId` 추출 → IdP outbound span 확인
|
||||
- 가능한 원인:
|
||||
- 새 kid가 JWKS에 publish되기 전 token 발급 → 24h overlap window 안에 있는지 확인
|
||||
- JWKS endpoint 장애 (5xx, timeout) → IdP status page 확인
|
||||
- 시계 skew로 인한 만료 오판 → NTP sync 상태 확인
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: old kid를 임시 재허용 (rollback). overlap window를 48h로 일시 확장
|
||||
- IdP에 새 JWKS publish 재시도 요청
|
||||
- 장기: rotation 절차에 "publish → 24h 대기 → switch" 단계 강제. observability에 kid 분포 metric 추가
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- P2 → P1 격상 조건: 401 error rate > 20% 또는 다중 tenant에 동시 발생
|
||||
- 다음 on-call로 page: 10분 내 회복 안 되면 IdP team 또는 platform-security team page
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: `auth_401_error_rate < 1%` 5분 지속, `AUTH_KID_UNKNOWN` 건수 0
|
||||
- post-incident:
|
||||
- rotation 절차 RCA 작성
|
||||
- JWKS overlap window 정책 문서 업데이트
|
||||
- kid 분포 dashboard 영구화
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `AUTH_TOKEN_EXPIRED`, `AUTH_KID_UNKNOWN`, `AUTH_JWKS_UNAVAILABLE`, `AUTH_TOKEN_INVALID_SIGNATURE`
|
||||
- metrics.yaml: `auth_jwks_cache_hit_ratio`, `auth_jwks_refresh_failure_total`
|
||||
- 관련 branch: [[feature-security-operational-baseline]]
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 도메인 도입 시 실제 IdP 종류·rotation 정책·JWKS endpoint URL·dashboard 링크로 보강 필요.
|
||||
@@ -0,0 +1,72 @@
|
||||
---
|
||||
title: Runbook — cross-tenant 접근 시도 감지
|
||||
category: AUTHZ
|
||||
error_codes: [AUTHZ_INSUFFICIENT_PERMISSION, AUTHZ_TENANT_MISMATCH]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-05-22
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: cross-tenant 접근 시도 감지
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `authz_cross_tenant_violation` 또는 `authz_403_spike`
|
||||
- alert payload 필수 field: `operation`, `error.code`, `error.category`, `principal_id_pseudonymized`, `tenant_id`, `runbook_link`
|
||||
- 임계:
|
||||
- P2: 403 with `error.code=AUTHZ_TENANT_MISMATCH` > 10건/5분
|
||||
- P1 격상: 동일 principal에서 3개 이상 tenant 시도 OR 5분 내 100건 초과
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. log query로 위반 principal 식별 (pseudonymized): `error.code=AUTHZ_TENANT_MISMATCH`
|
||||
2. principal의 정상 tenant scope 확인 (IdP claim 또는 entitlement table)
|
||||
3. `CROSS_TENANT_ADMIN` capability 보유 여부 확인 — 보유자라면 false positive 가능성
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- 명백한 위반 패턴이면 principal session 강제 만료 (token revocation list 추가)
|
||||
- security incident channel 통보 (`#sec-incident`)
|
||||
- 위반 request의 source IP / user-agent 기록
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="api"} | error.category="AUTHZ" | principal_id_pseudonymized="<hash>"`
|
||||
- metric panel: `authz_denied_total{reason="tenant_mismatch"}`, `authz_principal_tenant_distribution`
|
||||
- trace: 위반 request의 `traceId`로 호출 chain 확인. token claim의 `tenant_id`와 요청 path의 `tenant_id` 비교
|
||||
- 가능한 원인:
|
||||
- account takeover (계정 탈취) → 즉시 session revoke + 비밀번호 reset 요구
|
||||
- client bug (잘못된 tenant id 전송) → product team에 통보
|
||||
- 정상 admin operation 누락된 capability → entitlement 보정
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: principal session revoke, source IP rate-limit 강화
|
||||
- 위반이 client bug면 client patch release 협조
|
||||
- 장기: tenant boundary 검증 layer를 controller가 아닌 repository 진입점에서 강제 ([[feature-repository-access-permission-contract]])
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 다음 on-call로 page: 보안 incident channel 즉시 page. 5분 내 security on-call 응답 없으면 CISO escalation
|
||||
- legal/compliance 통보 필요 여부 판단 (개인정보 noted시)
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: `AUTHZ_TENANT_MISMATCH` 건수 정상 baseline 복귀
|
||||
- post-incident:
|
||||
- account takeover면 forensics 수행 + audit log 보존
|
||||
- cross-tenant 검증 unit test 추가
|
||||
- 위반 패턴 detection rule 영구화
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `AUTHZ_INSUFFICIENT_PERMISSION`, `AUTHZ_TENANT_MISMATCH`
|
||||
- metrics.yaml: `authz_denied_total`, `authz_principal_tenant_distribution`
|
||||
- 관련 branch: [[feature-tenant-context-policy]], [[feature-repository-access-permission-contract]]
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 도메인 도입 시 실제 tenant 모델·capability 정의·security team 연락 체계로 보강 필요.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTHZ_INSUFFICIENT_PERMISSION (권한 부족)
|
||||
category: AUTHZ
|
||||
error_codes: [AUTHZ_INSUFFICIENT_PERMISSION]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTHZ_INSUFFICIENT_PERMISSION (`runbook://authz/insufficient-permission`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 403 with `error.code=AUTHZ_INSUFFICIENT_PERMISSION`
|
||||
- Valid token but missing required role or permission
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check user's assigned roles in IdP
|
||||
- Review endpoint's required permission annotation
|
||||
|
||||
## Action
|
||||
|
||||
- Grant correct role/permission to user
|
||||
- Verify endpoint permission requirement is correct
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to access-management team if bulk users affected
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTHZ_TENANT_MISMATCH (테넌트 cross-access 시도)
|
||||
category: AUTHZ
|
||||
error_codes: [AUTHZ_TENANT_MISMATCH]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTHZ_TENANT_MISMATCH (`runbook://authz/tenant-mismatch`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 403 with `error.code=AUTHZ_TENANT_MISMATCH`
|
||||
- `log_level=ERROR` — cross-tenant access attempt detected
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Extract `traceId`, check `X-Tenant-Id` vs token tenant claim
|
||||
- Determine if this is misconfigured client or intentional attack
|
||||
|
||||
## Action
|
||||
|
||||
- Block repeat offenders at gateway level
|
||||
- Alert security team for investigation
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if confirmed malicious cross-tenant access attempt
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — CACHE_STAMPEDE_LOCK_TIMEOUT (캐시 스탬피드 락 타임아웃)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [CACHE_STAMPEDE_LOCK_TIMEOUT]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: CACHE_STAMPEDE_LOCK_TIMEOUT (`runbook://cache/stampede-lock-timeout`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=CACHE_STAMPEDE_LOCK_TIMEOUT`
|
||||
- Multiple concurrent cache misses on same key; lock contention
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check cache hit ratio metrics
|
||||
- Identify cache keys with high miss rates
|
||||
|
||||
## Action
|
||||
|
||||
- Verify stampede lock TTL is configured appropriately
|
||||
- Pre-warm cache for high-traffic keys on startup
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate if backend load spike accompanies stampede
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — CACHE_UNAVAILABLE (캐시 연결 불가)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [CACHE_UNAVAILABLE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: CACHE_UNAVAILABLE (`runbook://cache/unavailable`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=CACHE_UNAVAILABLE`
|
||||
- Redis connection errors in logs
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check Redis cluster health
|
||||
- Verify network connectivity from app to Redis
|
||||
|
||||
## Action
|
||||
|
||||
- Check Redis sentinel/cluster status
|
||||
- Enable cache degradation path if configured for optional caches
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if required cache is down and no degradation path exists
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DB_DEADLOCK (데드락)
|
||||
category: CONFLICT
|
||||
error_codes: [DB_DEADLOCK]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DB_DEADLOCK (`runbook://db/deadlock`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 409 with `error.code=DB_DEADLOCK`
|
||||
- SQLState 40P01 in Postgres logs
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `pg_locks` and `pg_stat_activity` during deadlock
|
||||
- Identify conflicting transaction lock order
|
||||
|
||||
## Action
|
||||
|
||||
- Client should retry (retryable=true)
|
||||
- Fix lock ordering in code if recurring
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to DBA if deadlock rate is sustained > 1% of transactions
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DB_IDLE_IN_TX_TIMEOUT (트랜잭션 idle 타임아웃)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DB_IDLE_IN_TX_TIMEOUT]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DB_IDLE_IN_TX_TIMEOUT (`runbook://db/idle-in-tx-timeout`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DB_IDLE_IN_TX_TIMEOUT`
|
||||
- SQLState 25P03; transaction held open too long without activity
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `idle_in_transaction_session_timeout` Postgres setting
|
||||
- Look for application-level long-running transaction holders
|
||||
|
||||
## Action
|
||||
|
||||
- Reduce transaction scope in application code
|
||||
- Verify `spring.jpa.properties.hibernate.connection.timeout` is bounded
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to DBA if connection pool exhaustion results
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DB_QUERY_CANCELED (쿼리 취소)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DB_QUERY_CANCELED]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DB_QUERY_CANCELED (`runbook://db/query-canceled`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DB_QUERY_CANCELED`
|
||||
- SQLState 57014; query exceeds statement timeout
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `statement_timeout` in Postgres
|
||||
- Identify slow queries in `pg_stat_statements`
|
||||
|
||||
## Action
|
||||
|
||||
- Optimize slow query or add index
|
||||
- Adjust statement timeout if query is legitimately long
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to DBA for query optimization if recurring
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DB_SERIALIZATION_FAILURE (직렬화 실패)
|
||||
category: CONFLICT
|
||||
error_codes: [DB_SERIALIZATION_FAILURE]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DB_SERIALIZATION_FAILURE (`runbook://db/serialization-failure`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 409 with `error.code=DB_SERIALIZATION_FAILURE`
|
||||
- SQLState 40001; high concurrent transaction contention
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check DB transaction isolation level
|
||||
- Identify hot rows / hot tables under high concurrency
|
||||
|
||||
## Action
|
||||
|
||||
- Client should retry with exponential backoff (retryable=true)
|
||||
- Optimize transaction scope if spike is sustained
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to DBA if sustained serialization failure rate > 5%
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DB_UNAVAILABLE (데이터베이스 연결 불가)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DB_UNAVAILABLE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DB_UNAVAILABLE (`runbook://db/unavailable`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DB_UNAVAILABLE`
|
||||
- SQLState 08* connection errors in logs
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check DB server health and connection pool exhaustion
|
||||
- Review network connectivity from app pods to DB
|
||||
|
||||
## Action
|
||||
|
||||
- Check DB primary health; failover to replica if available
|
||||
- Drain connection pool and reconnect
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1: immediate if DB primary is down
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_4XX_CLIENT (업스트림 클라이언트 오류)
|
||||
category: PERMANENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_4XX_CLIENT]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_4XX_CLIENT (`runbook://dependency/4xx-client`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 502 with `error.code=DEPENDENCY_4XX_CLIENT`
|
||||
- Upstream returned 401/403/400 — credential, scope, or request format issue
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check upstream response body in logs for error detail
|
||||
- Verify API credentials and scopes are valid
|
||||
|
||||
## Action
|
||||
|
||||
- Rotate credentials if expired
|
||||
- Fix request format if API contract changed
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to upstream API owner if contract change is suspected
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_5XX_SERVER (업스트림 서버 오류)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_5XX_SERVER]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_5XX_SERVER (`runbook://dependency/5xx-server`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 502 with `error.code=DEPENDENCY_5XX_SERVER`
|
||||
- Upstream returned 5xx; transient server-side failure
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `dependency_name` tag for which upstream is failing
|
||||
- Review upstream service status page
|
||||
|
||||
## Action
|
||||
|
||||
- Client should retry (retryable=true)
|
||||
- Monitor upstream recovery
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if critical upstream is in sustained 5xx state
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_CIRCUIT_OPEN (서킷 브레이커 개방)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_CIRCUIT_OPEN]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_CIRCUIT_OPEN (`runbook://dependency/circuit-open`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DEPENDENCY_CIRCUIT_OPEN`
|
||||
- Circuit breaker (Resilience4j) in OPEN state for a dependency
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check Resilience4j circuit breaker metrics for the dependency
|
||||
- Check upstream health; circuit opens after failure threshold breached
|
||||
|
||||
## Action
|
||||
|
||||
- Wait for circuit half-open probe (automatic after wait duration)
|
||||
- Resolve upstream issue to allow circuit to close
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if circuit remains open > 5 minutes on a critical dependency
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_CONNECT_FAILED (외부 의존성 연결 실패)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_CONNECT_FAILED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_CONNECT_FAILED (`runbook://dependency/connect-failed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DEPENDENCY_CONNECT_FAILED`
|
||||
- TCP connection refused or network unreachable to upstream
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `dependency_name` tag for which upstream is unreachable
|
||||
- Verify network path and firewall rules
|
||||
|
||||
## Action
|
||||
|
||||
- Check upstream service availability
|
||||
- Verify service discovery / DNS resolution
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if upstream is a critical service dependency
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_DNS_FAILED (DNS 조회 실패)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_DNS_FAILED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_DNS_FAILED (`runbook://dependency/dns-failed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DEPENDENCY_DNS_FAILED`
|
||||
- DNS resolution failure for upstream hostname
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Test DNS resolution from app pod: `nslookup <upstream-host>`
|
||||
- Check cluster DNS (CoreDNS) health
|
||||
|
||||
## Action
|
||||
|
||||
- Verify upstream hostname configuration
|
||||
- Check CoreDNS / cluster DNS health
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if cluster DNS is degraded
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_TIMEOUT (외부 의존성 타임아웃)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_TIMEOUT]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_TIMEOUT (`runbook://dependency/timeout`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 504 with `error.code=DEPENDENCY_TIMEOUT`
|
||||
- Upstream service did not respond within configured timeout (default: global 10s)
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `dependency_name` in log for which upstream is timing out
|
||||
- Review upstream service latency metrics
|
||||
|
||||
## Action
|
||||
|
||||
- Check upstream service health
|
||||
- Verify timeout settings match SLA expectations
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if critical upstream is timing out at scale
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,81 @@
|
||||
---
|
||||
title: Runbook — 외부 의존성 unavailable
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_TIMEOUT, DEPENDENCY_CONNECT_FAILED, DEPENDENCY_DNS_FAILED, DEPENDENCY_CIRCUIT_OPEN, DEPENDENCY_5XX_SERVER, CACHE_UNAVAILABLE, DB_UNAVAILABLE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-05-22
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: 외부 의존성 unavailable
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `dependency_error_rate_critical` 또는 `circuit_breaker_open`
|
||||
- alert payload 필수 field: `operation`, `error.code`, `error.category`, `dependency_name`, `dependency_kind`(required|optional), `runbook_link`
|
||||
- 임계:
|
||||
- P1: required dependency의 error rate > 50% 1분 OR circuit_open state 활성
|
||||
- P2: optional dependency degraded (fail-open으로 동작 중)
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. `dependency_name` 별 status page 확인 (외부 SaaS면 vendor status, internal이면 해당 service dashboard)
|
||||
2. log query로 실패 패턴 확인: timeout / connect / DNS / 5xx 중 어떤 모드인지
|
||||
3. runtime-health Dependency Matrix에서 required vs optional 분류 확인
|
||||
4. circuit breaker state 확인 (Resilience4j metric)
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- required dep이면 readiness probe로 traffic 차단 (회복 대기) — cascade failure 방지
|
||||
- optional dep이면 fail-open with degraded mode 확인. degraded banner를 client에 노출
|
||||
- DNS failure면 resolver/coredns 상태 확인. cache 강제 flush 검토
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | dependency_name="<name>" | stats count by error.code`
|
||||
- metric panel:
|
||||
- `resilience4j_circuitbreaker_state{name="<name>"}`
|
||||
- `resilience4j_retry_calls_total{kind="failed_without_retry"}`
|
||||
- `hikaricp_connections_active`, `hikaricp_connections_pending` (DB_UNAVAILABLE)
|
||||
- `http_client_requests_seconds_count{outcome="SERVER_ERROR"}`
|
||||
- trace: 실패 request의 outbound span에서 timeout/connect/DNS 분류, target endpoint 확인
|
||||
- 가능한 원인:
|
||||
- vendor outage → status page 확인, 회복 대기
|
||||
- 네트워크 문제 (DNS, security group, NAT) → infra team 확인
|
||||
- connection pool 고갈 (Hikari) → pool size/timeout 점검
|
||||
- circuit breaker open 후 half-open 전환 실패 → 수동 reset 검토
|
||||
- retry-storm으로 인한 self-DoS → retry budget 축소
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: required면 회복 대기 + traffic 차단, optional이면 degraded mode로 유지
|
||||
- pool 고갈이면 일시 pool size 상향 + leak detection 활성화
|
||||
- circuit이 stuck이면 수동 reset (`actuator/circuitbreakerevents`)
|
||||
- 장기: retry budget·timeout·circuit 임계 재조정, fallback path 보강, vendor SLA 재협상
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 다음 on-call로 page: required dep 5분 내 회복 안 되면 외부 dep team 또는 vendor에 page
|
||||
- 다중 dep 동시 장애면 incident commander 호출 (네트워크 전반 문제 의심)
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: dependency error rate < 1% 5분 지속, circuit_breaker_state = CLOSED, pool utilization 정상
|
||||
- post-incident:
|
||||
- vendor postmortem 요청 (외부 SaaS면)
|
||||
- timeout/retry/circuit 설정 재검토
|
||||
- degraded mode가 사용자 경험에 미친 영향 측정
|
||||
- chaos test에 해당 시나리오 추가
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `DEPENDENCY_TIMEOUT`, `DEPENDENCY_CONNECT_FAILED`, `DEPENDENCY_DNS_FAILED`, `DEPENDENCY_CIRCUIT_OPEN`, `DEPENDENCY_5XX_SERVER`, `CACHE_UNAVAILABLE`, `DB_UNAVAILABLE`
|
||||
- metrics.yaml: `resilience4j_circuitbreaker_state`, `hikaricp_connections_active`, `http_client_requests_seconds_count`
|
||||
- 관련 branch: [[feature-outbound-http-client-baseline]], [[feature-persistence-failure-baseline]]
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 도메인 도입 시 실제 dependency 목록·required/optional 분류·vendor 연락 체계·circuit/timeout 임계로 보강 필요.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DOWNLOAD_STREAMING_FAILURE (스트리밍 다운로드 실패)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DOWNLOAD_STREAMING_FAILURE]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DOWNLOAD_STREAMING_FAILURE (`runbook://file/download-streaming-failure`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DOWNLOAD_STREAMING_FAILURE`
|
||||
- Streaming response truncated; backpressure or timeout (60s / 100MB limit)
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check streaming response timeout configuration
|
||||
- Review download size vs 100MB limit
|
||||
|
||||
## Action
|
||||
|
||||
- Verify storage backend is reachable
|
||||
- Check for network congestion on download path
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to infra if storage backend is degraded
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,74 @@
|
||||
---
|
||||
title: Runbook — 5xx Internal error spike
|
||||
category: INTERNAL
|
||||
error_codes: [INTERNAL_ERROR, INTERNAL_AUTH_MISCONFIGURATION, JVM_OOM]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-05-22
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: 5xx Internal error spike
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `http_5xx_error_rate_critical`
|
||||
- alert payload 필수 field: `operation`, `error.code`, `error.category`, `request_id`, `traceId`, `runbook_link`
|
||||
- 임계: 5xx error rate > 5% 5분 지속 OR > 10% 1분
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. 가장 최근 deploy 시각 확인 (CI/CD dashboard, artifact registry digest)
|
||||
2. JVM metric 확인: heap usage, GC pause, CPU, thread count
|
||||
3. log에서 실패 request 1건 추출 → `request_id`, `traceId` 확보
|
||||
4. error.code 분포 확인: `INTERNAL_ERROR` vs `JVM_OOM` vs `INTERNAL_AUTH_MISCONFIGURATION`
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- 직전 deploy가 의심되면 즉시 rollback (artifact registry에서 직전 image digest pin)
|
||||
- OOM 패턴이면 affected pod evict → ASG/HPA로 replacement 유도
|
||||
- LB에서 unhealthy pod 격리 (readiness probe failure 유도)
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | http.status>=500 | stats count by error.code`
|
||||
- metric panel: `jvm_memory_used_bytes{area="heap"}`, `jvm_gc_pause_seconds`, `process_cpu_seconds_total`, `http_server_requests_seconds_count{status=~"5.."}`
|
||||
- trace: 실패 request의 `traceId`로 span chain 확인 → stack trace에서 root exception 추출
|
||||
- heap dump 위치: `/var/tmp/heap/heapdump-<pid>.hprof` (JVM ergonomics: `-XX:MaxRAMPercentage=75 -XX:+HeapDumpOnOutOfMemoryError`)
|
||||
- 가능한 원인:
|
||||
- 직전 deploy의 회귀 버그 → rollback
|
||||
- JVM OOM (메모리 leak 또는 부하 증가) → heap dump 분석
|
||||
- 외부 의존성 설정 오류 (`INTERNAL_AUTH_MISCONFIGURATION`) → config secret 확인
|
||||
- thread starvation (pool 고갈) → thread dump (`jstack <pid>`)
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: 직전 deploy rollback, OOM pod replacement, traffic 일시 감소(scale-out 또는 rate-limit 강화)
|
||||
- config 오류면 secret/configmap rollback
|
||||
- 장기: heap dump 기반 leak 수정, capacity planning 재검토
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 다음 on-call로 page: 10분 내 회복 안 되면 incident commander 호출, severity 1 incident 선언
|
||||
- 데이터 손상 의심되면 DBA team page
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: 5xx rate < 0.5% 5분 지속, JVM heap usage < 70%, GC pause p99 < 500ms
|
||||
- post-incident:
|
||||
- rollback 원인 RCA 작성 (배포 게이트 강화 필요 여부)
|
||||
- heap dump 분석 결과 공유
|
||||
- JVM ergonomics(`-XX:MaxRAMPercentage`) 재검토
|
||||
- rollback 자동화 절차 점검
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `INTERNAL_ERROR`, `INTERNAL_AUTH_MISCONFIGURATION`, `JVM_OOM`
|
||||
- metrics.yaml: `jvm_memory_used_bytes`, `jvm_gc_pause_seconds`, `http_server_requests_seconds_count`
|
||||
- 관련 branch: [[feature-operational-error-observability-foundation]], [[feature-container-runtime-contract]]
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 도메인 도입 시 실제 deploy 파이프라인·heap dump 보관 경로·rollback 자동화 명령으로 보강 필요.
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
title: Runbook — background job dead letter
|
||||
category: INTERNAL
|
||||
error_codes: [JOB_DEAD_LETTER]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-13
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: background job dead letter (`runbook://job/dead-letter`)
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `job_dead_letter`
|
||||
- alert payload 필수 field: `error.code=JOB_DEAD_LETTER`, `job_name`, `correlation_id`, `runbook_link`
|
||||
- 임계: `job.dlq.total` > 0 for 5m (p1) — retry 소진 후 DLQ 진입은 자동 회복이 없으므로 점검 대상
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. ERROR log에서 `JOB_DEAD_LETTER` 라인 확인: `job_name`, 최종 실패 원인 예외, `correlation_id` 추출
|
||||
2. `job.retry.total{outcome=EXHAUSTED}` 추이로 DLQ 유입 규모 파악
|
||||
3. DLQ 적재 위치(향후 retry carrier 확정 시 DB 테이블/큐) 확인 — 현재 skeleton은 vocabulary 단계
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- DLQ는 max attempts(3) 소진의 최종 상태 — 자동 재시도 없음, 수동 개입 필수
|
||||
- 비즈니스 크리티컬 job이면 §4의 수동 처분(재처리 또는 폐기)을 우선 수행
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | error.code="JOB_DEAD_LETTER" | stats count by job_name`
|
||||
- metric panel: `job.dlq.total{job_name}`, `job.retry.total{job_name, outcome}`
|
||||
- 최종 실패 원인 분류:
|
||||
- poison input(직렬화/계약 위반) → 입력 결함, 재처리해도 실패 — 수정 후 재처리 또는 폐기
|
||||
- 외부 의존성 장기 outage 중 attempts 소진 → 의존성 회복 후 재처리로 해결 가능
|
||||
- non-transient error(권한/도메인/스키마)인데 retry된 경우 → 분류기 보강 필요(WAF-REL05-C3: 즉시 DLQ가 정답)
|
||||
|
||||
## 4. Mitigation (수동 처분 — 둘 중 하나)
|
||||
|
||||
- **재처리 (기본)**: 원인 해소 후 해당 job을 다시 enqueue. 소비자는 멱등(idempotencyKey dedupe) 의무가 있으므로 중복 처리 안전
|
||||
- **폐기 (영구)**: 작업이 더 이상 유효하지 않으면 DLQ에서 제거. ⚠ 비즈니스 오너 승인 후에만 수행하고 incident 기록에 남김
|
||||
- 장기: poison input 재발 방지(입력 계약 테스트 보강), non-transient error는 retry 없이 즉시 DLQ로 분류
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 처분 판단(재처리 vs 폐기)이 불가하면 해당 job의 비즈니스 오너에게 escalate
|
||||
- DLQ 누적이 특정 `job_name`에 집중되면 해당 job 코드 오너에게 page
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인: `job.dlq.total` 증가 멈춤, 재처리분의 소비자 dedupe 동작 확인
|
||||
- post-incident: DLQ 원인 분류 기록, 같은 원인의 재발 방지 테스트 추가
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `JOB_DEAD_LETTER` (INTERNAL, 500, retryable=false)
|
||||
- metrics.yaml: `job.dlq.total{job_name}`, `job.retry.total{job_name, outcome=DLQ}`
|
||||
- 코드: `app-bootstrap` `async/BackgroundJobMetrics`(retry/DLQ vocabulary 기록 seam — D2/D4)
|
||||
- 관련 runbook: [[job-executor-rejected]], [[job-timeout]], [[outbox-dead-letter]]
|
||||
- 관련 branch: [[feature-background-job-async-contract]] (D4 retry/DLQ vocabulary SSOT — outbox/outbound가 consume)
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. retry carrier(Spring Retry / Resilience4j / 자체) 확정 후 DLQ 저장소·재처리 절차 보강 필요.
|
||||
@@ -0,0 +1,71 @@
|
||||
---
|
||||
title: Runbook — async executor rejected
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [JOB_EXECUTOR_REJECTED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-13
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: async executor rejected (`runbook://job/executor-rejected`)
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `executor_rejected`
|
||||
- alert payload 필수 field: `error.code=JOB_EXECUTOR_REJECTED`, `executor_name`, `policy`, `runbook_link`
|
||||
- 임계: `executor.rejected.total` > 0 for 1m (p1) — bounded pool이 saturation으로 task를 거부
|
||||
- 보조 신호: `executor.saturation` gauge > queue capacity의 80% for 5m (p2)
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. ERROR log에서 `JOB_EXECUTOR_REJECTED` 라인 확인: `executor_name`, `policy=AbortPolicy`, `queue_size` 추출
|
||||
2. `executor.saturation` 패널에서 큐 점유율 추이 확인 — 일시적 burst인지 지속 saturation인지 판별
|
||||
3. 동시 유입 원인 파악: 신규 배포 / 트래픽 spike / 다운스트림 지연으로 worker가 장기 점유되는지
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- AbortPolicy 거부는 호출부에 `RejectedExecutionException`으로 surface됨 — fire-and-forget `@Async` 호출이면 호출부의 async-exception 처리(log/metric)로 흡수됐는지 확인
|
||||
- 지속 saturation이면 유입 측(트래픽/스케줄러 빈도)을 우선 감속
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | error.code="JOB_EXECUTOR_REJECTED" | stats count by executor_name`
|
||||
- metric panel: `executor.saturation{executor_name}`, `executor.rejected.total{executor_name, policy}`
|
||||
- 가능한 원인 우선순위:
|
||||
- 다운스트림 의존성 지연 → worker가 반납되지 않아 큐 포화 (가장 흔함)
|
||||
- 트래픽 spike → 정상 부하 한계 초과
|
||||
- pool 과소 설정 (`APP_ASYNC_EXECUTOR_*`)
|
||||
- non-idempotent 작업이 retry로 누적
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: 유입 감속(상위 rate-limit / 스케줄러 interval 확대) 또는 다운스트림 의존성 회복
|
||||
- pool 재조정(restart-only): `APP_ASYNC_EXECUTOR_CORE_SIZE` / `APP_ASYNC_EXECUTOR_MAX_SIZE` / `APP_ASYNC_EXECUTOR_QUEUE_CAPACITY`
|
||||
— ⚠ queue를 무한정 키우지 말 것(unbounded 금지, D7). 부하테스트로 수치 검증 후 변경
|
||||
- CallerRunsPolicy로의 전환은 use-case 차원의 명시적 결정 필요(request thread latency 침식 — TPE-JDK21-C6)
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 다운스트림 의존성 장애가 근본 원인이면 해당 의존성 오너에게 escalate
|
||||
- pool 재조정으로도 saturation이 지속되면 용량 계획(capacity planning) 오너에게 page
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인: `executor.rejected.total` 증가 멈춤, `executor.saturation` < 80% 정상화
|
||||
- 거부된 작업의 재처리 경로(멱등 retry / 다음 스케줄 cycle) 정상 동작 확인
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `JOB_EXECUTOR_REJECTED` (TRANSIENT_DEPENDENCY, 503, retryable=true, retry_after 5s)
|
||||
- metrics.yaml: `executor.rejected.total{executor_name, policy}`, `executor.saturation{executor_name}`
|
||||
- 코드: `app-bootstrap` `async/AsyncExecutorConfig`(bounded executor), `async/LoggingAbortPolicy`(reject log+metric), `async/BackgroundJobMetrics`
|
||||
- env: `APP_ASYNC_EXECUTOR_CORE_SIZE` / `APP_ASYNC_EXECUTOR_MAX_SIZE` / `APP_ASYNC_EXECUTOR_QUEUE_CAPACITY`
|
||||
- 관련 runbook: [[job-timeout]], [[job-dead-letter]]
|
||||
- 관련 branch: [[feature-background-job-async-contract]] (D7 saturation policy)
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 실제 부하 프로파일·alert 채널·pool 수치 확정 시 보강 필요.
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: Runbook — background job timeout
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [JOB_TIMEOUT]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-13
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: background job timeout (`runbook://job/timeout`)
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `job_timeout`
|
||||
- alert payload 필수 field: `error.code=JOB_TIMEOUT`, `job_name`, `correlation_id`, `runbook_link`
|
||||
- 임계: `job.retry.total{outcome=RETRY}` 급증 또는 graceful-shutdown 중 in-flight job interrupt 발생
|
||||
- 연관: shutdown phase에서 19s await 초과로 interrupt된 job (D8)
|
||||
|
||||
## 2. First Response (10분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. ERROR log에서 `JOB_TIMEOUT` 라인 확인: `job_name`, 마지막 단계, 소요 시간 추출
|
||||
2. timeout이 정상 실행 중 발생인지, graceful-shutdown(배포/스케일다운) 중 interrupt인지 구분
|
||||
3. 해당 job이 멱등(retry-on-next-cycle 안전)인지 확인 — 비멱등이면 §4에서 신중히 처리
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- shutdown 중 interrupt면: 다음 기동 시 재시도 대상인지(멱등 전제) 확인, 중복 부작용 여부 점검
|
||||
- 정상 실행 중 timeout이면: 해당 job의 외부 의존성(DB/HTTP) 지연 여부 확인
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | error.code="JOB_TIMEOUT" | stats count by job_name`
|
||||
- metric panel: `job.retry.total{job_name, outcome}`
|
||||
- 가능한 원인 우선순위:
|
||||
- 외부 의존성(DB lock / 느린 HTTP) 지연으로 job p99 상승
|
||||
- job 작업량 증가로 단일 cycle이 19s 예산 초과 (D8 — interrupt 노출)
|
||||
- interrupt 미반응 blocking call(JDBC 등) → awaitTermination 초과 (K8S-POD-LC-C2 SIGKILL 경로)
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: 의존성 회복 / job 입력 배치 크기 축소
|
||||
- job p99가 구조적으로 19s를 넘으면: 작업을 분할하거나, grace period 연장 검토(parent project 운영 계약 소유자 승인 필요 — OUT_OF_BRANCH_SCOPE)
|
||||
- 비멱등 job이 재시도로 중복 부작용을 내면 멱등키/dedupe 도입 우선
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 의존성 지연이 근본 원인이면 해당 의존성 오너에게 escalate
|
||||
- shutdown 예산(20s) vs k8s `terminationGracePeriodSeconds`(30s) 정합 이슈면 플랫폼/런타임 오너에게 escalate
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인: `JOB_TIMEOUT` 신규 발생 멈춤, `job.retry.total{outcome=SUCCESS}` 정상 비율 회복
|
||||
- 멱등 재시도분의 부작용 중복 없음 확인
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `JOB_TIMEOUT` (TRANSIENT_DEPENDENCY, 500, retryable=true, retry_after 10s)
|
||||
- metrics.yaml: `job.retry.total{job_name, outcome}`
|
||||
- 코드: `app-bootstrap` `async/AsyncExecutorConfig`(awaitTermination 19s — D8 graceful shutdown)
|
||||
- env: `APP_SERVER_SHUTDOWN_TIMEOUT`(owner: feature-env-driven-runtime-configuration D2)
|
||||
- 관련 runbook: [[job-executor-rejected]], [[job-dead-letter]]
|
||||
- 관련 branch: [[feature-background-job-async-contract]] (D4 retry / D8 shutdown)
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 실제 retry carrier·job p99·shutdown 예산 확정 시 보강 필요.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — LOCK_ACQUISITION_TIMEOUT (분산 락 획득 타임아웃)
|
||||
category: CONFLICT
|
||||
error_codes: [LOCK_ACQUISITION_TIMEOUT]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: LOCK_ACQUISITION_TIMEOUT (`runbook://lock/acquisition-timeout`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 409 with `error.code=LOCK_ACQUISITION_TIMEOUT`
|
||||
- Distributed lock wait exceeded configured timeout; high contention on a resource
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `lock.acquisition` metric for lock name and duration
|
||||
- Identify lock holders (check DB `integration_lock` table)
|
||||
|
||||
## Action
|
||||
|
||||
- Client should retry with backoff (retryable=true)
|
||||
- Optimize critical section holding time if lock contention is systemic
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate if lock holder appears stuck (potential deadlock in distributed lock)
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — ACTUATOR_FORBIDDEN (Actuator 접근 거부)
|
||||
category: AUTHZ
|
||||
error_codes: [ACTUATOR_FORBIDDEN]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: ACTUATOR_FORBIDDEN (`runbook://management/actuator-forbidden`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 403 with `error.code=ACTUATOR_FORBIDDEN`
|
||||
- Attempt to access restricted actuator endpoint (env/configprops/heapdump/shutdown)
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Identify which actuator endpoint was accessed
|
||||
- Check caller identity (internal tooling vs external)
|
||||
|
||||
## Action
|
||||
|
||||
- Verify management port is not exposed externally
|
||||
- For heapdump/threaddump: follow break-glass runbook procedure
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if forbidden actuator access appears to be external attack
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,96 @@
|
||||
---
|
||||
title: Runbook — MIGRATION_FAILED (DB 마이그레이션 실패)
|
||||
category: INTERNAL
|
||||
error_codes: [MIGRATION_FAILED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-07-29
|
||||
status: active
|
||||
---
|
||||
|
||||
# Runbook: MIGRATION_FAILED (`runbook://migration/failed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Container exits with code 70 (migration failure exit)
|
||||
- Structured log with `error.code=MIGRATION_FAILED`, `startup.phase=migration`
|
||||
- App refuses to start (fail-fast)
|
||||
- JPA capability adapter refuses activation because its
|
||||
`capability_schema_registry.lifecycle_state` is not `ACTIVE`
|
||||
|
||||
## Diagnosis
|
||||
|
||||
1. Stop rollout and keep the failed revision out of readiness. Do not route traffic to a partially
|
||||
migrated instance.
|
||||
2. Identify the exact stream from `src/config/jpa/readiness-cards.yaml`. Each stream has an
|
||||
independent `location` and `history-table`; do not infer ownership from a broad
|
||||
`classpath:db/migration` scan.
|
||||
3. From a privileged migration session, capture the stream state before changing anything:
|
||||
|
||||
```sql
|
||||
select installed_rank, version, description, success
|
||||
from <owned_history_table>
|
||||
order by installed_rank;
|
||||
|
||||
select capability_id, installation_origin, core_epoch, feature_revision, lifecycle_state
|
||||
from capability_schema_registry
|
||||
where capability_id = '<card-id>';
|
||||
```
|
||||
|
||||
4. Check whether any owned relation was created without a successful history entry. Compare only
|
||||
against the owned tables in the reviewed migration; do not drop unrelated relations.
|
||||
5. Classify the failure:
|
||||
- lock/statement timeout: remove the blocker or reduce rollout concurrency, then rerun;
|
||||
- SQL/data precondition: create a new forward migration that makes the precondition explicit;
|
||||
- checksum mismatch: compare the deployed artifact with the already applied script before
|
||||
considering repair;
|
||||
- connection/TLS failure: fix transport or credentials without changing Flyway history.
|
||||
|
||||
## Action
|
||||
|
||||
1. Prefer forward recovery. Fix the environmental blocker or add a new immutable migration, then
|
||||
rerun the same owned stream with its exact history table.
|
||||
2. For an optional stream that never installed successfully, keep the capability marker absent and
|
||||
the runtime adapter disabled until migration succeeds.
|
||||
3. After a successful migration, validate:
|
||||
- the history contains only successful expected versions;
|
||||
- `core_epoch` and `feature_revision` match the readiness registry;
|
||||
- the marker is `INSTALLED_INACTIVE`;
|
||||
- owned objects and constraints exist.
|
||||
4. Change the marker to `ACTIVE` only after the compatible application revision is deployed and its
|
||||
readiness check succeeds. Disabling or rolling back application code changes the marker to
|
||||
`INSTALLED_INACTIVE`; it does not drop history or owned data.
|
||||
5. Re-run the candidate evidence task before promoting:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:persistence-jpa:verifyJpaCandidateEvidence --console=plain
|
||||
```
|
||||
|
||||
6. Record the failed revision, stream/history table, root cause, recovery migration, elapsed time
|
||||
and verification artifact in the incident.
|
||||
|
||||
Do not:
|
||||
|
||||
- edit an already applied migration;
|
||||
- delete or rewrite Flyway history to make validation green;
|
||||
- run `flyway repair` before checksum provenance is proven and reviewed;
|
||||
- use `clean`, destructive rollback, or schema-wide restore as the first response;
|
||||
- mark a capability `ACTIVE` before its migration and adapter readiness succeed.
|
||||
|
||||
If commit outcome was indeterminate during the failure, reconcile by the application
|
||||
`OperationId`/idempotency reference before retrying business work. Never blind-retry a commit whose
|
||||
result is unknown.
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 immediate: the required application revision cannot become ready.
|
||||
- Escalate to the database owner before Flyway history repair, destructive DDL, point-in-time
|
||||
recovery, or primary failover.
|
||||
- R3 restore/PITR and failover rehearsal requires a target-like backup topology; local
|
||||
Testcontainers evidence is not a substitute.
|
||||
|
||||
---
|
||||
|
||||
This runbook is forward-only. The reviewed migration artifact and the per-card evidence manifest
|
||||
are the audit sources.
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
title: Runbook — outbox dead letter
|
||||
category: INTERNAL
|
||||
error_codes: [OUTBOX_DEAD_LETTER]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-11
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: outbox dead letter (`runbook://outbox/dead-letter`)
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `outbox_dead_letter`
|
||||
- alert payload 필수 field: `error.code=OUTBOX_DEAD_LETTER`, `event_type`, `event_id`, `correlation_id`, `runbook_link`
|
||||
- 임계: `outbox.publisher.published.total{outcome=DEAD}` > 0 (DEAD 전이는 자동 회복이 없으므로 단건도 점검 대상)
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. ERROR log에서 `OUTBOX_DEAD_LETTER` 라인 확인: `event_id`, `event_type`, `correlation_id`, 마지막 실패 원인 예외 추출
|
||||
2. DB에서 DEAD row 확인: `SELECT * FROM outbox_event WHERE status = 'DEAD' ORDER BY occurred_at;`
|
||||
3. **차단 영향 파악 (중요)**: strict per-aggregate FIFO 정책상 DEAD row는 같은 `aggregate_id`의 후행 이벤트를 계속 차단함 —
|
||||
`SELECT count(*) FROM outbox_event b WHERE b.status <> 'PUBLISHED' AND EXISTS (SELECT 1 FROM outbox_event d WHERE d.status='DEAD' AND d.aggregate_id=b.aggregate_id AND d.occurred_at < b.occurred_at);`
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- DEAD는 max attempts(3) 소진의 최종 상태 — 자동 재시도 없음, 수동 개입 필수
|
||||
- 차단된 aggregate가 비즈니스 크리티컬하면 아래 §4의 수동 처분(재발행 또는 skip)을 우선 수행
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | error.code="OUTBOX_DEAD_LETTER" | stats count by event_type`
|
||||
- 마지막 실패 원인 분류:
|
||||
- poison event (payload 직렬화/계약 위반) → payload 자체 결함, 재발행해도 실패 — 수정 후 재발행 또는 skip
|
||||
- broker 장기 outage 중 attempts 소진 → broker 회복 후 재발행으로 해결 가능
|
||||
- 구성 오류 (Kafka disabled 상태에서 producer 활성) → 구성 수정 후 재발행
|
||||
- 가능한 원인 우선순위: 구성 오류 > broker outage > poison payload
|
||||
|
||||
## 4. Mitigation (수동 처분 — 둘 중 하나)
|
||||
|
||||
- **재발행 (기본)**: 원인 해소 후 해당 row를 다시 claim 가능 상태로 되돌림 —
|
||||
`UPDATE outbox_event SET status = 'PENDING', attempt_count = 0, next_attempt_at = now() WHERE event_id = '<id>' AND status = 'DEAD';`
|
||||
(consumer는 at-least-once + idempotencyKey dedupe 의무가 있으므로 중복 발행은 안전)
|
||||
- **skip (영구 폐기)**: 이벤트가 더 이상 유효하지 않으면 PUBLISHED로 마킹해 FIFO 차단을 해제 —
|
||||
`UPDATE outbox_event SET status = 'PUBLISHED' WHERE event_id = '<id>' AND status = 'DEAD';`
|
||||
⚠️ skip은 다운스트림에 영구 이벤트 갭을 만든다 — 비즈니스 오너 승인 후에만 수행하고 incident 기록에 남김
|
||||
- 장기: poison event 재발 방지(payload 계약 테스트 보강), DEAD 빈발 event_type의 producer 검증 강화
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 처분 판단(재발행 vs skip)이 불가하면 해당 이벤트의 비즈니스 오너에게 escalate
|
||||
- DEAD 누적이 특정 event_type에 집중되면 producer 코드 오너에게 page
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인: `SELECT count(*) FROM outbox_event WHERE status='DEAD';` = 0, 차단됐던 aggregate의 후행 이벤트가 PUBLISHED로 전이
|
||||
- `outbox.publisher.lag` 정상화(< 60s), 재발행분의 consumer dedupe 동작 확인
|
||||
- post-incident: DEAD 원인 분류 기록, 같은 원인의 재발 방지 테스트 추가
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `OUTBOX_DEAD_LETTER` (INTERNAL, retryable=false)
|
||||
- metrics.yaml: `outbox.publisher.published.total{outcome=DEAD}`, `outbox.pending.size{status=DEAD}`, `outbox.publisher.lag`
|
||||
- 코드: `application-core` `PublishPendingOutboxEventsUseCase`(FAILED→DEAD 전이), `adapter-persistence` `outbox/OutboxEventJpaRepository`(FIFO 게이트 — DEAD가 후행 차단)
|
||||
- 관련 runbook: [[outbox-publish-failed]]
|
||||
- 관련 branch: [[feature-domain-event-outbox-contract]], [[feature-background-job-async-contract]] (max attempts/DLQ vocabulary SSOT)
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 실제 broker·DLQ 토픽·승인 체계 확정 시 보강 필요.
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
title: Runbook — outbox publish 일시 실패
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [OUTBOX_PUBLISH_FAILED]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-11
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: outbox publish 일시 실패 (`runbook://outbox/publish-failed`)
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `outbox_publish_failed_rate` 또는 `outbox_publisher_lag`
|
||||
- alert payload 필수 field: `error.code=OUTBOX_PUBLISH_FAILED`, `event_type`, `correlation_id`, `runbook_link`
|
||||
- 임계 (metrics.yaml verbatim):
|
||||
- P2: `outbox.publisher.published.total{outcome=FAILED}` rate > 1% for 10m
|
||||
- P2: `outbox.publisher.lag` > 60s for 10m / P1: > 300s for 5m
|
||||
- P2: `outbox.pending.size{status=PENDING}` growing for 10m
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. ERROR log에서 `OUTBOX_PUBLISH_FAILED` 라인 확인: `event_type`, `event_id`, `correlation_id`, `attempt_count` 추출
|
||||
2. broker(기본 Kafka adapter) 상태 확인: `APP_MESSAGING_KAFKA_ENABLED` 값과 broker endpoint 가용성
|
||||
- Kafka disabled(default) 상태에서 outbox 이벤트가 append 되고 있으면 publish 경로가 `AdapterDisabledException`으로 전부 실패하는 구성 오류 — 이 경우 producer use case 쪽 활성화/구성을 먼저 의심
|
||||
3. `outbox.pending.size` status 분포 확인 (FAILED 누적 vs PENDING 누적)
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- 일시 실패는 자동 backoff 재시도(30s × 2^(attempt-1) + jitter, max attempts 3)가 동작 — 즉시 수동 개입 불필요
|
||||
- broker 장기 다운이면 DEAD 전이 누적 전에 broker 회복을 우선 (max attempts 소진 시 `runbook://outbox/dead-letter`로 이관)
|
||||
- relay 자체를 멈춰야 하면 `ca-skeleton.outbox.relay-enabled=false`로 스케줄러 비활성 (이벤트는 outbox 테이블에 안전하게 보존됨 — 유실 없음)
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | error.code="OUTBOX_PUBLISH_FAILED" | stats count by event_type`
|
||||
- metric panel:
|
||||
- `outbox.publisher.published.total{outcome}` — FAILED 비율
|
||||
- `outbox.publisher.lag{event_type}` — 최고령 미발행 이벤트 age
|
||||
- `outbox.pending.size{status}` — 상태별 분포
|
||||
- DB 확인: `SELECT status, count(*) FROM outbox_event GROUP BY status;`
|
||||
- 가능한 원인:
|
||||
- broker outage/네트워크 → broker 측 회복 대기
|
||||
- Kafka adapter 미구성(enabled인데 brokers 누락은 기동 시 차단됨) / disabled 상태에서 producer 활성화
|
||||
- poison event (직렬화 불가/payload 계약 위반) → 재시도 무의미, attempts 소진 후 DEAD로 흘러감 (의도된 동작)
|
||||
- 동일 aggregate head 실패로 후행 이벤트가 FIFO 게이트에 차단되어 lag 증가 (strict per-aggregate FIFO — 설계 의도)
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: broker 회복 후 backoff 만료 시 자동 재발행 — `outcome=PUBLISHED` 회복 확인
|
||||
- IN_FLIGHT orphan(claim 후 crash)은 in-flight-timeout(기본 PT5M) 경과 후 자동 재claim — at-least-once이므로 중복 발행 가능, consumer dedupe(idempotencyKey)가 흡수
|
||||
- 장기: `ca-skeleton.outbox.poll-interval`/`batch-size` 조정, broker 가용성 SLA 점검, 빈발 event_type의 payload 계약 검토
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- P1 lag(>300s 5m) 지속 + broker 회복 불가면 broker/infra 팀에 page
|
||||
- DEAD 전이가 발생하기 시작하면 `runbook://outbox/dead-letter` 절차로 이관
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: `outcome=FAILED` rate < 1% 10분 지속, `outbox.publisher.lag` < 60s, `outbox.pending.size{status=FAILED}` 감소 추세
|
||||
- post-incident: 실패 구간의 DEAD row 유무 확인, consumer 측 중복 처리량 확인(dedupe 동작 검증), backoff/attempts 상수 재평가
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `OUTBOX_PUBLISH_FAILED` (TRANSIENT_DEPENDENCY, retryable=true, retry_after 30s)
|
||||
- metrics.yaml: `outbox.publisher.published.total`, `outbox.publisher.lag`, `outbox.pending.size`
|
||||
- 코드: `application-core` `PublishPendingOutboxEventsUseCase`(상태머신), `adapter-persistence` `outbox/OutboxEventJpaRepository`(SKIP LOCKED claim + FIFO 게이트), `adapter-outbound` `messaging/outbox/KafkaOutboxMessagePublishAdapter`(fail-closed)
|
||||
- 관련 runbook: [[outbox-dead-letter]]
|
||||
- 관련 branch: [[feature-domain-event-outbox-contract]], [[feature-background-job-async-contract]] (retry/DLQ vocabulary SSOT)
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 실제 broker 채택·alert 라우팅·대시보드 링크 확정 시 보강 필요.
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
title: Runbook — Rate limit 초과 spike
|
||||
category: RATE_LIMIT
|
||||
error_codes: [RATE_LIMIT_EXCEEDED, IDEMPOTENT_IN_FLIGHT]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-05-22
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: Rate limit 초과 spike
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `rate_limit_429_high`
|
||||
- alert payload 필수 field: `operation`, `error.code`, `error.category`, `rate_limit_key_type`(ip|principal|tenant), `runbook_link`
|
||||
- 임계:
|
||||
- P3: 429 rate > 1% 10분 지속 (일상적 abuse 차단 효과 정상)
|
||||
- P2 격상: 정상 client(known principal/tenant)에서 spike 또는 spike와 함께 5xx 동반
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. rate-limit key 분포 확인: IP/principal/tenant 중 어디서 spike가 발생했는지
|
||||
- log query: `error.code=RATE_LIMIT_EXCEEDED | stats count by rate_limit_key_type, rate_limit_key`
|
||||
2. top-N offending key 추출 (상위 10건)
|
||||
3. 정상 client 식별 — 알려진 partner/internal service면 P2 격상
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- abuse traffic 패턴이면 WAF/gateway에서 IP block (geo, ASN 단위)
|
||||
- IDEMPOTENT_IN_FLIGHT 다발이면 client의 retry-storm 의심 → client에 retry-after 협조 요청
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="gateway"} | error.code="RATE_LIMIT_EXCEEDED" | stats count by rate_limit_key`
|
||||
- metric panel: `gateway_rate_limit_dropped_total`, `gateway_rate_limit_bucket_utilization`
|
||||
- trace: 429 응답의 `Retry-After` 헤더 값, `rate_limit_remaining` header 확인
|
||||
- 가능한 원인:
|
||||
- abuse / bot traffic → IP/ASN block
|
||||
- 정상 client의 traffic 증가 (캠페인, 신규 feature) → limit 일시 상향
|
||||
- retry-storm (client backoff 미적용) → client에 idempotency-key + exponential backoff 권고
|
||||
- limit 설정 오류 (잘못된 정량 threshold) → config rollback
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: abuse면 IP/ASN block, 정상 client면 해당 key의 limit 일시 상향(예: 2x, 1시간 TTL)
|
||||
- IDEMPOTENT_IN_FLIGHT 다발: idempotency-key 정책 점검, client 협조 요청
|
||||
- 장기: limit 정책을 tenant tier별 차등으로 재설계, abuse pattern detection 자동화
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 다음 on-call로 page: 30분 내 정상 client 회복 안 되면 product team 통보
|
||||
- 정상 client에 SLO 위반 가능성 있으면 CSM/계정담당 통보
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: 429 rate < 0.5% 10분 지속, 정상 client의 success rate 정상화
|
||||
- post-incident:
|
||||
- 일시 상향한 limit 원복 (TTL 만료 확인)
|
||||
- abuse pattern을 detection rule에 영구 등록
|
||||
- retry-storm이면 client SDK 가이드 보완
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `RATE_LIMIT_EXCEEDED`, `IDEMPOTENT_IN_FLIGHT`
|
||||
- metrics.yaml: `gateway_rate_limit_dropped_total`, `gateway_rate_limit_bucket_utilization`
|
||||
- 관련 branch: [[feature-rate-limit-idempotency-contract]]
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 도메인 도입 시 실제 gateway 제품(NGINX/Envoy/Kong 등)·tenant tier 정책·WAF 연동 절차로 보강 필요.
|
||||
@@ -0,0 +1,229 @@
|
||||
---
|
||||
title: Runbook — Redis capability incident
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: []
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-07-29
|
||||
status: active
|
||||
---
|
||||
|
||||
# Runbook: Redis capability incident (`runbook://redis/capability-incident`)
|
||||
|
||||
이 runbook은 Redis 전체를 하나의 상태로 취급하지 않는다. 먼저 영향받은 capability와 role을
|
||||
식별한다.
|
||||
|
||||
| Role | Capability | 기본 안전 결정 |
|
||||
| --- | --- | --- |
|
||||
| `CACHE` | cache, cache refresh soft lease | source fallback 예산 안에서 degraded serving 허용 |
|
||||
| `COORDINATION` | edge rate limit, request-replay idempotency, efficiency lease | 새 mutation/claim을 fail closed하고 결과 불확실성을 보존 |
|
||||
| `SESSION` | Redis session | 인증을 fail open하지 않고 재인증 또는 503으로 전환 |
|
||||
|
||||
Redis liveness 실패만으로 pod를 재시작하지 않는다. 재시작 폭주는 reconnect와 source fallback
|
||||
부하를 키울 수 있다.
|
||||
|
||||
## Detection
|
||||
|
||||
- readiness detail에서 affected role과 `required` 여부를 확인한다. endpoint, key, token, secret
|
||||
reference는 detail에 포함되면 안 된다.
|
||||
- semantic reason을 구분한다: read/write failure, program ACL denial, unsupported server
|
||||
version, program failure, command saturation, recent command failure, probe-in-progress,
|
||||
stale observation, closed route, command unavailable. `semanticObservedAt`,
|
||||
`semanticAgeMillis`, `semanticStale`를 함께 확인한다. PING 성공만으로 role이 ready라는 뜻은
|
||||
아니다.
|
||||
- `evictionValidation=CONFIGURED_EXPECTATION_ONLY`와
|
||||
`externalEvictionAttestation=INCOMPLETE`는 effective server policy가 증명되지 않았다는
|
||||
뜻이다. 이를 정상 attestation으로 해석하지 않는다.
|
||||
- `redis.capability.operations.total`과 `redis.capability.duration.seconds`에서 affected
|
||||
capability/role/operation의 실제 반환 outcome을 확인한다. mutation의
|
||||
`certainty=indeterminate`는 timeout이나 연결 끊김을 미실행 증거로 바꾸지 않는다.
|
||||
- `redis.capability.admission.rejected.total`에서 `rejected_saturated`와
|
||||
`rejected_closed`를 구분하고, `redis.capability.inflight.total`의 같은 role에 대해 현재 0이
|
||||
아닌 state를 확인한다. in-flight gauge는 bounded command count이며 byte 수나 queue depth가
|
||||
아니다.
|
||||
- `redis.capability.readiness.total`은 현재 상태 gauge가 아니라 exact sanitized
|
||||
`RoleHealth` 관측 횟수다. 최신 health detail의 state/reason/requirement와 함께 해석한다.
|
||||
optional cache의 degraded serving과 required coordination/session의 fail-closed 결정을
|
||||
같은 availability 의미로 합치지 않는다.
|
||||
- 종료 시 `redis.capability.lifecycle.drain.total`에서 `drained`,
|
||||
`forced_after_timeout`, `interrupted`를 구분한다. repeated close는 새 drain을 시작하거나
|
||||
중복 관측을 만들지 않는다.
|
||||
- reconnect, cache source-load, session repository error 지표의 변화를 함께 본다.
|
||||
- Redis server 측에서는 memory/eviction, rejected clients, replication link/lag,
|
||||
persistence error, Cluster coverage를 operator dashboard에서 확인한다.
|
||||
- `NOSCRIPT`, result-schema mismatch, ACL denial, TLS/auth failure, OOM, timeout을 서로 다른
|
||||
incident category로 분류한다. timeout은 command 미실행 증거가 아니다.
|
||||
|
||||
### Observability and lifecycle boundaries
|
||||
|
||||
- 여섯 `redis.capability.*` meter의 tag는 닫힌 enum에서만 생성된다. key, subject, session id,
|
||||
token, endpoint, exception text, script/SHA, value 같은 identity/wire material을 metric이나
|
||||
ticket에 복사하지 않는다.
|
||||
- semantic operation 계측은 logical provider가 실제로 반환한 hit/miss/denied/conflict/
|
||||
unavailable/indeterminate 결과를 기록한다. cache의 `stale`/`skipped`, session의
|
||||
`tombstoned`/`absolute_expired`도 정상 hit/miss와 분리한다. meter registry, classifier,
|
||||
monotonic ticker 장애는 command 결과나 원래 exception instance를 바꾸지 않는다.
|
||||
- route 응답이 설정된 byte/collection bound를 넘으면 동일 logical operation을
|
||||
`unavailable`로 종료한다. GET/read-only 응답은 `not_applied`, mutation VALUE/MULTI 응답은
|
||||
서버 실행 여부를 되돌릴 수 없으므로 `indeterminate`다. 앞선 `success` 표본과 이 실패를 두
|
||||
operation으로 합산하지 않는다.
|
||||
- Spring 종료의 dependency order는 invalidation subscription 같은 capability dependent를 먼저
|
||||
닫고, capability bean을 닫은 다음 canonical registry가 router admission을 닫아 in-flight를
|
||||
bounded drain하고 마지막에 runtime을 닫는 순서다. 종료 중 새 command를 허용하거나 drain
|
||||
timeout 뒤 무기한 기다리지 않는다.
|
||||
- 현재 composition에는 active Redis scheduler나 dormant credential-rotation coordinator가 없다.
|
||||
존재하지 않는 lifecycle coordinator를 복구 절차에서 찾거나 수동 호출하지 않는다.
|
||||
- 이 meter와 단일-process lifecycle test는 Sentinel/Cluster failover, TLS/ACL 배포 적합성,
|
||||
k3s multi-node, L1/L2 분산 일관성, distributed session 동작의 qualification 증거가 아니다.
|
||||
해당 label은 별도 topology/conformance lane의 실제 증거가 있어야 한다.
|
||||
|
||||
## Immediate mitigation
|
||||
|
||||
1. 새 배포나 credential/program 전환 직후라면 해당 rollout을 중지한다. 이미 실행된 mutation을
|
||||
무조건 재시도하지 않는다.
|
||||
2. optional cache만 영향을 받으면 source bulkhead와 stale/source fallback 예산을 확인한 뒤
|
||||
degraded serving을 유지한다. source가 포화되면 cache miss를 더 많은 source 요청으로
|
||||
증폭시키지 않는다.
|
||||
3. rate limit이 불확실하면 정책에 정의된 fail-closed 또는 bounded local-emergency만 사용한다.
|
||||
local provider를 조용한 primary fallback으로 바꾸지 않는다.
|
||||
4. idempotency claim/complete 응답이 유실됐으면 같은 operation token으로 inspect/reconcile한다.
|
||||
record를 삭제하거나 새 owner를 추측하지 않는다.
|
||||
5. lease 결과가 불확실하면 소유권이 있다고 가정하지 않는다. fencing 없는 efficiency lease를
|
||||
correctness lock으로 승격하지 않는다.
|
||||
6. session repository 장애에서는 기존 요청을 인증된 것으로 간주하지 않는다. fail closed 또는
|
||||
재인증으로 전환하고 JWT와 Redis Session filter를 동시에 활성화하지 않는다.
|
||||
|
||||
## Diagnosis
|
||||
|
||||
### Connectivity, TLS, ACL
|
||||
|
||||
- 배포 설정이 올바른 role을 참조하고 TLS, hostname verification, explicit trust bundle, named ACL
|
||||
user를 사용하는지 확인한다.
|
||||
- runtime identity로 `CONFIG`, `KEYS`, `FLUSH*`, arbitrary program deployment를 시도하지
|
||||
않는다. Catalog digest로 닫힌 recovery 외 ACL 점검은 별도 operator/deployer identity의
|
||||
`ACL DRYRUN` 또는 동등한 관리 절차로 수행한다.
|
||||
- runtime readiness identity에는 bounded probe namespace `~ca-health:*`, SET/GET/DEL,
|
||||
PING/EVALSHA와 catalog recovery에 필요한 SCRIPT LOAD, 그리고 선택 capability manifest의 exact
|
||||
command set이 필요하다. broad `~*`/`+@all`로 장애를 우회하지 않는다.
|
||||
- readiness probe는 5초 TTL의 opaque key만 사용한다. `ca-health:*` key가 5초를 넘겨 남는다면
|
||||
cleanup/expiry 이상으로 분류하되 key나 value를 ticket/log에 복사하지 않는다.
|
||||
- 기본 semantic cadence는 minimum interval 5초, maximum staleness 15초다. refresh follower는
|
||||
blocking하지 않는다. maximum staleness를 넘은 관측을 backend 정상으로 해석하지 말고,
|
||||
probe 부하를 줄이기 위해 interval을 1초 미만으로 낮추지 않는다.
|
||||
- optional CACHE의 typed temporary connect/PING outage만 dormant degraded startup과
|
||||
health-triggered reconnect를 허용한다. reconnect 후보는 full semantic qualification 뒤에만
|
||||
설치된다. auth/TLS/material/version/ACL/schema mismatch를 transient로 재분류하거나 required
|
||||
role에 같은 fallback을 적용하지 않는다.
|
||||
- credential rotation 중이라면 new credential 검증, traffic switch, old connection drain,
|
||||
old credential revoke 순서를 확인한다. secret 값은 ticket, log, shell history에 복사하지 않는다.
|
||||
|
||||
### Program or schema
|
||||
|
||||
- checked-in program manifest digest와 배포 artifact digest를 대조한다.
|
||||
- `semantic-capability-acl-v1` contract와 Redis minimum 7.2를 확인한다. 이 프로그램은
|
||||
Redis Lua API의 `redis.acl_check_cmd`로 선택 capability의 exact command/key 권한을
|
||||
비변경 방식으로 검사하고 `redis.REDIS_VERSION_NUM`의 explicit >=7.2 gate를 먼저 적용한다.
|
||||
두 API는 7.0부터 존재하지만 repository support policy minimum은 7.2다.
|
||||
- `NOSCRIPT`는 bounded `SCRIPT LOAD -> digest verify -> EVALSHA` recovery가 수행됐는지 확인한다.
|
||||
arbitrary `EVAL`로 우회하지 않는다.
|
||||
- result-schema/key/codec future version은 장애가 아니라 호환성 위반으로 분류하고 writer rollout을
|
||||
중지한다.
|
||||
- `BUSY` 또는 slow program이면 affected capability admission을 줄이고 isolated environment에서만
|
||||
재현한다. shared Redis에 장시간 script를 추가 실행하지 않는다.
|
||||
|
||||
### Memory and eviction
|
||||
|
||||
- `CACHE` 배포와 `COORDINATION`/`SESSION` 배포가 물리적으로 분리됐는지 확인한다.
|
||||
- correctness role에서 eviction이 관측되면 P1이다. 새 write를 중지하고 record loss를 전제로
|
||||
idempotency/session reconciliation 또는 재인증 범위를 산정한다.
|
||||
- noeviction OOM은 성공으로 변환하지 않는다. cache write는 degraded/indeterminate, coordination
|
||||
mutation은 unavailable/indeterminate로 유지한다.
|
||||
- big key를 찾을 때 production request path에서 `KEYS`나 unbounded collection read를 사용하지
|
||||
않는다. 승인된 operator job의 bounded `SCAN`/sampling을 사용한다.
|
||||
|
||||
### Topology and persistence
|
||||
|
||||
- 현재 구현 후보 card의 promotion topology는 readiness registry의 `selected-topology`가 정본이다.
|
||||
이는 selection 또는 R2 qualification을 뜻하지 않는다. Sentinel/Cluster evidence가 없는
|
||||
상태에서 standalone 증거를 HA 증거로 재사용하지 않는다.
|
||||
- Cluster same-slot semantic probe는 해당 hash slot owner 한 노드만 검증한다. 이를 cluster-wide
|
||||
또는 failover target version/ACL/program 증거로 해석하지 말고, promotion 전에 모든 target을
|
||||
별도 conformance lane으로 검증한다.
|
||||
- failover 뒤에는 in-flight mutation의 certainty, primary role, program availability, replication
|
||||
offset/lag, persistence status를 각각 확인한다.
|
||||
- restore 후 session/idempotency/lease record를 자동으로 신뢰하지 않는다. security epoch,
|
||||
tombstone, durable receipt/fencing high-watermark가 필요한 capability는 별도 reconciliation을
|
||||
수행한다.
|
||||
|
||||
### Sentinel failover
|
||||
|
||||
1. affected role의 semantic readiness가 unavailable인지 확인하고 단순 PING success로 정상 판정하지
|
||||
않는다. required coordination/session은 새 mutation admission을 닫는다.
|
||||
2. 세 Sentinel 중 응답 수와 같은 master에 동의한 수를 확인한다. 2-of-3 동의 전에는 임의 endpoint,
|
||||
최초 응답 또는 DNS 추측으로 data runtime을 바꾸지 않는다.
|
||||
3. Sentinel discovery credential/CA와 Redis data credential/CA가 분리되어 있는지 확인한다.
|
||||
장애 우회를 위해 trust-all, hostname verification off, plaintext 또는 broad ACL을 열지 않는다.
|
||||
4. election, discovered primary qualification, new runtime install, old runtime admission close/drain의
|
||||
순서를 확인한다. old runtime을 강제로 닫아야 했다면 그 시점의 mutation을 성공/미실행으로
|
||||
추정하지 않는다.
|
||||
5. response-only cut, timeout, disconnect가 있었던 rate/idempotency/session mutation은
|
||||
`INDETERMINATE`를 보존한다. rate evaluation replay, 같은 idempotency/session operation token의
|
||||
inspect/reconcile 또는 재인증을 사용하고 blind retry하지 않는다.
|
||||
6. semantic readiness 복구 전에는 traffic을 정상화하지 않는다. 복구 뒤 old primary의 replica
|
||||
재합류, replication lag/acknowledgement, program digest, actor runtime generation을 확인한다.
|
||||
|
||||
Sentinel은 asynchronous replication의 zero-data-loss나 strong consistency를 보장하지 않는다.
|
||||
`min-replicas-to-write`, lag bound, replica acknowledgement가 설정돼도 acknowledgement 결과가
|
||||
불명확한 mutation은 여전히 `INDETERMINATE`다.
|
||||
|
||||
`min-replicas-to-write 1` + `min-replicas-max-lag 1`은 선택이 아니라 **필수**다. 미설정 시
|
||||
promotion 중 교체된 구 primary가 계속 `+OK`를 반환하고 그 write는 resync에서 폐기된다. 7.4
|
||||
레인 실측: 승격 후 강등까지 11초, 그 사이 **2,086건이 acknowledge된 뒤 소실**, 실패한 명령은
|
||||
1건. 클라이언트는 이를 감지할 수단이 없다 — 서버가 응답했으므로 driver·SDK·호출자 모두
|
||||
정상 성공으로 기록한다. 설정 후 동일 promotion에서 소실 1건, 나머지 2,020건은 `NOREPLICAS`로
|
||||
명시 거부됐다. 근거: `docs/redis/operations.md`, `LiveRedisSentinelPromotionTest`.
|
||||
|
||||
### Disposable Multipass k3s qualification safety
|
||||
|
||||
qualification lab은 host k3s incident 조치 도구가 아니다. VM exact allowlist는
|
||||
`ca-redis-lab-server`, `ca-redis-lab-agent-1`, `ca-redis-lab-agent-2`이며 전용 kubeconfig와
|
||||
`ca-redis-lab` context만 사용한다.
|
||||
|
||||
- 시작 전 host context/API/node/CIDR/NodePort와 Multipass inventory fingerprint를 기록한다.
|
||||
- lab pod/service CIDR `10.52.0.0/16`, `10.53.0.0/16`이 host와 겹치면 생성하지 않는다.
|
||||
- default kubeconfig를 merge/overwrite하거나 host context에 write command를 실행하지 않는다.
|
||||
- cleanup은 exact 세 VM만 대상으로 한다. global `multipass purge`, wildcard delete를 사용하지
|
||||
않는다.
|
||||
- 성공/실패 뒤 postflight fingerprint와 VM resource 0을 확인한다. local retain-on-failure가
|
||||
명시적으로 활성화됐으면 보존 이유와 exact inventory를 기록하며 CI에서는 보존하지 않는다.
|
||||
- 이 한 물리 host의 3 VM 결과를 k3s control-plane HA, physical host/AZ failure 또는
|
||||
multi-region 증거로 승격하지 않는다.
|
||||
|
||||
## Recovery and verification
|
||||
|
||||
1. affected role의 connection/auth/TLS와 `ca-health:` SET/GET/cleanup probe가 정상인지
|
||||
확인한다. probe 잔여 key가 있으면 최대 TTL 5초 뒤 소멸하는지도 확인한다.
|
||||
2. 선택 capability의 대표 program digest/result schema, semantic ACL contract와 Redis minimum
|
||||
version 7.2를 재확인한다.
|
||||
3. capability별 smoke를 수행한다: cache generation guarded write, rate evaluation replay,
|
||||
idempotency same-operation inspect, lease stale-owner reject, session create/read/logout.
|
||||
4. queue saturation, indeterminate outcome, source fallback, re-auth 지표가 incident 전 범위로
|
||||
돌아온 뒤에만 rollout을 재개한다.
|
||||
5. `CONFIGURED_EXPECTATION_ONLY`인 eviction은 operator/deployer identity의 외부 conformance
|
||||
job 또는 서명 attestation으로 effective policy를 별도 검증한다. runtime user에 CONFIG/ACL
|
||||
권한을 추가하지 않는다.
|
||||
6. production label을 변경하기 전 repository readiness task를 실행한다. Sentinel/Cluster task가
|
||||
zero-evidence로 실패한다면 topology를 낮춰 표기하거나 실제 evidence를 먼저 추가한다.
|
||||
7. Sentinel qualification에서는 actual image ID/digest와 fault/election/runtime-swap/readiness
|
||||
timeline, capability certainty, teardown 결과가 sanitizer/reconciler를 통과했는지 확인한다.
|
||||
clean committed source와 실제 remote CI가 없으면 `implemented-candidate`,
|
||||
`releaseQualification=NOT_CLAIMED`를 유지한다.
|
||||
|
||||
## Escalation
|
||||
|
||||
- `COORDINATION` 또는 `SESSION` required role이 5분 이상 unavailable이면 P1로 Redis/platform,
|
||||
application on-call을 동시에 호출한다.
|
||||
- data loss, stale session resurrection, conflicting idempotency completion, duplicate correctness
|
||||
side effect가 의심되면 security/business owner까지 즉시 확대한다.
|
||||
- 한 물리 host의 VM 세 개 또는 standalone container 결과를 AZ/host failure 증거로 해석하지
|
||||
않는다. 그 증거가 필요한 release는 별도 disposable multi-node qualification을 요구한다.
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: Runbook — JVM_OOM (JVM OutOfMemoryError)
|
||||
category: INTERNAL
|
||||
error_codes: [JVM_OOM]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: JVM_OOM (`runbook://runtime/jvm-oom`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Container exits with code 137 (ExitOnOutOfMemoryError triggered)
|
||||
- Structured log entry with `error.code=JVM_OOM` before exit
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check heap dump if `-XX:HeapDumpOnOutOfMemoryError` is configured
|
||||
- Review memory usage trends before crash
|
||||
- Check for memory leaks: large cache growth, unbounded lists, session accumulation
|
||||
|
||||
## Action
|
||||
|
||||
- Restart container immediately (k8s will auto-restart with liveness probe)
|
||||
- If recurring: increase heap `-Xmx` or fix memory leak
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1: immediate if multiple pods crashing simultaneously
|
||||
- Page SRE / infra team for heap analysis
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
title: Runbook — PROFILE_MISMATCH (프로파일 불일치)
|
||||
category: INTERNAL
|
||||
error_codes: [PROFILE_MISMATCH]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: PROFILE_MISMATCH (`runbook://startup/profile-mismatch`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Container exits with code 71 (profile mismatch exit)
|
||||
- Structured log with `error.code=PROFILE_MISMATCH`, `startup.phase=profile-check`
|
||||
- Production profile active with local-only settings enabled
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check active Spring profiles (`spring.profiles.active`)
|
||||
- Identify which local-only setting is incorrectly enabled in prod profile
|
||||
|
||||
## Action
|
||||
|
||||
- Remove local-only setting from production deployment config
|
||||
- Ensure prod profile does not inherit local/dev profile settings
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1: security risk if local settings expose debug endpoints in production
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — REQUIRED_ADAPTER_DISABLED (필수 어댑터 비활성화)
|
||||
category: INTERNAL
|
||||
error_codes: [REQUIRED_ADAPTER_DISABLED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: REQUIRED_ADAPTER_DISABLED (`runbook://startup/required-adapter-disabled`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Container exits with code 72 (required adapter disabled exit)
|
||||
- Structured log with `error.code=REQUIRED_ADAPTER_DISABLED`, `startup.phase=adapter-check`
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Identify which adapter is disabled but required
|
||||
- Check adapter enable flags in environment config
|
||||
|
||||
## Action
|
||||
|
||||
- Enable required adapter in deployment configuration
|
||||
- If adapter is intentionally disabled, update the required/optional designation
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1: app cannot start; coordinate with deployment team
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — STARTUP_VALIDATION_FAILED (환경 변수 검증 실패)
|
||||
category: INTERNAL
|
||||
error_codes: [STARTUP_VALIDATION_FAILED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: STARTUP_VALIDATION_FAILED (`runbook://startup/validation-failed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Container exits with code 78 (env validation failure exit)
|
||||
- Structured log with `error.code=STARTUP_VALIDATION_FAILED`, `startup.phase=env-validation`
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check which required env variable is missing or malformed
|
||||
- Review container environment and secrets injection
|
||||
|
||||
## Action
|
||||
|
||||
- Supply missing environment variables to deployment
|
||||
- Verify secrets are correctly mounted / injected
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1: app cannot start; coordinate with deployment/secrets team
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
title: Runbook — <TITLE>
|
||||
category: <CATEGORY>
|
||||
error_codes: [<ERROR_CODE_1>, <ERROR_CODE_2>]
|
||||
severity: <P1|P2|P3>
|
||||
owner: oncall
|
||||
last_updated: <YYYY-MM-DD>
|
||||
status: <stub|active>
|
||||
---
|
||||
|
||||
# Runbook: <TITLE> (`runbook://<area>/<scenario>`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- What observable signals trigger this runbook?
|
||||
- Alert name, metric thresholds, log patterns
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Step-by-step diagnostic commands and queries
|
||||
- Log queries (Loki/CloudWatch)
|
||||
- Metric panels to check
|
||||
- Trace investigation approach
|
||||
|
||||
## Action
|
||||
|
||||
- Immediate mitigation steps
|
||||
- Configuration changes
|
||||
- Manual intervention procedures
|
||||
|
||||
## Escalation
|
||||
|
||||
- Conditions for severity upgrade (e.g., P2 → P1)
|
||||
- Who to page and when
|
||||
- Fallback procedures if on-call cannot resolve
|
||||
|
||||
---
|
||||
|
||||
> **Note**: This is the canonical runbook template.
|
||||
> Copy this file, rename it to match the `runbook://area/scenario` pattern (→ `area-scenario.md`),
|
||||
> fill in the frontmatter fields, replace section bodies with operational content,
|
||||
> then set `status: active`. `LEGACY_STUB_DEBT` in `RunbookCoverageContractTest` is temporary
|
||||
> containment for existing debt only; do not add a new stub there. Complete the runbook or adopt
|
||||
> the future owned, expiring debt ledger.
|
||||
@@ -0,0 +1,4 @@
|
||||
# feature-security-operational-baseline D5 — deny-by-default public path snapshot.
|
||||
# SSOT: SECURITY_PUBLIC_PATHS (src/.env) -> SecurityConfig permitAll(); anyRequest authenticated.
|
||||
# Update only after review with: ./gradlew updatePublicPathSnapshot -PapprovePublicPathChange
|
||||
/api/healthcheck
|
||||
@@ -0,0 +1,131 @@
|
||||
> **SUPERSEDED — HISTORICAL PROVENANCE ONLY (2026-07-25):** The user-approved harness-free
|
||||
> Mode B amendment supersedes this plan. Retain the body as historical provenance; it is not
|
||||
> executable instruction.
|
||||
|
||||
# Harness Policy Engine Implementation Plan
|
||||
|
||||
> **Spec:** `docs/superpowers/specs/2026-07-20-harness-policy-engine-design.md`
|
||||
|
||||
**Goal:** Replace topology- and platform-specific duplicated harness rules with a registry,
|
||||
strict evidence validators, generated platform variants, and risk-based review policies.
|
||||
|
||||
**Working policy:** human-only commits. Each task leaves changes in the working tree.
|
||||
|
||||
## Task 1 — Registry, resolver, and Gradle SSOT
|
||||
|
||||
**Files:**
|
||||
|
||||
- Add `.harness/project/modules.yaml`
|
||||
- Add `.harness/lib/module_registry.py`
|
||||
- Add `.harness/validators/validate_modules.py`
|
||||
- Add `.harness/tests/test_module_registry.py`
|
||||
- Modify `src/settings.gradle`
|
||||
- Modify the dependency-verifier section of `src/build.gradle`
|
||||
|
||||
**Steps:**
|
||||
|
||||
- [ ] Write failing tests for 19-leaf loading, nested owner resolution, nearest `CLAUDE.md`,
|
||||
unknown paths, and settings/registry parity.
|
||||
- [ ] Add the registry and stdlib loader/resolver.
|
||||
- [ ] Make Gradle settings and dependency verification consume registry data.
|
||||
- [ ] Run Python tests and `./gradlew projects verifyCleanArchitectureDependencies`.
|
||||
|
||||
## Task 2 — Registry-driven import gate and mutation suite
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify `.claude/hooks/ca_import_gate.py`
|
||||
- Modify `.claude/hooks/test_ca_import_gate.py`
|
||||
- Add `.harness/tests/test_import_gate_mutations.py`
|
||||
|
||||
**Steps:**
|
||||
|
||||
- [ ] Add failing real-path tests for every registered production module.
|
||||
- [ ] Replace flat-path regex/prefix rules with registry owner and role policy.
|
||||
- [ ] Normalize Claude snake_case and Antigravity camelCase tool events.
|
||||
- [ ] Fail closed on malformed in-scope events and marker failures.
|
||||
- [ ] Run all import-gate tests.
|
||||
|
||||
## Task 3 — Verdict schema, evidence artifacts, and platform adapters
|
||||
|
||||
**Files:**
|
||||
|
||||
- Add `.harness/schemas/verdict.schema.json`
|
||||
- Add `.harness/schemas/evidence.schema.json`
|
||||
- Add `.harness/lib/verdict.py`
|
||||
- Add `.harness/validators/validate_verdict.py`
|
||||
- Add `.harness/validators/validate_evidence.py`
|
||||
- Add `.harness/adapters/antigravity_hook.py`
|
||||
- Add `.harness/tests/test_verdict.py`
|
||||
- Modify `.claude/hooks/ca_verdict_gate.py`
|
||||
- Modify `.claude/hooks/test_ca_verdict_gate.py`
|
||||
- Add `.agents/plugins/ca-superpowers/hooks.json`
|
||||
|
||||
**Steps:**
|
||||
|
||||
- [ ] Write negative tests for missing required enums, negative counts, Gradle arithmetic,
|
||||
behavior change without red, missing upstream artifacts, malformed input, and revision
|
||||
mismatch.
|
||||
- [ ] Implement strict validation and evidence recording with source/diff hashes.
|
||||
- [ ] Adapt Claude fenced verdicts to the common model.
|
||||
- [ ] Add Antigravity Stop/pre-tool adapter and plugin hook wiring.
|
||||
- [ ] Run validator, adapter, and JSON syntax tests.
|
||||
|
||||
## Task 4 — Canonical agents and deterministic rendering
|
||||
|
||||
**Files:**
|
||||
|
||||
- Add `.harness/agents/*.md`
|
||||
- Add `.harness/project/platforms.yaml`
|
||||
- Add `.harness/generators/render_agents.py`
|
||||
- Add `.harness/tests/test_platform_parity.py`
|
||||
- Regenerate `.claude/agents/*`, `.codex/agents/*.toml`, `.agents/agents/*/agent.json`
|
||||
- Update `.agents/plugins/ca-superpowers/README.md` and `plugin.json`
|
||||
- Update `.codex/agents/README.md`
|
||||
|
||||
**Steps:**
|
||||
|
||||
- [ ] Seed canonical sources from the newest human-only Claude policy, then update module
|
||||
discovery and runner validation to use the registry.
|
||||
- [ ] Add generated metadata and stable output ordering.
|
||||
- [ ] Render all variants and add a `--check` parity mode.
|
||||
- [ ] Assert commit policy, source hashes, tool permissions, and body parity in tests.
|
||||
|
||||
## Task 5 — Risk/profile policies and guidance drift cleanup
|
||||
|
||||
**Files:**
|
||||
|
||||
- Add `.harness/manifest.yaml`
|
||||
- Add `.harness/core/risk-policy.yaml`, `.harness/core/evidence-policy.yaml`
|
||||
- Add current architecture/language/build/framework/capability profile files
|
||||
- Add `.harness/validators/resolve_task.py` and tests
|
||||
- Modify `AGENTS.md`, root `CLAUDE.md`, clean-architecture rule, workflow skill,
|
||||
advisory-depth rule, reporting-standards rule, and plugin README
|
||||
- Modify stale module `CLAUDE.md` files and add missing leaf-module guidance where useful
|
||||
|
||||
**Steps:**
|
||||
|
||||
- [ ] Add failing task-classification tests for high-risk one-file changes and low-risk
|
||||
multi-file fixture/docs changes.
|
||||
- [ ] Implement profile resolution.
|
||||
- [ ] Replace `N!`, routine all-quote grep, file-count report split, and unconditional
|
||||
counterargument policies with the design profiles.
|
||||
- [ ] Replace flat module documentation and focused commands with registry-backed nested names.
|
||||
- [ ] Run policy grep assertions and harness tests.
|
||||
|
||||
## Task 6 — Full review and verification
|
||||
|
||||
- [ ] Run harness unit/mutation/parity suite.
|
||||
- [ ] Run `./gradlew projects` and `./gradlew verifyCleanArchitectureDependencies`.
|
||||
- [ ] Run the focused ArchUnit suite.
|
||||
- [ ] Run `./gradlew check`.
|
||||
- [ ] Audit the working-tree diff in order: architecture → spec → quality.
|
||||
- [ ] Fix findings and restart the review chain, up to three loops.
|
||||
|
||||
## Task 7 — LLM Wiki capture
|
||||
|
||||
- [ ] Read the LLM Wiki authority and branch-note template.
|
||||
- [ ] Update/create the detached-HEAD branch note with implementation decisions, changed files,
|
||||
verification evidence, failures, and open risks.
|
||||
- [ ] Create/link derived error, interview, or blog-topic raw notes only when supported by the
|
||||
completed work; otherwise record an explicit “none” judgment in the branch note.
|
||||
+144
@@ -0,0 +1,144 @@
|
||||
# Application Outbox Failure Reporting Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development
|
||||
> (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use
|
||||
> checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make `application-core` framework/logging-free while preserving one safe structured ERROR
|
||||
after each confirmed outbox FAILED/DEAD transition.
|
||||
|
||||
**Architecture:** The application owns a narrow typed reporting port and safe report value.
|
||||
Messaging renders the report through SLF4J, and bootstrap only injects it. Transition state remains
|
||||
authoritative; reporter failures are non-authoritative and contained.
|
||||
|
||||
**Tech Stack:** Java 21 records, JUnit Jupiter, AssertJ, Spring Boot 4 configuration, SLF4J 2 fluent
|
||||
logging, Logback capture tests, ArchUnit, Gradle Groovy DSL, dependency locking.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Safe Application Report Contract
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/application-core/src/test/java/dev/caskeleton/application/outbox/OutboxRelayFailureReportTest.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayFailureReport.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/outbox/OutboxRelayFailureReportPort.java`
|
||||
|
||||
- [ ] Write factory, invariant, and reflection-whitelist tests for the exact eight record components.
|
||||
- [ ] Run `./gradlew :application-core:test --tests '*OutboxRelayFailureReportTest' --console=plain`
|
||||
and record the expected missing-type RED.
|
||||
- [ ] Implement the immutable record, exact invariants, factories, and functional port.
|
||||
- [ ] Re-run the focused value test and record GREEN.
|
||||
|
||||
### Task 2: Relay Reporting Behavior
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/application-core/src/test/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCaseTest.java`
|
||||
- Modify: `src/application-core/src/main/java/dev/caskeleton/application/outbox/PublishPendingOutboxEventsUseCase.java`
|
||||
- Modify direct test constructor sites under
|
||||
`src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/integration/outbox/`
|
||||
|
||||
- [ ] Add recording/throwing reporters and tests for exact FAILED/DEAD reports, all no-report paths,
|
||||
transition failure propagation, and reporter-failure continuation.
|
||||
- [ ] Run the relay test and record constructor/behavior RED.
|
||||
- [ ] Inject the reporter after the publish port, remove SLF4J, report only after successful
|
||||
transition, and contain reporter `RuntimeException`.
|
||||
- [ ] Update test-only direct constructors with explicit lambdas and re-run relay tests GREEN.
|
||||
|
||||
### Task 3: Structured Messaging Adapter and Publish-Adapter Deduplication
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create:
|
||||
`src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/Slf4jOutboxRelayFailureReportAdapterTest.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/Slf4jOutboxRelayFailureReportAdapter.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/messaging/src/test/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapterTest.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/outbox/OutboxMessagePublishAdapter.java`
|
||||
|
||||
- [ ] Write Logback capture tests for exact ERROR count, fixed fields, throwable, retry-only time,
|
||||
unsafe-data absence, internal logging failure containment, and the adapter contract that
|
||||
`report(null)` never throws.
|
||||
- [ ] Run
|
||||
`./gradlew :adapter:outbound:messaging:test --tests '*Slf4jOutboxRelayFailureReportAdapterTest' --console=plain`
|
||||
and record missing-type RED.
|
||||
- [ ] Implement the SLF4J 2 fluent adapter and re-run GREEN.
|
||||
- [ ] Replace outbox publish WARN expectations with no-log and propagation expectations; run RED.
|
||||
- [ ] Remove `FailOpenDependencyLogger` from the outbox adapter and re-run its tests GREEN, leaving
|
||||
`OutboundMessagePublisher` unchanged.
|
||||
|
||||
### Task 4: Unconditional Reporter Wiring
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/adapter/outbound/messaging/src/main/java/dev/caskeleton/adapter/outbound/messaging/MessagingConfig.java`
|
||||
- Modify: `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/outbox/OutboxConfig.java`
|
||||
- Modify: `src/app-bootstrap/src/test/java/dev/caskeleton/adapter/outbound/OptionalAdapterBeanGatingTest.java`
|
||||
- Modify: `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/outbox/OutboxConfigTest.java`
|
||||
|
||||
- [ ] Add disabled and active context assertions for exactly one structured reporter bean.
|
||||
- [ ] Run `OptionalAdapterBeanGatingTest` and record RED.
|
||||
- [ ] Add the unconditional messaging reporter bean, use `disabled` for blank broker, update outbox
|
||||
publish adapter construction, and inject the port through bootstrap.
|
||||
- [ ] Re-run the gating and outbox configuration tests GREEN.
|
||||
|
||||
### Task 5: Application Dependency Purity
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/build.gradle`
|
||||
- Modify: `src/application-core/build.gradle`
|
||||
- Mechanically regenerate only: `src/application-core/gradle.lockfile`
|
||||
|
||||
- [ ] Add `verifyApplicationCoreDependencyPurity`, wire it into `:application-core:check`, and run it
|
||||
against the current starter declaration to record RED.
|
||||
- [ ] Give `application-core` only JUnit Jupiter and AssertJ test dependencies while retaining the
|
||||
shared Boot test dependencies for every other leaf.
|
||||
- [ ] Remove the application Spring Boot starter and re-run the purity task GREEN.
|
||||
- [ ] Run
|
||||
`./gradlew :application-core:resolveAndLockAll --write-locks --console=plain`; confirm no other
|
||||
lockfile changes.
|
||||
- [ ] Run application lock verification, tests, and compile/test runtime dependency reports.
|
||||
|
||||
### Task 6: Non-Vacuous Diagnostic Architecture Rule
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify:
|
||||
`src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/CleanArchitectureTest.java`
|
||||
- Create:
|
||||
`src/app-bootstrap/src/test/java/dev/caskeleton/application/architecture/violations/ApplicationDiagnosticFrameworkViolation.java`
|
||||
- Modify:
|
||||
`src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/ArchitectureViolationFixtureTest.java`
|
||||
|
||||
- [ ] Add the violation fixture inside the exact `dev.caskeleton.application..` rule scope and its
|
||||
mutation assertion; run it before the rule to record RED.
|
||||
- [ ] Add `APPLICATION_HAS_NO_DIAGNOSTIC_FRAMEWORK`, scoped exactly to
|
||||
`dev.caskeleton.application..`, for SLF4J, JUL, Logback, Log4j, and Micrometer.
|
||||
- [ ] Run the mutation test and production `CleanArchitectureTest` GREEN.
|
||||
|
||||
### Task 7: Documentation and Verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/application-core/CLAUDE.md`
|
||||
- Modify: `src/application-core/README.md`
|
||||
- Modify: `src/adapter/outbound/messaging/CLAUDE.md`
|
||||
- Modify: `src/adapter/outbound/messaging/README.md`
|
||||
- Modify relevant wiring guidance in `src/app-bootstrap/README.md`
|
||||
|
||||
- [ ] Document the framework-free application contract, typed report semantics, messaging ownership,
|
||||
duplicate-log rule, and bootstrap wiring-only role.
|
||||
- [ ] Run focused application, messaging, gating, architecture mutation, production architecture,
|
||||
and available outbox integration tests.
|
||||
- [ ] Run `verifyCleanArchitectureDependencies`, dependency evidence reports, and `check`.
|
||||
- [ ] Run required safety greps, `git diff --check`, and `git status --short`; report any skip or
|
||||
remaining risk.
|
||||
- [ ] Hand the exact LLM Wiki capture responsibility and evidence back to the top-level controller;
|
||||
do not write the vault from this dispatched scope.
|
||||
|
||||
No step authorizes staging, committing, amending, pushing, public-path changes, CI changes, module
|
||||
registry changes, or `.harness` changes.
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,99 @@
|
||||
# Harness-Free Mode B Amendment Implementation Plan
|
||||
|
||||
> **For agentic workers:** Execute this plan task-by-task with
|
||||
> `superpowers:executing-plans`; use `superpowers:test-driven-development` for the build behavior
|
||||
> change and `superpowers:verification-before-completion` before reporting results.
|
||||
|
||||
**Goal:** Restore Gradle bootstrap and Clean Architecture dependency enforcement without recreating
|
||||
the absent development harness.
|
||||
|
||||
**Architecture:** One strict JSON registry under `src/config/architecture/` owns all 19 leaf
|
||||
identities, paths, and allowed production project edges. Gradle settings validate and include the
|
||||
registry fail-closed; the root dependency verification task reads the same file and checks actual
|
||||
production project dependencies against it.
|
||||
|
||||
**Tech Stack:** Gradle Groovy DSL, Groovy `JsonSlurper`, strict JSON, Java 21.
|
||||
|
||||
**Working policy:** Human-only git handling. Do not stage, commit, amend, push, or create a PR.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Capture the broken bootstrap
|
||||
|
||||
**Files:**
|
||||
|
||||
- Read: `src/settings.gradle`
|
||||
|
||||
- [x] Run `cd src && ./gradlew help --console=plain`.
|
||||
- [x] Confirm exit 1 is caused by the missing `.harness/project/modules.yaml`, not dependency
|
||||
resolution or an unrelated build failure.
|
||||
|
||||
### Task 2: Add the Gradle-owned registry
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `src/config/architecture/modules.json`
|
||||
- Read: each of the 19 leaf-module `build.gradle` files
|
||||
|
||||
- [x] Record exactly 19 unique module IDs, Gradle paths, and repository-relative source paths.
|
||||
- [x] Set `allowed_dependencies` from each leaf's current `api`, `implementation`, `compileOnly`,
|
||||
and `runtimeOnly` project dependencies.
|
||||
- [x] Exclude test/fixture configurations from production policy and keep `sample-portfolio` a
|
||||
fixture consumer that no production leaf may depend on.
|
||||
- [x] Parse the file with Python's strict JSON parser and compare its edges with the checked-in
|
||||
leaf build declarations.
|
||||
|
||||
### Task 3: Restore Gradle bootstrap and dependency enforcement
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `src/settings.gradle`
|
||||
- Modify: `src/build.gradle`
|
||||
|
||||
- [x] Make settings load only `config/architecture/modules.json`.
|
||||
- [x] Fail closed on a missing registry, wrong root/module/field types, empty values, duplicate
|
||||
identities or paths, unsafe path shapes, unknown/self dependencies, count drift, or missing
|
||||
source directories.
|
||||
- [x] Include every registered Gradle path and map it to its repository-root-relative source
|
||||
directory.
|
||||
- [x] Make `verifyCleanArchitectureDependencies` read the same registry without a second module
|
||||
list.
|
||||
- [x] Preserve all-leaf coverage and forbidden-edge checks, explicitly reject a production edge
|
||||
to `sample-portfolio`, and replace stale error wording with actionable registry guidance.
|
||||
|
||||
### Task 4: Align active repository guidance
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `AGENTS.md`
|
||||
- Modify: `CLAUDE.md`
|
||||
- Modify: `README.md`
|
||||
- Modify: `src/README.md`
|
||||
- Modify: all 19 nearest leaf-module `CLAUDE.md` files that name the old registry
|
||||
- Annotate as superseded: the 2026-07-20 harness design and plan
|
||||
|
||||
- [x] Point active topology and allowed-edge guidance to
|
||||
`src/config/architecture/modules.json`.
|
||||
- [x] State that focused commands are derived from the owning Gradle path rather than a task
|
||||
packet.
|
||||
- [x] Keep all eight local HARD-STOP meanings, architecture boundaries, human-only git policy,
|
||||
verification discipline, and LLM Wiki capture requirements.
|
||||
- [x] Make the earlier harness documents explicit historical provenance rather than active
|
||||
reconstruction instructions.
|
||||
|
||||
### Task 5: Verify from a fresh Gradle invocation
|
||||
|
||||
**Files:**
|
||||
|
||||
- Verify: all changed files
|
||||
|
||||
- [ ] Run `cd src && ./gradlew help --console=plain`.
|
||||
- [ ] Run `cd src && ./gradlew projects --console=plain`.
|
||||
- [ ] Run `cd src && ./gradlew verifyCleanArchitectureDependencies --console=plain`.
|
||||
- [ ] Run a deterministic strict-JSON script proving exactly 19 unique IDs/Gradle paths and
|
||||
existing source directories.
|
||||
- [ ] Run a deterministic comparison between registry edges and leaf production project
|
||||
dependencies.
|
||||
- [ ] Run `git diff --check` and `git status --short`.
|
||||
- [ ] Report exact exits, any unavailable checks, LLM Wiki capture outcome, and remaining risks
|
||||
without claiming the broader Phase A/refactor is complete.
|
||||
@@ -0,0 +1,117 @@
|
||||
# Harness-Free Quality and Security CI Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use `superpowers:executing-plans` to implement this
|
||||
> plan task-by-task, `superpowers:test-driven-development` for executable drift controls, and
|
||||
> `superpowers:verification-before-completion` before reporting. Git remains human-only: do not
|
||||
> stage, commit, amend, or push.
|
||||
|
||||
**Goal:** Reconstruct a harness-free, repository-internal quality and dependency-security CI
|
||||
control plane that is truthful to the current Gradle build and `main` branch.
|
||||
|
||||
**Architecture:** Canonical workflows live only under `.github/workflows`. A small YAML gate matrix
|
||||
maps current controls to real Gradle tasks/plugins/tests and workflow jobs, while a portable Bash
|
||||
verifier rejects drift; vulnerability policy is enforced by a platform-neutral Trivy filesystem
|
||||
job with guarded GitHub-only complements.
|
||||
|
||||
**Tech Stack:** GitHub Actions-compatible YAML, Bash, Gradle 9 Groovy DSL, Java/Temurin 21, Trivy,
|
||||
jq, lychee.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Capture missing-control RED
|
||||
|
||||
**Files:**
|
||||
|
||||
- Verify absent: `.trivyignore.yaml`
|
||||
- Verify absent: `.github/ci-gate-matrix.yml`
|
||||
- Verify absent: `.github/scripts/verify-gate-matrix.sh`
|
||||
|
||||
- [ ] Run `cd src && ./gradlew verifyTrivyignore --console=plain`.
|
||||
- [ ] Confirm the failure names the missing repository-root `.trivyignore.yaml`.
|
||||
- [ ] Confirm the matrix, verifier, and canonical workflows are absent.
|
||||
|
||||
### Task 2: Add repository baselines
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `.tool-versions`
|
||||
- Create: `.gitattributes`
|
||||
- Create: `.trivyignore.yaml`
|
||||
|
||||
- [ ] Pin `java temurin-21.0.11+10`, matching candidate evidence and the local Gradle launcher JDK.
|
||||
- [ ] Normalize source, YAML, Markdown, Gradle, and shell text to LF; keep `gradlew.bat` CRLF and
|
||||
mark common binary formats `-text`.
|
||||
- [ ] Add the four structured empty Trivy sections with suppression governance comments.
|
||||
- [ ] Run `cd src && ./gradlew verifyTrivyignore --console=plain` and expect zero suppressions
|
||||
validated.
|
||||
|
||||
### Task 3: Add quality governance and drift verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `.github/CODEOWNERS`
|
||||
- Create: `.github/pull_request_template.md`
|
||||
- Create: `.github/ci-gate-matrix.yml`
|
||||
- Create: `.github/scripts/verify-gate-matrix.sh`
|
||||
- Create: `.github/workflows/ci-quality-gates.yml`
|
||||
- Create: `.github/workflows/link-check.yml`
|
||||
|
||||
- [ ] Record only current Gradle/task/test/job mechanisms in the matrix.
|
||||
- [ ] Implement repository-root-safe matrix parsing with schema, uniqueness, task/plugin/test, and
|
||||
workflow-job checks.
|
||||
- [x] Before Java/Gradle, fail unless `docs/security/public-paths-snapshot.txt` is committed and
|
||||
non-empty; do not let the Gradle task create a first-run CI baseline.
|
||||
- [ ] Have a human track and commit the canonical snapshot; agents do not stage or commit, and CI's
|
||||
`git ls-files` precondition rejects an untracked worktree file.
|
||||
- [ ] Add required `quality-gates`, `sample-off`, and `gate-matrix-lint` jobs plus the advisory
|
||||
quarantine job.
|
||||
- [ ] Make `release-gate` depend exactly on the three required jobs and fail unless all succeeded.
|
||||
- [ ] Add path-scoped link checking for PR and `main` push.
|
||||
- [ ] Pin every workflow `uses:` reference to a verified full commit SHA and retain its immutable
|
||||
release label in an inline comment.
|
||||
- [ ] Run Bash syntax and gate-matrix checks.
|
||||
|
||||
### Task 4: Add dependency-vulnerability controls
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `.github/dependency-review-config.yml`
|
||||
- Create: `.github/dependency-vulnerability-policy.md`
|
||||
- Create: `.github/scripts/install-jq.sh`
|
||||
- Create: `.github/workflows/dependency-vulnerability.yml`
|
||||
|
||||
- [ ] Configure PR dependency review to block new High/Critical runtime vulnerabilities and
|
||||
forbidden strong/network-copyleft licenses without posting PR summary comments.
|
||||
- [ ] Document High/Critical blocking, Medium/Low advisory, KEV fail-closed handling, suppression
|
||||
review, GitHub/Gitea differences, egress, and mirror requirements.
|
||||
- [ ] Install checksum-pinned jq and version-pinned Trivy under `${RUNNER_TEMP}`, adding them through
|
||||
`${GITHUB_PATH}` without privileged writes.
|
||||
- [ ] Guard GitHub-only review/submission and keep `trivy-fs` platform-neutral on all required
|
||||
triggers.
|
||||
- [ ] Pass `--ignorefile .trivyignore.yaml` to every Trivy invocation.
|
||||
- [ ] Reject KEV catalogs with blank metadata, non-positive/non-integral or mismatched counts,
|
||||
empty vulnerability arrays, invalid CVE identifiers, or duplicate identifiers before
|
||||
intersection.
|
||||
- [ ] Reject malformed or empty Trivy JSON before extracting candidate vulnerability identifiers.
|
||||
|
||||
### Task 5: Verify the reconstructed slice
|
||||
|
||||
**Files:**
|
||||
|
||||
- Verify: all files created by this plan
|
||||
|
||||
- [ ] Parse strict policy/matrix YAML with an available parser and document GitHub `on` parser
|
||||
limitations if applicable.
|
||||
- [ ] Prove only `main` is an active branch trigger and no active `master` remains.
|
||||
- [ ] Prove every Trivy scan consumes the root ignore file.
|
||||
- [ ] Prove the release fan-in is exact and excludes quarantine.
|
||||
- [x] Prove the missing/empty/untracked snapshot precondition exits non-zero; the canonical
|
||||
`/api/healthcheck` snapshot now exists in the worktree but still requires a human commit.
|
||||
- [ ] Exercise the KEV predicate with empty/malformed/count/CVE/duplicate failures and a valid
|
||||
synthetic catalog.
|
||||
- [ ] Exercise the Trivy JSON predicate with malformed Results/Vulnerabilities/IDs and a realistic
|
||||
valid Results array.
|
||||
- [ ] Prove no harness call or `.gitea/workflows` shadow was introduced.
|
||||
- [ ] Run `git diff --check` and `git status --short`.
|
||||
- [ ] Capture the work in the required LLM Wiki branch note, including evidence and external
|
||||
blockers, without claiming server Actions or full Phase A completion.
|
||||
@@ -0,0 +1,103 @@
|
||||
# Harness-Free Module and Gradle Hygiene Implementation Plan
|
||||
|
||||
**Goal:** Apply the approved 19-leaf dependency and boundary cleanup without `.harness`.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-07-25-module-gradle-hygiene-harness-free-design.md`
|
||||
|
||||
**Policy:** TDD for behavior/boundary changes; focused proof before dependency removal; human-only
|
||||
Git operations.
|
||||
|
||||
## Task 1: Lock Phase B and characterize the Phase C baseline
|
||||
|
||||
- [ ] Confirm the Phase B focused tests, dependency-purity gate, spec review, and quality review
|
||||
are green.
|
||||
- [ ] Record the current 19-leaf registry and affected lockfiles.
|
||||
- [ ] Run the existing OpenAPI runtime tests before changing springdoc.
|
||||
|
||||
## Task 2: Isolate pure-core tests
|
||||
|
||||
- [ ] Change the root test convention so `domain-core`, `application-core`, and
|
||||
`shared-contract` receive only JUnit Jupiter, AssertJ, and the platform launcher.
|
||||
- [ ] Run the three core test suites and dependency reports.
|
||||
- [ ] Regenerate only their affected locks and prove no Spring coordinate remains on their test
|
||||
runtime classpaths.
|
||||
|
||||
## Task 3: Prune core/inbound declarations and align Boot 4
|
||||
|
||||
- [ ] Before editing, run and record each affected leaf's `compileJava`, `compileTestJava`, `test`,
|
||||
runtime dependency report, and relevant dependency insight.
|
||||
- [ ] Remove the approved unused project edges from application and inbound leaves.
|
||||
- [ ] Upgrade springdoc to `3.0.0`.
|
||||
- [ ] Remove unused GraphQL/WebSocket Jackson 2 declarations and unused gRPC direct declarations.
|
||||
- [ ] Characterize `jackson-databind-nullable` with dependency insight and focused
|
||||
present/null/undefined Jackson 3 tests; exclude its Jackson 2 transitive dependency only if the
|
||||
tests and real-server OpenAPI contract remain green.
|
||||
- [ ] Run each affected leaf test plus the two real-server `/v3/api-docs` tests.
|
||||
- [ ] Update the OpenAPI snapshot only if the generated public contract is semantically unchanged.
|
||||
|
||||
## Task 4: Prune outbound declarations
|
||||
|
||||
- [ ] Before editing, run and record each affected leaf's `compileJava`, `compileTestJava`, `test`,
|
||||
runtime dependency report, and relevant dependency insight.
|
||||
- [ ] Apply the approved support/cache/httpclient/identifier/messaging/notification project-edge
|
||||
removals.
|
||||
- [ ] Remove Groovy/Spock only from leaves with no Groovy tests.
|
||||
- [ ] Narrow fileserver/objectstorage from the broad Boot starter to autoconfigure plus SLF4J API.
|
||||
- [ ] Remove the JPA domain edge and remove explicit Flyway core only if focused compile/test proves
|
||||
it is redundant.
|
||||
- [ ] Run affected compile/tests before and after each dependency group.
|
||||
|
||||
## Task 5: Enforce configuration-processor parity
|
||||
|
||||
- [ ] Add a failing verification fixture or temporary mutation proving the exact
|
||||
`@ConfigurationProperties(` parity check detects missing and extra processors.
|
||||
- [ ] Register `verifyConfigurationPropertiesProcessor` from the JSON registry and wire it into
|
||||
leaf `check`.
|
||||
- [ ] Add processors to settings-owning leaves and remove the unused GraphQL processor.
|
||||
- [ ] Run the new gate and affected settings tests.
|
||||
|
||||
## Task 6: Remove the Mongo example domain
|
||||
|
||||
- [ ] Add tests for disabled mode, enable-flag binding, and enabled infrastructure with a mock
|
||||
`MongoClient`.
|
||||
- [ ] Delete all production/test `Example*` types and remove the fixed example bean/repository
|
||||
scanning.
|
||||
- [ ] Remove obsolete project and Testcontainers dependencies.
|
||||
- [ ] Run the Mongo tests and an `rg` assertion that production contains no `Example*`.
|
||||
|
||||
## Task 7: Invert sample correlation access
|
||||
|
||||
- [ ] Add framework-free `CorrelationIdPort` contract tests/fakes.
|
||||
- [ ] Add and test the inbound web MDC implementation.
|
||||
- [ ] Change the two sample application collaborators to use the port while retaining event-id
|
||||
fallback behavior.
|
||||
- [ ] Add an architecture assertion that sample application source has no SLF4J dependency.
|
||||
- [ ] Run application, web, sample outbox/poster, and architecture focused tests.
|
||||
|
||||
## Task 8: Clean generated state and composition documentation
|
||||
|
||||
- [ ] Delete tracked `src/sample-portfolio/.jqwik-database` and ignore future files.
|
||||
- [ ] Correct app-bootstrap “every module” wording and document default versus opt-in runtime
|
||||
composition.
|
||||
- [ ] Preserve the existing default runtime dependency set.
|
||||
|
||||
## Task 9: Locks, full verification, and review
|
||||
|
||||
- [ ] Regenerate strict lockfiles only with each affected leaf's
|
||||
`:leaf-path:resolveAndLockAll --write-locks`; do not run the root all-leaf writer.
|
||||
- [ ] Run all commands in the design verification section.
|
||||
- [ ] Run `git diff --check` and inspect the complete unstaged/untracked status.
|
||||
- [ ] Request spec and code-quality review; fix all actionable findings.
|
||||
- [ ] Update the mandated LLM Wiki raw branch note and derived raw notes, or record the exact
|
||||
missing-vault blocker.
|
||||
|
||||
## Final review hardening
|
||||
|
||||
- [x] Pin the Springdoc 3 `ApiError.details` widening with a real-server RED test.
|
||||
- [x] Add a web-owned OpenAPI customizer, import it in both real-server test applications, and
|
||||
restore the committed `type: object` snapshot without adding Swagger to `shared-contract`.
|
||||
- [x] Reproduce starter-driven Mongo activation through an actual `@EnableAutoConfiguration`
|
||||
context in both default and explicit-false modes.
|
||||
- [x] Register a module-level Boot 4 `AutoConfigurationImportFilter` that blocks Mongo
|
||||
auto-configuration until the module enable flag is true.
|
||||
- [x] Re-run affected formatting, locks, focused tests, and all design verification commands.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,66 @@
|
||||
# Fileserver Durable Recovery Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this
|
||||
> plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. Repository policy is
|
||||
> human-only, so no step stages or commits changes.
|
||||
|
||||
**Goal:** Make the local publication provider restart-safe for completed and sealed operations
|
||||
without re-running the row producer.
|
||||
|
||||
**Architecture:** Keep the application port unchanged. The adapter owns a private operation journal
|
||||
under `.ca-fileserver/operations`, writes records through forced temp files and atomic rename, and
|
||||
uses a deterministic request fingerprint. A retry restores a verified terminal receipt or resumes a
|
||||
sealed staged artifact; disagreement is a conflict or indeterminate outcome, never an overwrite.
|
||||
|
||||
**Tech Stack:** Java 21 NIO, JUnit 5, AssertJ, existing Gradle quality gates.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Define deterministic journal records and request fingerprints
|
||||
|
||||
**Files:**
|
||||
- Create: `src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPublicationJournalRecord.java`
|
||||
- Create: `src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPublicationJournalCodec.java`
|
||||
- Create: `src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FilePublishRequestFingerprint.java`
|
||||
- Test: `src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPublicationJournalTest.java`
|
||||
|
||||
- [x] Write a failing test proving stable request fingerprints and different fingerprints for
|
||||
source/schema changes.
|
||||
- [x] Write a failing test proving journal round-trip and rejection of corrupt/newer records.
|
||||
- [x] Run
|
||||
`./gradlew :adapter:outbound:fileserver:test --tests '*LocalPublicationJournalTest' --console=plain`
|
||||
and confirm the missing types fail compilation.
|
||||
- [x] Implement a bounded flat JSON codec with schema version, state, fingerprint, locator token,
|
||||
checksum/counts and receipt snapshot fields. It must reject duplicate/unknown keys and never
|
||||
serialize absolute paths or row data.
|
||||
- [x] Run the focused test and confirm GREEN.
|
||||
|
||||
### Task 2: Add forced atomic journal persistence and recovery
|
||||
|
||||
**Files:**
|
||||
- Create: `src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPublicationJournal.java`
|
||||
- Modify: `src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalFilePublicationAdapter.java`
|
||||
- Test: `src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalFilePublicationRecoveryTest.java`
|
||||
|
||||
- [x] Write a failing test where a completed operation is retried with a producer that throws; the
|
||||
original receipt must be returned and the producer must remain uncalled.
|
||||
- [x] Write a failing test that reconstructs a new adapter over a sealed journal plus staged bytes
|
||||
and resumes publication without calling the producer.
|
||||
- [x] Write a failing test proving the same operation ID with a different request is a conflict and
|
||||
a digest mismatch is indeterminate.
|
||||
- [x] Run the recovery test and confirm RED.
|
||||
- [x] Persist `WRITING`, `SEALED`, and `PUBLISHED` records with temp + force + atomic move. Verify
|
||||
the target size and SHA-256 before terminal reconstruction.
|
||||
- [x] Run all Fileserver tests and confirm GREEN.
|
||||
|
||||
### Task 3: Report the exact readiness boundary
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/adapter/outbound/fileserver/README.md`
|
||||
- Modify: `src/adapter/outbound/fileserver/CLAUDE.md`
|
||||
- Modify: `docs/superpowers/specs/2026-07-26-fileserver-production-capability-design.md`
|
||||
|
||||
- [x] Mark single-node local restart recovery as implemented.
|
||||
- [x] Keep multi-node fencing, bounded background reaper, SFTP, NFS and HA evidence explicitly
|
||||
unimplemented.
|
||||
- [x] Run `./gradlew :adapter:outbound:fileserver:check --console=plain`.
|
||||
@@ -0,0 +1,236 @@
|
||||
# Fileserver Production Capability Foundation Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace the list-materializing CSV demo boundary with the Phase 1 framework-free publication contract and a bounded, staged local CSV R1 provider without claiming crash-safe R2 guarantees.
|
||||
|
||||
**Architecture:** `application-core` owns typed publication requests, rows, cells, producer/sink callbacks, opaque references, and receipts. `adapter:outbound:fileserver` owns CSV encoding, spreadsheet-formula mitigation, staging, digest/count limits, and local atomic publication. The legacy `FileExportPort` remains temporarily for compatibility and is explicitly documented as deprecated R0/R1 behavior.
|
||||
|
||||
**Tech Stack:** Java 21, JUnit 5, AssertJ, Spring Boot configuration properties, JDK NIO filesystem and SHA-256.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add the framework-free publication contract
|
||||
|
||||
**Files:**
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/FilePublicationPort.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/FilePublishRequest.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/FilePublishOperationId.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/FileDestinationId.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/LogicalFileName.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/SourceRevision.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/ExportSchema.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/TabularCell.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/TabularRow.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/TabularRowProducer.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/TabularRowSink.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/FilePublishReceipt.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/PublishedFileReference.java`
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/filepublication/FileVersion.java`
|
||||
- Test: `src/application-core/src/test/java/dev/caskeleton/application/filepublication/FilePublicationContractTest.java`
|
||||
|
||||
- [ ] **Step 1: Write the failing contract test**
|
||||
|
||||
```java
|
||||
@Test
|
||||
void requestRejectsPathLikeLogicalNamesAndSchemaRejectsDuplicateColumns() {
|
||||
assertThatThrownBy(() -> new LogicalFileName("../report.csv"))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
assertThatThrownBy(
|
||||
() ->
|
||||
new ExportSchema(
|
||||
"worklog-v1",
|
||||
1,
|
||||
List.of(
|
||||
new ExportSchema.Column(
|
||||
"id", ExportSchema.CellType.INTEGER, false,
|
||||
ExportSchema.FormulaPolicy.REJECT, 64),
|
||||
new ExportSchema.Column(
|
||||
"id", ExportSchema.CellType.TEXT, false,
|
||||
ExportSchema.FormulaPolicy.MITIGATE, 128))))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify RED**
|
||||
|
||||
Run: `cd src && ./gradlew :application-core:test --tests '*FilePublicationContractTest' --console=plain`
|
||||
|
||||
Expected: compilation failure because the `filepublication` contract does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement immutable validated values**
|
||||
|
||||
The contract must expose this shape and no `Path`, `File`, stream, Spring, or provider type:
|
||||
|
||||
```java
|
||||
public interface FilePublicationPort {
|
||||
FilePublishReceipt publish(FilePublishRequest request, TabularRowProducer producer);
|
||||
}
|
||||
|
||||
@FunctionalInterface
|
||||
public interface TabularRowProducer {
|
||||
void produce(TabularRowSink sink);
|
||||
}
|
||||
|
||||
public interface TabularRowSink {
|
||||
void write(TabularRow row);
|
||||
void checkpoint();
|
||||
}
|
||||
```
|
||||
|
||||
`TabularCell` is a sealed interface with nested records for text, integer, decimal, boolean, date,
|
||||
instant, and null. `ExportSchema` owns ordered columns, cell type, nullability, formula policy, and
|
||||
per-cell byte bounds. Records reject null/blank IDs, path separators in `LogicalFileName`, duplicate
|
||||
column names, empty schemas, and non-positive limits.
|
||||
|
||||
- [ ] **Step 4: Verify GREEN**
|
||||
|
||||
Run: `cd src && ./gradlew :application-core:test --tests '*FilePublicationContractTest' --console=plain`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 2: Add streaming CSV encoding and staged local publication
|
||||
|
||||
**Files:**
|
||||
- Create: `src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/StreamingCsvEncoder.java`
|
||||
- Create: `src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalFilePublicationAdapter.java`
|
||||
- Create: `src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalFilePublicationPolicy.java`
|
||||
- Test: `src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalFilePublicationAdapterTest.java`
|
||||
|
||||
- [ ] **Step 1: Write the failing streaming publication tests**
|
||||
|
||||
```java
|
||||
@Test
|
||||
void publishesRowsThroughTheSinkAndReturnsAnOpaqueReceipt() {
|
||||
AtomicInteger calls = new AtomicInteger();
|
||||
FilePublishReceipt receipt =
|
||||
adapter.publish(
|
||||
request(),
|
||||
sink -> {
|
||||
calls.incrementAndGet();
|
||||
sink.write(new TabularRow(List.of(new IntegerCell(1), new TextCell("=cmd"))));
|
||||
});
|
||||
|
||||
assertThat(calls).hasValue(1);
|
||||
assertThat(receipt.reference().value()).doesNotContain(tempDir.toString());
|
||||
assertThat(Files.readString(publishedFile(receipt), UTF_8)).contains("1,'=cmd");
|
||||
}
|
||||
|
||||
@Test
|
||||
void abortsBeforeFinalPublicationWhenTheByteLimitIsExceeded() {
|
||||
assertThatThrownBy(
|
||||
() -> adapter.publish(request(), sink -> sink.write(oversizedRow())))
|
||||
.isInstanceOf(FilePublicationException.class);
|
||||
assertThat(finalArtifacts()).isEmpty();
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify RED**
|
||||
|
||||
Run: `cd src && ./gradlew :adapter:outbound:fileserver:test --tests '*LocalFilePublicationAdapterTest' --console=plain`
|
||||
|
||||
Expected: compilation failure because the staged provider does not exist.
|
||||
|
||||
- [ ] **Step 3: Implement the minimum staged provider**
|
||||
|
||||
`LocalFilePublicationPolicy` validates a fixed destination ID, base directory, maximum rows,
|
||||
maximum encoded bytes, and the only initial format profile `csv-rfc4180-v1`.
|
||||
|
||||
`LocalFilePublicationAdapter` must:
|
||||
|
||||
```text
|
||||
validate request/schema before producer invocation
|
||||
create a private .staging directory
|
||||
exclusive-create an operation-scoped .part file
|
||||
write header and each row directly through StreamingCsvEncoder
|
||||
enforce schema/cell/row/byte limits at each sink call
|
||||
prefix dangerous spreadsheet text with a single quote when policy is MITIGATE
|
||||
compute SHA-256 and counts while writing
|
||||
flush and FileChannel.force(true)
|
||||
move staging to the final operation-scoped file with ATOMIC_MOVE
|
||||
delete staging on pre-publish failure
|
||||
return an opaque reference and never an absolute path
|
||||
```
|
||||
|
||||
The first release is labelled local R1. Existing final artifacts cause a typed conflict; durable
|
||||
operation journals, crash reconciliation, replace semantics, and SFTP/NFS remain unimplemented and
|
||||
must not be advertised.
|
||||
|
||||
- [ ] **Step 4: Verify GREEN**
|
||||
|
||||
Run: `cd src && ./gradlew :adapter:outbound:fileserver:test --tests '*LocalFilePublicationAdapterTest' --console=plain`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 3: Add opt-in R1 composition and truthful documentation
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileExportProperties.java`
|
||||
- Modify: `src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileExportConfig.java`
|
||||
- Create: `src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FilePublicationConfigTest.java`
|
||||
- Modify: `src/adapter/outbound/fileserver/README.md`
|
||||
- Modify: `src/adapter/outbound/fileserver/CLAUDE.md`
|
||||
|
||||
- [ ] **Step 1: Write the failing composition test**
|
||||
|
||||
```java
|
||||
@Test
|
||||
void disabledConfigurationCreatesNoPublicationPort() {
|
||||
contextRunner
|
||||
.withUserConfiguration(FileExportConfig.class)
|
||||
.run(context -> assertThat(context).doesNotHaveBean(FilePublicationPort.class));
|
||||
}
|
||||
|
||||
@Test
|
||||
void enabledConfigurationCreatesExactlyOneLocalR1PublicationPort() {
|
||||
contextRunner
|
||||
.withUserConfiguration(FileExportConfig.class)
|
||||
.withPropertyValues(
|
||||
"ca-skeleton.fileserver.enabled=true",
|
||||
"ca-skeleton.fileserver.destination-id=local-export",
|
||||
"ca-skeleton.fileserver.base-directory=" + tempDir)
|
||||
.run(context -> assertThat(context).hasSingleBean(FilePublicationPort.class));
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify RED**
|
||||
|
||||
Run: `cd src && ./gradlew :adapter:outbound:fileserver:test --tests '*FilePublicationConfigTest' --console=plain`
|
||||
|
||||
Expected: FAIL because the new port is not composed.
|
||||
|
||||
- [ ] **Step 3: Wire only the local R1 provider**
|
||||
|
||||
Add validated destination ID, row limit, byte limit, and format-profile settings. Contribute
|
||||
`FilePublicationPort` only when explicitly enabled. Keep `FileExportPort` as a deprecated compatibility
|
||||
bean and document that it materializes caller rows and is not R2 evidence.
|
||||
|
||||
- [ ] **Step 4: Verify module and architecture gates**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :application-core:test :adapter:outbound:fileserver:check --console=plain
|
||||
./gradlew verifyCleanArchitectureDependencies --console=plain
|
||||
./gradlew :app-bootstrap:test --tests '*CleanArchitectureTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: all commands PASS.
|
||||
|
||||
### Task 4: Record the unfinished R2 boundary
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/superpowers/specs/2026-07-26-fileserver-production-capability-design.md`
|
||||
|
||||
- [ ] **Step 1: Update implementation status without weakening completion criteria**
|
||||
|
||||
Record Phase 0–1/local R1 foundation as implemented. Keep Phase 2 durable journal/reconciliation,
|
||||
Phase 3 operations, Phase 4 SFTP, Phase 5 NFS/HA/bootstrap, and Phase 6 optional operations marked
|
||||
unimplemented. The document must still say that local R1 is not Fileserver R2.
|
||||
|
||||
- [ ] **Step 2: Verify documentation structure**
|
||||
|
||||
Run: `rg -n 'R1|R2|구현 상태|미구현' docs/superpowers/specs/2026-07-26-fileserver-production-capability-design.md`
|
||||
|
||||
Expected: explicit R1 implementation and remaining R2 gaps are both present.
|
||||
@@ -0,0 +1,882 @@
|
||||
# Fileserver R2 Control Plane and Provider Selection Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
||||
> `superpowers:subagent-driven-development` to implement this plan task-by-task. Steps use checkbox
|
||||
> (`- [ ]`) syntax for tracking. Repository policy is `human-only`: do not stage, commit, amend, or
|
||||
> push.
|
||||
|
||||
**Goal:** Add an explicit provider-neutral Fileserver R2 control plane and qualify
|
||||
`local-persistent` as the first provider without making local filesystem the production default.
|
||||
|
||||
**Architecture:** `application-core` keeps the existing `FilePublicationPort` and gains only one
|
||||
provider-neutral achieved-durability value. The fileserver leaf compiles `app.fileserver`
|
||||
destination/provider settings into an exact registry, routes requests through one port bean, and
|
||||
coordinates versioned operation, manifest, and reference records. A strict
|
||||
`local-persistent` provider attests its root before use and advances the durable publication state
|
||||
machine in forced, recoverable steps.
|
||||
|
||||
**Tech Stack:** Java 21, Spring Boot 4 configuration properties/autoconfiguration, JDK NIO/POSIX,
|
||||
JUnit 5, AssertJ, ApplicationContextRunner, Gradle quality gates.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add the provider-neutral achieved durability
|
||||
|
||||
**Files:**
|
||||
- Modify:
|
||||
`src/application-core/src/main/java/dev/caskeleton/application/filepublication/FilePublishReceipt.java`
|
||||
- Modify:
|
||||
`src/application-core/src/test/java/dev/caskeleton/application/filepublication/FilePublicationContractTest.java`
|
||||
|
||||
- [x] **Step 1: Write the failing contract test**
|
||||
|
||||
Add a test that constructs a receipt with the new achieved value and proves no provider or path type
|
||||
is introduced:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void receiptCanReportFileAndDirectorySyncWithoutExposingAProviderType() {
|
||||
FilePublishReceipt receipt =
|
||||
receiptWith(DurabilityGuarantee.FILE_AND_DIRECTORY_SYNC);
|
||||
|
||||
assertThat(receipt.durabilityGuarantee())
|
||||
.isEqualTo(DurabilityGuarantee.FILE_AND_DIRECTORY_SYNC);
|
||||
assertThat(FilePublishReceipt.class.getDeclaredFields())
|
||||
.allSatisfy(field -> assertThat(field.getType().getName())
|
||||
.doesNotContain("java.nio.file", "fileserver", "sftp"));
|
||||
}
|
||||
```
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :application-core:test --tests '*FilePublicationContractTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation failure because `FILE_AND_DIRECTORY_SYNC` does not exist.
|
||||
|
||||
- [x] **Step 3: Implement the minimum contract change**
|
||||
|
||||
Add only this enum member:
|
||||
|
||||
```java
|
||||
public enum DurabilityGuarantee {
|
||||
PROCESS_LOCAL_SYNC,
|
||||
FILE_AND_DIRECTORY_SYNC,
|
||||
PROVIDER_ACK_ONLY
|
||||
}
|
||||
```
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Compile exact destination/provider settings with no local fallback
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverR2Settings.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/CompiledFileDestination.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverBindingCompiler.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FileserverBindingCompilerTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing exact-binding tests**
|
||||
|
||||
Cover:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void enabledSettingsRequireAnExplicitDestinationAndProvider() {
|
||||
assertThatThrownBy(() -> FileserverBindingCompiler.compile(enabled(Map.of(), Map.of())))
|
||||
.isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessageContaining("destination");
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsUnknownOrUnimplementedProviderTypes() {
|
||||
assertThatThrownBy(() -> compile("shared-mounted"))
|
||||
.isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessageContaining("local-persistent");
|
||||
}
|
||||
|
||||
@Test
|
||||
void compilesOnlyAnExactLocalPersistentBinding() {
|
||||
Map<FileDestinationId, CompiledFileDestination> result =
|
||||
FileserverBindingCompiler.compile(validSettings());
|
||||
|
||||
assertThat(result).containsOnlyKeys(new FileDestinationId("local-export"));
|
||||
assertThat(result.get(new FileDestinationId("local-export")).providerId())
|
||||
.isEqualTo("local-primary");
|
||||
}
|
||||
```
|
||||
|
||||
Also reject blank IDs, unknown `provider-ref`, duplicate normalized IDs, non-absolute root, enabled
|
||||
`auto-create`, unsupported publication/durability values, and non-positive row/byte bounds.
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*FileserverBindingCompilerTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation failure because the settings/compiler do not exist.
|
||||
|
||||
- [x] **Step 3: Implement typed settings**
|
||||
|
||||
Use one public configuration-properties record:
|
||||
|
||||
```java
|
||||
@ConfigurationProperties(prefix = "app.fileserver")
|
||||
public record FileserverR2Settings(
|
||||
boolean enabled,
|
||||
Map<String, DestinationSettings> destinations,
|
||||
Map<String, ProviderSettings> providers) {
|
||||
|
||||
public record DestinationSettings(
|
||||
String providerRef,
|
||||
String requiredPublication,
|
||||
String requiredDurability,
|
||||
long maximumRows,
|
||||
long maximumEncodedBytes) {}
|
||||
|
||||
public record ProviderSettings(
|
||||
String type,
|
||||
String rootDirectory,
|
||||
boolean autoCreate,
|
||||
boolean strictPathSecurity,
|
||||
String expectedFileStoreName,
|
||||
String expectedFileStoreType,
|
||||
String mountSentinelName,
|
||||
String mountSentinelSha256,
|
||||
String expectedOwner,
|
||||
String maximumRootMode) {}
|
||||
}
|
||||
```
|
||||
|
||||
The compiler accepts exactly:
|
||||
|
||||
```text
|
||||
type=local-persistent
|
||||
required-publication=unique-atomic-create
|
||||
required-durability=file-and-directory-sync
|
||||
auto-create=false
|
||||
strict-path-security=true
|
||||
```
|
||||
|
||||
`CompiledFileDestination` contains validated application destination ID, provider ID, absolute
|
||||
root, limits, root attestation inputs, and no Spring type.
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Attest a pre-provisioned persistent root
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentRootEvidence.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentRootAttestor.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentRootAttestorTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing attestation tests**
|
||||
|
||||
Create a real POSIX temporary root and sentinel. Test successful evidence and each fail-closed
|
||||
condition:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void attestsOwnerModeStoreSentinelSecureDirectoryAndSyncPrimitives() {
|
||||
CompiledFileDestination destination = destinationFor(attestedRoot());
|
||||
|
||||
LocalPersistentRootEvidence evidence =
|
||||
new LocalPersistentRootAttestor().attest(destination);
|
||||
|
||||
assertThat(evidence.root()).isEqualTo(root.toRealPath());
|
||||
assertThat(evidence.secureDirectoryStream()).isTrue();
|
||||
assertThat(evidence.directorySync()).isTrue();
|
||||
assertThat(evidence.exclusiveHardLink()).isTrue();
|
||||
}
|
||||
```
|
||||
|
||||
Separate tests reject:
|
||||
|
||||
- relative or missing root;
|
||||
- symlink root/ancestor;
|
||||
- owner mismatch;
|
||||
- group/world-writable root;
|
||||
- FileStore name/type mismatch;
|
||||
- missing, symlinked, non-regular, or digest-mismatched sentinel;
|
||||
- staging/data/control on a different FileStore;
|
||||
- unavailable `SecureDirectoryStream`, hard-link, or directory-force probe.
|
||||
|
||||
Probe collaborators may be package-private injectable functions so negative paths do not depend on
|
||||
the host filesystem lacking a feature.
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentRootAttestorTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation failure because attestation types do not exist.
|
||||
|
||||
- [x] **Step 3: Implement strict attestation**
|
||||
|
||||
The attestor must:
|
||||
|
||||
```text
|
||||
reject before creating anything when root/sentinel/owner/mode/store mismatch
|
||||
capture root real path, file key, FileStore name/type, sentinel digest
|
||||
create private .ca-fileserver, data, staging, operations, manifests, references, probe directories
|
||||
set newly-created directories to 0700
|
||||
force each created parent directory
|
||||
open a SecureDirectoryStream on root
|
||||
run unique exclusive-create + force + hard-link + directory-force probe
|
||||
delete probe artifacts and force the probe directory
|
||||
return immutable evidence used for pre/post identity checks
|
||||
```
|
||||
|
||||
Do not silently downgrade to R1.
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS on the supported Linux/POSIX lane.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add strict reference, journal-v2, manifest, and reference records
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/R2PublishedReferenceCodec.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/DurablePublicationRecord.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/PrivateFileManifest.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/PublishedReferenceRecord.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverControlRecordCodec.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FileserverControlRecordCodecTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing codec tests**
|
||||
|
||||
Test:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void referenceRoundTripRejectsForgeryUnknownRouteAndTruncation() {
|
||||
PublishedFileReference reference = codec.encode("routea1", fixedFileId());
|
||||
|
||||
assertThat(codec.decode(reference, Set.of("routea1")).fileId()).isEqualTo(fixedFileId());
|
||||
assertThatThrownBy(() -> codec.decode(tamper(reference), Set.of("routea1")))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
assertThatThrownBy(() -> codec.decode(reference, Set.of("routeb2")))
|
||||
.isInstanceOf(IllegalArgumentException.class);
|
||||
}
|
||||
```
|
||||
|
||||
For all three records prove:
|
||||
|
||||
- canonical encode/decode round trip;
|
||||
- maximum encoded length;
|
||||
- exact schema version;
|
||||
- state and revision invariants;
|
||||
- single-segment internal locators;
|
||||
- lowercase SHA-256 fields;
|
||||
- no absolute path, raw row/cell, credential, URI, or control character;
|
||||
- newer schema and duplicate/unknown fields fail closed.
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*FileserverControlRecordCodecTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation failure because R2 records/codecs do not exist.
|
||||
|
||||
- [x] **Step 3: Implement bounded canonical records**
|
||||
|
||||
Use a strict flat canonical JSON codec owned by this leaf. The record state is:
|
||||
|
||||
```java
|
||||
enum State {
|
||||
WRITING,
|
||||
SEALED,
|
||||
DATA_PUBLISHED,
|
||||
MANIFEST_PUBLISHED,
|
||||
REFERENCE_PUBLISHED,
|
||||
PUBLISHED,
|
||||
QUARANTINED
|
||||
}
|
||||
```
|
||||
|
||||
`R2PublishedReferenceCodec` uses:
|
||||
|
||||
```text
|
||||
fsr1.<route-token>.<32-lower-hex-file-id>.<first-12-hex-of-sha256(prefix)>
|
||||
```
|
||||
|
||||
The check digits detect corruption only and are not authentication.
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Persist forced control records and operation locks
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentControlPlane.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentControlPlaneTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing control-plane tests**
|
||||
|
||||
Test direct lookup and forced revision handling:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void storesAndDirectlyLoadsOperationManifestAndReferenceRecords() {
|
||||
controlPlane.storeOperation(writingRecord());
|
||||
controlPlane.storeManifest(manifest());
|
||||
controlPlane.storeReference(referenceRecord());
|
||||
|
||||
assertThat(controlPlane.findOperation(OPERATION_ID)).contains(writingRecord());
|
||||
assertThat(controlPlane.findManifest(FILE_ID)).contains(manifest());
|
||||
assertThat(controlPlane.findReference(FILE_ID)).contains(referenceRecord());
|
||||
}
|
||||
```
|
||||
|
||||
Also prove:
|
||||
|
||||
- lower/equal incompatible state revision is rejected;
|
||||
- request fingerprint mismatch is conflict;
|
||||
- temp file is force-written before atomic replace;
|
||||
- target parent is forced after replace;
|
||||
- shard creation forces its parent;
|
||||
- symlink shard/record is rejected with `NOFOLLOW_LINKS`;
|
||||
- reads, temporary creation, stat, and delete use attested directory-relative names through
|
||||
`SecureDirectoryStream`; operations without a portable secure hard-link/flagged atomic-replace
|
||||
overload remain limited to the private-owner root and require pre/post identity checks;
|
||||
- same operation is serialized by JVM stripe plus OS `FileLock`;
|
||||
- record corruption is never treated as absent.
|
||||
|
||||
Use a package-private fault-point callback to observe/throw at:
|
||||
|
||||
```text
|
||||
TEMP_FORCED
|
||||
RECORD_REPLACED
|
||||
PARENT_FORCED
|
||||
```
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentControlPlaneTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation failure because the control plane does not exist.
|
||||
|
||||
- [x] **Step 3: Implement durable storage**
|
||||
|
||||
All writes follow:
|
||||
|
||||
```text
|
||||
CREATE_NEW sibling temp
|
||||
write all bytes
|
||||
FileChannel.force(true)
|
||||
ATOMIC_MOVE + REPLACE_EXISTING for the control record only
|
||||
force parent directory
|
||||
read-back and verify identity/revision/digest
|
||||
```
|
||||
|
||||
Payload publication must never use overwrite-capable move. Control record replacement is safe only
|
||||
under the operation lock and monotonically increasing `stateRevision`.
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Implement the local-persistent R2 provider and deterministic recovery
|
||||
|
||||
**Files:**
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/CompiledFileDestination.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverBindingCompiler.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentControlPlane.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPublicationJournalCodec.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FilePublicationProvider.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentPublicationProvider.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentPayloadOperations.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FilePublicationCanonicalDigests.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentRecoveryVerifier.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FileserverBindingCompilerTest.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentControlPlaneTest.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentPayloadOperationsTest.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentPublicationProviderTest.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentPublicationRecoveryTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing publication-order tests**
|
||||
|
||||
First add failing compiler/control-plane assertions for:
|
||||
|
||||
```text
|
||||
deterministic route token = "r" + first 31 lowercase hex of canonical policy digest
|
||||
same startup allowlist route-token collision -> startup failure
|
||||
length-prefixed effective policy/schema/format digest stability
|
||||
same secure operation lookup -> typed canonical v1 or v2
|
||||
v1 is read-only; malformed UTF-8/non-canonical/newer schema is indeterminate, never absent
|
||||
control fault context identifies record kind, identity,
|
||||
applicable operation state/revision, and force boundary
|
||||
```
|
||||
|
||||
Then use a deterministic file ID/clock and a fault recorder. Prove exact order:
|
||||
|
||||
```text
|
||||
J_WRITING
|
||||
STAGE_FORCED
|
||||
J_SEALED
|
||||
DATA_LINKED
|
||||
DATA_DIRECTORY_FORCED
|
||||
J_DATA_PUBLISHED
|
||||
MANIFEST_FORCED
|
||||
J_MANIFEST_PUBLISHED
|
||||
REFERENCE_FORCED
|
||||
J_REFERENCE_PUBLISHED
|
||||
J_PUBLISHED
|
||||
```
|
||||
|
||||
Verify the receipt has an opaque `fsr1` reference,
|
||||
`UNIQUE_ATOMIC_CREATE`, and `FILE_AND_DIRECTORY_SYNC`.
|
||||
|
||||
Also test producer once, streaming bounds, formula mitigation, target collision no overwrite,
|
||||
root-identity change indeterminate, and manifest/reference locator non-disclosure. The stored
|
||||
`internalLocator` is the generated filename only; its data shard is derived from the first two
|
||||
hex characters of `fileId`.
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*FileserverBindingCompilerTest' \
|
||||
--tests '*LocalPersistentControlPlaneTest' \
|
||||
--tests '*LocalPersistentPublicationProviderTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation/test failure because the compiled identity, typed compatibility lookup,
|
||||
contextual fault seam, payload operations, and provider do not exist.
|
||||
|
||||
- [x] **Step 3: Implement prerequisites and minimal R2 publication**
|
||||
|
||||
Compile one restart-stable destination identity without adding a config key:
|
||||
|
||||
```text
|
||||
effectivePolicyDigest = SHA-256(length-prefixed canonical descriptor fields)
|
||||
routeToken = "r" + first 31 lowercase hex of effectivePolicyDigest
|
||||
```
|
||||
|
||||
The canonical descriptor includes destination/provider IDs, limits, required guarantees, and the
|
||||
format/encoder revision. The schema and format policy use the same length-prefixed digest helper.
|
||||
Reject route-token collisions across the compiled startup allowlist. Keep digest/token derivation
|
||||
on the production SHA-256 path only. Exercise the otherwise impractical collision branch through
|
||||
the same package-private pure route-registry check used by production, using two different test
|
||||
digests whose first 31 hex characters collide; expose no digest/token runtime override.
|
||||
|
||||
Extend `LocalPersistentControlPlane` with one secure relative typed operation lookup. It returns
|
||||
schema-v1 only through strict UTF-8 plus canonical v1 re-encode byte equality and never writes v1;
|
||||
schema-v2 remains the only write format. Enrich its package-private fault callback with record kind,
|
||||
identity, operation state/revision, and force boundary so Task 8 can stop at an exact record force.
|
||||
|
||||
The provider:
|
||||
|
||||
```text
|
||||
validates destination and request before producer invocation
|
||||
acquires operation lock
|
||||
loads operation by direct ID
|
||||
allocates fileId/name before WRITING
|
||||
streams with existing StreamingCsvEncoder
|
||||
forces stage and stores SEALED
|
||||
exclusive hard-links data and forces data directory
|
||||
publishes private manifest
|
||||
publishes reference index
|
||||
stores terminal receipt snapshot
|
||||
returns only after terminal journal parent force/read-back
|
||||
```
|
||||
|
||||
`LocalPersistentPayloadOperations` owns restrictive staging/data shard creation, secure relative
|
||||
stage create/write/force, stable no-follow artifact inspection/digest, exact stage deletion,
|
||||
exclusive no-replace hard-link, standalone recovery-time data-shard directory force, and
|
||||
attested-root-relative R1 artifact inspection. Absolute hard-link/directory-force calls are allowed
|
||||
only inside the attested private-owner boundary with file/root/directory identity checks. An
|
||||
existing matching data artifact discovered from `SEALED` must have its shard directory forced
|
||||
again before the journal may advance; it is never republished through a collision path. A
|
||||
root-level R1 artifact is restored only after bounded SDS-relative no-follow inspection matches the
|
||||
terminal R1 journal.
|
||||
|
||||
Before and after the hard-link commit, compare root real path, file key, FileStore, and sentinel
|
||||
digest to `LocalPersistentRootEvidence`.
|
||||
|
||||
- [x] **Step 4: Write failing recovery matrix tests**
|
||||
|
||||
For every non-terminal state construct matching/missing artifacts and retry with a producer that
|
||||
throws if called. Expected:
|
||||
|
||||
```text
|
||||
SEALED + stage -> resume data publish
|
||||
SEALED + matching data -> resume manifest
|
||||
DATA_PUBLISHED -> resume manifest
|
||||
MANIFEST_PUBLISHED -> resume reference
|
||||
REFERENCE_PUBLISHED -> finish terminal journal
|
||||
PUBLISHED + all matching -> restore exact receipt
|
||||
non-terminal data/manifest/reference mismatch -> QUARANTINED / integrity failure
|
||||
PUBLISHED artifact/metadata/receipt mismatch -> preserve all terminal evidence; integrity / indeterminate
|
||||
required artifact missing -> fail-closed indeterminate / quarantine, never success
|
||||
fingerprint mismatch -> CONFLICT
|
||||
root identity mismatch -> PUBLISH_INDETERMINATE
|
||||
WRITING producer/stage failure -> exact cleanup + unsealed QUARANTINED
|
||||
retry with existing WRITING -> producer is not invoked; indeterminate / quarantine
|
||||
retry of unsealed QUARANTINED -> producer is not invoked
|
||||
```
|
||||
|
||||
`LocalPersistentRecoveryVerifier` must cross-check the operation, incoming request, stable data
|
||||
digest, canonical manifest/reference digests, all locators/counts/timestamps, and guarantees.
|
||||
Because operation schema v2 does not carry a standalone format-policy snapshot, it must require an
|
||||
exact current compiled effective-policy revision/digest match before using the current
|
||||
format-policy digest; it must fail closed instead of guessing across an encoder-policy change.
|
||||
Current configured byte/row limits apply to a new attempt. Recovery inspection is bounded by the
|
||||
already frozen operation byte size (with overflow-safe equality), so a later lower configuration
|
||||
limit does not reinterpret a sealed artifact. If both stage and data exist, their stable file keys
|
||||
must match before exact stage deletion; equal bytes alone are insufficient.
|
||||
Restore a terminal receipt only when it equals the full receipt reconstructed from the verified
|
||||
manifest/reference; checking only operation ID/count/SHA is insufficient. Reuse a verified
|
||||
immutable manifest/reference `publishedAt` after a crash instead of generating a conflicting time.
|
||||
`QUARANTINED` journal transitions are limited to non-terminal operations. A mismatch discovered
|
||||
from `PUBLISHED` must not replace the terminal journal or delete/overwrite data, manifest, or
|
||||
reference records; return typed integrity/indeterminate and preserve all terminal evidence. A
|
||||
separate immutable quarantine incident record is outside this increment.
|
||||
|
||||
- [x] **Step 5: Write failing R1 compatibility tests**
|
||||
|
||||
Pre-provision an existing R1 root so it passes every R2 root attestation condition, then configure
|
||||
that same root as the R2 destination. Place a valid journal schema-v1 terminal record at the shared
|
||||
hashed operation path and a matching root-level R1 artifact.
|
||||
The R2 reader may restore its original `PROCESS_LOCAL_SYNC` receipt, but must not create an R2
|
||||
manifest/reference, change its guarantee, or rewrite the record as schema v2. Newer/corrupt R1
|
||||
records remain indeterminate. Also prove malformed UTF-8 and a decodable but non-canonical v1
|
||||
encoding fail, and that simultaneous R1/R2 bean activation is not required for migration.
|
||||
|
||||
- [x] **Step 6: Verify compatibility RED, then implement read-only compatibility**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentPublicationRecoveryTest' --console=plain
|
||||
```
|
||||
|
||||
Expected before implementation: the R1 restoration assertion fails. Reuse the existing schema-v1
|
||||
model/codec behind an added strict UTF-8 and canonical re-encode equality guard, only as a read-only
|
||||
compatibility reader; do not add schema-v1 write paths or an unconfigured second root.
|
||||
|
||||
- [x] **Step 7: Verify recovery RED, then implement recovery**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentPublicationRecoveryTest' --console=plain
|
||||
```
|
||||
|
||||
Expected before recovery implementation: failures at each resume assertion. Implement only the
|
||||
matrix and verifier rules above. When producer or staging fails after `J_WRITING`, preserve the
|
||||
original exception, attach cleanup/control failures as suppressed, exact-delete the partial stage,
|
||||
and store unsealed `QUARANTINED` evidence so retry cannot replay the producer. A retry that finds
|
||||
`WRITING` after a process crash also must not invoke the producer. Then rerun. Expected: PASS.
|
||||
|
||||
- [x] **Step 8: Verify provider GREEN**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentPublicationProviderTest' \
|
||||
--tests '*LocalPersistentPublicationRecoveryTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Add one routing port bean and reject ambiguous R1/R2 activation
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/RoutingFilePublicationAdapter.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverR2Config.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileserverActivationValidator.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FileserverR2ConfigTest.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileExportConfig.java`
|
||||
- Rename:
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileExportProperties.java`
|
||||
to
|
||||
`src/adapter/outbound/fileserver/src/main/java/dev/caskeleton/adapter/outbound/fileserver/FileExportSettings.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FilePublicationConfigTest.java`
|
||||
- Modify:
|
||||
`src/app-bootstrap/build.gradle`
|
||||
- Modify:
|
||||
`src/config/architecture/modules.json`
|
||||
- Modify:
|
||||
`src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/OptionalAdapterBeanGatingTest.java`
|
||||
- Modify:
|
||||
`src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/architecture/DisabledAdapterArchitectureTest.java`
|
||||
|
||||
- [x] **Step 1: Write failing composition/routing tests**
|
||||
|
||||
Prove:
|
||||
|
||||
```java
|
||||
@Test
|
||||
void disabledR2CreatesNoPortOrFilesystemSideEffect() {}
|
||||
|
||||
@Test
|
||||
void enabledR2CreatesExactlyOneRoutingPortForExplicitBindings() {}
|
||||
|
||||
@Test
|
||||
void requestForUnknownDestinationFailsBeforeProducerInvocation() {}
|
||||
|
||||
@Test
|
||||
void enablingLegacyR1AndR2TogetherFailsStartup() {}
|
||||
|
||||
@Test
|
||||
void configuredButUnimplementedSharedOrSftpProviderFailsStartup() {}
|
||||
```
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*FileserverR2ConfigTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: compilation/test failure because R2 composition does not exist.
|
||||
|
||||
Execution note: the production composition skeleton had already been introduced before the
|
||||
delegated test task returned, so a standalone RED Gradle run was no longer reproducible without
|
||||
reverting work. The tests still exposed the missing method-level conditional gate through the
|
||||
bootstrap architecture check; that failure was observed and fixed before GREEN.
|
||||
|
||||
- [x] **Step 3: Implement exact routing composition**
|
||||
|
||||
`RoutingFilePublicationAdapter` contains an immutable
|
||||
`Map<FileDestinationId, FilePublicationProvider>` and delegates only after exact lookup.
|
||||
`FileserverR2Config`:
|
||||
|
||||
- is conditional on `app.fileserver.enabled=true`;
|
||||
- enables `FileserverR2Settings`;
|
||||
- compiles and attests every configured binding at startup;
|
||||
- creates one provider instance per provider ID;
|
||||
- creates exactly one `FilePublicationPort`;
|
||||
- rejects `ca-skeleton.fileserver.enabled=true` in the same environment before either R1 root
|
||||
creation or R2 attestation, independently of Spring bean creation order;
|
||||
- rejects different provider IDs that resolve to the same normalized root;
|
||||
- never creates directories/connections when disabled.
|
||||
|
||||
The same package-private activation validator runs first in both R1 bean factories and the R2
|
||||
routing factory; conditional precedence is not an acceptable substitute for an ambiguity failure.
|
||||
Use strict configuration-properties binding (`ignoreUnknownFields = false`). Wire the fileserver
|
||||
leaf into `app-bootstrap` through the architecture registry and Gradle dependency in this task so
|
||||
the runtime composition is real, while keeping all local provider/control types private to the
|
||||
leaf. Rename the legacy configuration-properties type to the repository-required `*Settings`
|
||||
suffix before exposing this leaf to bootstrap naming checks.
|
||||
|
||||
- [x] **Step 4: Verify GREEN**
|
||||
|
||||
Run the command from Step 2. Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Add process-crash qualification, docs, and full gates
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/FileserverCrashScenarioMain.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/fileserver/src/test/java/dev/caskeleton/adapter/outbound/fileserver/LocalPersistentCrashRecoveryTest.java`
|
||||
- Modify: `src/adapter/outbound/fileserver/README.md`
|
||||
- Modify: `src/adapter/outbound/fileserver/CLAUDE.md`
|
||||
- Modify:
|
||||
`docs/superpowers/specs/2026-07-26-fileserver-production-capability-design.md`
|
||||
- Modify:
|
||||
`docs/superpowers/specs/2026-07-28-fileserver-r2-control-plane-provider-selection-design.md`
|
||||
- Modify:
|
||||
`docs/superpowers/plans/2026-07-28-fileserver-r2-control-plane-provider-selection.md`
|
||||
- Modify: `docs/registries/env-keys.yaml`
|
||||
|
||||
- [x] **Step 1: Write the failing forked-process crash test**
|
||||
|
||||
Launch a new JVM with the test runtime classpath. The helper receives a fault point and calls
|
||||
`Runtime.getRuntime().halt(91)` immediately after that point. Cover:
|
||||
|
||||
```text
|
||||
J_WRITING
|
||||
STAGE_FORCED
|
||||
J_SEALED
|
||||
DATA_LINKED
|
||||
DATA_DIRECTORY_FORCED
|
||||
MANIFEST_FORCED
|
||||
MANIFEST_DIRECTORY_FORCED
|
||||
REFERENCE_FORCED
|
||||
REFERENCE_DIRECTORY_FORCED
|
||||
TERMINAL_JOURNAL_FORCED
|
||||
TERMINAL_JOURNAL_DIRECTORY_FORCED
|
||||
```
|
||||
|
||||
Restart in a second JVM/process and assert exact receipt restoration or a documented typed
|
||||
indeterminate/quarantine outcome, never producer replay or partial final bytes.
|
||||
|
||||
Also run a forked cross-process operation-lock proof using the same attested root and operation ID:
|
||||
process A acquires and reports the OS lock, process B uses a bounded non-blocking/timed attempt and
|
||||
must not enter the critical section while A is alive, then must acquire after A releases or is
|
||||
forcibly terminated. This proof must exercise the OS `FileLock`; the same-JVM stripe test is not a
|
||||
substitute and every wait requires a timeout.
|
||||
|
||||
- [x] **Step 2: Verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:fileserver:test \
|
||||
--tests '*LocalPersistentCrashRecoveryTest' --console=plain
|
||||
```
|
||||
|
||||
Expected: failure until every fault point is injectable and recoverable.
|
||||
|
||||
Execution note: the contextual control-plane and payload fault seams introduced in Task 6 already
|
||||
covered all eleven boundaries. The first complete forked-process run therefore passed without a
|
||||
new production hook; no implementation was reverted merely to manufacture a RED result.
|
||||
|
||||
- [x] **Step 3: Implement only missing fault hooks/recovery transitions**
|
||||
|
||||
Fault hooks remain package-private test collaborators. No runtime setting or production bean may
|
||||
allow arbitrary process termination.
|
||||
|
||||
- [x] **Step 4: Verify focused and module checks**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :application-core:check :adapter:outbound:fileserver:check --console=plain
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [x] **Step 5: Update readiness documentation**
|
||||
|
||||
Record:
|
||||
|
||||
- provider-neutral control plane and exact selector implemented;
|
||||
- `local-persistent` is the only qualified R2 provider;
|
||||
- `FILE_AND_DIRECTORY_SYNC` does not claim physical device power-loss protection;
|
||||
- `shared-mounted`, SFTP, reaper/retention/quota/observability remain unimplemented;
|
||||
- R1 compatibility artifacts are never auto-promoted.
|
||||
|
||||
Register the exact local provider environment keys from the design (`ROOT`, expected FileStore
|
||||
name/type, sentinel digest, expected owner) with restart-only policy and conditional
|
||||
`app.fileserver.enabled` validation. Do not add SFTP/NFS keys before those providers exist.
|
||||
|
||||
- [x] **Step 6: Run full repository gates**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew check --console=plain
|
||||
./gradlew \
|
||||
:application-core:verifyDependencyLocks \
|
||||
:adapter:outbound:fileserver:verifyDependencyLocks \
|
||||
:app-bootstrap:verifyDependencyLocks \
|
||||
:sample-portfolio:verifyDependencyLocks \
|
||||
verifyCleanArchitectureDependencies \
|
||||
verifyPublicPathSnapshot \
|
||||
verifyEnvKeys --console=plain
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Expected: all commands PASS.
|
||||
|
||||
- [x] **Step 7: Request final independent review**
|
||||
|
||||
Review against:
|
||||
|
||||
- the R2 design spec;
|
||||
- HARD-STOP rules;
|
||||
- provider fallback/activation ambiguity;
|
||||
- path/symlink/mount identity;
|
||||
- crash ordering and recovery;
|
||||
- receipt guarantee truthfulness;
|
||||
- R1 compatibility and no unrelated adapter dependency.
|
||||
|
||||
Fix every Critical/Important issue and rerun the affected focused test plus full gates.
|
||||
@@ -0,0 +1,120 @@
|
||||
# HTTP Client Canonical Zero-Binding Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
||||
> `superpowers:subagent-driven-development` or `superpowers:executing-plans`. Repository policy
|
||||
> overrides the skill's commit steps: do not stage, commit, amend, or push.
|
||||
|
||||
**Goal:** Make HTTP client activation an explicit canonical composition decision and prove that the
|
||||
default zero-binding state creates no client, executor, shutdown guard, retry/circuit-breaker
|
||||
registry, or transport resource.
|
||||
|
||||
**Architecture:** `adapter:outbound:httpclient` owns strict canonical configuration, immutable
|
||||
binding/provider/catalog/readiness registries, and a pure activation resolver. `app-bootstrap` owns
|
||||
the composition root that binds canonical properties and publishes an inert capability descriptor.
|
||||
The existing JDK `OutboundHttpClient` remains an explicitly constructed R1 migration facade; its
|
||||
legacy settings and infrastructure configuration must no longer be discovered automatically.
|
||||
|
||||
**Scope boundary:** This increment does not add Apache HC5, a provider factory, a real semantic
|
||||
upstream binding, hard wire cancellation, TLS/DNS/proxy/auth, or an R2 readiness claim. Every current
|
||||
ACTIVE selection must fail closed because the only derived readiness card remains
|
||||
`NOT_IMPLEMENTED`.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add strict canonical selection and provider binding models
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/activation/HttpClientExpectedState.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/activation/HttpClientCanonicalConfiguration.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/activation/HttpClientCanonicalConfigurationBinder.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/activation/HttpClientCanonicalConfigurationBinderTest.java`
|
||||
|
||||
- [x] Write RED tests for the canonical YAML shape under
|
||||
`ca-skeleton.capabilities.http-client` and `ca-skeleton.providers.http-client`.
|
||||
- [x] Reject unknown fields, malformed IDs, unknown expected state, and any legacy input entering
|
||||
canonical composition, including the DISABLED state.
|
||||
- [x] Preserve `OutboundHttpSettings` constructors as migration API, but remove its global
|
||||
`@ConfigurationPropertiesScan` participation.
|
||||
- [x] Keep provider definitions inert data; configuration alone must not create a transport.
|
||||
|
||||
### Task 2: Add catalog/readiness registries and pure fail-closed activation resolution
|
||||
|
||||
**Files:**
|
||||
- Create:
|
||||
`src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/activation/HttpOperationCatalogRegistry.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/activation/HttpClientReadinessCardRegistry.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/activation/ResolvedHttpClientCapability.java`
|
||||
- Create:
|
||||
`src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/activation/HttpClientActivationResolver.java`
|
||||
- Test:
|
||||
`src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/activation/HttpClientActivationResolverTest.java`
|
||||
|
||||
- [x] Prove `DISABLED + bindings 0 + provider definitions 0` resolves to
|
||||
`DISABLED_VERIFIED`, selected binding/card count 0.
|
||||
- [x] Reject `DISABLED` with bindings or provider resources.
|
||||
- [x] Reject `ACTIVE` with zero bindings.
|
||||
- [x] For every binding, require an exact provider, provider destination, and registered operation
|
||||
catalog for the same destination.
|
||||
- [x] Derive the `httpclient-static-buffered` card from each current buffered classic profile.
|
||||
- [x] Mark that card `NOT_IMPLEMENTED`; reject ACTIVE before any provider resource/factory exists.
|
||||
|
||||
### Task 3: Move HTTP Spring activation to the composition root
|
||||
|
||||
**Files:**
|
||||
- Modify:
|
||||
`src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/OutboundHttpClientConfig.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/OutboundHttpSettings.java`
|
||||
- Modify:
|
||||
`src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/resilience/OutboundHttpResilienceConfig.java`
|
||||
- Create:
|
||||
`src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/httpclient/HttpClientCompositionConfig.java`
|
||||
- Modify: `src/app-bootstrap/src/main/resources/application.yml`
|
||||
- Modify:
|
||||
`src/app-bootstrap/src/test/java/dev/caskeleton/adapter/outbound/OptionalAdapterBeanGatingTest.java`
|
||||
- Test:
|
||||
`src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/httpclient/HttpClientCompositionConfigTest.java`
|
||||
|
||||
- [x] Detach legacy HTTP infrastructure from component/configuration-properties scanning while
|
||||
preserving direct constructors/factory methods used by forks and existing unit tests.
|
||||
- [x] Register only canonical configuration, immutable registries, resolver, and inert descriptor
|
||||
in the composition root.
|
||||
- [x] Default application YAML to canonical `expected-state: DISABLED`, empty bindings, and empty
|
||||
provider definitions; keep legacy migration keys out of both main and test application YAML.
|
||||
- [x] Assert zero `OutboundHttpClient`, `RestClient`, `OutboundCallExecutor`,
|
||||
`OutboundHttpShutdownGuard`, `OutboundHttpResilience`, `RetryRegistry`, and
|
||||
`CircuitBreakerRegistry` beans/resources in the default context.
|
||||
- [x] Assert contradictory/ACTIVE configurations fail startup before resource construction.
|
||||
- [x] Load the real `application.yml` in composition tests and prove ACTIVE reaches the
|
||||
`NOT_IMPLEMENTED` readiness card rather than a legacy conflict.
|
||||
|
||||
### Task 4: Document exact readiness and verify
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/adapter/outbound/httpclient/README.md`
|
||||
- Modify: `src/adapter/outbound/httpclient/CLAUDE.md`
|
||||
- Modify: `docs/superpowers/specs/2026-07-27-httpclient-production-capability-design.md`
|
||||
- Modify:
|
||||
`docs/superpowers/plans/2026-07-28-httpclient-production-capability-foundation.md`
|
||||
|
||||
- [x] Mark canonical zero-binding as implemented without marking HTTP R2 complete.
|
||||
- [x] Keep HC5/provider resources/security/real-network qualification explicitly unimplemented.
|
||||
- [x] Run focused tests:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:httpclient:check --rerun-tasks --console=plain
|
||||
./gradlew :app-bootstrap:check --rerun-tasks --console=plain
|
||||
./gradlew :sample-portfolio:test --rerun-tasks --console=plain
|
||||
./gradlew verifyCleanArchitectureDependencies verifyConfigurationPropertiesProcessor \
|
||||
verifyEnvKeys verifyPublicPathSnapshot --console=plain
|
||||
```
|
||||
|
||||
Do not edit unrelated notification, messaging, object-storage, JPA, MongoDB, GraphQL, gRPC, web, or
|
||||
WebSocket files.
|
||||
@@ -0,0 +1,80 @@
|
||||
# HTTP Client Production Capability Foundation Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development
|
||||
> (recommended) or superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Establish the framework-free call-budget and typed operation/target foundation, then close
|
||||
two proven safety defects in the legacy JDK provider without claiming Apache HC5, hard total
|
||||
deadline, egress security, or R2 readiness.
|
||||
|
||||
**Architecture:** `application-core` owns only a monotonic `CallBudget`. Product forks continue to
|
||||
own feature-specific semantic ports. `adapter:outbound:httpclient` owns destination/operation IDs,
|
||||
immutable operation descriptors, relative target construction, status/retry/body semantics, and
|
||||
legacy provider fixes. The generic `OutboundHttpClient` remains a migration facade.
|
||||
|
||||
**Scope boundary:** This applies Phase 0 and a bounded Phase 1 foundation. Canonical zero-binding
|
||||
composition and active logical cancellation were implemented by later tracked plans. Exact
|
||||
readiness tuple registry, Apache HC5 pool, TLS/DNS/proxy, auth, codec, and real-network
|
||||
qualification remain unimplemented.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add a framework-free monotonic call budget
|
||||
|
||||
**Files:**
|
||||
- Create: `src/application-core/src/main/java/dev/caskeleton/application/outbound/CallBudget.java`
|
||||
- Test: `src/application-core/src/test/java/dev/caskeleton/application/outbound/CallBudgetTest.java`
|
||||
|
||||
- [x] Write RED tests for expiry, remaining time, finite bounds, and parent/child intersection.
|
||||
- [x] Implement without Spring, wall-clock timestamps, scheduler, or HTTP types.
|
||||
- [x] Verify GREEN.
|
||||
|
||||
### Task 2: Add typed operation catalog and safe target construction
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/adapter/outbound/httpclient/build.gradle`
|
||||
- Create: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/operation/HttpDestinationId.java`
|
||||
- Create: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/operation/HttpOperationId.java`
|
||||
- Create: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/operation/HttpOperationDescriptor.java`
|
||||
- Create: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/operation/HttpOperationCatalog.java`
|
||||
- Create: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/target/FixedHttpDestination.java`
|
||||
- Create: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/target/HttpTargetBuilder.java`
|
||||
- Test: `src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/operation/HttpOperationCatalogTest.java`
|
||||
- Test: `src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/target/HttpTargetBuilderTest.java`
|
||||
|
||||
- [x] Write RED tests for ID/uniqueness/cross-field operation invariants.
|
||||
- [x] Write RED tests rejecting absolute, scheme-relative, traversal, user-info, query/fragment, and
|
||||
multi-segment variables.
|
||||
- [x] Implement closed immutable descriptors and one-pass path-segment encoding.
|
||||
- [x] Verify GREEN.
|
||||
|
||||
### Task 3: Correct characterized legacy provider safety defects
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/OutboundHttpClient.java`
|
||||
- Modify: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/OutboundHttpRestClientFactory.java`
|
||||
- Test: `src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/OutboundHttpClientSafetyRegressionTest.java`
|
||||
|
||||
- [x] Reproduce streaming 5xx body delivery and logical-call-only circuit-breaker counting.
|
||||
- [x] Make streaming validate status before exposing the body and discard error bodies.
|
||||
- [x] Put circuit breaker around each physical attempt and retry around the attempt loop.
|
||||
- [x] Set JDK redirects to `NEVER` explicitly and validate legacy base URI/relative request targets.
|
||||
- [x] Verify focused regressions and the full legacy test suite.
|
||||
|
||||
### Task 4: Record exact readiness and verify
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/adapter/outbound/httpclient/README.md`
|
||||
- Modify: `src/adapter/outbound/httpclient/CLAUDE.md`
|
||||
- Modify: `docs/superpowers/specs/2026-07-27-httpclient-production-capability-design.md`
|
||||
|
||||
- [x] Mark the implemented foundation and fixed legacy defects.
|
||||
- [x] Track later total-deadline and canonical-zero-binding increments separately while keeping
|
||||
Apache pool, fixed egress, TLS/auth, bounded decoded streaming, and R2 cards unimplemented.
|
||||
- [x] Run:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :application-core:check :adapter:outbound:httpclient:check --console=plain
|
||||
./gradlew verifyCleanArchitectureDependencies --console=plain
|
||||
```
|
||||
@@ -0,0 +1,59 @@
|
||||
# HTTP Client Total Deadline Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this
|
||||
> plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. Repository policy is
|
||||
> human-only, so no step stages or commits changes.
|
||||
|
||||
**Goal:** Enforce `CallBudget` across the legacy HTTP logical call, including retry wait and blocking
|
||||
I/O, and cancel the executing task when the absolute monotonic deadline wins.
|
||||
|
||||
**Architecture:** Preserve the current migration facade but inject a bounded executor owned by each
|
||||
client. Every call intersects the caller budget with the configured maximum, passes the same
|
||||
absolute deadline to retry policy, waits through `Future.get(remaining)`, and cancels on timeout or
|
||||
shutdown. This is R1 cancellation evidence, not Apache pool or hard-wire-cancellation R2 evidence.
|
||||
|
||||
**Tech Stack:** Java 21 virtual-thread executor, Spring RestClient/JDK HttpClient, Resilience4j,
|
||||
JUnit loopback HTTP server.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add deadline execution and explicit timeout vocabulary
|
||||
|
||||
**Files:**
|
||||
- Create: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/OutboundCallDeadlineExceededException.java`
|
||||
- Create: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/OutboundCallExecutor.java`
|
||||
- Test: `src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/OutboundCallExecutorTest.java`
|
||||
|
||||
- [x] Write failing tests proving an expired budget does not start work, a running task is
|
||||
interrupted on expiry, and completion wins before the deadline.
|
||||
- [x] Confirm RED.
|
||||
- [x] Implement absolute monotonic remaining-time calculation, `Future.get`, cancellation and
|
||||
exact exception mapping.
|
||||
- [x] Confirm GREEN.
|
||||
|
||||
### Task 2: Connect the budget to buffered and streaming calls
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/OutboundHttpClient.java`
|
||||
- Modify: `src/adapter/outbound/httpclient/src/main/java/dev/caskeleton/adapter/outbound/httpclient/OutboundRetryPolicy.java`
|
||||
- Test: `src/adapter/outbound/httpclient/src/test/java/dev/caskeleton/adapter/outbound/httpclient/OutboundHttpClientDeadlineTest.java`
|
||||
|
||||
- [x] Write a failing loopback test where response delay exceeds the budget and confirm bounded
|
||||
return; record that JDK-provider server-side hard close is not proven by this lane.
|
||||
- [x] Write a failing test proving a shorter caller budget wins and retry cannot start after expiry.
|
||||
- [x] Confirm RED.
|
||||
- [x] Add overloads accepting `CallBudget`; existing methods create a configured maximum budget.
|
||||
Intersect budgets once and use the same deadline for retry and blocking execution.
|
||||
- [x] Confirm GREEN and run the complete HTTP leaf tests.
|
||||
|
||||
### Task 3: Record provider limits and verify
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/adapter/outbound/httpclient/README.md`
|
||||
- Modify: `src/adapter/outbound/httpclient/CLAUDE.md`
|
||||
- Modify: `docs/superpowers/specs/2026-07-27-httpclient-production-capability-design.md`
|
||||
|
||||
- [x] Record active logical-call deadline/cancellation as implemented.
|
||||
- [x] Keep explicit pool lease, Apache exact provider, DNS rebinding, TLS/auth/proxy and R2 hard
|
||||
cancellation evidence unimplemented.
|
||||
- [x] Run the HTTP leaf check and architecture/public-path gates.
|
||||
@@ -0,0 +1,484 @@
|
||||
# JPA/PostgreSQL Production Capability Implementation Plan
|
||||
|
||||
> 상태: Phase 0~3 기반과 Phase 4의 idempotency/outbox polling/inbox 후보 구현 및 전체
|
||||
> local/real PostgreSQL 검증을 마쳤다. 검증을 통과한 항목은 `implemented-candidate`이며
|
||||
> immutable 운영 evidence가 없는 항목을 R2로 승격하지 않는다. Phase 5~7은 외부 topology와
|
||||
> policy prerequisite가 없어 `not-implemented`를 유지한다.
|
||||
|
||||
- 작성일: 2026-07-28
|
||||
- 구현 branch: `codex/jpa-production-capability`
|
||||
- worktree:
|
||||
`/home/donghyeon/workspace/clean-architecture-backend-template-jpa`
|
||||
- 시작 revision: `b3add0162df8d4a0a11e749e514901defe0a62a3`
|
||||
- 설계 원본:
|
||||
`/home/donghyeon/workspace/clean-architecture-backend-template/docs/superpowers/specs/2026-07-28-jpa-production-capability-design.md`
|
||||
- 설계 SHA-256:
|
||||
`c02eaef2a193a6ca66f4814087cc4d6bce723509aec251f40ea7b029046fd234`
|
||||
|
||||
설계 문서는 `main` worktree의 untracked 사용자 변경이므로 stage/commit/copy하지 않는다. 구현
|
||||
중에는 위 절대 경로와 hash를 승인된 정본 snapshot으로 사용한다. 정본이 바뀌면 hash drift를
|
||||
먼저 보고하고 해당 task의 설계를 재검토한다.
|
||||
|
||||
## 1. 목표와 완료 경계
|
||||
|
||||
목표는 JPA/PostgreSQL leaf의 각 capability를 독립적으로 구현·검증하는 것이다.
|
||||
|
||||
```text
|
||||
truthful baseline
|
||||
-> transaction/failure/deadline
|
||||
-> entity/query discipline
|
||||
-> migration/lifecycle/security
|
||||
-> owner-safe reliability
|
||||
-> optional replica
|
||||
-> optional tenant/coordination
|
||||
-> R3 rehearsal
|
||||
```
|
||||
|
||||
한 phase의 unit test 통과를 전체 JPA R2로 확대하지 않는다. card가 R2가 되려면 설계 §31.3의
|
||||
prerequisite, real PostgreSQL task, zero-skip sentinel과 immutable evidence manifest를 모두
|
||||
충족해야 한다.
|
||||
|
||||
현재 구현 작업의 완료 경계는 다음과 같다.
|
||||
|
||||
1. 독립 worktree와 계획이 존재한다.
|
||||
2. Phase 0의 SQLState, Duration, OSIV/DDL, machine-readable readiness baseline이
|
||||
fail-closed한다.
|
||||
3. named transaction policy, absolute deadline, PostgreSQL local timeout, phase-aware outcome,
|
||||
bounded serialization/deadlock retry가 구현된다.
|
||||
4. PostgreSQL 16 real test source set에서 lifecycle/security/migration/transaction/
|
||||
aggregate/query가 무-skip로 실행된다.
|
||||
5. owner-safe idempotency V2, immutable outbox storage V2, polling delivery V2, same-store
|
||||
inbox가 독립 migration stream과 real PostgreSQL concurrency test를 가진다.
|
||||
6. 외부 CDC, replica, tenant/RLS, R3는 토폴로지/evidence 없이 선택하거나 R2로 광고하지 않는다.
|
||||
7. 전체 test/check와 Wiki capture 결과를 기록한다.
|
||||
|
||||
## 2. 공통 구현 규칙
|
||||
|
||||
- `src/config/architecture/modules.json`의 19개 leaf와 edge를 유지한다.
|
||||
- `domain-core`에는 Spring/JPA/JDBC/PostgreSQL type을 추가하지 않는다.
|
||||
- application contract에는 framework-neutral Java type만 둔다.
|
||||
- transaction boundary는 application use case가 `TransactionPort`로 소유한다.
|
||||
- controller/repository/mapper/configuration에 business policy를 두지 않는다.
|
||||
- PostgreSQL 전용 code/import는 persistence-jpa leaf의 `.postgresql` package에 둔다.
|
||||
- 동작 변경은 failing test를 먼저 확인한 뒤 최소 production code를 작성한다.
|
||||
- applied Flyway V1/V3/V4/V5는 수정하지 않는다.
|
||||
- agent는 stage/commit/amend/push하지 않는다.
|
||||
- 다른 worktree의 dirty/untracked 변경을 복사하거나 되돌리지 않는다.
|
||||
|
||||
worktree 생성 직후 `src/gradlew.bat`는 CRLF blob과 checkout/attribute line-ending
|
||||
normalization 차이 때문에 dirty로 표시된다. 비교 결과 의미 있는 텍스트 변경은 없지만 raw
|
||||
worktree hash와 HEAD blob hash는 EOL 표현 때문에 다르다. targeted restore로도 사라지지 않는
|
||||
known baseline drift이므로 구현 diff와 완료 판정에서 분리하고 stage하지 않는다.
|
||||
|
||||
## 3. Phase 0 — Truthful baseline과 contract freeze
|
||||
|
||||
### Task 0.1 SQLState mapping duplicate fail-fast
|
||||
|
||||
상태: 2026-07-28 구현 및 focused/architecture 검증 완료.
|
||||
|
||||
소유 leaf: `adapter-outbound-persistence-jpa`
|
||||
|
||||
파일:
|
||||
|
||||
- 수정:
|
||||
`src/adapter/outbound/persistence-jpa/src/test/java/dev/caskeleton/adapter/outbound/persistence/failure/PersistenceExceptionTranslatorTest.java`
|
||||
- 수정:
|
||||
`src/adapter/outbound/persistence-jpa/src/main/java/dev/caskeleton/adapter/outbound/persistence/failure/PersistenceExceptionTranslator.java`
|
||||
- 필요 시 수정:
|
||||
`src/adapter/outbound/persistence-jpa/README.md`
|
||||
|
||||
TDD:
|
||||
|
||||
1. 서로 다른 두 `SqlStateErrorMapping`이 같은 exact SQLState에 같은
|
||||
`OperationalError`를 등록해도 constructor가 실패하는 test를 작성한다.
|
||||
2. 같은 SQLState에 서로 다른 `OperationalError`를 등록하면 실패하는 test를 작성한다.
|
||||
3. error message가 raw SQL, credential, endpoint 없이 duplicate SQLState와 mapping
|
||||
contributor type을 식별하는지 검증한다.
|
||||
4. focused test를 실행해 RED를 확인한다.
|
||||
5. `putAll`을 explicit merge로 바꾸고 first/duplicate provenance를 보존한다.
|
||||
6. null mapping/map/key/value와 `08*` pseudo-entry를 fail-fast할지 현재 SPI 계약에 맞춰
|
||||
validation test를 추가한다. 이 세부 계약은 범위를 키우지 않고 constructor invariant로
|
||||
한정한다.
|
||||
7. focused test를 GREEN으로 만든다.
|
||||
|
||||
검증:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :adapter:outbound:persistence-jpa:test \
|
||||
--tests 'dev.caskeleton.adapter.outbound.persistence.failure.PersistenceExceptionTranslatorTest' \
|
||||
--console=plain
|
||||
./gradlew :adapter:outbound:persistence-jpa:test --console=plain
|
||||
./gradlew verifyCleanArchitectureDependencies --console=plain
|
||||
```
|
||||
|
||||
### Task 0.2 Duration/OSIV/DDL production safety
|
||||
|
||||
상태: 2026-07-28 strict Duration와 prod DDL guard 구현 완료. OSIV guard는 기존 구현을
|
||||
재사용하고 함께 회귀 검증했다.
|
||||
|
||||
소유 leaf:
|
||||
|
||||
- `app-bootstrap`: runtime settings/startup validator
|
||||
- `adapter-outbound-persistence-jpa`: typed provider settings가 필요할 때만
|
||||
|
||||
선행 조사 파일:
|
||||
|
||||
- `src/app-bootstrap/src/main/java/dev/caskeleton/bootstrap/runtime/HikariPoolConstraintValidator.java`
|
||||
- `src/app-bootstrap/src/test/java/dev/caskeleton/bootstrap/runtime/HikariPoolConstraintValidatorTest.java`
|
||||
- `src/app-bootstrap/src/main/resources/application.yml`
|
||||
- `src/app-bootstrap/CLAUDE.md`
|
||||
|
||||
TDD:
|
||||
|
||||
1. `5s`, `PT5S`, millisecond number의 canonical/legacy 허용 matrix를 test로 고정한다.
|
||||
2. invalid/unknown Duration을 skip하지 않고 startup failure로 만드는 RED를 확인한다.
|
||||
3. `spring.jpa.open-in-view=true`를 거절한다.
|
||||
4. production profile의 `ddl-auto=update|create|create-drop`을 거절한다.
|
||||
5. local/sample compatibility를 별도 test로 유지한다.
|
||||
|
||||
검증:
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :app-bootstrap:test \
|
||||
--tests 'dev.caskeleton.bootstrap.runtime.HikariPoolConstraintValidatorTest' \
|
||||
--console=plain
|
||||
./gradlew :app-bootstrap:test --tests '*CleanArchitectureTest' --console=plain
|
||||
./gradlew verifyEnvKeys --console=plain
|
||||
```
|
||||
|
||||
### Task 0.3 Machine-readable readiness baseline
|
||||
|
||||
상태: 2026-07-28 구현 및 mutation/registry 검증 완료.
|
||||
|
||||
파일:
|
||||
|
||||
- 추가: `src/config/jpa/readiness-cards.yaml`
|
||||
- 수정: `src/build.gradle`
|
||||
- 추가: persistence-jpa readiness registry parser/validation tests
|
||||
|
||||
구현:
|
||||
|
||||
1. 설계 §31.3의 15 card와 7 owned migration stream을 exact key로 옮긴다.
|
||||
2. unknown/missing card, duplicate task, cycle, missing prerequisite, duplicate
|
||||
location/history를 fail-closed한다.
|
||||
3. 현재 구현되지 않은 task/card는 `not-implemented`로 유지한다.
|
||||
4. 존재하지 않는 target task를 통과 증거로 만들지 않는다.
|
||||
5. registry structural verification task를 `check`의 architecture policy chain에 연결하되
|
||||
real PostgreSQL readiness를 거짓으로 통과시키지 않는다.
|
||||
|
||||
## 4. Phase 1 — Transaction/failure/deadline foundation
|
||||
|
||||
상태: 2026-07-28 application contract, Spring executor, local timeout, phase-aware outcome,
|
||||
bounded retry/backoff 후보 구현 완료. commit fault injection과 immutable R2 manifest는 남아 있다.
|
||||
|
||||
### Task 1.1 Additive application transaction contract
|
||||
|
||||
소유 leaf: `application-core`
|
||||
|
||||
예상 파일:
|
||||
|
||||
- 추가: `transaction/TransactionPolicy.java`
|
||||
- 추가: `transaction/CallBudget.java`
|
||||
- 추가: `transaction/OperationId.java`
|
||||
- 추가: `transaction/TransactionOutcome.java`
|
||||
- 추가: `transaction/PolicyTransactionPort.java`
|
||||
- 수정: `transaction/TransactionPort.java`
|
||||
- tests: 같은 package의 pure unit tests
|
||||
|
||||
계약:
|
||||
|
||||
- 기존 `inWrite`, `inRead`, `inNew` source compatibility 유지
|
||||
- named write policy는 stable operation ID 요구
|
||||
- legacy facade는 non-replayable/uncorrelated policy로 격리
|
||||
- absolute deadline과 finite timeout intersection
|
||||
- core에는 Spring `TransactionDefinition`/`DurationStyle`을 노출하지 않음
|
||||
|
||||
### Task 1.2 Spring policy executor와 propagation ownership
|
||||
|
||||
소유 leaf: `adapter-outbound-persistence-jpa`
|
||||
|
||||
예상 파일:
|
||||
|
||||
- 수정: `transaction/SpringTransactionPort.java`
|
||||
- 추가: `transaction/SpringPolicyTransactionPort.java`
|
||||
- 추가: transaction phase/outcome collaborator
|
||||
- tests: unit + real PostgreSQL task
|
||||
|
||||
검증:
|
||||
|
||||
- REQUIRED physical owner와 participant 구분
|
||||
- REQUIRES_NEW depth/capacity admission
|
||||
- read/write route mismatch fail-fast
|
||||
- commit callback ordering
|
||||
- locale 없는 `toLowerCase()` 제거
|
||||
|
||||
### Task 1.3 Deadline와 PostgreSQL local timeout
|
||||
|
||||
- Hikari acquisition은 fixed pool timeout으로 유지
|
||||
- action 시작 전 remaining budget pre-gate
|
||||
- first statement 전 `SET LOCAL statement_timeout`, `lock_timeout`
|
||||
- transaction/statement/lock rounding boundary test
|
||||
- pool wait 뒤 total budget overshoot negative test
|
||||
|
||||
### Task 1.4 Phase-aware failure/retry
|
||||
|
||||
- operation/query executor를 모든 production persistence path에 연결
|
||||
- constraint name allowlist
|
||||
- begin/action/flush/commit/after-completion phase 분류
|
||||
- `COMMIT_INDETERMINATE`는 blind retry 금지
|
||||
- pre-commit + replay-safe + budget 조건에서만 whole-transaction retry
|
||||
|
||||
## 5. Phase 2 — Entity/query discipline
|
||||
|
||||
상태: production template에 임의 business aggregate를 추가하지 않고 sample의 기존 entity/
|
||||
mapper/query discipline을 실제 PostgreSQL aggregate CAS와 query-plan fixture로 검증했다.
|
||||
|
||||
### Task 2.1 Aggregate persistence baseline
|
||||
|
||||
- domain aggregate와 persistence entity 분리
|
||||
- mapper round-trip과 invariant failure test
|
||||
- optimistic version/expected-version conflict
|
||||
- audit creation carry-forward와 bulk DML guard
|
||||
- bounded persistence-context batch
|
||||
|
||||
### Task 2.2 Purpose-built query model
|
||||
|
||||
- application projection `*QueryPort`
|
||||
- allowlisted query ID
|
||||
- max page/IN bound와 signed/versioned keyset cursor
|
||||
- N+1 statement budget
|
||||
- native/JDBC query는 `.postgresql` package
|
||||
- representative `EXPLAIN` invariant task
|
||||
|
||||
## 6. Phase 3 — Migration/lifecycle/security
|
||||
|
||||
상태: legacy V1/V3/V4/V5/V6 adoption, independent core stream, PostgreSQL 16 lifecycle/security/
|
||||
migration/transaction/aggregate/query candidate task와 content-addressed manifest producer 구현
|
||||
완료. TLS verify-full/role/redaction, pool lifecycle, fresh/interrupted/rolling migration,
|
||||
transaction concurrency/fault dimension을 실제 PostgreSQL과 transport test로 채웠다. clean CI
|
||||
provenance와 외부 restore rehearsal이 없으면 R2/R3 aggregation은 계속 fail-closed한다.
|
||||
|
||||
### Task 3.1 Legacy adoption과 independent streams
|
||||
|
||||
- legacy V1/V3/V4/V5 checksum/object fingerprint
|
||||
- `capability_schema_registry`
|
||||
- explicit target stream version-0 adoption command
|
||||
- core/optional history table ownership
|
||||
- fresh/LEGACY_ADOPTED/interrupted paths
|
||||
- old/target dual authority rejection
|
||||
|
||||
### Task 3.2 Real PostgreSQL qualification source set
|
||||
|
||||
canonical tasks:
|
||||
|
||||
```text
|
||||
postgresqlLifecycleIntegrationTest
|
||||
postgresqlSecurityBaselineIntegrationTest
|
||||
postgresqlMigrationIntegrationTest
|
||||
postgresqlTransactionIntegrationTest
|
||||
postgresqlAggregateIntegrationTest
|
||||
postgresqlQueryIntegrationTest
|
||||
verifyJpaPrimaryFoundationEvidence
|
||||
```
|
||||
|
||||
Docker/Testcontainers가 없으면 R2 lane은 skip이 아니라 fail이다. local optional task와 evidence
|
||||
producer를 분리한다.
|
||||
|
||||
구현된 evidence task:
|
||||
|
||||
```text
|
||||
verifyJpaEvidenceHarnessContract
|
||||
generateJpaEvidenceManifests
|
||||
verifyJpaCandidateEvidence
|
||||
verifyJpaPrimaryFoundationEvidence
|
||||
```
|
||||
|
||||
candidate task는 11개 active card의 exact JUnit selector, zero-skip count, source/이미지/의존성
|
||||
version과 prerequisite manifest ID를 SHA-256 filename manifest로 남긴다. primary task는
|
||||
`-PjpaEvidenceProfile=r2`, clean revision, CI job/artifact metadata, 모든 base dimension과
|
||||
prerequisite R2를 추가로 요구한다.
|
||||
|
||||
### Task 3.3 Lifecycle/security
|
||||
|
||||
- migration/runtime role 분리
|
||||
- trusted schema/search_path, `PUBLIC CREATE`/`TEMP` revoke
|
||||
- TLS verify-full profile
|
||||
- startup/readiness/shutdown/quiesce
|
||||
- bounded/redacted metric/trace/log
|
||||
- restore/forward-recovery runbook
|
||||
|
||||
## 7. Phase 4 — Owner-safe same-store reliability
|
||||
|
||||
상태: idempotency V2, outbox storage V2, polling delivery V2, inbox V1은 각각
|
||||
`implemented-candidate`. 네 stream 모두 fresh-disabled/first-enable/disable/re-enable/
|
||||
interrupted-recovery의 non-destructive lifecycle을 실제 PostgreSQL에서 검증한다. CDC는 external
|
||||
messaging prerequisite가 없어 `not-implemented`다.
|
||||
|
||||
독립 implementation slice:
|
||||
|
||||
1. `jpa-idempotency-owner-safe-v2`
|
||||
2. `jpa-outbox-storage-v2`
|
||||
3. `jpa-outbox-polling-delivery-v2` 또는 `jpa-outbox-cdc-retention-v1`
|
||||
4. `jpa-inbox-same-store-v1`
|
||||
|
||||
각 slice는 자기 migration stream/task/manifest를 가진다.
|
||||
|
||||
outbox storage 구현은:
|
||||
|
||||
- V3 `outbox_event`를 수정하지 않음
|
||||
- `outbox_publication_control_v2`
|
||||
- `outbox_publication_cutover_v2`
|
||||
- `outbox_event_identity_v2`
|
||||
- `outbox_event_log_v2`
|
||||
- polling 선택 시에만 `outbox_delivery_v2`
|
||||
- fresh/legacy genesis sentinel
|
||||
- legacy mutation trigger/ACL fence
|
||||
- paused old writer와 cutover barrier test
|
||||
|
||||
를 포함한다.
|
||||
|
||||
## 8. Phase 5–7
|
||||
|
||||
상태: 선택된 replica topology, tenant mode/RLS policy, target-like backup/failover environment가
|
||||
없으므로 registry에서 `not-implemented`를 유지한다. 로컬 단일 PostgreSQL 테스트를 해당
|
||||
운영 보장의 대체 evidence로 사용하지 않는다.
|
||||
|
||||
### Phase 5 — Primary/replica
|
||||
|
||||
- 별도 pool/route context
|
||||
- explicit `ReadConsistency`
|
||||
- endpoint-bound lag evidence
|
||||
- strong/RYW primary default
|
||||
- failover authority reconciliation
|
||||
|
||||
### Phase 6 — Tenant/RLS와 JDBC coordination
|
||||
|
||||
- tenant-prefixed unique/FK/query
|
||||
- missing context fail-closed
|
||||
- optional FORCE RLS
|
||||
- runtime role bypass negative test
|
||||
- JDBC coordination은 `EFFICIENCY_ONLY`
|
||||
|
||||
### Phase 7 — R3
|
||||
|
||||
- target-like load/capacity
|
||||
- failover, rolling migration, certificate rotation
|
||||
- backup/PITR restore
|
||||
- outbox/idempotency/inbox reconciliation
|
||||
- measured RPO/RTO와 operator game day
|
||||
|
||||
## 9. 공통 verification ladder
|
||||
|
||||
변경 leaf focused test부터 실행한다.
|
||||
|
||||
```bash
|
||||
cd src
|
||||
./gradlew :application-core:test --console=plain
|
||||
./gradlew :adapter:outbound:persistence-jpa:test --console=plain
|
||||
./gradlew :app-bootstrap:test --tests '*CleanArchitectureTest' --console=plain
|
||||
./gradlew verifyCleanArchitectureDependencies --console=plain
|
||||
./gradlew verifyPublicPathSnapshot --console=plain
|
||||
./gradlew verifyEnvKeys --console=plain
|
||||
```
|
||||
|
||||
전체 `test`/`check`와 real PostgreSQL task는 해당 phase가 경계를 실제로 변경하거나 required
|
||||
task를 추가한 시점에 실행한다. 실행하지 못한 명령은 이유와 남은 위험을 branch-note와 최종
|
||||
응답에 기록한다.
|
||||
|
||||
## 10. Wiki capture
|
||||
|
||||
각 의미 있는 slice가 끝날 때 실제 vault의 branch-note:
|
||||
|
||||
```text
|
||||
raw/branch-notes/codex-jpa-production-capability.md
|
||||
```
|
||||
|
||||
에 다음을 누적한다.
|
||||
|
||||
- design hash와 plan path
|
||||
- 변경 파일/decision ID
|
||||
- RED/GREEN/architecture command와 결과
|
||||
- 실패/차단/known baseline drift
|
||||
- evidence grade와 아직 R2가 아닌 이유
|
||||
- 실제 파생 raw interview/blog/error 판단
|
||||
|
||||
canonical 문서는 별도 요청 전 생성하지 않는다.
|
||||
|
||||
## 11. 최종 실행 결과
|
||||
|
||||
2026-07-28:
|
||||
|
||||
- `./gradlew :sample-portfolio:test --console=plain`
|
||||
→ 성공, 176 tests.
|
||||
- `./gradlew test --console=plain`
|
||||
→ 성공, 1m 59s.
|
||||
- PostgreSQL readiness task 10개
|
||||
(`lifecycle`, `security`, `migration`, `transaction`, `aggregate`, `query`, `idempotency`,
|
||||
`outbox-storage`, `outbox-polling`, `inbox`)
|
||||
→ 성공, 49s. XML 합계 23 tests, `skipped=0`, `failures=0`, `errors=0`.
|
||||
- `./gradlew check --console=plain`
|
||||
→ 성공, 2m 9s, 209 actionable tasks. 같은 실행에서 root architecture policy,
|
||||
Checkstyle, Spotless, SpotBugs와 custom PostgreSQL source set 검증을 통과했다.
|
||||
- `./gradlew verifyCleanArchitectureDependencies verifyPublicPathSnapshot verifyEnvKeys
|
||||
verifyJpaReadinessRegistry --console=plain`
|
||||
→ 성공. 19개 leaf edge, 1개 public path, 113 env keys, exact 15 cards/7 streams 검증.
|
||||
- `git diff --check`
|
||||
→ 진단 없음.
|
||||
- `./gradlew :adapter:outbound:persistence-jpa:verifyJpaPrimaryFoundationEvidence --console=plain`
|
||||
→ 기존 unconditional sentinel을 제거했다. content-addressed candidate manifest를 검증한 뒤
|
||||
candidate profile과 observability, TLS/role/redaction, fresh/interrupted/rolling migration,
|
||||
transaction concurrency 누락을 card별 blocker로 보고 R2를 차단한다.
|
||||
- `./gradlew :adapter:outbound:persistence-jpa:verifyJpaCandidateEvidence --console=plain`
|
||||
→ 성공, active card 11개 manifest 생성. PostgreSQL 23 tests와 primary base aggregation
|
||||
7 tests 모두 zero-skip이고 content hash/prerequisite link를 검증했다.
|
||||
- `bash .github/scripts/verify-gate-matrix.sh`
|
||||
→ 성공, 21 gates verified. PR candidate evidence job과 conditional R2 workflow가 registry에
|
||||
반영됐다.
|
||||
- CI metadata를 주입한
|
||||
`verifyJpaPrimaryFoundationEvidence -PjpaEvidenceProfile=r2`
|
||||
→ PostgreSQL 23 tests와 r2-profile manifest 11개 생성 뒤 의도된 실패, 1m 21s.
|
||||
`worktree-is-dirty`, observability, TLS/roles/redaction, migration
|
||||
fresh/interrupted/rolling, transaction concurrency를 실제 blocker로 보고했다.
|
||||
- `./gradlew test --console=plain`
|
||||
→ 성공, 15s, 78 tasks up-to-date. 직전 evidence lane에서 persistence/app test는 강제
|
||||
재실행했다.
|
||||
- `./gradlew check verifyPublicPathSnapshot verifyDependencyLocks --console=plain`
|
||||
→ 성공, 9s, 230 actionable tasks(37 executed, 193 up-to-date).
|
||||
|
||||
전체 test에서 발견한 sample Flyway 회귀는 independent `V1` stream을 broad
|
||||
`classpath:db/migration`으로 합친 문제와 production/sample `V6` 충돌이었다. sample slice를
|
||||
legacy PostgreSQL location으로 한정하고 disposable poster migration을 `V7`로 이동했다. 세부
|
||||
재현·해결 기록은 Wiki
|
||||
`raw/errors/flyway-independent-stream-broad-root-collision-2026-07-28.md`에 남겼다.
|
||||
|
||||
2026-07-29 completion pass:
|
||||
|
||||
- primary foundation의 pool lifecycle/observability, TLS verify-full/role/redaction,
|
||||
fresh/interrupted/rolling migration, transaction concurrency/fault evidence를 추가했다.
|
||||
- idempotency/outbox storage/outbox polling/inbox 네 독립 stream에
|
||||
fresh-disabled/first-enable/disable/re-enable/interrupted-recovery 실제 PostgreSQL
|
||||
lifecycle test를 추가했다.
|
||||
- `./gradlew :adapter:outbound:persistence-jpa:verifyJpaCandidateEvidence --console=plain`
|
||||
→ **BUILD SUCCESSFUL in 1m 55s**. 11개 manifest 모두 `missing=none`, zero-skip.
|
||||
PostgreSQL producer 38 tests와 web redaction support 2 tests가 실행됐으며 primary
|
||||
aggregation은 20 tests다.
|
||||
- `./gradlew test --console=plain`
|
||||
→ **BUILD SUCCESSFUL in 55s**, 78 actionable tasks.
|
||||
- `./gradlew check verifyPublicPathSnapshot verifyDependencyLocks --console=plain`
|
||||
→ 포맷과 test fixture SQL construction을 수정한 뒤 **BUILD SUCCESSFUL in 12s**,
|
||||
231 actionable tasks. 19 leaf architecture, Checkstyle, Spotless, SpotBugs, dependency lock,
|
||||
env/readiness/public-path gate를 통과했다.
|
||||
- CI 메타데이터 형식만 주입한
|
||||
`verifyJpaPrimaryFoundationEvidence -PjpaEvidenceProfile=r2`
|
||||
→ **의도된 BUILD FAILED in 2m 6s**. missing evidence는 없고 root blocker는
|
||||
`worktree-is-dirty`; 다른 blocker는 prerequisite R2 전파뿐이다.
|
||||
- `bash .github/scripts/verify-gate-matrix.sh`
|
||||
→ **OK**, 21 gates/21 verified.
|
||||
- `git diff --check`
|
||||
→ 진단 없음.
|
||||
|
||||
현재 환경에서 선택된 Phase 0~4 후보의 로컬 구현·검증은 완료됐다. R2 승격은 사람의
|
||||
commit/push, clean revision에서의 retained CI artifact가 필요하고, R3는 target-like
|
||||
backup/failover/load/operator rehearsal 환경이 필요하다.
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user